Abstract
Addressing the challenging behavior of students requires evidence-based interventions that can be implemented in school settings; however, there is a relative lack of literature investigating effective strategies for high school students in secondary settings. Behavior contracts have been used to address challenging behavior in elementary and middle settings but less so in high school settings. Furthermore, the extent to which they have involved student input at the high school level has been unclear despite research indicating that collaborative intervention development processes may be associated with several additional benefits beyond effectiveness. Despite these empirical shortcomings, behavior contracts may be well suited to address individual high school students’ challenging behavior given their flexibility, collaborative nature, and use of goal setting and clearly stated contingencies (i.e., rules) to promote behavior change. Thus, the purpose of this study was to evaluate the acceptability and effectiveness of a structured interview-informed behavior contract intervention to address the disruptive behavior of high school students. Specifically, we used a multiple baseline design across three students and found that the interview-informed behavior contracts were effective in increasing their academically engaged behavior (Tau = .62) and decreasing disruptive and passive off-task behavior. Limitations and future directions are discussed.
Keywords
Students exhibiting challenging behavior within the classroom jeopardize the learning environment for themselves and others, reducing their ability to be academically successful and increasing their risk for a host of negative long-term outcomes (Chafouleas et al., 2010; Evertson & Emmer, 1982). For this reason, it is imperative that school staff members have access to a range of evidence-based interventions to address students’ challenging behavior as it arises. It is common for such interventions to emphasize increasing student academic engagement, rather than merely decreasing challenging behavior, due to the former’s link to improved academic outcomes (e.g., Lei et al., 2018). That is, because academic engagement is often defined as being incompatible with challenging behavior, meaning the two cannot co-occur, increases in academic engagement necessitate a decrease in challenging or otherwise off-task behaviors, both improving the student’s likelihood for academic success and reducing the behavior from which the concern originated.
Unfortunately, there is a trend in the school-based intervention literature toward addressing the behavioral concerns of elementary and middle school students with far fewer studies devoted to identifying effective strategies for high school students. In their comprehensive review of school-based behavior interventions research, Wilson and Lipsey (2007) found that only 20% of studies evaluated such strategies within a high school setting. This has been a consistent finding in more recent syntheses of school-based behavior intervention research (Blair et al., 2021; Lory et al., 2020; Maggin et al., 2017; Van Camp et al., 2020). This trend is concerning as 42% of high school teachers report that student misbehavior interferes with their teaching (National Center for Education Statistics [NCES], 2018), suggesting a need for strategies to address such behaviors. Most recently, school staff members reported a 44% increase in classroom disruptions from misconduct at the high school level due to the effects of the COVID-19 pandemic (NCES, 2022), highlighting the continued, and perhaps more pressing, need for support.
Many school-based behavioral interventions operate by generating hypotheses about the temporally proximal environmental factors that may be maintaining a challenging behavior (i.e., function-based interventions; Blair et al., 2021; Lloyd et al., 2019); however, the literature supporting function-based interventions in high school settings is sparse (e.g., Lane et al., 2009). Walker and colleagues (2018) noted the limited support for function-based interventions in high school settings and suggested that “older students may have more established patterns of behavior that are resistant to intervention” (p. 212) and that alternative approaches may need to be considered. One alternative approach is to incentivize appropriate behavior through a contrived contingency in the hopes that its reinforcing value outweighs the reinforcing value of the long learning histories maintaining a students’ challenging behavior.
Diaz (2010) directly compared these two approaches (i.e., function-based interventions and contrived contingencies) to behavior intervention development in a sample of three adults with intellectual disabilities. Using a multiple-baseline with an embedded reversal design they demonstrated that contingencies using preferred stimuli (i.e., rewards based on the participants’ choice) were generally as effective in reducing challenging behavior as contingencies using function-based reinforcers (i.e., rewards based on the hypothesized function of the challenging behavior). Furthermore, when given the choice between which contingency they wanted to be implemented, all participants selected the preference-based reward condition over the function-based reward condition more often than now. Although they might not be generalizable to typically developing high school students, the results suggest that an acceptable and effective behavior intervention for adolescents and adults might incorporate client decision-making and preference rather than functional assessment-derived contingencies. Notably, the intervention strategy used to arrange the contingencies in the Diaz (2010) was a behavior contract, which has seen use in school settings, making it a viable option for further exploration.
Behavior Contracts
A behavior contract, also known as a contingency contract, is “a document that specifies a contingent relationship between the completion of a target behavior and access to, or delivery of, a specified reward” (Cooper et al., 2019, p. 672). In other words, a behavior contract specifies a verbal rule that contrives a reinforcement contingency between the target behavior and the specified reward. Critical components of a behavior contract include the target behavior to be performed, who is responsible for performing, and the reward for doing so. Optionally, a task record can be included to track the individual’s performance under the contract. The contracting component refers to an agreement between two or more parties, most often a teacher and student, that the reinforcement contingency will be enforced. Behavior contracts capitalize on the behavior analytic concept of positive reinforcement (Cooper et al., 2019) and the goal-setting theory of motivation (Locke & Latham, 2012), both of which predict an increase in the behavior targeted by the contract.
To provide an illustrative example of these mechanisms at work, consider Flood and Wilder’s (2002) investigation of a contingency contract to address the off-task behavior of an 11-year-old student. At the start of each academic session, the student was told that he could access one preferred item (i.e., positive reinforcement) for every 2 division problems or 10 word problems he completed correctly (i.e., goal-setting). The student was able to earn multiple items in this way throughout each 10 min academic session, and the researchers documented a substantial decrease in off-task behavior when the behavior contract was in place compared to pre-treatment baseline sessions.
Behavior contracts have been identified as an evidence-based practice for classroom management (Simonsen et al., 2008). Bowman-Perrott and colleagues (2015) conducted a meta-analysis of the school-based behavior contract literature (n = 18) and identified a moderate overall effect size (Tau-U = .57). Furthermore, a moderator analysis revealed nearly no differences across age, gender, disability status, or target behavior, suggesting behavior contracts are equally effective for students across Grades K–12 settings. Although this review identified more behavior contract studies at the secondary level (n = 11) than the primary level (i.e., Grades K–6; n = 9), secondary was defined as Grades 7 to 12 and beyond, including both students in middle school and adult high school dropouts. The number of behavior contract studies conducted in high school buildings was only five, just under 30% of the school-based behavior contract literature and mirroring the proportion of high school studies across the broader school-based behavior intervention literature. Furthermore, the scope of these five studies is somewhat restricted, making it unclear whether the results generalize to broader applications.
Behavior Contracts in High School Settings
Williams et al. (1972) first used behavior contracts in a secondary setting to increase the appropriate behavior and reduce the off-task and disruptive behavior of academically gifted seniors in a single classroom within a private parochial high school. A single behavior contract was developed between the teacher and all of the students whereby any student could earn additional free time through a point system by meeting certain behavioral expectations (e.g., bringing appropriate books to class); however, due to low baseline levels of disruptive and off-task behavior and high levels of appropriate behavior, it is difficult to determine whether the behavior contract intervention was responsible for substantial behavior change in this study. A similar study was conducted in a public high school ninth grade English classroom by Arwood et al. (1974). They provided students the opportunity to respond to three questions related to classroom expectations (e.g., “What do you consider as appropriate classroom behaviors?”) and incorporated most of the responses into a contingency contract that was then signed by each student. The contingency contract was layered on top of a teacher-developed token economy and was shown to be more effective than the token economy alone in improving students’ appropriate behavior. Neither study provided much detail regarding how the contingency contracts were developed from student input, making it difficult to determine the extent to which students were a part of the process. Furthermore, because both studies developed a single contract that applied to every student within the classroom it is not clear whether the specified contingencies were relevant to all students.
A brief report by Trice (1990) demonstrated that contingency contracting was more effective than counseling in reducing truancy and equally effective in reducing disruptive behavior exhibited by high school students. Unfortunately, the report provides nearly no methodological detail, obfuscating important details about the contingency contracts and their development that might provide insight into why they were differentially effective. On the other hand, Newstrom et al. (1999) provided a detailed report on the use of a contingency contract to improve a ninth grade student’s use of two academic writing skills (i.e., correct capitalization and correct punctuation). The participant was provided information about the core components of contingency contracts by their teacher and the first author and was encouraged to identify their own reward (i.e., free computer time). Access to the reward was made contingent on accurate capitalization and punctuation during a daily journal writing activity. The contract was agreed upon and signed by all parties and the student was reminded of the contract at the start of each language arts period. Upon implementation of the contract in a staggered fashion across skills, the students’ correct capitalization and punctuation during the journaling activity increased dramatically, suggesting the contingency contract was responsible for the improvement and highlighting the versatile nature of this strategy.
A final example of contingency contracting in secondary settings is provided by Din and colleagues (2003). They used researcher-developed contracts to reduce tardiness in a group of high school students who frequently showed up to class late. Compared to a no-treatment control group, the students performing under the contingency contracts demonstrated a notable decrease in tardiness over the course of a 12-week period. Unlike some of the previously reviewed literature, Din and colleagues (2003) did not seek student input when developing their contingency contracts. Although this is not a necessary component of the strategy, student input into behavior contracts is common and some have suggested that a more collaborative approach to behavior intervention development might lead to more acceptable and effective interventions.
Collaborative Behavior Interventions
Early efforts to involve students in the intervention development process are evidenced by Salend and Ehrlich (1983), who examined a procedure in which the behaviors targeted by an intervention were explained to the students for whom the intervention was developed. They found that this procedure resulted in enhanced outcomes for the students, providing evidence that a collaborative approach could yield improved outcomes; however, a review by Bruhn and colleagues (2016) revealed that less than 30% of school-based behavior interventions involving goal setting included student input, suggesting that they are more often than not excluded from the intervention development process. Most recently, H. N. Johnson and Carpenter (2022) articulated numerous benefits to including students in the intervention development process, specifically through the use of student interviews to inform functional behavior assessments, including enhanced self-advocacy and self-determination, increased likelihood of generalizing the intervention effects to other settings, and facilitating the learning of skills and behaviors to enable students to become more independent. They also report on a case study detailing how a structured student interview was used to inform the development of an effective behavioral intervention to address a 16-year-old student’s inappropriate vocalizations.
The link between collaborative development and improved effectiveness likely runs through students’ perceptions of the intervention’s acceptability. Although there is literature investigating the perceived feasibility and acceptability of classroom behavioral strategies from the perspective of high school classroom teachers (e.g., State et al., 2017), little evidence of high school students’ perceptions of these constructs is available. Incorporating assessments of students’ perceptions of collaboratively developed behavior contracts would provide a more comprehensive understanding of this strategy and its appropriateness for continued use in secondary settings.
Purpose
Addressing the challenging classroom behavior of high school students requires the availability of evidence-based interventions designed to promote academic engagement while reducing the challenging behavior. Despite a relatively sparse literature base, behavior contracts appear to be a promising candidate due to their effectiveness across a wide range of students (Bowman-Perrott et al., 2015) and the regularity with which student input is integrated (e.g., Newstrom et al., 1999). However, as previously noted, the extent to which high school students’ input has been integrated into behavior contracts is unclear, leaving questions about the effectiveness of this strategy. Furthermore, the behavior contract literature in secondary settings has been limited to addressing issues of truancy (e.g., Din et al., 2003), academic skills (Newstrom et al., 1999), and class-wide contracts (e.g., Arwood et al., 1974), leaving questions about whether a behavior contract would be successful in improving the academically engaged behavior of a targeted high school student within the classroom. The purpose of this study was to examine the acceptability and effectiveness of a behavior contract intervention that was informed by brief structured interviews with the high school students for whom they were developed. The following research questions were used to guide the investigation:
Methods
Setting
All participants were recruited from a rural high school in the southeastern United States. This high school had been implementing a Positive Behavioral Interventions and Supports (PBIS) framework for several years that included SW-PBIS at Tier 1, universal screening to identify students identified as at risk, and Check-in/Check-out (CICO) as a Tier 2 strategy. Fidelity of school-wide PBIS implementation was not available at the time the study was conducted. Approximately 600 students were enrolled at the school, with nearly 70% of students qualifying for free and reduced lunch.
Participants
Participants were identified via the school’s universal screening process, which involved administration of the Student Risk Screening Scale–Externalizing (SRSS-E7; Lane & Menzies, 2009). Inclusion criteria for participation in the study were as follows: (a) one or fewer absences, (b) an SSRS-E7 score in the moderate range, (c) not currently receiving Tier 2 services within the school’s PBIS framework and (d) not currently receiving special education services under an eligibility category where behavior problems are prevalent (e.g., Emotional Disturbance). The attendance criterion was included to increase the likelihood that students were present at school to participate in the intervention. Moderate SSRS-E7 scores were used because it is recommended that students scored in the moderate to high risk categories should be provided with additional supports beyond what is provided at Tier 1 (Lane et al., 2014). We excluded students who were already receiving Tier 2 services in order to increase internal validity by removing those services as a potential explanation for any observed behavior change. Similarly, students receiving special education services under a category where behavior problems are prevalent were excluded because these students would likely already have comprehensive behavior intervention plans as part of their Individualized Education Program.
Using these criteria, 16 students were identified as eligible for participation in the study after the school’s fall screening period. In order to select students, course schedules were generated and compared to the data collectors’ availability. That is, data collectors were present at the school on specific days and times of day, so it was necessary to identify students who were in an instructional period during the same time. Eleven of the identified students did not fit this constraint and the remaining five students were selected for participation. At this time, a request for parental consent to participate was sent to the students’ respective parents or guardians. The consent procedures were reviewed and approved by the authors’ institutional review board as were all other study procedures. Due to high levels of on-task behavior during baseline observations (i.e., greater than 80% on-task behavior), two students were dropped from the study and their teachers were provided with indirect behavioral support (i.e., consultation). The remaining three students were included in this study. Please refer to Table 1 for participant demographics, including each participants’ SSRS-E7 score.
Demographic Information for Participants in the Study on Interview-Informed Behavior Contracts
Note. SRSS-E = Student Risk Screening Scale—Externalizing.
Brennan
Brennan was a 14-year-old Black male enrolled in the ninth grade. Brennan was not receiving special education services nor was he under evaluation to receive such services. Observations for Brennan were conducted at the same time every day in his Algebra course while the teacher was lecturing, and students were taking notes.
Chuck
Chuck was a 16-year-old Black male enrolled in the 11th grade. Chuck was not receiving special education services nor was he under evaluation to receive such services. Observations for Chuck were conducted immediately following his lunch period in his Algebra course, which was different than Brennan’s, while the students listened to a brief lecture from the teacher and completed independent math work.
Dale
Dale was a 15-year-old Hispanic male enrolled in the 10th grade. Dale was receiving special education services under the disability category of Speech or Language Impairment. He was pulled out of the general education classroom for 30 min weekly for speech therapy. Additionally, Dale was also an English language learner, and was nearing proficiency on the LAS Links test for the English Language. Dale’s observations were conducted at the same time every day in his biology course while the teacher was lecturing and the students were taking notes.
The interventionists and data collectors were four graduate students completing their doctoral degrees in school psychology. These individuals were part of a university-based research team as a component of their graduate study and did not have an active role in the school outside the scope of this study. All interventionists and data collectors had completed graduate coursework in behavioral intervention and assessment as well as at least 500 hr of school-based experience serving as behavioral consultants. To reduce the likelihood of reactivity during observations, the interventionists and observers for each student participant were not the same person. Teachers were not selected to act as primary interventionists as the purpose of this study was to determine the initial efficacy of the student-informed behavior contract within an ideal implementation environment.
Measures
Several measures were used throughout the course of this study. Below we report the assessments that were used for screening, intervention development, intervention evaluation and monitoring, and social validity evaluations.
Student Risk Screening Scale (SRSS-E7)
The SRSS-E7 (Drummond, 1994; Lane & Menzies, 2009) is a universal screener that was used by the district to identify students who are considered at risk for externalizing behavioral problems. The SSRS-E7 is composed of seven items assessing externalizing behavior risk from the broader SSRS-IE12 (Lane et al., 2012). It is completed by teachers who are asked to rate how often each student in their class exhibits each behavior on a scale of 0 (Never) to 3 (Frequently). The teacher’s ratings are added together for each student, and students are placed in a “risk category.” Total scores of 0–3 are low risk, 4–8 are moderate risk, and 9–21 are high risk (Lane et al., 2017). A recent psychometric evaluation of the SRSS-E7 in secondary settings revealed adequate internal consistency (α = .84) and a unidimensional factor structure measuring externalizing behavior (Lane et al., 2017). Investigations of the SRSS-E7 in elementary and middle school settings support its classification accuracy compared to other externalizing screeners (Lane et al., 2019) and its ability to predict end-of-year office discipline referrals (Gregory et al., 2021).
Problem Identification Interview (PII)
The Problem Identification Interview (Kratochwill & Bergan, 1990) was used to obtain information about each of the participating students’ problem behaviors. The interview was adapted to obtain the following information: presenting problem behaviors, examples and descriptions of problem behaviors, hypothesized antecedents and consequences, as well as frequencies and duration of the behaviors. The information was utilized to assist in the development of operational definitions for student target behaviors but was not used to develop the behavior contracts. The PII form used in this study is included as an online supplemental file.
Children’s Usage Rating Profile (CURP)
The CURP (Briesch & Chafouleas, 2009) is a 21-item questionnaire designed for students to self-report their perceptions an intervention’s acceptability on the following factors: Personal Desirability (α = .92), Understanding (α = .75), and Feasibility (α = .82). Higher scores on the Personal Desirability and Understanding factors and lower scores on the Feasibility factor indicate greater approval of the intervention. Dale, Chuck, and Brennan were asked to complete the CURP 1 week after the end of the intervention phase.
Teacher Social Validity Assessment
Because teachers were not responsible for implementation, a less formal assessment of social validity was developed by the authors to assess the teachers’ perception of the intervention’s impact on their students’ behavior. The assessment was composed of four questions assessing the percentage of student’s on-task behavior (0%–100%) before the intervention, during the intervention, and after the intervention. Teachers were asked to complete this assessment 1 week after the conclusion of data collection when the intervention was no longer being implemented by the researchers and observers had not entered the classroom for 1 week. This assessment asked them to complete a Likert-type scale estimating the percentage of time their student exhibited academically engaged behavior before the intervention was implemented, throughout the implementation phase, and one week after implementation ended. Additionally, teachers were asked to complete one item related to their perceptions about whether the intervention resulted in improvement in their student’s behavior using a Likert-type scale of 1 (strongly disagree) to 5 (strongly agree).
Student Interview Form
A brief student interview to inform the behavior contract intervention was also developed by the authors. Development of the interview form was informal but based on the general principles of applied behavior analysis, specifically drawing from the concepts of behavior function, positive reinforcement, and rule-governed behavior (Cooper et al., 2019). The interview form provided a brief introduction to behavior function, and then asked students why they typically engage in the identified problem behavior in class. If students were unable to clearly identify the function of their behavior, they were provided with a more detailed description of behavior function in addition to examples of functions of behavior (e.g., access attention, escape or avoid task). Students were then asked about possible antecedents and consequences to their problem behaviors. Finally, students were asked what would motivate them to stop engaging in the problem behavior and start engaging in on-task behavior in the classroom. The interviewer was responsible for developing a rule for the student to engage in on-task behavior (e.g., “If I engage in on-task behavior, then I can earn . . .”). This rule was used as the basis of the behavior contract intervention and was negotiated between the student and the interviewer. The interviewer allowed, but did not encourage, students to identify reinforcing consequences that were unconventional, yet realistic. The purpose of this was to maximize the potential reinforcing value of the contingency specified within the contract and invite authentic collaboration between the interviewer and the student. The functional information was used solely for comparative purposes during analysis. The complete interview form can be found in Appendix.
Dependent Variables
The primary dependent variable was academically engaged behavior (AEB). We adapted a definition from the Behavioral Observation of Students in Schools (BOSS; Shapiro, 2010). Specifically, AEB was defined as the student exhibiting active or passive engagement with the current academic task demand. Active engagement included observable responses to academically relevant stimuli including but not limited to asking questions, completing independent work, and talking with the teacher or peers about academically relevant topics. Passive engagement included less salient responding such as steady state orientation of the head and body toward academically relevant stimuli (e.g., speaking teacher, worksheet, textbook) in the absence of an accompanying motor (e.g., writing) or vocal response (e.g., answering a question). For the purposes of this study, both active and passive responding were treated as AEB with no distinction made between them. Students were not considered academically engaged if they left their seat unless the teacher asked them to do so, or the academic task required such movement and they were also actively or passively engaged.
Secondary dependent variables included disruptive behavior (DB) and passive off-task behavior (POT). DB was defined individually for each target student based on the results of the Problem Identification Interview with the teacher. Brennan’s DB included talking out of turn (e.g., making noises when the teacher is lecturing, talking to friends about unrelated topics), being out of seat without permission (e.g., walking around the classroom), completing assignments from other courses not related to the current task demand (e.g. completing English assignment in Algebra class), and playing games on his Chromebook. Chuck’s DB included being out of seat without permission (e.g., laying on the floor, walking around the classroom) and talking out of turn (e.g., speaking when the teacher is lecturing, talking to friends about unrelated topics), and playing games on his Chromebook. Dale’s DB included playing with objects (e.g., empty water bottles, keys, window blinds), talking out of turn (e.g., talking to friends about unrelated topics), out of seat without permission (e.g., walking around the classroom), and playing games on his Chromebook. POT was defined identically for each participant and included the student orienting toward anything but the current academic task demand or appearing otherwise unengaged (e.g., resting head on desk). AEB, DB, and POT were all defined to be mutually exclusive, meaning a student could not simultaneously be exhibiting more than one of these behaviors, and comprehensive, meaning the sum of these three represented.
Data Collection
Using these definitions, student behavior data were collected during 15-min observations within the identified class period using a 15-s momentary time sampling method. A 15-s momentary time sampling was selected to reduce observer demand due to its relative accuracy compared to other interval recording schemes (e.g., Harrop & Daniels, 1986). The 15 min observation lengths were selected based on observer variability. During observations, the observers stood in an unobtrusive location in the back of the classroom where they were still able to view the target students and their academic materials (e.g., Chromebook screen). At the end of each 15-s interval, the data collectors determined whether the student was engaged in AEB, DB, or POT and scored the data sheet accordingly. This process continued for a total of 15 min, yielding 60 intervals of observation data each session. After each session, the data were transformed into the percentage of intervals during which each behavior occurred. This was calculated by dividing the total number of intervals in which one target behavior occurred by the total number of intervals (i.e., 60).
Design and Analysis
This study used a concurrent multiple baseline design across participants to evaluate the effectiveness of the student-informed behavior contract intervention on academically engaged behavior. This particular design was selected because it allows for direct replication of treatment effects across participants, minimizing common threats to internal validity while allowing for a demonstration of experimental control, and does not require a potentially effective treatment to be removed while doing so (Gast et al., 2014). Participants were introduced to the intervention following a minimum of five baseline sessions with a three-point stagger minimum across participant panels. All criteria for What Works Clearinghouse (WWC, 2013) multiple baseline design standards were met (i.e., a minimum of six phases with a minimum of five data points per phase, and a minimum of 20% inter-observer agreement across all participants and phases).
After a minimum of five baseline sessions were completed, visual analysis was used to examine the trend, level, and variability of participants’ AEB. Using these data, we engaged in dynamic decision making (A. H. Johnson & Cook, 2019) to determine when individual participants would change phases from baseline to intervention, ensuring a minimum of 3 data points of staggered implementation across participants. Completed data sets were analyzed primarily using visual analysis of level, trend, variability, immediacy of effect, consistency and nonoverlap (Horner et al., 2005). Secondary analyses included the calculation of treatment effect sizes using Baseline Corrected Tau (BCT; Tarlow, 2017) for changes in students AEB, DB, and POT. BCT was selected due to the improvements it makes compared to other single-case effect size estimates based on nonoverlap (Tarlow, 2017). Effect sizes were calculated by comparing each participant’s baseline data to their behavior contract intervention phase data using an online calculator (Tarlow, 2016). Interpretive guidelines were adapted from Vannest and Ninci (2015) and are as follows: small (0.20 or less), moderate (0.21–0.60), large (0.61–0.80), and very large (0.81 or larger).
Treatment Integrity
Treatment integrity was assessed using an author-developed 5-item checklist with each item corresponding to one of the intervention components. Data collectors were responsible for completing the treatment integrity checklists during every observation session of the intervention phase by marking whether or not each component was implemented by the interventionist. The checklists were identical for each participant and included the following steps: (a) interventionist reminds the student of their behavior goal, (b) interventionist reminds student of the reward for meeting the behavior goal, (c) interventionist calculates the percentage of intervals in which the student was academically engaged, (d) interventionist meets with the student following the 15-min observation to report their performance, and (e) if the goal is met, the interventionist provides the student with the reward, otherwise, no reward is delivered. Treatment integrity was collected for at least 80% of all intervention sessions across participants and averaged 100%.
Interobserver Agreement
The graduate student observers were trained in the observation procedures through a didactic presentation followed by a 10-min practice video in which observers were responsible for collecting data on a target student. Agreement was calculated using an exact interval-by-interval method whereby each interval was scored as an agreement or a disagreement between observers. Percent agreement (i.e., agreements / total number of intervals) was used as an index of interobserver agreement (IOA). Observers had to obtain a minimum of 90% IOA with the lead author on the video trainings in order to collect data for the study. All data collectors met this criterion on their first trial. Observers were then provided with didactic instructions for the current target students and their respective operational definitions. If the observers fell below 80% IOA throughout the course of the study, they would be retrained. There were no observation sessions in which IOA fell below 80%. IOA was calculated for at least 20% of observations across baseline and intervention phases for each participant. IOA was calculated separately for each dependent variable as well as an overall overage. Overall IOA averaged 96.49% (range = 88.23%–100%). Average agreement for AEB was 96.30% (range = 93.33%–98.33%), DB was 96.99% (range = 93.33%–100%), and POT was 97.24% (range = 93.33%–100%).
Procedures
Identification of Problem Behaviors
Once student participants had been identified, the graduate student interventionists met with each student’s teacher to conduct the PII and obtain information related to the student’s problem behaviors. Results of the PII informed operational definitions for each target student. The graduate students also had teachers report a time in the day in which the student’s behavior was most likely to occur, so that the observers could identify an appropriate time to observe the target student in the classroom consistently.
Baseline
During baseline procedures, the teacher was instructed to continue their classroom management routine as normal. Data collectors entered the room approximately 5 min prior to the start of each observation to reduce observer reactivity. Data collection procedures explained above were used throughout each phase of the study. A minimum of five data points were collected for the baseline phase and phase changes to the treatment condition were made upon stability of academically engaged behavior.
Student Interview
Following a minimum of five data points in baseline, and at least a three-point stagger for the participants in Panels 2 and 3, students were introduced to the intervention. Each student met with the primary interventionist to complete the student interview. The results of each interview are presented below.
After the initial introduction to behavior function, Brennan was unable to determine why he engaged in sleeping behaviors; therefore, the interventionist provided an additional explanation and asked questions related to the antecedents and consequences of his behavior. According to Brennan, an antecedent to his problem behavior was falling behind in the class lecture and/or not understanding the current class material. Brennan was unable to identify any consequences to his problem behaviors; however, Brennan’s answer to the final question was that he would be motivated to stay on-task in class if he was provided with a tutor to help him learn the material he does not understand. Therefore, Brennan and the researchers jointly developed the rule that if Brennan remained on-task during algebra instruction, then he would gain access to a brief tutoring session. Specifically, it was decided that Brennan could have access to 15 min of algebra tutoring from the interviewer during a free period immediately after algebra class if he was academically engaged for at least 80% of the observed intervals that day. As mentioned previously, students were allowed to identify unconventional reinforcing consequences as part of their behavior contract. Brennan’s selection of tutoring is an example of such a consequence. We want to make it very clear that these tutoring services were provided in addition to any other academic supports Brennan was receiving and emphasize that they were offered during a free period. They did not replace any academic services provided to Brennan by school personnel and did not serve a functional role in the academic data-based decision-making process. Most often these services involved supporting Brennan in completing his daily algebra homework.
Chuck stated that every time he engages in problem behavior, he believes that his teacher yells at him. When the secondary description about behavior function was provided and Chuck was asked about potential antecedents and consequences to his problem behavior, he identified that when the workload increases in class, he is less likely to remain on-task. Chuck again stated that after he engages in problem behaviors in class, his teacher typically yells at him. The antecedent stimulus (i.e., increased workload) suggests an escape function; however, the consequent stimulus (i.e., teacher reprimands) suggests an attention function. When asked what would motivate Chuck to stop engaging in problem behaviors and start engaging in on-task behavior, Chuck responded that he would be motivated by nap time. Together, the interviewer and Chuck developed the rule that if he remained academically engaged during class, then he would gain access to a brief nap period. Specifically, it was decided that Chuck could have access to 20-min of nap time at the end of class, if he completed all classwork with a minimum of 80% accuracy in the 40-min period following his lunch break. In other words, Chuck’s contract specified a work completion and accuracy goal rather than a duration-based academic engagement goal. Chuck’s teacher was responsible for checking his independent math work for both completion and accuracy at the end of the 40-min period; however, observers continued to assess his AEB during this time. In effort to prevent Chuck from cheating, he was required to show his work and was not allowed to utilize his Chromebook to check his answers.
After reading the introduction to behavior function during Dale’s interview, he stated that he engaged in problem behaviors in class because he is often tired and does not like the class. Because there was not a clear function identified, the interventionist continued with the second part of the interview. Dale was unable to determine any antecedents to his behavior and stated that his consequence is that he typically avoids doing classwork, suggesting an escape function; however, when asked what would motivate him to stop engaging in problem behaviors and start engaging in academically engaged behavior, Dale stated that he would work for money. Together, the interviewer and Dale agreed that if Dale was on-task in class then he would earn a small amount of money. Specifically, it was agreed that if Dale was academically engaged for a minimum of 80% of the observed intervals during the 15 min observation then he would earn $2. Similar to Brennan, Dale’s selection of money may seem like an unconventional stimulus to use as a reinforcer; however, it is considered a generalized conditioned reinforcer (Cooper et al., 2019), and there is a precedent for using it to reinforcer appropriate behavior at school (MacDonald et al., 1970). Furthermore, the amount selected was deemed comparable to the value of other more traditional tangible stimuli that might have been selected otherwise.
Student Intervention
Each student met with the primary interventionist outside of the classroom prior to each intervention session to remind them of their behavior contract, including their daily goal and reward. Once inside the classroom, the observers waited approximately 5 min before beginning the observation to reduce reactivity. Data collection procedures continued as described above. At the end of the 15 min, the student met again with the interventionist to review the data and obtain their reward.
Results
Figures 1 and 2 present the AEB and combined challenging behavior (i.e., POT and DB) for each student participant, respectively. After five baseline data points, Dale’s data exhibited a therapeutic trend (i.e., increasing AEB) so it was decided that he would continue in an extended baseline. Brennan was then randomly selected between himself and Chuck to begin the intervention. After three more data points had been collected, Chuck was randomly selected between himself and Dale to being the intervention. Finally, after three additional data points, Dale begin the intervention. Overall, there was evidence of a functional relation between implementation of the behavior contract intervention, increases in student AEB, and decreases in student DB and POT. Although effects were generally more immediate for reductions in DB and POT, responding under the behavior contract condition was associated with consistently higher AEB compared to the baseline phase and pronounced changes in each DV were only observed when the behavior contract was implemented. A more detailed analysis of individual results are presented in detail below.
Academically engaged behavior for Brennan, Chuck, and Dale.
Challenging behavior for Brennan, Chuck, and Dale.
Brennan
During baseline, Brennan displayed variable levels of AEB, DB, and POT. AEB averaged 24.33% of observed intervals (range = 0%–56.67%), DB averaged 6.67% of observed intervals (range = 0%–20%), and POT averaged 69% of observed intervals (range = 30%–100%). At the start of the intervention phase, Brennan requested that he receive tutoring at the end of his class to avoid being further behind in lecture. Upon implementation of the student intervention, there was no immediate change in AEB, which was slightly variable at first but ended with a stable increasing trend. Brennan was observed engaging in AEB for an average of 57.62% of observed intervals, demonstrating an overall increase in the average percentage of intervals spent engaging in AEB during the intervention phase (range = 5%–83.33%). Brennan demonstrated variable levels of DB at first, however, the level of DB decreased on the third day of intervention and the level of DB remained stable throughout the rest of the intervention phase. Brennan was observed engaging in DB for an average of 18.09% of observed intervals (range = 0%–70%). Following implementation of the student intervention, Brennan demonstrated an immediate decrease in POT, starting with an increasing trend, and ending with a decreasing trend. Brennan was observed engaging in POT for an average of 24.29% of observed intervals, which demonstrates an overall decrease in comparison to baseline level of POT (range= 5%–75%).
Chuck
During baseline, Chuck displayed slightly variable levels of AEB, DB, and POT. AEB remained low during baseline with variability at the end of the baseline phase. Chuck was observed engaging in AEB for an average of 13.75% of observed intervals (range = 0%–63.33%). Chuck was observed engaging in DB for an average of 64.58% of observed intervals (range = 6.67%–100%). Chuck was observed engaging in POT for an average of 21.67% of observed intervals (range = 0%–65%). Following implementation of the student intervention, there was an immediate increase in the level of AEB. Chuck demonstrated slight variability in AEB, however, there was an overall increase in AEB (M = 65.67%; range = 31.67%–83.33%). Chuck demonstrated an overall decrease in DB following the implementation of the student intervention (M = 26%, range = 15%–43.33%). For DB, there was a slight increasing trend following implementation of the student intervention but demonstrated a decreasing trend at the end of the intervention phase. There was an immediate decrease in POT and lower levels of POT remained stable throughout the intervention phase (M = 10.34%; range = 0%–25%). Throughout the intervention phase, there were two instances in which Chuck chose to remain in class rather than take his earned 20-min nap. In one of those instances, he chose to complete his homework assignment in class rather than take it home. In the other instance, he chose to play a Kahoot game to review before an upcoming test.
Dale
During baseline, Dale demonstrated variable levels of AEB, DB, and POT. Dale was observed engaging in AEB for an average of 20.46% of observed intervals (range = 6.67%–51.67%). Dale was observed engaging in DB for an average of 39.33% of observed intervals (range = 17.67%–76.67%). On average, Dale was observed engaging in POT for 41.11% of observed intervals (range = 16.67%–70.59%). At the start of the intervention phase,
Dale requested to collect his earned money in one lump sum every 5 days (i.e., up to $10 dollars weekly). Following implementation of the student intervention, there was an immediate increase in AEB and decrease in DB and POT. Additionally, levels of AEB, DB, and POT remained stable throughout the student intervention phase. Dale was observed engaging in AEB for an average of 90% of observed intervals (range = 81.67%–98.33%). On average, Dale was observed engaging in DB for 7% of observed intervals (range = 1.67%–18.33%). Dale was observed engaging in POT for an average of 3% of observed intervals (range = 0%–8.33%).
Baseline Corrected Tau
Neither Brennan, Chuck, nor Dale’s baseline AEB data exhibited significant trends, thus, Tau was calculated without any baseline corrections. Effect size estimates are displayed in Table 2. For Brennan, BCT calculations demonstrate a moderate effect size for AEB, a small effect size for DB, and a large effect size for POT. For Chuck, BCT calculations demonstrate a large effect size for AEB, a moderate effect size for DB, and a small effect size for POT. For Dale, BCT calculations demonstrate a large effect size for AEB, DB, and POT.
Effect Size for the Three Study Participants’ Behaviors
Note. Effect sizes were classified as either moderate (0.2–0.6), large (0.6–0.8) or very large (0.8–1.00).
Social Validity
Teacher social validity data are displayed in Table 3. All teachers reported perceiving the intervention as effective and noted a clear increase in perceived academically engaged behavior once the intervention was in effect. Dale, Chuck, and Brennan’s CURP data appear in Table 4. Their scores were uniformly very high in support of personal desirability, feasibility, and understanding, indicating general acceptance and support of the behavior contract procedures.
Teacher Social Validity.
Note. Before, during, and after: on-task 0%––19% (1), on-task 20%–39% (2), on-task 40%–59% (3), on task 60%–79% (4), and on-task 80%–100% (5). Improvement: Likert Scale of 1(Strongly Disagree) to 5 (Strongly Agree).
Student Social Validity.
Note. Likert Scale of 1(totally disagree) to 4 (totally agree).
Discussion
Behavior contracts have a long and effective history of use in schools (Bowman-Perrott et al., 2015); however, the extent to which they have been implemented with high school students exhibiting challenging behaviors is limited. The school-based behavior contract literature to date has largely supported their use in elementary and middle school settings with high school applications targeting a restricted range of student behaviors like truancy (e.g., Din et al., 2003) and academic skills (e.g., Newstrom et al., 1999) or focusing on class-wide behavior change (e.g., Williams et al., 1972). The data from this study support the effectiveness of behavior contracts for improving the academically engaged behavior of high school students who exhibit challenging behaviors. This empirical support provides additional information about for whom behavior contracts are effective as well as demonstrating their flexibility in targeting student behaviors not previously addressed in the literature (i.e., AEB).
Further, although we did not conduct a direct comparison, as Diaz (2010) did, our results are consistent with theirs in suggesting that the identification of preferred stimuli (as opposed to functionally-relevant stimuli) may be sufficient to drive desirable behavior change through a behavior contract. This process was made easier given that our participants were typically developing high school students who were capable of providing verbal reports of preferred reinforcers through a brief interview, eliminating the need for lengthier and more structured preference assessments (e.g., Daly et al., 2009; Diaz, 2010).
In comparison, the interview between the student and interviewer was a relatively informal process that prioritized a single question about potentially motivating stimuli. Each behavioral contract was individualized by the collaborative identification of a putative reinforcing consequence to be provided should the behavioral expectation be met. This individualization strategy (i.e., reinforcer modification) is similar to the one used by Kilgus and colleagues (2016) to address the escape-maintained behavior of elementary school students; however, a key difference here is that the assessment used to make the modification in this study was a 5-min interview with the student rather than a teacher-completed checklist (i.e., Functional Assessment Checklist for Teachers and Staff; McIntosh et al., 2008) and 60 min of systematic direct observation.
Because classroom teachers did not play a large role in the development and implementation of the behavior contracts, we did not assess their perceptions of the intervention’s feasibility and usage properties; however, their positive perceptions of student behavior change lend some degree of social validity (Wolf, 1978) to the procedures and outcome data collected by the researchers. Additionally, the students’ positive perceptions of the behavior contracts were encouraging and likely driven in part by their collaborative involvement in the development of each contract. As mentioned previously, it is less common for students to be given such a powerful voice during the intervention development process (e.g., Bruhn et al., 2016). In this study, all three student participants agreed to the goal of 80% of intervals academically engaged, indicating that it was perceived as feasible and fair.
Limitations
There are several limitations to address in order to fully contextualize these findings. First, although the study was conducted in a high school reportedly implementing behavior interventions within a PBIS framework, the school did not routinely collect fidelity data regarding implementation of Tier 1 or Tier 2 supports. Thus, it is not known whether the student participants in this study were adequately exposed to behavioral prevention strategies before being identified as at-risk through universal screening. Additionally, all of the assessment and intervention work was conducted by researchers with almost no involvement from teachers or other school staff members. The benefit of this decision was a tightly controlled experiment in which the efficacy of the student interview-informed behavior contracts could be rigorously evaluated; however, it is not clear whether these procedures would be implemented with fidelity on a larger scale or if turned over fully to teachers. Future research should investigate whether less intensive behavior assessment methods that could reasonably be used by teachers (e.g., direct behavior rating; e.g., Volpe et al., 2020) allow for behavior contract contingency management.
Second, the stimuli and activities selected by students as reinforcers (i.e., tutoring, naps, money) were unconventional and may not be perceived as acceptable to all involved parties. Although we received consent to provide them to students from their caregivers and involved teachers, they do not reflect what is typically available within a school setting. However, the purpose of this study was to work collaboratively with students to develop a contractual agreement that would be reinforcing. Limiting students’ selections to those stimuli most often used in school-based intervention research (e.g., candy, pencils, homework passes, etc.) would have undermined true collaboration. Future research could investigate this issue by comparing the effectiveness of less conventional reinforcers to those used more frequently. Similarly, concerns related to reinforcer magnitude could be addressed through future research examining a process by which the reinforcement schedule could be thinned to reduce the frequency or amount provided.
Third, it is possible that the feedback provided to the student participants by the researchers regarding their behavioral performance at the end of each session contributed to their increases in AEB, preventing definitive conclusions to be drawn about the effects of the reinforcement contingency in isolation. Although previous studies investigating behavior contracts have integrated components using similar, and sometimes more frequent, behavior-specific feedback (e.g., Mruzek et al., 2007), the effect of the feedback component in this study is unknown. Future research should examine whether it is a necessary or meaningful component and, if so, whether it would be possible to fade the feedback over time. There is some empirical support for the continued effectiveness of other school-based behavioral interventions (e.g., CICO) as the number of feedback sessions provided to students is gradually reduced (e.g., Campbell & Anderson, 2011). Reducing the feedback component of this intervention would also lessen the implementation burden associated.
Implications for Practice
In light of these limitations, the results of this study hold two primary implications for practitioners. First, these data support the use of a student-interview informed behavior contract intervention in secondary settings. As discussed previously, there is a dearth of behavior intervention research in secondary settings. Practitioners should consider this strategy to address the challenging behavior of students in high school. Second, because the intervention was implemented with students who had been identified as at-risk using a universal screener, the results may support the use of this intervention as a Tier 2 strategy within a multi-tiered system of supports (MTSS). Although the reinforcing consequence was individualized, the core components of the intervention were standardized, and the assessment used to inform the intervention’s development was very brief. These characteristics make the student-informed behavior contract an ideal option as a Tier 2, which should be accessible within 72 hr of referral, flexible, based on assessment, and allow students to choose to participate (Center on PBIS, 2023).
Conclusion
In sum, these data support the need for further investigation into behavior contracts as efficient and effective interventions to support the academically engaged behavior of high school students identified as at-risk. Furthermore, the efficiency and flexibility of the brief structured student interviews may confer additional practical value over other interventions that may require lengthier assessments (e.g., functional assessments) while maintaining the strong theoretical mechanisms (i.e., positive reinforcement and goal-setting theory of motivation) that drive behavior change in students. Future research is needed to determine if this strategy remains effective when implementation is turned over to teachers and school staff members and it is contextualized within a school building’s broader behavior prevention work.
Supplemental Material
sj-doc-1-bhd-10.1177_01987429231184808 – Supplemental material for Student Interview-Informed Behavior Contracts for High School Students Identified as At-Risk
Supplemental material, sj-doc-1-bhd-10.1177_01987429231184808 for Student Interview-Informed Behavior Contracts for High School Students Identified as At-Risk by Stefanie R. Schrieber, Mary E. Ware and Evan H. Dart in Behavioral Disorders
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
