Effects of Obtrusive Observation and Rules on Classroom Behavior of Adolescents in a Juvenile Residential Treatment Setting

Abstract

Several studies have used interdependent group contingencies to decrease disruptive behavior and increase appropriate behavior for groups of adolescents. In addition, one study demonstrated that rules plus feedback about rule violations, without additional group contingencies, decreased problem behavior and increased appropriate behavior for adolescents in three classrooms within a residential juvenile facility. Given the rapid behavior change observed in the aforementioned study, it is possible behavior changes were produced by reactivity to obtrusive observation from program implementers. To address this question, we used two A-B designs in conjunction with the conservative dual-criterion (CDC) method to evaluate the extent to which obtrusive observation alone and rules, without systematic consequences, decreased problem behaviors in two classrooms within a residential juvenile facility. Results from visual and CDC analyses indicate that (a) obtrusive observation did not affect problem behavior in either classroom and (b) rules decreased problem behavior in both classrooms and increased appropriate behavior in one classroom. In addition, a measure of social validity indicated that the procedures and outcomes were acceptable to the classroom teacher.

Keywords

adolescent problem behavior classroom rules group contingencies juvenile justice problem behavior residential juvenile facilities rule-following behavior

In general, there are three categories of group contingencies: dependent, independent, and interdependent. Within a dependent group contingency, the performance of a selected individual or small group of individuals determines the outcome for the larger group (Litow & Pumroy, 1975). In contrast, within an independent group contingency, implementers deliver consequences to each group member dependent on their individual performance (Litow & Pumroy, 1975). The third category is an interdependent group contingency wherein the implementer combines aspects of the previously mentioned categories (Groves & Austin, 2017; Little et al., 2015). Specifically, the implementer sets the performance criterion and delivers consequences contingent on the collective behavior of all the members of the group (Brogan et al., 2017; Litow & Pumroy, 1975). In addition to decreasing problem behavior, researchers have found that interdependent group procedures increase prosocial behaviors (Groves & Austin, 2019; Skinner et al., 1996) and are highly rated by both instructors and students (e.g., Brogan et al., 2017).

In a seminal study of independent group contingencies, Barrish et al. (1969) evaluated the effects of the Good Behavior Game (GBG) in classrooms with young children. During the GBG, an implementer (a) divided students into teams, (b) provided rules to individuals in the classroom, (c) denoted when members violated those rules, and (d) awarded the winning team special privileges (e.g., 30 min of free time at the end of the day). Barrish et al. found that disruptive classroom behavior decreased while the GBG as in place. Subsequently, researchers have consistently demonstrated the effectiveness of the GBG, and similarly designed interdependent group contingencies, for young children in a variety of settings (e.g., Flower et al., 2014; Groves & Austin, 2017; Kelshaw-Levering et al., 2000; Lee et al., 2017; Little et al., 2015; Sy et al., 2016; Wiskow et al., 2019; Wright & McCurdy, 2012). Based on standards for determining the collective strength of a procedure in the literature (e.g., Lanovaz & Rapp, 2016), there is clearly strong empirical support of the use of the GBG and other group procedures with young children. However, the effects of interdependent group procedures with adolescents are not as clear due to the limited number of studies with this population. Moreover, only a handful of studies have evaluated group procedures for adolescents in juvenile residential facilities.

Recently, researchers have applied the GBG to classrooms containing adolescent students. For example, Hernan et al. (2019) used the GBG plus a structural antecedent (i.e., a box in which instructors asked students to place mobile devices during class) to decrease inappropriate use of mobile devices by adolescents in typical high school classrooms (see also Kleinman, & Saigh, 2011). In series of studies, Joslyn and colleagues showed the GBG decreased problem behavior in several classrooms for delinquent adolescents in alternative placement high schools (Joslyn et al., 2014, Joslyn, Vollmer et al., 2019; Rubow et al., 2018). Although these results are promising, they may have limited generality to classrooms in residential juvenile facilities. That is, classrooms in residential juvenile facilities present unique challenges including (a) the presence of multiple students in each classroom with histories of severe problem behavior, (b) students with varying levels of academic ability, and (c) facility guidelines that may limit the use of certain consequent events (e.g., edible items as rewards for appropriate behavior¹).

Brogan et al. (2017) and McDougale et al. (2019; Experiment 2) were among the first studies to evaluate interdependent group contingency procedures, not the GBG per se, with adolescents in a juvenile residential treatment facility. Brogan et al. found that a group contingency procedure comprised of pre-sessions rules, within-session feedback (regarding rule violations), and group consequences decreased problem behavior across two 50-min therapy groups. McDougale et al. found a similar group contingency procedure decreased problem behavior and increased appropriate line walking during relatively brief (e.g., 5 min to 7 min) transitions between buildings within the campus of the same residential facility. McDougale et al. also showed that rules alone (i.e., without consequences) supported appropriate group behavior after the group had a history of receiving consequences for both rule violations and appropriate line walking.

A few studies have evaluated the effectiveness of rules alone to either increase appropriate or decrease inappropriate group behavior in classroom settings (Foley et al., 2019; Greenwood et al., 1974; Hernan et al., 2019; Madsen et al., 1968; but see Moore et al., 2019). Notably, results from of these studies suggest rules alone are not sufficient for reducing disruptive group behavior for either young children or adolescents unless researchers combined a secondary component (e.g., feedback or consequences) with the rules.

As an extension of the Brogan et al. (2017) and McDougale et al. (2019; Experiment 2) studies, Chinnappan et al. (2019) evaluated the effects of rules and feedback for rule violations on disruptive behavior displayed by adolescent males in the three classrooms within a residential juvenile facility. At the start of each intervention session, an experimenter verbally reviewed the rules for each classroom. During each session, the same experimenter placed a tally on the white board at the front of the classroom to denote each rule violation. At the end of the session, the experimenter provided positive and constructive feedback based on the observed behaviors. Chinnappan et al. found problem decreased in each classroom and appropriate behavior (e.g., hand raising) increased in two classrooms (they did not collect baseline data for hand raising in one classroom). Although the Chinnappan et al. and McDougale et al. studies decreased problem behavior by providing only rules and feedback, both studies also implemented one or more consequent components in conjunction with the rules. Thus, it is not clear whether the antecedent components, alone, were sufficient for reducing problem behavior.

Interestingly, problem behavior in the Chinnappan et al. (2019) study decreased rapidly following the introduction of the intervention phase for each classroom. Likewise, for each classroom the frequency of rule violations in the first and last treatment sessions was comparable, suggesting the effect of providing feedback (i.e., tallies) about rule violations was minimal. In other words, the absence of a transition state (e.g., Brogan et al., 2019; Johnston & Pennypacker, 1993) in the problem-behavior data paths suggests an antecedent component, without a programmed consequent component, was the essential treatment component.

Importantly, Chinnappan et al. (2019) included two antecedent components (i.e., presence of researcher and verbal rules) which may have facilitated the rapid decrease of disruptive classroom behavior independent of the feedback component of the intervention. With respect to the first antecedent component, the presence of the researcher within the classroom could have functioned as an inhibitory stimulus for disruptive behavior. For example, Kazdin (1979) noted individuals behave reactively when they are aware they are monitored or assessed. Most direct observations within applied behavior analysis (ABA) are obtrusive insofar as the observer is typically discernable to others within the given context. For this reason, presence of the observer may affect behavior of adolescents in classroom settings.

The second antecedent component the implementer provided to students in the Chinnappan et al. (2019) study was the clearly stated classroom rules. Within this residential juvenile facility, there are rigid rules and contingencies in place to enforce inhibitory control (Brogan et al., 2018). Thus, it is possible rules alone may effectively control residents’ behavior in this environment due to specific histories of reinforcement for rule-following behavior and punishment for rule-breaking behavior.

There are at least two broad reasons to evaluate the contributions of obtrusive observation to behavior changes produced with group procedures. On a practical level, teachers in settings with heavy regulations may be more likely to implement interventions that require the least modification to their daily teaching strategies (Chinnappan et al., 2019). To that end, decreasing problem behavior with just the presence of an observer could obviate more effortful group intervention components (e.g., tracking rule violations). On a conceptual level, identifying active components of group procedures could extend the current knowledge base on group procedures and give rise to further technological advances (Joslyn, Donaldson et al., 2019). Thus, the purpose of the present study was to evaluate the effects of obtrusive observation, followed by rules without feedback, for two classrooms within a residential juvenile rehabilitation center.

Methods

Participants and Intervention Team

Participants were males aged 14 to 19 years who resided in a juvenile rehabilitation facility for sex offenders. As described by Brogan et al. (2018), the treatment program included individual, family, and multi-family group treatment, educational and vocational services, and recreational activities. Clinical staff made referrals to the applied behavior analysis (ABA) unit for various problem behaviors displayed by residents in the treatment facility. The ABA unit was comprised of (a) graduate students in a master’s ABA program (hereafter denoted as “ABA interns”) who conducted sessions and collected data, (b) undergraduate students in departments of psychology and rehabilitation who helped collect data, and (c) an onsite Board Certified Behavior Analyst who provided oversight and supervision for the individuals in (a) and (b).

Sessions

ABA interns conducted all sessions within two classrooms located on the campus of the residential facility. We identified classrooms for this study via teacher referral indicating high rates of problematic classroom behavior. The first classroom (C1) was a history class that consisted of eight to 11 students during baseline sessions and four to 11 students during intervention sessions. The teacher reported that high levels of students’ disruptive behavior prevented her from instructing the class. Each session in C1 was 30 min in duration. The second classroom (C2) was also a history class with seven to 12 students during baseline sessions and eight to 13 students during intervention sessions. Importantly, the teacher of C1 was also the teacher of C2; none of the students in C2 had spent time in C1. ABA interns conducted baseline sessions with C2 while they conducted the last few intervention sessions with C1. As in C1, the teacher noted that multiple students in C2 were engaging in high rates of disruptive behavior that prevented her from instructing the class. Each session in C2 was 30 min in duration.

Target Behaviors

Observers collected data on interruptions, talking out of turn, out of seat behavior, junk behavior, hand raising, teacher attention to problem behavior, and teacher attention to hand raising. We defined interruptions any instance of student behavior that resulted in the teacher ending or delaying instruction or an activity for 5 s. This included a student asking the teacher for help more than one time after the teacher had instructed the student to wait. We defined talking out of turn as any instance of a student making any sounds or mouthing words when there was an ongoing demand to remain silent. We defined out of seat behavior as standing, kneeling, or squatting beside a desk such that a student’s bottom was out of his chair or more than 1.33 m from an assigned seat. This included sitting in a desk other than the one assigned by the classroom teacher.² We defined junk behavior as hitting, banging, throwing, tearing, crumpling, or breaking any classroom materials or furniture. This included dancing or kicking any part of the desk in front of them. Most observation intervals contained multiple types of problem behavior from multiple students. Similar to Chinnappan et al. (2019), we combined data for interruptions, talking out of turn, out of seat behavior, and junk behavior into a single dependent variable hereafter referred to as “problem behavior.”

Observers also collected data on appropriate classroom behavior and teacher attention to appropriate and inappropriate behaviors. Specifically, we defined hand raising as any instance of a student raising a hand above shoulder level to ask a question from the teacher. We defined teacher attention to problem behavior as any instance of the teacher providing any kind of attention following any problem behavior (e.g., talking out of turn, out of seat behavior). Teacher attention included eye contact, comments, or utterances (e.g., a sigh) from the teacher within 3 s of a student’s behavior (as described above). Finally, we defined teacher attention to hand raising as any instance of teacher attention following hand raising by a student. In the event that a student engaged in hand raising while talking out of turn (e.g., calling the teacher’s name while simultaneously engaging in hand raising), observers scored both an instance of hand raising and talking out of turn.

Data Collection

As in prior studies (e.g., Brogan et al., 2017; Chinnappan et al., 2019; McDougale et al., 2019), observers collected data on students in each classroom as one behaving unit. Specifically, observers collected data for hand raising using continuous frequency recording and other target behaviors using 10-s partial interval recording (PIR). As noted by Chinnappan et al. (2019), studies have shown 10-s PIR detects a high percentage of changes in frequency events with visual analysis during 30-min observations in single-case experimental designs (Devine et al., 2011). Observers also collected data on the teacher’s attention following problem behavior and hand raising as secondary dependent variables (i.e., we did not use data paths for these variables to makes decisions about phase changes). Observers paused the session timer if interruptions occurred for fire drills or resident counts and subsequently resumed the timer after the interruption abated.

Interobserver Agreement

Secondary observers scored 46.15% and 35.7% of the sessions for C1 and C2, respectively. We calculated interobserver agreement (IOA) scores for hand raising using the proportional method within 10-s bins (Mudford et al., 2009). We divided each 30-min session into 180, 10-s bins. For each bin, we compared the scoring for the secondary observer to the primary observer, divided the smaller count by the larger count, and multiplied by 100. Subsequently, we totaled the percentages and divided that value by the total number of bins across sessions to arrive upon the mean IOA score. We calculated IOA scores for target behaviors scored with 10-s PIR using the interval-by-interval method (Cooper et al., 2007). We scored an agreement if both the primary (e.g., first author) and secondary observers (e.g., trained graduate or undergraduate students) scored an interval as either an occurrence or nonoccurrence of the specific target event. For each target behavior, we calculated IOA scores for each session by dividing the total number of agreements by the total number of intervals in the observation period and converting that into a percentage (e.g., multiplying by 100). For C1, mean IOA scores for interruptions, talking out of turn, out of seat behavior, junk behavior, teacher attention to problem behavior, hand raising, and teacher attention to hand raising³ were 99.86% (range, 98.8 to 100), 92.36% (range, 78.8 to 97.2), 98.4% (range, 93.8 to 100), 99.6% (range, 98.8 to 100), 97.5% (range, 94 to 99.3), 99.5% (range, 98.3 to 100), and 99.02% (range, 97.7 to 100), respectively. For C2, mean IOA scores for interruptions, talking out of turn, out of seat behavior, junk behavior, teacher attention to problem behavior, hand raising, and teacher attention to hand raising were 100%, 93.77% (range, 86.1 to 99.4), 97.6% (range, 92.2 to 100), 99.2% (range, 97.2 to 100), 98.1% (range, 93.8 to 100), 100%, and 100%, respectively.

Experimental Design

We evaluated the effects of obtrusive observation and rules on problem behaviors in each classroom using two A-B designs. Prior to beginning the study, we planned to use (a) visual analysis to make decisions about phase changes based on problem behavior as we conducted the study and (b) supplemental statistical analysis upon completion of the study for both problem behavior and hand raising. It was possible that obtrusive observation could exert small or inconsistent effects (i.e., change behavior in one classroom but not the other), which could disrupt the demonstration of experimental control in a nonconcurrent multiple baseline design (e.g., Coon & Rapp, 2018). Thus, we opted to use the conservative dual-criterion (CDC) method to evaluate the effects of the interventions upon completion of data collection. Lanovaz et al. (2017) found that A-B designs with A phases containing three or more data points and B phases containing five or more data points produced very low proportions of false positives when analyzed with the CDC method. Thus, we planned to conduct at least three sessions in the baseline phase and at least five sessions in the treatment phases for each classroom.

Visual analysis

As previously noted, Chinnappan et al. (2019) found changes in their dependent measures during the first session of their intervention in each classroom. Lanovaz et al. (2019) found most A-B designs that showed rapid and large changes in the dependent variable during the first B phase later yielded replicable changes in the dependent variable. Thus, we expected any changes in the dependent variable would be visually detectable within one or two sessions after the introduction of the antecedent component.

Conservative dual criterion analysis

As a supplement to visual analysis, we used the CDC analysis method (Fisher et al., 2003⁴; Lanovaz et al., 2017) to evaluate the extent which obtrusive intervention and rules decreased problem behavior in each classroom. Initially, we used data in the baseline phase to determine both the mean and regression lines that we projected into the obtrusive observation phase. After the CDC analysis revealed that there was not a significant reduction in disruptive behavior in the obtrusive observation phase in either classroom (described below), we combined the baseline and obtrusive observation phases and recalculated the mean and regression lines for comparison to the last two intervention phases. The combined baseline and obtrusive observation phase for C1 and C2 contained nine and eight data points, respectively.

Procedures

Prior to the baseline phase, the participating teacher granted permission for ABA team members to conduct classroom observations. ABA interns discussed the procedures and the schedule with the teacher. ABA interns conducted classroom observations once per day, one to three times per week, for six to nine weeks (excluding the maintenance probe for C2; described below).

Baseline

In this phase, ABA interns instructed the teacher to conduct class as she normally would. Observers stood or sat in different areas of the classroom. Neither the observers nor the teacher provided students with information regarding their presence in the classroom. If students attempted to interact with an observer, the observer continued to focus on her clipboard and did not respond. Following the observation, observers left the classroom setting without delivering any feedback to either the students or teacher. The purpose of this phase was to (a) conduct unobtrusive observations and (b) establish a baseline for disruptive behavior against which we could compare the effects of obtrusive observations. We planned to conduct three to five sessions in this phase.

Obtrusive observation

Prior to beginning this phase, ABA interns reminded the teacher of the procedures. Similar to baseline, ABA interns instructed the teacher to teach and to respond to all problem behavior as she normally would. Before each observation began, the primary observer solicited the attention of the students and provided a statement indicating she was there to observe their behavior (e.g., “Today I will be observing you throughout the class period.”). Following the statement, observers started the session timer. During the session, observers responded the same way as they did in the baseline phase. Immediately after each session, observers left the classroom without providing feedback to the students. The purpose of this phase was to determine the extent to which obtrusive observation of the students in their classroom decreased their disruptive behavior. We planned to conduct five or more sessions in this phase.

Rules alone

Prior to beginning this phase, ABA interns met with the classroom teacher to develop up to three classroom rules (see Appendix A). At the beginning of this phase, ABA interns asked the teacher to instruct and respond to all problem behavior as she normally would. Before the first intervention session began, an ABA intern stated the rules for the respective classroom and solicited questions from the students. This process took approximately 5 min. After the ABA intern stated the rules, observers started the session timer and began collecting data. At the beginning of subsequent sessions, the ABA intern briefly reviewed the rules, which took approximately 1–3 min. During each session, observers responded toward students in the same manner as in the prior two phases. Immediately after each session, observers left the classroom without providing feedback to the students. The purpose of this phase was to determine the extent to which clearly stated rules decreased students’ disruptive behavior in each classroom. We planned to conduct at least five sessions in this phase.

Teacher implementation of rules (C1 only)

Procedures for this phase were similar to those in the rules alone phase with the exception the teacher provided the rules at the beginning of the session. Prior to this phase, ABA interns provided instructions to the classroom teacher on implementing the rules. Observers collected data on student behavior in the same manner as during the rules alone phase. We planned to conduct at least five sessions in this phase. In addition, observers collected treatment integrity data on the teacher’s performance. For each session, we calculated a treatment integrity score by dividing the number of correctly implemented components by the total number of treatment components and multiplying by 100.

Maintenance probes (C2 only)

ABA interns conducted a 6-week maintenance probe in C2 using the same procedures as the rules alone phase. ABA interns were unable to conduct maintenance probes in C1 due to schedule changes within the facility.

Social Validity

Following the final intervention phase, the researcher delivered a modified version of the social validity questionnaire used by Chinnappan et al. (2019). The classroom teacher completed and later returned a questionnaire for each classroom to the first author. The questionnaire (see Appendix B) included items in which the teacher rated on a 5-point scale of value in which 0 indicated “No,” and 5 indicated “Yes.” The survey items included the following: the teacher’s satisfaction with the intervention, feasibility of implementing the procedures for the teacher, and the ease of implementation for the teacher.

Results

Figures 1 through 4 depict the results for C1 and C2. For each figure and dependent measure, we first describe the results of visual analysis. Subsequently, we describe the results of the CDC analysis (see also Figures S1-S8 in supporting files). Figure 1 shows the percentage of 10-s intervals with problem behavior (primary y-axis) and the responses per minute of hand raising (secondary y-axis) across phases for C1. During the baseline phase, students engaged in a moderate to high level of problem behavior (M = 50.9%) and zero instances of hand raising. During the obtrusive observation phase, students continued to engage in moderate levels of problem behavior (M = 41.1%) and they emitted one instance of hand raising (M = 0.006 responses per minute [rpm]). During the rules alone phase, students’ problem behavior decreased (M = 14.2%) and hand raising increased (M = 0.2 rpm) relative to both the baseline and the obtrusive observation phases. During the teacher implemented rules phase, students’ problem behavior (M = 19.2%) and hand raising (M = 0.18 rpm) were comparable to the rules only phase. For each session in this phase, the treatment integrity score for the teacher’s implementation of the rules components was 100%.

Figure 1.

Percentage of 10-s intervals with problem behavior (primary y-axis) and rate of hand raising (secondary y-axis) across baseline, obtrusive observation, rules alone, and teacher implemented rules phases for Classroom 1. C1 = classroom 1.

Figure 2.

Percentage of 10-s intervals with teacher attention to hand raising (open circles) and teacher attention to problem behavior (closed squares) across baseline, obtrusive observation, rules alone, and teacher implemented rules phases for Classroom 1. C1 = classroom 1.

Figure 3.

Percentage of 10-s intervals with problem behavior (primary y-axis) and rate of hand raising (secondary y-axis) across baseline, obtrusive observation, and rules alone phases for Classroom 2. C2 = Classroom 2. MP = Maintenance probe at 6 weeks after observation 13.

Figure 4.

Percentage of 10-s intervals with teacher attention to hand raising (open circles) and teacher attention to problem behavior (closed squares) across baseline, obtrusive observation, and rules alone phases for Classroom 2. MP = Maintenance probe at 6 weeks after observation 13.

We conducted additional analyses of the data in Figure 1 with the CDC method. Using the mean and trend lines projected from the baseline phase to the obtrusive observation phase, results of the CDC analysis indicated that there was not a statistically significant decrease in problem behavior during the obtrusive observation phase compared to the baseline phase. Subsequently, we combined data from the baseline and obtrusive observation phases, and then projected the new mean and trend lines into the rules only and teacher implemented rules phases. For the decrease in problem behavior to be statistically significant, at least 12 of 16 data points in the combined rules phases had to be below both the mean and regression lines from the combined baseline and obtrusive observation phases. Results of the CDC analysis showed that problem behavior was lower for 15 of 16 sessions, thus indicating problem behavior was significantly lower in the two rules phases compared to the baseline and obtrusive observation phases. We repeated the same analyses for hand raising. Specifically, results of the CDC analyses indicated that hand raising (a) did not change in the obtrusive observation phase relative to the baseline phase and (b) increased significantly during the combined rules phases compared to the combined baseline and obtrusive observation phases. In summary, obtrusive observation did not affect C1’s problem behavior; however, rules from either an ABA intern or the teacher decreased C1’s problem behavior and increased C1’s appropriate behavior.

Figure 2 depicts the percentage of 10-s intervals the teacher provided attention following problem behavior and hand raising across phases for C1. Across both the baseline and obtrusive observation phases, only one instance of hand raising occurred. As such, the teacher had limited opportunity to attend to hand raising. During the baseline phase, the teacher attended to a low percentage of problem behavior (M = 14.7%). During the obtrusive observation phase, there was a slight increase in teacher attention to problem behavior (M = 22.47%). During the rules alone phase, teachers’ attention to problem behavior continued to increase relative to baseline and the obtrusive observation phases (M = 39.67%). As hand raising increased (see Figure 1), so did teacher attention to hand raising (M = 46%). During the teacher implemented rules phase, teacher attention to hand raising remained higher (M = 67.64%) than teacher attention to problem behavior (M = 19.8%). Collectively, these results suggest that during the two rules phases, the teacher attended to student’s hand raising more frequently than to students’ problem behavior.

Figure 3 depicts the percentage of 10-s intervals in which students engaged in problem behavior (primary y-axis) and the rate of hand raising (secondary y-axis) across phases for C2. During the baseline phase, students engaged in a moderate to high level of problem behavior (M = 51.5%) and low rates of hand raising (M = 0.07 rpm). During the obtrusive observation phase, students’ problem behavior (M = 35%) decreased slightly, but remained relatively high, and hand raising, with the exception of session 4, remained relatively low and comparable to the baseline rate (M = 0.18 rpm). During the rules alone phase, students’ problem behavior decreased relative to previous phases (M = 14.5%) and rates of hand raising increased (M = 0.33 rpm). During the 6-week maintenance probe, students’ problem behavior remained low (4.44% of 10-s intervals) and they did not engage in any instances of hand raising.

As with C1, we conducted additional analyses of the data in Figure 3 for C2 with the CDC method. Using the mean and trend lines projected from the baseline phase to the obtrusive observation phase, results of the CDC analysis indicated that there was not a statistically significant decrease in problem behavior during the obtrusive observation phase compared to the baseline phase. As before with C1, we combined the baseline and obtrusive observation phases, and then projected the new mean and trend lines into the rules only phase (including the maintenance probe). In order for the decrease in problem behavior to be statistically significant, all six data points in the rules only phase had to be below both the mean and regression lines from the baseline and obtrusive observation phases. Results of the CDC analysis indicated that problem behavior was significantly lower in the rules only phase when compared the mean and projected lines from combined baseline and obtrusive observation phases. As with C1, we repeated the same analyses for hand raising. Results of the CDC analyses for C2 indicated that hand raising did not change to a statistically significant extent in either the obtrusive observation phase relative to the baseline phase or during the rules phases compared to the combined baseline and obtrusive observation phases. In summary, obtrusive observation did not affect C2’s problem behavior; however, rules from an ABA intern decreased C2’s problem behavior but did not significantly change C2’s hand raising.

Figure 4 depicts the percentage of 10-s intervals in which the teacher attended to problem behavior and hand raising across phases for C2. During the baseline phase, even though students displayed more problem behavior than hand raising (see Figure 3), teacher attention following problem behavior (M = 11.6%) and hand raising (M =11.67%) was comparable. During the obtrusive observation phase, the teacher attended to hand raising (M = 73.4%) more often than problem behavior (M = 10.5%). During the rules alone phase, teacher attention to hand raising (M = 67.5%) remained higher than teacher attention to problem behavior (M= 39.5%); however, teacher attention to hand raising decreased across sessions in this phase. During the 6-week maintenance probe for the rules alone phase, the teacher continued to provide attention to problem behavior at a comparable level to other sessions. By contrast, teacher attention to hand raising was at zero (students did not engage in hand raising; see Figure 3). Taken together, these results suggest that the teacher was more likely to attend to hand raising, and less likely to attend to problem behavior, after ABA interns implemented the obtrusive observation phase. Table 1 shows that the teacher provided high ratings of approval for both the intervention and the outcomes in C1 and C2.

Table 1.

Mean Rating by Teacher for Classroom 1 and 2.

Survey item	Mean Ratings
Do you think the rules intervention was acceptable?	5
Do you think the amount of the decrease in problem behavior (the effect of the rules) was acceptable or sufficient?	5
Do you feel that the overall goals of this intervention were acceptable, appropriate, and important for the group?	5
Would you recommend the rules intervention to other teachers who are attempting to decrease disruptive behavior in classroom settings?	5
Do you feel like you would utilize this intervention in the future?	5
Do you feel like you were able to spend more time teaching and less time addressing problem behavior with the rules being implemented?	5
Do you feel like your interactions with the students have become more positive since beginning this intervention?	5

Note. Responders used a 1 to 5 scale with the following anchors: 1 = no, 3 = kind of, 5 = yes.

Discussion

The current study evaluated the extent to which obtrusive observation and, subsequently, clearly stated classroom rules, decreased problem behavior and increased appropriate behavior in two classrooms comprised of adolescent males within a residential treatment facility. We evaluated data for problem behavior and appropriate behavior (i.e., hand raising) from each classroom using both visual analysis and the CDC method. Results from these analyses indicated that obtrusive observation did not decrease problem behavior in either classroom. Subsequently, additional analyses indicated that rules from either an ABA intern or the classroom teacher (a) decreased problem behavior in C1 and C2 and (b) increased hand raising in C1. Results also suggest that after students began to raise their hands during class, the teacher was more likely to provide attention for hand raising than for problem behavior. In addition, results from the social validity measure indicate that the teacher approved of both the rules intervention and the corresponding behavior changes.

Findings from this study contribute to the literature in at least two ways. First, this study shows the mere presence of an individual who observes and records students’ behavior in a classroom did not facilitate desirable changes in those students’ behavior. Thus, although behavioral observations are required to document behavior change, results from this study suggest the observation process, in isolation, was not an active treatment component. Second, results of this study replicate and extend the Chinnappan et al. (2019) and Moore et al. (2019) studies by showing rules without additional feedback decreased problem behavior in both classrooms and increased hand raising in one classroom. As in the Chinnappan et al. study, results from this study suggest that (a) rules evoked student’s hand raising and (b) teacher attention following hand raising was likely necessary to support the behavior. Results of the current study differ somewhat from those in the Chinnappan et al. study insofar as problem behavior did not decrease below 10% of intervals for either C1 or C2 (see also Joslyn et al., 2014 and Joslyn, Vollmer et al. 2019). Thus, it is possible feedback following rule violations is necessary to reduce problem behavior by some individuals (e.g., students with weak histories of rule governance), subtle forms of problem behavior (i.e., response forms that may be scored as problem behavior by observers but not detected as such by students), or both.

Although we did not design this clinically driven study to isolate the variables that accounted for the behavior changes, there are at least three possibilities. First, even though our data suggest obtrusive intervention did not influence problem behavior, it is possible some aspect of our manipulation primed students to be receptive to rules ABA interns delivered in the subsequent rules phases. In this way, being informed their behavior was being observed may have strengthened both the evocative (e.g., for hand raising) and inhibitory (e.g., talking out of turn) stimulus control of rules provided prior to each session in the rules alone phase. Thereafter, adherence to those rules may have contacted social positive reinforcement, avoided social positive punishment, or both. Second, the process of providing rules at the beginning of each session may have functioned to increase the value of attention from the teacher for specific behavior. In this way, outlining the rules before each session could function as a “motivative augmental” which temporarily alters the value of teacher attention (e.g., Kissi et al., 2017). That is, attention from the teacher may have been available in prior sessions but noting teacher attention was available for engaging in a simple behavior enhanced the value of that attention. In turn, increasing the value of teacher attention may have devalued or displaced competing reinforcers (e.g., attention from peers). Third, the process of outlining rules may indirectly serve as a form of feedback regarding rule violations. Put differently, students in the classroom may infer they had previously violated rules because ABA interns provided rules to them about specific behaviors they should and should not emit.

Results of this study have important implications for instructors in classrooms with adolescent students. First, results suggest providing clear rules on a regular basis may decrease problem behavior to low, tolerable levels. Importantly, this treatment component required very little effort from the teacher. Second, results suggest obtrusive observation neither increased nor decreased problem behavior in either classroom. Put differently, students did not appear to be reactive to in vivo observations. Due to the nature of the detention setting, which contained numerous videos cameras, it is possible residents had habituated to observations by authority figures. Alternatively, it is possible students in these classrooms lacked skills to evaluate their own recent behavior. Thus, teaching students to report their recent behavior accurately may alter their responsiveness to obtrusive observation. Future research could address this question.

At least two outcomes from this study warrant specific discussion. First, results of the 6-week maintenance probe in C2 indicated students’ problem behavior was low and they did not engage in hand raising. We had not previously observed this combination in either classroom and, thus, the circumstances behind this outcome were not clear. It is possible one or more unreported events (e.g., an investigation of sexual misconduct) within the classroom or facility between observation 13 and 14 influenced this outcome. Second, in C1 the teacher typically did not provide attention for students’ hand raising unless they simultaneously made a verbal request. Informally, it appeared the vocal request, which was technically talking-out-of-turn, served as an orienting response for the teacher, who did not frequently orient toward the students in the class. Relatedly, although student behavior did not appear to be influenced by the obtrusive observation manipulation, results for teacher attention to hand raising in C2 suggest the teacher’s behavior may have been sensitive to the presence of the observer. This outcome is surprising given the ABA interns were in the classroom at the request of the teacher. That is, the teacher was aware that ABA interns would observe her and her students before the obtrusive observation phase. Nonetheless, the increase in teacher attention to hand raising suggests the dynamic between students and teacher may be influenced by rules.

We should also note some limitations of this study. First, we conducted only five sessions in the obtrusive observation phase with each classroom. It is possible students’ behavior could have changed after additional sessions in the obtrusive observation phase. Relatedly, it is possible the presence of observers in the baseline phase decreased students’ problem behavior below the true level. In this way, we simply did not detect the effect of obtrusive observation. Nevertheless, this outcome is unlikely because the teacher indicated problem behavior during the baseline sessions was typical of class periods wherein observers were absent. Second, the number of students in the classrooms varied from session to session; this variability in classroom students may have contributed to the variability in the dependent variables. Notably, this limitation is common to nearly every study of interdependent group contingencies with adolescents (Chinnappan et al., 2019; Hernan et al., 2019; Joslyn et al., 2014, Joslyn, Donaldson et al., 2019; McDougale et al., 2019). Third, results with respect to rules are not clear from this study. As in the Chinnappan et al. (2019) study, results suggest the consequences provided by the teacher for rule-following behavior by students are likely important for maintaining behavior changes. Apart from instructing the teacher to state rules before each class, this study did not attempt to manipulate teacher behavior. It is possible additional components such as feedback for violations or praise for following the rules could produce incremental, but socially important, improvements in behavior. Fourth, the generality of the findings regarding rules may be limited due to our use of a common teacher in both classrooms. Similarly, due to her history of teaching in this setting, the teacher in this study may have had a higher tolerance threshold for problem behavior than would a typical high school teacher. More broadly, this study did not evaluate the extent to which participants’ behavior changes in either C1 or C2 generalized to other classroom or dormitory settings. As suggested by McDougale et al. (2019) following successful transitions, future research should evaluate both the persistence and generalization of behavior changes produced with group procedures.

Results from this study give rise to at least two avenues for future research on group procedures for adolescents in classrooms and other group contexts. First, it is possible there is a critical mass for changing behavior of multiple individuals in groups setting. That is, change in the behavior of one or two students who are strategically located in a classroom may evoke changes in the behavior of other students. For example, if rules evoked hand raising by an individual sitting in the front of a classroom, students sitting behind this individual may observe the behavior and consequence, and thereafter imitate this behavior to contact the same outcome. In this way, rules could indirectly influence the behavior of some individuals. Future research should track the flow of behavior changes across the classroom setting for clues about changes in behavior dynamics following the introduction of rules. Second, future research should evaluate the type of attention the teacher provides for problem behavior and appropriate behavior before and after providing classroom rules. A recent study shows teachers in elementary classrooms were more likely to provide reprimands to “at risk” students than to peer-comparison students (Downs et al., 2019). Many, if not all, of the residents in this juvenile facility would have met criteria for being “at risk.” Future research should determine if strategic placements of attention and instructional support for students in residential facilities could increase academic performance and indirectly decrease problem behavior.

Supplemental Material

Supporting_Information_with_Figures_12-30-19 – Supplemental material for Effects of Obtrusive Observation and Rules on Classroom Behavior of Adolescents in a Juvenile Residential Treatment Setting

Supplemental material, Supporting_Information_with_Figures_12-30-19 for Effects of Obtrusive Observation and Rules on Classroom Behavior of Adolescents in a Juvenile Residential Treatment Setting by Sally A. Hamrick, Sarah M. Richling, Kristen M. Brogan, John T. Rapp and William T. Davis in Behavior Modification

Footnotes

Appendix A

Appendix B

Acknowledgements

The authors thank the Alabama Department of youth Services for their support. We also thank Adam Almanza, Peta Kimber, Emily Longino, Kaleem Morrow, and Rachel Peters for their assistance with data collection.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The fourth author received financial support from the Alabama Department of Youth Services.

ORCID iDs

Sarah M. Richling

John T. Rapp

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Sally A. Hamrick, MS, BCBA, recently graduated from Auburn University. She is currently fulfilling a two-year fellowship as a behavior analyst at the Arc of Central Alabama, Birmingham, AL. Her research interests include application of behavior analytic treatments to children and adolescents in state custody.

Sarah M. Richling, PhD, BCBA-D, is a clinical assistant professor in the Department of Psychological Sciences at Auburn University. Her current scholarly interests include criminal recidivism, cultural and social issues, evidence-based practice and continuing education, the empirical evaluation of common clinical practices, scientific system-building, Interbehaviorism, and the philosophical bases of behavioral science.

Kristen M. Brogan, MS, BCBA, is a doctoral candidate in the Cognitive and Behavioral Science Program at Auburn University. Her research interests include the assessment and treatment of problem behavior emitted by justice-involved youth, problematic sexual behavior, and delay discounting.

John T. Rapp, PhD, BCBA-D, is an alumni professor in the Department of Psychological Sciences at Auburn University. His research interests include the assessment and treatment of stereotypy for children with autism spectrum disorder, treatment of problem behavior display by foster children and justice-involved youth, and applications of single-subject experimental designs.

William T. Davis, BS, is a graduate student in the Applied Behavior Analysis program in the Department of Psychological Sciences at Auburn University. His research interests include application of behavior analytic treatments to adolescents in state custody and conjugate reinforcement.

References

Barrish

H. H.

Saunders

Wolf

M. M.

(1969). Good behavior game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis, 2(2), 119–124. https://doi.org/10.1901/jaba.1969.2-119

Brogan

K. M.

Falligant

J. M.

Rapp

J. T.

(2017). Interdependent group contingencies decrease adolescents’ disruptive behaviors during group therapy: A practitioner’s demonstration. Behavior Modification, 41(3), 405–421. https://doi.org/10.1177/0145445517693812

Brogan

K. M.

Rapp

J. T.

Sturdivant

B. R.

(2019). Transition states in single case experimental designs. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445519839213

Brogan

K. M.

Richling

S. M.

Rapp

J. T.

Thompson

K. R.

Burkhart

B. R.

(2018). Activism and advocacy note: Collaborative efforts by the Auburn University applied behavior analysis program in the treatment of adolescents adjudicated for illegal sexual behavior. Behavior and Social Issues, 27, 11–13. https://doi.org/10.5210/bsi.v.27i0.8267

Chinnappan

Rapp

J. T.

Burkhart

(2019). Effects of rules and feedback on classroom behavior of adolescents in a residential treatment setting. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445519834637

Coon

J. C.

Rapp

J. T.

(2018). Application of multiple baseline designs in behavior analytic research: Evidence for the influence of new guidelines. Behavioral Interventions, 33(2), 160–172. https://doi.org/10.1002/bin.1510

Cooper

J. O.

Heron

T. E.

Heward

W. L.

(2007). Applied behavior analysis (2nd ed.). Pearson.

Devine

S. L.

Rapp

J. T.

Testa

J. R.

Henrickson

M. L.

Schnerch

(2011). Detecting changes in simulated events using partial-interval recording and momentary time sampling III: Evaluating sensitivity as a function of session length. Behavioral Interventions, 26(2), 103–124. https://doi.org/10.1002/bin.328

Downs

K. R.

Caldarella

Larsen

R. A.

Charlton

C. T.

Wills

H. P.

Kamps

D. M.

Wehby

J. H.

(2019). Journal of Positive Behavior Interventions, 21(3), 135–147. https://doi.org/10.1177/1098300718800824

10.

Fisher

W. W.

Kelley

M. E.

Lomas

J. E.

(2003). Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. Journal of Applied Behavior Analysis, 36(3), 387–406. https://doi.org/10.1901/jaba.2003.36-387

11.

Flower

McKenna

Muething

C. S.

Pedrotty-Bryant

Bryant

B. R.

(2014). Effects of the Good Behavior Game on class wide off-task behavior in a high school basic algebra resource classroom. Behavior Modification, 38(1), 45–68. https://doi.org/10.1177/0145445513507574

12.

Foley

Dozier

Lessor

(2019). Comparison of components of the Good Behavior Game in a preschool classroom. Journal of Applied Behavior Analysis, 52(1), 84–104. https://doi.org/10.1002/jaba.506

13.

Greenwood

C. R.

Hops

Delquadri

Guild

(1974). Group contingencies for group consequences in classroom management: A further analysis. Journal of Applied Behavior Analysis, 7(3), 413–425. https://doi.org/10.1901/jaba.1974.7-413

14.

Groves

E. A.

Austin

J. L.

(2017). An evaluation of interdependent and independent group contingencies during the Good Behavior Game. Journal of Applied Behavior Analysis, 50(3), 552-566. https://doi.org/10.1002/jaba.393

15.

Groves

E. A.

Austin

J. L.

(2019). Does the Good Behavior Game evoke negative peer pressure? Analyses in primary and secondary classrooms. Journal of Applied Behavior Analysis, 52(1), 3–16. https://doi.org/10.1002/jaba.513

16.

Hernan

C. J.

Collins

T. A.

Morrison

J. Q.

Kroeger

S. D.

(2019). Decreasing inappropriate use of mobile devices in urban high school classrooms: Comparing an antecedent intervention with and without the Good Behavior Game. Behavior Modification, 43(3), 439–463. https://doi.org/10.1177/0145445518764343

17.

Johnston

J. M.

Pennypacker

H. S.

(1993). Strategies and tactics of behavioral research: 2nd ed. Lawrence Erlbaum Associates, Inc.

18.

Joslyn

P. R.

Donaldson

J. M.

Austin

J. L.

Vollmer

T. R.

(2019). The Good Behavior Game: A brief review. Journal of Applied Behavior Analysis, 52(3), 811–815. https://doi.org/10.1002/jaba.572

19.

Joslyn

P. R.

Vollmer

T. R.

Hernández

(2014). Implementation of the Good Behavior Game in classrooms for children with delinquent behavior. Acta de Investigación Psicológica, 4(3), 1673–1682. https://doi.org/10.1016/S2007-4719(14)70973-1

20.

Joslyn

P. R.

Vollmer

T. R.

Kronfli

F. R.

(2019). Interdependent group contingencies reduce disruption in alternative high school classrooms. Journal of Behavioral Education. Advance online publication. https://doi.org/10.1007/s10864-019-09321-0

21.

Kazdin

A. E.

(1979). Unobtrusive measures in behavioral assessment. Journal of Applied Behavior Analysis, 12(4), 713–724. https://doi.org/10.1901/jaba.1979.12-713

22.

Kelshaw-Levering

Sterling-Turner

H. E.

Henry

J. R.

Skinner

C. H.

(2000). Randomized interdependent group contingencies: Group reinforcement with a twist. Psychology in the Schools, 37(6), 523–533. https://doi.org/10.1002/1520-6807(200011)37:6<523::AID-PITS5>3.0.CO;2-W

23.

Kissi

Hughes

Mertens

Barnes-Holmes

De Houwer

Crombez

(2017). A systematic review of pliance, tracking, and augmenting. Behavior Modification, 41(5), 683–707. https://doi.org/10.1177/0145445517693811

24.

Kleinman

K. E.

Saigh

P. A.

(2011). The effects of the good behavior game on the conduct of regular education New York City high school students. Behavior Modification, 35(1), 95–105. https://doi.org/10.1177/0145445510392213

25.

Lanovaz

M. J.

Huxley

S. C.

Dufour

M. M.

(2017). Using the dual-criteria methods to supplement visual inspection: An analysis of non-simulated data. Journal of Applied Behavior Analysis, 50(3), 662–667.https://doi.org/10.1002/jaba.394

26.

Lanovaz

M. J.

Rapp

J. T.

(2016). Using single-case experiments to support evidence-based decisions: How much is enough? Behavior Modification, 40(3), 377–395. https://doi.org/10.1177/0145445515613584

27.

Lanovaz

M. J.

Turgeon

Cardinal

Wheatley

T. L.

(2019). Using single case designs in practical settings: Is within-subject replication always necessary? Perspectives on Behavior Science, 42(1), 153–162. https://doi.org/10.1007/s40614-018-0138-9

28.

Lee

Penrod

Price

J. N.

(2017). A comparison of cost and reward procedures with interdependent group contingencies. Behavior Modification, 41(1), 21–44. https://doi.org/10.1177/0145445516650964

29.

Litow

Pumroy

D. K.

(1975). A brief review of classroom group oriented contingencies. Journal of Applied Behavior Analysis, 8(3), 341–347. https://doi.org/10.1901/jaba.1975.8-341

30.

Little

S. G.

Akin-Little

O’Neill

(2015). Group contingency interventions with children—1980-2010: A meta-analysis. Behavior Modification, 39(2), 322–341. https://doi.org/10.1177/0145445514554393

31.

Madsen

C. H.

Becker

W. C.

Thomas

D. R.

(1968). Rules, praise, and ignoring: Elements of elementary classroom control. Journal of Applied Behavior Analysis, 1(2), 139–150. http://doi.org/10.1901/jaba.1968.1-139

32.

McDougale

C. B.

Coon

J. C.

Richling

S. M.

O’Rourke

Rapp

J. T.

Thompson

K. R.

Burkhart

B. R.

(2019). Group procedures for decreasing problem behavior displayed by detained adolescents. Behavior Modification, 43(5), 615–638. https://doi.org/10.1177/0145445518781314

33.

Moore

T. C.

Alpers

A. J.

Rhyne

Coleman

M. B.

Gordon

J. R.

Daniels

Skinner

C. H.

Park

(2019). Brief prompting to improve classroom behavior: A first-pass intervention option. Journal of Positive Behavior Intervention, 21(1), 30–41. https://doi.org/10.1177/1098300718774881

34.

Mudford

O. C.

Martin

N. T.

Hui

J. K. Y.

Taylor

S. A.

(2009). Assessing observer accuracy in continuous recording of rate and duration: Three algorithms compared. Journal of Applied Behavior Analysis, 42(3), 527–539. https://doi.org/10.1901/jaba.2009.42-527

35.

Rubow

C. C.

Vollmer

T. R.

Joslyn

P. R.

(2018). The effects of the good behavior game on student and teacher behavior in an alternative school. Journal of Applied Behavior Analysis, 51(2), 382–392. https://doi.org/10.1002/jaba.455

36.

Skinner

C. H.

Cashwell

C. S.

Dunn

M. S.

(1996). Independent and interdependent group contingencies: Smoothing the rough waters. Special Services in the Schools, 12(1–2), 61–78. https://doi.org/10.1300/J008v12n01_04

37.

J. R.

Gratz

Donaldson

J. M.

(2016). The good behavior game with students in alternative educational environments: Interactions between reinforcement criteria and scoring accuracy. Journal of Behavioral Education, 25(4), 455–477. https://doi.org/10.1007/s10864-016-9257-0

38.

Wiskow

K. M.

Matter

A. L.

Donaldson

J. M.

(2019). The good behavior game in preschool classrooms. An evaluation of feedback. Journal of Applied Behavior Analysis, 52(1), 101–115. https://doi.org/10.1002/jaba.500

39.

Wright

R. A.

McCurdy

B. L.

(2012). Class-wide positive behavior support and group contingencies: Examining a positive variation of the Good Behavior Game. Journal of Positive Behavior Interventions, 14(3), 173–180. https://doi.org/10.1177/1098300711421008.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB