The Impact of the Contextual Fit Enhancement Protocol on Behavior Support Plan Fidelity and Student Behavior

Abstract

The contextual fit of a behavior support plan refers to the extent that the procedures of the plan are consistent with the knowledge, values, skills, resources, and administrative support of those who are expected to implement the plan. This study used a concurrent multiple baseline design across four participants to assess the presence of a functional relation between introduction of the Contextual Fit Enhancement Protocol, an intervention designed to improve contextual fit, and (a) an increase in fidelity of support plan implementation and (b) improved student behavior. Results indicate that following implementation of the Contextual Fit Enhancement Protocol, support plan implementation fidelity increased and student problem behavior decreased. In addition, teachers participating in the study rated the contextual fit intervention process as effective and efficient. Limitations and implications for future research, practice, and training are discussed.

Keywords

behavior support plans contextual fit implementation fidelity problem behaviors

Problem behaviors such as aggression, non-compliance, disruption, bullying, and withdrawal remain a major challenge in schools. Students with problem behavior experience high rates of negative educational consequences, including academic failure, peer rejection, and isolation (Algozzine et al., 2010; McIntosh et al., 2006; Walker et al., 1996). Students with problem behavior also are at risk for restrictive class placement and school dropout; have low rates of postsecondary employment; exhibit general adult adjustment problems; and experience high rates of delinquency, violence, and substance abuse during adolescence (Fabelo et al., 2011; Nelson et al., 2014; Sprague et al., 2014).

Behavior support plans (BSPs) are team-developed procedures for reducing problem behavior and improving appropriate behavior for individual students (O’Neill et al., 2015). BSPs have been found to be more successful if they are based on results from a functional behavioral assessment (FBA; Iwata et al., 1994; O’Neill et al., 2015). FBA involves an inductive process in which (a) antecedent-behavior-consequence data are collected, (b) behavior patterns in the data are identified, (c) patterns are used to identify the controlling antecedents and maintaining consequences of the problem behavior, and (d) the information is used to guide selection of behavior support strategies that match the pattern and behavioral function identified (Brown & De Pry, 2015; Martens & Lambert, 2014). Moreover, a compelling body of empirical evidence indicates that technically sound BSPs (e.g., those in which the intervention elements are conceptually consistent with the assessment findings) designed from an FBA are more likely to be effective and efficient than support plans not consistent with FBAs (Carr et al., 1999; Strickland-Cohen & Horner, 2015).

Albin et al. (1996) proposed that technically sound BSPs are more likely to be implemented with integrity (e.g., procedural fidelity) if the strategies, procedures, and elements of the plans are perceived to be in the best interest of the student and consistent with the values, knowledge, skills, perceived efficacy, resources, and administrative support of those who implement the plan. This emphasis on contextual fit offered by Albin et al. is based on the goodness-of-fit framework proposed by Bailey and colleagues (1990) as part of their family-centered, early intervention approach. Relatedly, implementation science researchers have indicated that fit between intervention elements and implementer capacity improves the chances of evidence-based practices being effectively implemented over time and leading to positive long-term outcomes (Fixsen et al., 2010; Horner et al., 2014).

Despite suggestions that the contextual fit of support plans affect the quality with which the plans are implemented (Benazzi et al., 2006; Crone et al., 2015; O’Neill et al., 2015; Sandler et al., 2003), systematic studies documenting the effects of contextual fit are rare. To address this need, we designed the Contextual Fit Enhancement Protocol (CFEP; Monzalve & Horner, 2016). The CFEP starts by determining if BSPs are based on the perceived best interest of a student and then examines six core components of contextual fit: (a) implementers have knowledge of the BSP procedures, (b) BSP procedures are consistent with implementers’ values, (c) implementers have the skills required by the BSP, (d) the BSP is perceived as effective and efficient, (e) implementers have the resources to deliver the BSP, and (f) administrative support is available for implementation. These six components were included because they are consistently discussed in the literature as being core elements of contextual fit (Albin et al., 1996; Horner et al., 2014; Sandler et al., 2003).

The CFEP is used within a 45- to 60-min BSP team meeting in which FBA data guide the initial design or modification of a BSP. The CFEP protocol adds to the BSP team’s initial effort to define BSP procedures that are likely to result in behavior change. Once a constellation of procedures has been defined, the team is encouraged to follow the CFEP steps of (a) determining if each of the six contextual fit elements (knowledge, values, skills, effectiveness, resources, and administrative support) is met, (b) modifying the BSP procedures in response to this assessment, and (c) revising the action plan for BSP implementation. Initial field testing of this protocol with two elementary school behavior support teams indicated that the process was perceived as both efficient and effective at adapting BSPs and resulted in BSPs that were implemented with integrity and produced positive change in student behavior.

The purpose of this study was to provide the first empirical analysis of the effect of the CFEP protocol on (a) the fidelity of BSP implementation and (b) student outcomes. The study addressed two experimental research questions and two descriptive research questions:

Is there a functional relation between implementation of the CFEP and the proportion of BSP elements implemented with fidelity by classroom staff?

Is there a functional relation between implementation of the CFEP and reduction in the level of student problem behavior?

Is implementation of the CFEP associated with improved perception of contextual fit for BSPs implemented by classroom teachers?

Is implementation of the CFEP perceived as socially valid by members of behavior support teams?

Method

Participants

Four student–teacher dyads in general education classrooms in two elementary schools participated in the study. Dyads 1 and 3 were in School 1, and Dyads 2 and 4 were in School 2. Dyads were considered for inclusion in the study if (a) the target student was currently receiving specialized supports to address problem behavior through a BSP and (b) the BSP was technically adequate but was not being implemented with high fidelity or impact. Verification of nominated dyads was achieved through two 20-min direct observations by the first author during which teacher behavior was monitored for BSP implementation, and student behavior was monitored for the frequency of problem behavior. The first author also conducted a preliminary assessment of the technical adequacy of each BSP using the BSP Critical Features Checklist. During the study, none of the four students were receiving special education services.

The student in Dyad 1 was a 5-year-old Caucasian male in a half-day general education kindergarten classroom with 27 students, one classroom teacher, and two teaching assistants. Both his regular classroom teacher and teaching assistants reported that he engaged in problem behaviors such as disruptive silly noises; out of seat; yelling; in-seat, off-task behavior; and work refusal. His BSP had been designed and initiated 6 months prior to the onset of the study. The teacher in Dyad 1 had 27 years of experience in working with preschool and elementary school students. She had a master’s degree in elementary education and 23 credit hours in language arts and math. She indicated that she had received district-wide training in positive behavior supports.

The student in Dyad 2 was a 10-year-old Hispanic/Latino male in a fourth-grade general education classroom with 25 students and one classroom teacher. According to his BSP and his regular classroom teacher, he engaged in disruptive noises, verbal refusal to work, and leaving his chair and wandering around the room. His BSP had been designed and initiated 10 months prior to the onset of the study. The classroom teacher in Dyad 2 held a master’s degree in education and had 10 years of experience as a general education elementary teacher. She had previously received training in positive behavior supports through university coursework and district-wide training.

The student in Dyad 3 was a 9-year-old Caucasian female student in a third-grade general education classroom with 22 students and one classroom teacher. According to her BSP and her classroom teacher, she engaged in noncompliant behavior that included work refusal, not following routines, not completing assignments, being out of seat, and disrupting peers. Her BSP had been put in place 6 months prior to the onset of this study. The classroom teacher in Dyad 3 had 23 years of experience as a general education elementary school teacher. She had a master’s degree in education and had received training in positive behavior supports through university coursework and district-wide training.

The student in Dyad 4 was a 7-year-old Caucasian female student in a second-grade general education classroom with 25 students and one classroom teacher. Her regular teacher reported that she engaged in disruptive behaviors such as unsolicited verbal outbursts and talking to peers when the teacher was engaged in instruction and during group activities when students were working. Her BSP had been designed and initiated 6 months prior to the onset of the study. The classroom teacher in Dyad 4 had a master’s degree in elementary education and 10 years of experience as a general classroom elementary school teacher. She indicated that she had received training through university coursework and district-wide training in positive behavior supports.

Students’ BSPs were examined by two expert behavior analysts who had professional expertise in function-based support and were not connected to the present research. Both had at least 5 years of research experience on FBA and its use in designing and implementing BSPs. Experts were selected based on (a) documented experience teaching FBA and BSP design at the university and (b) publication of at least two peer-reviewed original research papers examining the effects of FBA and function-based BSPs.

Setting

The study took place in two elementary schools (kindergarten through fifth grade) that were part of an urban public-school district located in a medium-sized town in the Pacific Northwest. This school district provides education from kindergarten through high school, and students attending these schools are from diverse socioeconomic backgrounds, ranging from lower-to-upper middle class. School 1 served 469 students, including 1% African American, 10% Hispanic/Latino, 9% Multi-racial, 79% White, and 1% other ethnicities. The enrollment for School 2 was 398 students, including 1% African American, 21% Hispanic/Latino, 6% Multi-racial, 69% White, and 3% other ethnicities. Each of the participating schools was implementing Positive Behavioral Interventions and Supports with documented Tier I fidelity as assessed by a score of 70% or higher on the Tiered Fidelity Inventory (Algozzine et al., 2014; McIntosh et al., 2017) during the year in which study data were collected.

Study Measures

Directly observed measures

Direct observation was used to collect data on problem behavior and fidelity of implementation. Each observation session was defined as the first 20 min of an instructional period. Data collection took place in the classroom at a consistent time each day, 3 days per week, per scheduling request of teachers. For each student, the same activity was observed across sessions and was chosen from the FBA interview and direct observation (O’Neill et al., 2015) as likely to be associated with problem behavior. For Dyad 1, observations took place during reading class from 10:15 a.m. to 10:35 a.m. while students completed independent work from their workbooks or different reading activities. For Dyad 2, observations took place during math class from 9:05 a.m. to 9:25 a.m. while the student worked on both tasks guided by the teachers and independent work from his workbook. For Dyad 3, observations took place during math class from 8:55 a.m. to 9:15 a.m. while the student completed math activities guided by the teachers and independent work from her bookwork. For Dyad 4, observations took place during reading class from 12:00 p.m. to 12:20 p.m. while the student participated in group activities, worked on tasks guided by the teachers, and completed independent work. The observers manually recorded their observations with pencil and paper, and they had a timing device (vibrating timer) to gauge the observation intervals.

Fidelity of implementation of BSP strategies was assessed via direct observation using checklists of each student’s BSP elements. Checklists consisted of eight to nine items scored as “yes,” “no,” or “no opportunity.” Items included (a) one-time discreet events (e.g., “When independent assignment is given, teacher pre-corrects by reminding the student of behavioral goals, rewards, and asking for help or a break if needed”); (b) conditional probabilities (e.g., “When working on an independent assignment, if the student appropriately asks for a break, teacher grants a 5-min break within 1 min of request”); and (c) rates of behavior (e.g., “During 20-min observation period, teacher provides 5:1 positive [reinforcing] to negative [corrective] statements”). Each implementation checklist produced a score indexing the percentage of BSP components implemented with fidelity by the classroom teacher during a 20-min observation session. Teachers 1 and 3 started a week later than Teachers 2 and 4 due to school schedule and informed consent processing.

Observers used a 10-s partial-interval observation system to collect data on student problem behavior. As noted above, problem behavior was defined uniquely for each student and recorded if any instance of problem behavior occurred during the 10-s interval. Problem behaviors included disruptive behavior, off-task behavior, out of seat, protest/task refusal, and elopement. Disruptive behavior was defined as,

verbal displays that interrupted the continuity of the class or the ability of students to hear the teacher, the focus student talking with other students and not listening to the teacher, or physical contact between the focus student and either other students or the teacher.

Off-task behavior was defined as “during work time, student engages in any tasks other than the assigned task or ongoing activity (e.g., looking around the room, playing with items, talking, head on the desk) for more than 5 seconds.” Out-of-seat behavior was defined as “student’s buttocks are not in contact with the seat for a minimum of 3 consecutive seconds (at a time student is expected to be seated).” Protest/task refusal was defined as “student does not initiate teacher request/direction within 5 seconds saying ‘no,’ ‘I don’t want to,’ ‘I won’t do it,’ or ‘not now’ to any academic or non-academic request.” Finally, elopement was defined as “student leaves designated area without permission (e.g., classroom).”

Technical adequacy

The technical adequacy of the BSPs was evaluated by the two-person expert panel using the BSP Critical Features Checklist (Strickland-Cohen, 2012), a 20-item instrument that outlines critical elements of a BSP. The experts were provided with the FBA and BSP for each student and asked to indicate whether the BSPs included 20 critical elements. Items were scored as component present or not present. Each item was worth one point. Strickland-Cohen (2012) recommended considering scores of 12–20 to indicate high technical adequacy, 6–11 indicating moderate technical adequacy, and low technical adequacy indicated by scores of 0–5. Formal assessment of the psychometric properties of the BSP Critical Features Checklist is ongoing. Experts A and B rated BSPs as highly technically adequate for Students 2 (scores of 13 and 18, respectively) and 4 (scores of 14 from both experts), and moderately technically adequate for Students 1 (scores of 9 and 7, respectively) and 3 (scores of 8 and 6, respectively). The weaknesses most frequently identified were related to precision in monitoring of problem behavior and absence of strategies for maintaining behavioral gains. However, the experts were in 100% agreement that each of the four plans (a) operationally defined the problem behavior; (b) defined the context most associated with the problem behavior; (c) defined the behavioral function of the problem behavior; (d) included procedures for teaching a competing alternative behavior, strategies for preventing problem behaviors; (e) included strategies for reinforcing alternative/desired behaviors; and (f) included plans for collecting data to assess the impact of the plans on student outcomes. BSP features where the experts disagreed were limited to strategies to minimize reinforcement of problem behaviors.

Contextual fit

The perceived contextual fit of each student’s BSP was evaluated by the classroom teachers who worked directly with the student using the Self-Assessment of Contextual Fit in Schools checklist (Horner et al., 2003) both before and after administration of the CFEP. The Self-Assessment of Contextual Fit in Schools examines the match between BSP elements and contextual fit features. The assessment consists of 16 items that generate scores defining the extent to which individuals implementing a BSP (a) are knowledgeable of elements in the plan, (b) have skills needed to implement the plan, (c) have values consistent with elements of the BSP, (d) have resources available to implement the plan, (e) have administrative support to implement the plan with fidelity, (f) perceive the BSP to be effective, (g) perceive the BSP to be in the best interest of the student, and (h) perceive the BSP to be efficient to implement. Each item is evaluated on a 6-point Likert-type scale (1 = strongly disagree to 6 = strongly agree).

We used the Self-Assessment of Contextual Fit in Schools checklist at the end of each baseline phase to examine the perceptions of teachers about the contextual fit of their poorly functioning Initial BSPs. Teachers also completed the checklist on the Modified BSP after 5 days of implementing the modified plan. For the purpose of comparison, teachers again completed the checklist for their Initial BSPs at this same time.

Social validity

At the end of the study, the four classroom teachers completed the Teacher Questionnaire on Social Validity (Monzalve & Horner, 2016). Although several measures of social validity with documented technical adequacy exist, we developed a measure specifically tailored to the contextual fit model (i.e., assessed each of the elements of teacher knowledge, skill, values, perceptions of efficacy and efficiency, resources, and administrative support) for this study. The questionnaire included eight items that assessed teachers’ perceptions about the role that the CFEP played in the implementation of BSPs in each of their classrooms. Teachers rated agreement with each statement on a 5-point Likert-type scale (1 = strongly disagree to 5 = strongly disagree). The questionnaire also included three open-ended questions assessing (a) overall satisfaction with the CFEP, (b) likelihood teachers would use the CFEP in the future, and (c) recommendations for improving the CFEP process. The Teacher Questionnaire on Social Validity is available from the first author and has not yet undergone psychometric analysis.

Design and Procedures

A concurrent single-subject, multiple-baseline design across participants was used to examine a functional relation between the implementation of the CFEP and (a) increased implementation of BSP elements by teachers and (b) decreased student problem behavior. The schedule of data collection and onset of intervention followed response-guided concurrent multiple baseline protocols (Kratochwill & Levin, 2014). Data collection included at least five data points per phase for each student, and ensured each student had at least five overlapping baseline data points, as recommended by the What Works Clearinghouse (Kratochwill et al., 2010).

Baseline

During baseline, trained observers collected data on student problem behavior and fidelity of BSP implementation. Classroom teachers continued implementing the BSPs as usual. No feedback was provided to the teachers regarding plan implementation or student behavior.

CFEP

The CFEP involved a 45- to 60-min team meeting led by the first author in which each student’s BSP team (e.g., classroom teachers, behavioral specialists) used the CFEP Manual to (a) review and affirm the goals and procedures of the BSP for that student, (b) use the Self-Assessment of Contextual Fit in Schools checklist to assess the contextual fit of their current plan, (c) define specific adaptations to the plan to improve contextual fit, and (d) build an action plan for implementing the revised plan.

The procedures followed during CFEP meetings were same for all four teacher participants. Once baseline data documented a consistent pattern of student problem behavior and implementation fidelity, the first author contacted members of the student’s BSP team to set up a meeting where the CFEP was used. Meetings were scheduled depending on teachers’ availability. For Teachers 1 and 3, the meeting was held at 8:00 a.m. at the conference room of the school. For Teachers 2 and 4, this process was conducted at 3 p.m. in their classrooms. At the beginning of each meeting, the first author explained to teacher participants the goals and procedures that would be addressed during the meeting. Teachers were given a copy of the BSP they were implementing and a copy of the Self-Assessment of Contextual Fit in Schools (Horner et al., 2003) and then asked to rate the contextual fit of the BSP. The first author identified any BSP element rated “3” or less and led a discussion about potential adaptations to the BSP to improve contextual fit.

Teacher 1 provided ratings of 3 or lower regarding “knowledge of BSP procedures,” “skill at implementing,” and “administrative support.” This led to review of the definition for “pre-correction” as a practical classroom strategy, agreement on specific times of day and procedures for using pre-correction, and brief role playing on how pre-correction could be used with Student 1. Teacher 1 also recruited (and received) greater clarity about the priority of the BSP from her administration.

Teacher 2 provided ratings of 3 or lower for (a) having the resources needed to implement the plan and (b) perceiving the plan was likely to be effective as proposed. The process led to review of the BSP procedures and a reallocation of the existing classroom schedule. In addition, Teacher 2 received a 15-min training session from the first author that focused on embedding the proposed behavior support strategies within the normal classroom routine.

Teacher 3 provided a rating of 3 or lower regarding valuing behavior-specific praise as an appropriate strategy for Student 3. The term “behavior-specific praise” was replaced in the BSP with “behavioral feedback,” and Teacher 3 specified forms of positive behavioral feedback that she felt were appropriate to follow targeted student behaviors. The Contextual Review process ended with a brief role-play of how acceptable behavioral feedback would be used.

Teacher 4 provided a rating of 3 or lower regarding administrative support in the implementation of the BSP. A schedule was developed for a weekly check-in between the teacher, building administrator, and team. The brief check-in allowed review of BSP implementation and student behavior (both academic and social). No other changes to the BSP were recommended.

It is important to note that (a) no intervention elements were added or removed from the four BSPs during this process and (b) teachers were not provided any information or feedback regarding the current implementation fidelity or level of student problem behavior. The day following the meeting, data collectors continued to collect data on implementation fidelity and student behavior exactly as they had during the baseline condition. Once each teacher had been exposed to the revised plan for at least five sessions, the researcher met again with teachers and behavioral specialists as during the baseline phase to re-assess contextual fit of the BSPs. This time they were asked to both (a) re-rate the initial plan and (b) rate the modified plan.

Interobserver Agreement

Evaluation of interobserver agreement (IOA) took place during the students’ regularly scheduled classes. The primary observers were graduate students in special education and school psychology trained using videos, examples, and on-site observations prior to beginning formal data collection. Five 1-hr training sessions were provided by the first author and included review of operational definitions and practice using the observation codes and materials with stock videos of unrelated students in classroom contexts. Observers then conducted classroom observations of the teacher and student participants simultaneously with the first author until they met a 90% IOA level for both fidelity of BSP implementation and student behavior (typically two sessions). Once an observer met the IOA criterion, observer data was included in the study. IOA was measured for a minimum of 33% of the observation sessions during both baseline and intervention sessions across variables and participants. For problem behavior, IOA was determined by calculating the number of agreements between the two observers on the occurrence and non-occurrence of problem behaviors, divided by the number of agreements plus disagreements and multiplied by 100. For implementation fidelity, IOA was calculated by taking the number of items on which the two observers agreed and dividing by the total number of items. Average IOA across students for problem behavior was 97%, ranging from 95% to 98%. For implementation fidelity, average IOA was 98% across teachers, ranging from 97% to 98%.

Visual and Data Analysis

The presence of functional relations was visually analyzed following recommendations given by Horner et al. (2005), which included (a) assessing within-phase level, trend, and variability of data; and (b) between-phase changes in level, trend, variability; immediacy of effect; overlapping data (across adjacent phases); and consistency of data patterns. Percent of non-overlapping data (PND) was calculated following guidelines from Parker et al. (2011).

Results

Direct Observation Data

Results for teacher BSP fidelity and student problem behavior are displayed in Figures 1 and 2, respectively.

Figure 1.

Percentage of behavior support plan components implemented during baseline and Contextual Fit Enhancement Protocol.

Figure 2.

Percentage of 10-s intervals with problem behaviors during baseline and Contextual Fit Enhancement Protocol.

Teacher Implementation Fidelity

Baseline

During baseline, a consistent, low level of implementation fidelity was observed across all four teachers. Teacher 1 was not implementing any components included in the BSP. Teacher 2 displayed a low overall implementation with an average level of fidelity at 15% of the BSP components performed and a range of 0% to 57%. Teacher 3 displayed a stable, low level of implementation fidelity for BSP components implemented with a mean level of 16%, ranging from 0% to 25%. Finally, Teacher 4 displayed high variability and low levels of implementation during the baseline phase with an average level of BSP fidelity implementation at 29% of the BSP components and a range of 0% to 71%.

CFEP Condition

Following implementation of the CFEP, all four teachers displayed an immediate and sustained increase in the level of BSP components implemented with fidelity. Teacher 1’s fidelity during the CFEP condition averaged 75%, with a range of 60% to 85% and no clear trend. The percentage of non-overlapping data between baseline and CFEP phases was 100%. Teacher 2’s percentage of BSP components implemented with fidelity increased following CFEP introduction. Although some variability was observed during the intervention phase, the mean level increased to 67%, ranging from 57% to 100%, with 92% non-overlapping data and no clear trend. Similarly, Teacher 3 displayed an instant and sustained increase in the level of BSP components following implementation of the CFEP, with an average of 76%; range of 75% to 85%; 100% non-overlapping data; and a stable, flat trend. Finally, Teacher 4 displayed an immediate change in trend and level of implementation fidelity. The mean percentage of BSP components implemented with fidelity was 83%, with a range of 71% to 100%. The percentage of non-overlapping data points between the baseline and interventions conditions was 96.4%. Across the four teachers, the mean percentage of BSP components implemented with fidelity was 83%, with a range of 57% to 100%. The percentage of non-overlapping data points between the baseline and interventions conditions was 96.4%.

Problem Behavior

Baseline

As shown in Figure 2, baseline levels of problem behavior were moderate to high across all four students. Student 1’s mean percentage of intervals with problem behaviors during 20-min observations was 62%, ranging from 33% to 80%. The mean percentage of intervals with problem behaviors for Student 2 was 43%, ranging from 24% to 80%. Student 2 displayed a moderate level of problem behavior with a flat trend characterized by low variability in baseline, with the exception of session 17. According to the classroom teacher, the high level of problem behavior observed during session 17 was associated with a confrontation about time spent in the preferred resource room. Problem behavior exhibited by Student 3 during the baseline phase was moderate (M = 38%, range = 27%–53%). Finally, the mean level of problem behaviors during baseline for Student 4 was 41% of the intervals, ranging from 25% to 78%.

CFEP Condition

Following introduction of the CFEP, an immediate and consistent decrease in problem behavior was observed across all four students. The percentage of intervals in which Student 1 displayed problem behavior dropped from 67% to 15% for the first data point collected in this phase, with a mean of 14% for the phase, ranging from 3% to 24%, 100% non-overlapping data with baseline, and a descending slope. For Student 2, mean percentage of intervals with problem behavior was 14%, with a range from 3% to 21%, 100% non-overlapping data, and a descending slope. The mean percentage of intervals with problem behavior for Student 3 was 21%, ranging from 10% to 25%, with 100% non-overlapping data and a stable slope. The average percentage of intervals with problem behaviors was 15% for Student 4, with a range of 7% to 19% of intervals, 100% non-overlapping data points, and a descending slope.

Contextual Fit

The contextual fit of the BSP procedures was assessed twice for the Initial BSP (once in baseline, and again after 5 days of implementation with the modified plan) and once for the Modified BSP after 5 days of plan implementation. Results of contextual fit ratings are provided in Table 1. During baseline, the teachers rated the contextual fit of their Initial BSPs as moderate to high. Each of the teachers rated the Modified BSP as having higher contextual fit than the Initial Plan, but the differences were modest for Teachers 1 and 4 (less than 0.5 change) and greater for Teachers 2 and 3 (1.6- and 1.2-point changes, respectively). Teachers 3 and 4 each rated the contextual fit of their Initial BSP lower after experience with the modified version of the plan, Teacher 1’s ratings of the Initial BSP stayed relatively constant after experience with the Modified BSP, and Teacher 2’s ratings of the Initial BSP increased after experience with the Modified BSP.

Table 1.

Teacher-Reported Contextual Fit Scores.

Contextual fit domain	Teacher 1			Teacher 2			Teacher 3			Teacher 4
Contextual fit domain	Initial BSP A	Initial BSP B	Modified BSP	Initial BSP A	Initial BSP B	Modified BSP	Initial BSP A	Initial BSP B	Modified BSP	Initial BSP A	Initial BSP B	Modified BSP
Knowledge of BSP elements	5.5	5.5	6.0	5.0	5.0	5.0	5.0	3.0	6.0	5.0	4.5	5.5
Skills to implement BSP	5.5	6.0	6.0	5.0	5.0	5.0	5.0	4.0	6.0	5.0	5.5	5.5
Values consistent with BSP	6.0	6.0	6.0	5.5	5.0	5.0	5.0	4.5	6.0	5.5	5.5	6.0
Resources to implement BSP	5.5	5.0	5.5	1.0	4.0	5.0	4.5	5.5	6.0	6.0	5.5	5.5
Administrative support	5.5	5.5	6.0	2.5	4.5	5.5	4.5	5.5	5.5	4.0	3.5	4.5
Effectiveness of BSP	5.0	6.0	6.0	1.0	4.0	4.5	3.5	3.0	5.5	5.0	4.5	5.5
BSP is in the best interest of student	6.0	6.0	6.0	3.5	4.0	4.5	4.5	3.5	6.0	5.5	5.5	6.0
Efficiency	5.0	5.0	5.0	3.0	5.0	5.0	5.0	3.5	5.5	5.0	5.0	5.0
M (SD)	5.5 (0.4)	5.6 (0.4)	5.8 (0.4)	3.3 (1.8)	4.6 (0.5)	4.9 (0.3)	4.6 (0.5)	4.1 (1.0)	5.8 (0.3)	5.1 (0.6)	4.9 (0.7)	5.4 (0.5)

Note. Initial BSP A refers to the original BSP assessed during baseline. Initial BSP B refers to the original BSP assessed 5 days after modified BSP was implemented. BSP = behavior support plan.

Social Validity

A summary of data from the Teacher Questionnaire on Social Validity survey is provided in Table 2. Three teachers “strongly agreed” and one “agreed” that the contextual fit process (a) improved the impact of the BSPs on student behavior (decreasing problem behaviors and increasing desired behaviors) and (b) improved teacher clarity about how to implement the BSP in their classrooms or other settings. When asked whether addressing contextual fit had improved implementation of the BSP, two teachers indicated strong agreement, and two of them agreed. Moreover, three of the four teachers agreed and one strongly agreed that the contextual fit process made elements of the BSP more consistent with their values about teaching and behavioral support and that core components of the BSP were not changed after the intervention, but the manner in which they implemented those components was changed.

Table 2.

Teacher Responses to the Teacher Questionnaire on Social Validity.

Item	Teacher 1	Teacher 2	Teacher 3	Teacher 4	M
I believe the process of addressing contextual fit improved implementation of the behavior support plan (BSP)	5	4	5	4	4.5
I believe that the core elements of the BSP did not change, but that the way we implemented those elements did change with the intervention	4	4	4	5	4.3
The contextual fit process made it more clear how to use the BSP in my classroom/setting	5	4	5	5	4.8
The contextual fit process made the elements of the BSP more consistent with my values about teaching and behavior support	4	4	5	4	4.3
The contextual fit process improved the level of administrative support I received to implement this BSP	3	3	4	3	3.3
The contextual fit process improved the impact of the BSP on my student’s behavior (decrease in problem behavior, increase in appropriate behavior)	5	4	5	5	4.8
The contextual fit process improved the degree of effort and stress needed to implement the BSP	4	4	4	3	3.8
The contextual fit process should be used in all future behavior support planning in the district	4	4	4	4	4.0
M	4.3	3.9	4.5	4.2

Note. Likert-type scale for participant responses ranged from 1 = Strongly Disagree to 5 = Strongly Agree.

Discussion

This study is the first formal assessment of CFEP and used a concurrent multiple baseline design across four teacher–student dyads to examine the effects of the CFEP on implementation fidelity of BSPs and student problem behavior. Results indicate that (a) during baseline BSPs were not implemented well and problem behavior maintained and (b) after one 45- to 60-min meeting where BSPs were modified using the CFEP, BSP implementation fidelity improved and student problem behaviors decreased. The design demonstrates the clinical effect at four separate points in time, thereby meeting standards for establishing a functional relation (Horner et al., 2005; Kratochwill et al., 2010). Furthermore, classroom teachers considered the CFEP practical and socially valid. These findings present initial evidence that the CFEP is an effective and efficient strategy that can be used by school staff to improve BSP implementation. In addition, this study’s results contribute to the implementation science field with new insights on the importance of contextual fit for successful implementation of evidence-based practices as well as how this construct may be assessed and adapted in applied settings.

Implications for Future Research

Although there is general agreement that contextual fit is a multifaceted construct, there is no evidence showing how and to what extent each of the contextual fit elements affects implementation of evidence-based practices. Future research should focus on defining and explaining which elements of contextual fit are related to, independently or in interaction with other elements, improved implementation. Teachers indicated in their social validity interviews that (a) clarifying exactly what implementation of the plan required, (b) rehearsing specific ways to implement the plan within their physical context and curriculum materials, and (c) clarifying the administrative priority and support for plan implementation were positive elements of the CFEP process. Future researchers should also examine which specific contextual fit variables are associated with improved generalization and maintenance.

In the future, researchers should focus on developing technically adequate measures of contextual fit. Although the Self-Assessment of Contextual Fit in Schools has been used in several studies (Benazzi et al., 2006; Pinkelman & Horner, 2017; Strickland-Cohen & Horner, 2015), its psychometric properties have not been investigated.

Implications for Practice

This study provides an example of how one 45- to 60-min meeting, in which the BSP team and first author worked together to assess and improve the contextual fit of procedures included in the BSPs, can be an efficient and effective way to improve BSPs. We propose that district and school teams establish systems to embed the contextual fit process as a key component that should be addressed during initial development and ongoing adaptation of BSPs. Thus, using the tools and procedures presented in the CFEP intervention, school teams may be able to (a) develop more contextually appropriate and efficient plans, (b) assess to what extent BSP procedures are contextually appropriate before investing in direct implementation efforts, (c) improve BSPs that are not obtaining desired outcomes, and (d) identify contextual issues to be modified to obtain high levels of implementation fidelity and improved student outcomes. Note that although we used the CFEP to adapt existing BSPs in the current analysis, we strongly recommend use of the same protocol during initial development of BSPs.

Moreover, district and school administrators should consider providing professional development and coaching activities that focus on core components of fit, and strategies to assess and enhance contextual fit of BSPs. We recommend that training and professional development also focus on providing school teams with the skill sets and sufficient knowledge about key components of contextual fit. Furthermore, coaching should be provided to assist school personnel as they implement new skills and practices required by a BSP (Kretlow et al., 2012). Some tasks that can be supported by coaches are (a) use of contextual fit assessment tools, (b) collection and interpretation of contextual fit data, (c) identifying procedures to improve the contextual fit of plans, and (d) coordination of different school parties (i.e., classroom teachers, behavioral specialists, administrative personnel, and family members).

Limitations

There are limitations that should be considered when interpreting study results. First, findings may not apply to other geographical regions, to teachers who had not received training in Positive Behavioral Interventions and Supports, or to students who exhibit more or less problem behaviors. Another limitation is that, although the Self-Assessment of Contextual Fit in Schools (Horner et al., 2003) includes items that examine all core components of fit discussed in the literature, to date no systematic studies have documented psychometric properties for this measure. Relatedly, the variables associated with contextual fit are based primarily on conceptual theory rather than empirical evidence. As such, future research is needed to test which elements of contextual fit most strongly predict improved implementation fidelity and student outcomes. Furthermore, we assessed implementers’ self-perceptions to assess the degree of fit of an intervention, which may not always be valid. We recommend that researchers also collect ratings from other stakeholders (e.g., other school teams, researchers).

Another limitation is the absence of generalization and maintenance data. This study did not include a phase that examined to what extent improvements in teacher implementation and student behavior maintained over time. It is also unclear if use of the CFEP protocol would have generalized to other students and BSPs. It is important, then, that future studies addressing contextual fit collect generalization and maintenance data.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Albin

R. W.

Lucyshyn

J. M.

Horner

R. H.

Flannery

K. B.

(1996). Contextual fit for behavior support plans. In Koegel

L. K.

Koegel

R. L.

Dunlap

(Eds.), Positive behavioral support: Including people with difficult behavior in the community (pp. 81–98). Paul H. Brookes.

Algozzine

Barrett

Eber

George

Horner

Lewis

Putnam

Swain-Bradway

McIntosh

Sugai

(2014). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. https://www.pbis.org/resource/tfi

Algozzine

Putnam

Horner

(2010). What we know about the relationship between achievement and behavior. In Algozzine

Daunic

A. P.

Smith

S. W.

(Eds.), Preventing problem behaviors (2nd ed., pp. 223–226). Corwin.

Bailey

D. B.

Simeonsson

R. J.

Winton

P. J.

Huntington

G. S.

Comfort

Isbell

O’Donnell

K. J.

Helm

J. M.

(1990). Family-focused intervention: A functional model for planning, implementing, and evaluating individualized family services in early intervention. Journal of the Division for Early Childhood, 10(2), 156–171. https://doi.org/10.1177/105381518601000207

Benazzi

Horner

R. H.

Good

R. H.

(2006). Effects of behavior support team composition on the technical adequacy and contextual fit of behavior support plans. The Journal of Special Education, 40(3), 160–170. https://doi.org/10.1177/00224669060400030401

Brown

De Pry

R. L.

(2015). Individual positive behavior supports: A standards-based guide to practices in school and community settings. Paul H. Brookes.

Carr

E. G.

Horner

R. H.

Turnbull

A. P.

Marquis

J. G.

Magito-McLaughlin

D. M.

McAtee

M. L.

Smith

C. E.

Anderson Ryan

Ruef

M. B.

Doolabh

(1999). Positive behavior support for people with developmental disabilities: A research synthesis. American Association on Mental Retardation.

Crone

D. A.

Hawken

L. S.

Horner

R. H.

(2015). Building positive behavior support systems in schools. Guilford Press.

Fabelo

Thompson

M. D.

Plotkin

Carmichael

Marchbanks

M. P.

Booth

E. A.

(2011). Breaking schools’ rules: A statewide study of how school discipline relates to students’ success and juvenile justice involvement. Council of State Governments Justice Center. https://justicecenter.csg.org/resources/juveniles

10.

Fixsen

D. L.

Blase

K. A.

Duda

M. A.

Naoom

S. F.

Van Dyke

(2010). Implementation of evidence-based treatments for children and adolescents: Research findings and their implications for the future. In Weisz

Kazdin

(Eds.), Implementation and dissemination: Extending treatments to new populations and new settings (2nd ed., pp. 435–450). Guilford Press.

11.

Horner

R. H.

Blitz

Ross

S. W.

(2014). The importance of contextual fit when implementing evidence-based interventions [Assistant Secretary for Planning and Evaluation Issue brief]. https://aspe.hhs.gov/pdf-report/importance-contextual-fit-when-implementing-evidence-based-programs

12.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177%2F001440290507100203

13.

Horner

R. H.

Salentine

Albin

R. W.

(2003). Self-assessment of contextual fit in schools. University of Oregon.

14.

Iwata

B. A.

Dorsey

M. F.

Slifer

K. J.

Bauman

K. E.

Richman

G. S.

(1994). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27(2), 197–209. https://doi.org/10.1901/jaba.1994.27-197

15.

Kratochwill

T. R.

Hitchcock

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2010). Single case designs technical documentation. In What works clearinghouse: Procedures and standards handbook (Version 2.0). https://ies.ed.gov.ncee.wwc.pdf.wwc_procedures_v2_standards_handbook.pdf

16.

Kratochwill

T. R.

Levin

J. R.

(2014). Single-case intervention research: Methodological and statistical advances. American Psychological Association.

17.

Kretlow

A. G.

Cooke

N. L.

Wood

C. L.

(2012). Using in-service training and coaching to increase teachers’ accurate use of research-based strategies. Remedial and Special Education, 33(6), 348–361. https://doi.org/10.1177%2F0741932510395397

18.

Martens

B. K.

Lambert

T. L.

(2014). Conducting functional behavioral assessments for students with emotional/behavioral disorders. In Walker

H. M.

Gresham

F. M.

(Eds.), Handbook of evidence-based practices for emotional and behavioral disorders: Application in schools (pp. 243–257). Guilford Press.

19.

McIntosh

Chard

D. J.

Boland

J. B.

Horner

R. H.

(2006). Demonstration of combined efforts in school-wide academic and behavioral systems and incidence of reading and behavioral challenges in early elementary grades. Journal of Positive Behavior Interventions, 8(3), 146–154. https://doi.org/10.1177%2F10983007060080030301

20.

McIntosh

Massar

M. M.

Algozzine

R. F.

George

H. P.

Horner

R. H.

Lewis

T. J.

Swain-Bradway

(2017). Technical adequacy of the SWPBIS Tiered Fidelity Inventory. Journal of Positive Behavior Interventions, 19(1), 3–13. https://doi.org/10.1177/1098300716637193

21.

Monzalve

Horner

(2016). Contextual Fit Enhancement Protocol (CFEP). https://figshare.com/s/46b517d2ab6efa9108b8

22.

Nelson

J. R.

Benner

G. J.

Bohaty

(2014). Addressing the academic problems and challenges of EBD students. In Walker

H. M.

Gresham

F. M.

(Eds.), Handbook of evidence-based practices for emotional and behavioral disorders: Applications in schools (pp. 363–377). Guilford Press.

23.

O’Neill

R. E.

Albin

R. W.

Storey

Horner

R. H.

Sprague

J. R.

(2015). Functional assessment and program development for problem behavior: A practical handbook (3rd ed.). Cengage Learning.

24.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

(2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35(4), 303–322. https://doi.org/10.1177%2F0145445511399147

25.

Pinkelman

Horner

(2017). Improving implementation of function-based interventions: Self-monitoring, data collection, and data review. Journal of Positive Behavior Interventions, 19(4), 228–238. https://doi.org/10.1177/1098300716683634

26.

Sandler

Albin

R. W.

Horner

R. H.

Yovanoff

(2003). Contextual fit and the viability of behavior support plans [Unpublished manuscript]. University of Oregon, Educational and Community Supports.

27.

Sprague

J. R.

Jolivette

Nelson

C. M.

(2014). Applying positive behavioral interventions and supports in alternative education programs and secure juvenile facilities. In Walker

H. M.

Gresham

F. M.

(Eds.), Handbook of evidence-based practices for emotional and behavioral disorders: Applications in schools (pp. 261–276). Guilford Press.

28.

Strickland-Cohen

M. K.

(2012). An examination of the efficacy of training school personnel to build behavioral interventions from functional behavioral assessment information [Dissertation]. College of Education, University of Oregon.

29.

Strickland-Cohen

M. K.

Horner

R. H.

(2015). Typical school personnel developing and implementing basic behavior support plans. Journal of Positive Behavior Interventions, 17(2), 83–94. https://doi.org/10.1177%2F1098300714554714

30.

Walker

H. M.

Horner

R. H.

Sugai

Bullis

Sprague

J. R.

Bricker

Kaufman

M. J.

(1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders, 4(4), 194–209. https://doi.org/10.1177%2F106342669600400401