Examining the Efficacy of a Basic Functional Behavioral Assessment Training Package for School Personnel

Schools and School Professionals

Ten elementary schools (kindergarten through fifth grade) in a school district in the Pacific Northwest and 12 school professionals with flexible roles (i.e., not directly responsible for instruction of students) in these schools were recruited to participate. All participants had experience as members of individual student behavior support teams, but none of the participants were behavior specialists or board certified behavior analysts. The participants were provided the opportunity to participate in this study through nomination by their school district administration. Each of the 12 school professionals formally consented to participation in the study, and signed up to attend one of three training cohorts (each cohort received four 1-hr training events). Cohort 1 included 4 school professionals who attended trainings in the early fall (late September to mid-October). Cohort 2 consisted of 3 participants who attended trainings in late fall (late October to early November). Cohort 3 was made up of 5 participants who attended trainings in the winter (mid-January to early February).

Table 1 shows demographic information of the school professionals that participated in the study. Seven of the school professionals were school counselors (58%), two were principals (17%), two were learning specialists (17%), and one was a vice principal (8%). The participants averaged 14 years of experience in education (range = 4–33 years). Seven of the 12 participants (58%; 5 counselors and 2 learning specialists) stated they had conducted an FBA before. All 12 of the participants indicated that they had experience as members of a team implementing behavioral interventions for students prior to the training. Participants rated themselves on average 3.4 out of 4 (0 = none to 4 = extensive) on their perceived knowledge of basic behavioral theory.

OPEN IN VIEWER

Table 1.

School Professional Demographics and Assessment Scores.

Participant	Position	Years	FBAs conducted	Behavior theory	Pretest (%)	Posttest (%)	% Change
1	Counselor	10	5+	4	65.70	97.10	+40.40
2	Counselor	19	2	4	34.20	92.90	+58.70
3	Counselor	14	0	3	51.40	100.00	+48.61
4	Learning specialist	9	2	2	68.50	97.10	+28.60
5	Counselor	17	3	4	42.90	94.30	+51.40
6	Principal	16	0	2	37.10	97.10	+60.00
7	Principal	4	0	2	11.40	97.10	+85.70
8	Counselor	6	0	3	37.14	89.00	+51.86
9	Learning specialist	26	1	3	22.86	92.00	+69.14
10	Counselor	6	1	4	60.00	97.00	+37.00
11	VP/teacher	33	0	3	17.14	80.00	+62.60
12	Counselor	9	1	3	25.71	77.00	+51.29
Overall M					39.50	92.55	+53.77

Note. FBA = functional behavioral assessment; VP = vice principal. Participants 11 and 12 completed the Basic FBA training but did not complete a Basic FBA for a student.

Following Basic FBA training, 10 of the 12 school professionals completed a FBA as part of the normal behavior support process within their school. The 2 professionals (1 vice principal and 1 counselor) who did not complete an FBA within the remaining months of the school year indicated that the need did not arise.

Students

After the Basic FBA training sessions were completed, 10 students from 10 different schools were identified by the trained school professionals for the second phase of the study based on typical behavior support procedures involving staff nomination of students whose behavior was a barrier to their academic and/or social success in school. Within the Basic FBA training, school professional participants were provided instruction on student’s for whom a Basic FBA would be appropriate compared with students who required a more extensive FBA. These criteria were (a) students did not engage in behaviors dangerous to themselves or others and (b) problem behaviors were not pervasive throughout the school day (i.e., behaviors did not occur in more than two school routines). Following staff nomination, the first author conducted brief preliminary staff interviews and student observations to verify that the students met the criteria for a Basic FBA. In addition, the first author ensured that nominated student participants provided parent consent and student assent following American Psychological Association (APA) standards for participation in research. All of the students nominated by their school personnel for participation in Phase 2 of the study were included in the study.

All 10 of the student participants were of elementary age: 1 kindergartener (Participant 6), 1 first grader (Participant 10), 5 second graders (Participants 1, 4, 5, 7, 9), 1 third grader (Participant 2), and 2 fourth graders (Participants 3, 9). Of the 10 participants, only 2 of the students were girls (Participants 1 and 8). Four of the students (Participants 1, 4, 6, 10) received special education services under the eligibility criteria of “learning disabled”; however, all students received instruction in typical general education classrooms. The classroom sizes ranged from 16 to 24 students (median = 21).

FBA Knowledge Assessment

Each of the 12 school professionals who received Basic FBA training was administered an assessment of FBA knowledge (available from the first author) prior to training and again as part of the final training session. The nine items of the FBA Knowledge Assessment generated a total score between 0 and 35 points. The first two items of the assessment were short-answer questions that evaluated each participant’s ability to list the steps and outcomes of an FBA (4 points; 11% of score) and compared an efficient versus a comprehensive FBA (3 points possible; 9% of score). For Items 3 and 4, participants were asked to operationally define behavior using multiple-choice and short-answer formats (4 points possible; 11% of score). In Item 5, participants were provided with two scenarios from which they were to use a short-answer format to identify (a) a setting event (1 point possible; 3% of score), (b) an antecedent (1 point possible; 3% of score), (c) a consequence (1 point possible; 3% of score), and (d) behavioral functions (2 points possible; 6% of score). Item 6 provided four boxes for participants and asked them to “label and briefly define the 4 key terms (i.e., 4-term contingency) that are included in a ‘summary of behavior’ or hypothesis statement developed from an FBA” (8 points possible; 23% of score). For Item 7, participants were asked to summarize student behavior based on a scripted scenario (4 points possible; 11% of score). Finally, for Items 8 and 9, participants were provided with a completed Functional Assessment Checklist for Teachers and Staff (FACTS interview form; March et al., 2000; McIntosh et al., 2008) that was created solely for the purpose of this assessment. Participants were asked to identify (a) the routine where the behavioral problems occurred (1 point; 3% of score) and (b) the FBA summary statement for the target problem behavior (6 points; 17% of score). The assessment took participants an average of 20 min to complete, but they were provided with as much time as they needed.

The participant’s scores on pre- and posttraining assessments were calculated. Twenty-five percent (6) of the pre- and posttraining tests were randomly selected and rated by a second rater. Based on an answer key indicating the answers for the tests (created before initial scoring of the assessments), the two raters achieved 99.05% total agreement ([Agreement / Agreement + Disagreement] × 100%).

Staff Interviews

Ten of the 12 school professionals who completed the Basic FBA training conducted an interview and completed the FACTS interview form (March et al., 2000; McIntosh et al., 2008) with the teacher of a student identified as requiring behavior supports. These interview data were used to develop a preliminary FBA summary statement hypothesizing (a) the setting events, (b) the antecedent events triggering (c) the problem behavior(s) and (d) maintaining the function of the student’s problem behavior in the defined antecedent context. Part of the Basic FBA process is confirmation of interview information via direct observation. As such, interview data guided selection of the routines and behaviors used in direct observations to validate FBA findings.

Direct Observations

School professionals who received the Basic FBA training observed students in those routines identified on the FACTS as most likely to involve problem behavior. The direct observation data were used to confirm, disconfirm, or enhance the hypothesis statement developed from the FACTS interview conducted with staff. Participants used a modified version of the Antecedent Behavior Consequence (ABC) recording form (Van Norman, 2007) to gather direct observation data (form available from the first author). The modified ABC form provided space for observers to input elements of an FBA summary statement. In addition, checkboxes of categories for the activities, antecedents, and consequences were provided on the form to enable observers to determine the frequency that these categories were observed. Finally, the form guided recorders to rate, on a scale of 1 (not sure) to 6 (100% sure), their confidence that the summary statement from the observation was accurate. Participants used the summary statement scale on the modified ABC recording form to determine whether more observations were required (i.e., rating of less than “4” [content taught within the Basic FBA training]). On average, participants conducted direct observations twice (five participants conducted one observation, three participants conducted two observations, one participant conducted six observations, and one participant conducted seven observations). The average total time participants spent observing a student was 49 min (range = 20–127 min).

FBA Procedural Adequacy Checklist

Each trained participant completing a Basic FBA at their school (N = 10) submitted their interview and observation results with a final summary statement defining the student’s behavior, antecedent events triggering the behavior, and hypothesized function of the student’s problem behaviors. An FBA Procedural Adequacy Checklist (available from the first author) was used by the first author to rate each FBA completed by the participants to determine whether the following five features were met:

Each FBA received a score between 0 and 5 for FBA Procedural Adequacy. Six of the 10 FBAs (60%) were rated with a second independent rater (a graduate research assistant blind to the purpose of the study) to determine the reliability of the scoring of the procedural adequacy. There was 100% agreement between the two raters on the procedural adequacy of these FBAs.

Acceptability Rating Questionnaire (ARQ)

At the conclusion of the study, the school professionals were given a questionnaire to identify the level of acceptability of the FBA training and process (the ARQ is available from the first author). The ARQ consisted of 10 questions concerning the acceptability of the training process, FBA materials, and FBA procedures used by the school professionals. Each question included a Likert-type scale ranging from 1 (strongly disagree) to 6 (strongly agree) producing a summary score between 0 and 60.

Record of  Time Expended in Conducting FBA

The school professionals were asked to document the amount of time they spent in scheduling, conducting, and completing the FBA process. This was completed using a time log with the dates, and start and end times for completing each task of the Basic FBA process (e.g., interviews, observations, summarizing results). The time log included a column titled “tasks that labeled the following rows”: (a) “scheduling FACTS interview with teacher(s),” (b) “conducting FACTS interview with teacher(s),” (c) “conducting student-guided FACTS interview,” (d) “observing student behavior using ABC form(s),” (d) “completing summary statements,” and (e) “other related tasks” with additional blank space. The remaining columns on the time log were “date(s),” “start time(s),” “end time(s),” and “total time.” Results were summed to determine total time required to complete the FBA process.

Direct Observations During Functional Analysis (FA)

During functional analyses, graduate students who had completed 4 hr of training in behavioral observation and documented interobserver agreement (IOA) at or above a 90% criterion collected observation data on the occurrence or nonoccurrence of target behaviors using a partial-interval recording system. The specific FA conditions varied slightly between students based on the specific hypotheses generated from the Basic FBA. Before observing a student, observers were provided with the procedures and specific conditions that were to be used in the FA. All observers were blind to the hypotheses being tested. Nearly 44% (43.64%) of all observations were observed by two observers. The total IOA ([Agreement / Agreement + disagreement] × 100%) across all of the observations was 99.11%. Individual IOA scores per student participant ranged from 93.3% to 100%.

Basic FBA Training

Twelve school professionals participated in the Basic FBA training, which consisted of four 1-hr training sessions guided by a training manual (Loman & Borgmeier, 2010) and delivered by the first author. Each of the training sessions followed a format based on recognized instructional design principles (e.g., Kame’enui & Simmons, 1990; Metz, Burkhauser, & Bowie, 2009; Wiggins & McTighe, 2005). Sessions started with a definition of the training objectives, then (for Sessions 2–4) a review of the content and tasks from the previous session. Reviews were followed by a presentation of new content embedded with multiple practice opportunities. During the practice opportunities, participants engaged with the content and with other participants allowing for feedback from the trainer. Sessions concluded with each participant completing and submitting a “Checks for Understanding” worksheet where they answered key questions assessing mastery of the content presented. The trainer reviewed each of the worksheets and provided written feedback to each participant at the successive session (for Sessions 2–4).

Between each session, participants were asked to complete a task where they could practice using the new tools and procedures presented with staff and students at their school site. None of the students used for these practice sessions were included in the study. School professional participants did not select their student participants until after completing the training. On average, it took participants 37 days to start the FBA process (i.e., obtain permission) after they had completed the trainings (range = 2–102 days). They may have had “in mind” whom they wanted to use for the actual FBA, but this was not made evident to the authors. Furthermore, no coaching or feedback was provided to school professionals at their school site at any point in the study.

The first session of the training provided an overview of the Basic FBA training series and introduced concepts, examples, and practice opportunities to (a) operationally define behavior, (b) identify the function of behavior, (c) construct functional behavioral summary statements, and (d) identify students for whom practical and complex FBA procedures are required (i.e., identifying conditions for which a behavior analyst should be consulted to provide further assessment of student behavior). The second training session briefly reviewed content from the first session and provided instruction, modeling, and practice opportunities in conducting FACTS interviews with staff and students (Borgmeier & Horner, 2006). Participants practiced completing a FACTS interview by role-playing as an interviewer and teacher guided by a script (concerning a fictional student) provided within the Basic FBA manual. During this training session, participants also practiced constructing behavioral summary statements from each interview. Between Sessions 2 and 3, participants were asked to practice completing a FACTS interview with a staff member within their school concerning an actual practice student (not the student for whom they conducted their FBA submitted for the study).

The third training session provided a brief overview of the previous trainings and then instruction and practice opportunities for participants to conduct ABC (Bijou Peterson, & Ault, 1968) observations (i.e., direct observations of students within routines identified as settings in which the problem behavior occurs most frequently based on the staff FACTS interviews). Participants used an ABC recording form provided within the Basic FBA manual to practice recording behaviors observed from short (2–3 min) video clips of three fictional student actors (from Scott, Liaupsin, & Nelson, 2005). During this third session, participants also practiced constructing summary statements based on data from their observations to verify or modify summary statements derived from their FACTS interviews.

Between the third and fourth sessions, participants were asked to practice using the ABC recording form to practice observing a student at their school site. The fourth and final training session included (a) an overview of all of the concepts and skills taught during the first three sessions; (b) opportunities for participants to practice the skills that they had learned in conducting interviews, observations, and constructing behavioral summary statements; (c) introduction to the competing behavior pathway (Crone & Horner, 2003); and (d) strategies for working with individual student support teams to link FBA data to design function-based behavioral supports. Specifically, participants were taught that all elements (setting events, antecedents, alternative and desired behaviors, and consequences) of a student’s behavior support plan should be based on the maintaining function of student behavior. Examples of interventions for attention-maintained behaviors and escape-maintained behaviors were presented, though participants were not expected to have mastered these concepts from this final session.

The first author, who had a 5-year history of conducting and using FA, FBA, and function-based interventions, administered the Basic FBA trainings. The first author completed fidelity checklists for every training session for each of the three cohorts. These training fidelity checklists noted the number of attendees, start time, end time, and the completion of the prescribed aspects of the training according to the manual. The prescribed aspects, according to the manual, were completion of each review activity, session activity, and final checks for understanding activity. For 33% of the training sessions, interrater reliability on the fidelity of training components was calculated. There was 100% agreement between the raters as to the fidelity of training components administered per session.

Prior to receiving the Basic FBA training, each participant completed an assessment of FBA knowledge and skills (available from the first author). During the training, participants did not receive any type of feedback regarding their pretests, nor did participants have access to their pretest results. After each participant completed all four sessions of the training, he or she completed a posttraining test of FBA knowledge and skills with the same vignettes and questions from the pretest. The assessment on average took participants 20 min to complete; however, participants were provided as much time as they needed to complete the assessment.

FA

FA procedures followed those recommended by Iwata et al. (1982, 1994) and adapted for FBA validation by Borgmeier and Horner (2006). Each FA involved a multielement design across maintaining conditions adapted to the specific characteristics of the identified routine and context in which the target behavior was most likely. The FA conditions were designed specifically to test the validity of the summary statement developed from FBAs.

To determine the conditions to be used within each FA, the authors reviewed each trained professionals completed FBA. Next, the first author spoke briefly (no more than 5 min) with the school professional who conducted the FBA to identify the classroom teacher and possible conditions for conducting the FA. The first author then spoke with the student’s teachers to describe procedures that were to occur for each condition and observation period. Seven of the 10 FAs were conducted within the student’s classrooms during the targeted routine identified by the FBA. The other 3 FAs (for Participants 5, 6, and 7) were conducted in small vacant classrooms where the researchers used the regular class activities and materials along with two to three peers from the target student’s class. The FAs that were not conducted within the student’s classrooms were due to the teachers being concerned about the level of disruption to the rest of the students, which may have been caused by the researcher and observers.

Each FA consisted of three conditions: control, attention, and escape. The control condition was designed to establish a condition in which the student would be predicted, based on the FBA hypothesis statement, to perform little to no problem behavior (e.g., the presumed reinforcer maintaining problem behavior was available as part of the antecedent context).

The attention condition involved establishing a context in which the presumed maintaining reinforcer (adult or peer attention) was available but accessed only contingent upon occurrence of the problem behavior. In the cases where peer attention was hypothesized as the function of student problem behavior, peers were present in the condition to allow for peer response to the target student’s problem behavior. If peers did not respond to the student following his or her problem behavior, the researcher ensured that contingent (adult) attention was provided to the student within the context of peers.

The escape condition examined the function of student behavior through the contingent removal of presumed aversive tasks following the occurrence of the problem behavior. The tasks for each individual student were part of the regular curriculum, and identified through teacher and student interviews as “nonpreferred.” Each FA condition consisted of ten 30-s trials and lasted a maximum of 5 min (total of 15 min per observation session). Observers utilized a partial-interval system to record data; therefore, some conditions were completed within less than 5 min. Conditions were randomly ordered within three session blocks (Control, Attention, and Escape). A brief transition period followed each condition. Before starting a new condition, the student was provided with a verbal description of the procedures that were to be used in that condition.