A sustainable model for training teachers to use pivotal response training

Abstract

The increase in the rate of autism diagnoses has created a growing demand for teachers who are trained to use effective interventions. The train-the-trainer model, which involves training supervisors to train others, may be ideal for providing cost-effective training and ongoing support to teachers. Although research supports interventions, such as pivotal response training, as evidence-based, dissemination to school environments has been problematic. This study assessed the benefits of using the train-the-trainer model to disseminate pivotal response training to school settings. A multiple-baseline design was conducted across three training groups, each consisting of one school staff member (trainer), three special education teachers, and six students. Trainers conducted the teacher-training workshop with high adherence to training protocol and met mastery criteria in their ability to implement pivotal response training, assess implementation of pivotal response training, and provide feedback to teachers. Six of the nine teachers mastered all components of pivotal response training. The remaining three teachers implemented 89% of the pivotal response training components correctly. The majority of trainers and teachers maintained their abilities at follow-up. These results support the use of the train-the-trainer model as an effective method of disseminating evidence-based practices in school settings.

Keywords

autism spectrum disorder education services professional development

Introduction

With the increased rate of diagnosis of autism, special education enrollment for autism has grown particularly quickly, quadrupling nationwide since 2000 (Scull and Winkler, 2011), and more than tripling from 17,508 students statewide in California (2.6% of special education students) in 2001 to 59,690 (8.8%) in 2010 (Lucile Packard Foundation for Children’s Health, 2011). This increased enrollment has led to growing demand on educational service providers to provide high-quality programming for children with autism. Systematic reviews have been conducted to evaluate the evidence base for specific autism spectrum disorder (ASD) interventions, and several strategies were determined by multiple reviews to be evidence-based practices (EBPs) for educating children with ASD (National Standards Project, 2009; Odom et al., 2010; Wong, 2014). However, although EBPs for educating children with ASD exist, they are often not incorporated into school-based programs (Hess et al., 2008; Morrier et al., 2011; Stahmer and Ingersoll, 2004) or are implemented with low fidelity (Suhrheinrich et al., 2007, 2013). There is an identified need to improve the availability and quality of training available to teachers of children with autism toward the goal of increasing use of EBPs for children with autism (California Department of Developmental Services, 2003; Simpson et al., 2003).

Post-certification professional development for teachers is likely to provide an immediate form of effective training in EBPs, especially those that are newly developed or recently modified (Bailey et al., 1990). Such training is typically provided after teachers are already “in service” as professional educators and is often completed during paid work time. Several effective training methods have been identified, including providing direct instruction, opportunities for teachers to practice skills while receiving feedback, and ongoing coaching and supervision (National Advisory Mental Health Council, 2001; Odom, 2009; Reid et al., 1989; Scheuermann et al., 2003; Suhrheinrich, 2011).

Equipping existing school district staff members to train and support teachers may provide substantial benefits to teachers with limited additional cost. School district personnel in roles as Autism Specialists or Behavior Specialists often lend support and training to classroom teachers as a regular part of their job. These specialists are in schools on a regular basis and can provide long-term training and support to teachers. High-quality in-service training provided by existing school staff is likely a cost-effective and sustainable method for educating teachers in EBP. However, school district staff may need specific training in management skills, because often they have entered their supervisory role with only clinical training (Reid et al., 1989). Training staff to both train teachers in EBPs and provide helpful feedback to teachers should address issues of both practicality and sustainability in school programs for children with autism.

The train-the-trainer (TTT) model, which has also been called pyramidal training, triadic training, and helper model training, focuses on initially training a person or people who, in turn, train other people at their home agency. The TTT model has promise of being both efficient and cost-effective (LaVigna et al., 2005) and may be especially beneficial in translating interventions to community settings and providing ongoing support to trainees. The TTT model has a sound body of literature supporting its effectiveness in a variety of contexts, including residential centers (Page et al., 1982; Parsons and Reid, 1995; Shore et al., 1995), hospitals for the mentally disabled (Whalen and Henker, 1971), experimental laboratories, and schools (Hester et al., 1995; Jones et al., 1977). TTT has also involved multiple types of trainees, including direct care providers (Page et al., 1982; Parsons and Reid, 1995; Shore et al., 1995), university students (Fremouw and Harmatz, 1975), parents (Hester et al., 1995; Kuhn et al., 2003; Neef, 1995), and teachers (Jones et al., 1977; LaVigna et al., 2005). The broad scope of the studies demonstrating the effectiveness of TTT highlights the strong potential for applicability of TTT methods to train teachers in EBP for children with autism in a cost-effective and sustainable manner.

One EBP for children with autism is pivotal response training (PRT). PRT is a naturalistic behavior intervention that was developed to facilitate generalization, increase spontaneity, reduce prompt dependency, and increase motivation. Specific components of PRT include providing clear and appropriate cues, allowing for child choice of and within an activity, turn-taking, interspersing maintenance tasks with acquisition tasks, reinforcing attempts, responding to multiple cues, and providing contingent reinforcement that is directly related to the child’s response. A review of the research base for use of PRT concludes that it is an efficacious EBP for children with autism (Humphries, 2003; National Advisory Mental Health Council, 2001; National Standards Project, 2009; Odom et al., 2010). PRT has been shown to improve language function (Koegel et al., 1987, 1992, 1998; Laski et al., 1988; Sze et al., 2003), play and social skills (Pierce and Schreibman, 1997; Stahmer, 2005; Thorp et al., 1995), and joint attention (Rocha et al., 2007; Whalen and Schreibman, 2003). The various skills that have been taught through PRT suggest that it is an appropriate intervention for teachers of children with autism. Additionally, because PRT was developed for use in the natural environment, school classrooms are an appropriate setting for PRT implementation. There is some preliminary evidence that special education teachers can be trained to implement PRT with moderate levels of fidelity of implementation (e.g. Suhrheinrich, 2011, 2013). Therefore, a next step in increasing access for children with autism and improving sustainability of skills is training trainers.

There is a growing need for teachers who are trained to educate children with autism. Specifically, teachers need access to training in EBPs, such as PRT. To ensure maximum benefit to students, training must prepare teachers to implement PRT with high fidelity (FI) and maintain high FI after training ends. This project addressed areas of need by employing school-based specialists as PRT trainers using the TTT model. Three primary research aims were explored.

Do school-based personnel learn to implement PRT themselves, train others to use PRT, assess others’ implementation of PRT, and provide feedback aligned with areas of strength and weakness?

Do teachers learn to implement PRT when trained by school-based personnel within their district?

Do trainers and teachers maintain their skills over a 3-month follow-up period?

Method

Participants

Three school staff participated as trainers in this investigation. School staff participants met the following inclusion criteria: (a) a primary job description that allowed them to provide training to special education teachers (e.g. behavior specialist, autism specialist), (b) no prior training in teaching others to use PRT, and (c) direct contact with at least three teachers willing to participate in this study as trainees. All three trainers were female with an age range of 29–32 years. Trainers A and C were Caucasian and Trainer B was Latino. Trainers A and B completed a Master’s degree in Special Education with a specialization in autism, and Trainer C held a Bachelor’s degree in Special Education. All trainers had some specialized training in autism and some exposure to or training in PRT. The trainers represented two school districts and each trainer worked with three teachers within her own district for the purposes of the current study.

Nine special education teachers participated in this investigation. Teachers met the following inclusion criteria: (a) a full- or part-time position as lead teacher in a special education classroom and (b) at least two students with a primary educational classification of autism who had parental consent to participate in this study. All nine teachers were female with an age range of 25–55 years. Six teachers were Caucasian, one teacher was Filipino, one teacher was Native American, and one teacher chose not to report ethnicity. One teacher (A2) had a master’s degree in education technology and elementary science; all other teachers had bachelor’s degrees. All teachers held teaching credentials. Number of years teaching children with autism ranged from 1 to 13. All teachers reported having some training on how to educate children with autism, described as either “on the job” training or through university coursework. Additionally, five teachers (A2, B1, B2, B3, and C3) reported having received some “on the job” training specifically in PRT.

In all, 21 students participated in this investigation. Students met the following inclusion criteria: (a) a primary educational classification of autism and (b) a chronological age of 3–8 years. This age group was chosen because it has been the focus of the majority of the evidence supporting PRT. Each teacher selected two students from her own class, and each trainer selected one student from within the school sites she served, for the purpose of demonstrating PRT throughout their participation in the study.

Study design

This study employed a single-case, multiple-baseline design across training groups. This type of design has the advantage of controlling for developmental maturation and exposure to the treatment (Kazdin, 1982; Kratochwill et al., 2010). Each training group consisted of one trainer, three teachers, and six students. Each group participated in a baseline condition for 3–7 weeks, according to the multiple-baseline design. Baseline durations were 3, 5, and 7 weeks, with seven weeks being about equal to the length of the intervention. Data were also obtained after a 3-month follow-up period.

Baseline

Because the overall aim of the study was to examine the effectiveness of the training model and resources, the baseline condition measured trainer and teacher behavior with training resources that are commonly available in school settings—the PRT manual, observation from a district staff member, and documents for assessing implementation of PRT. To allow time for reading, each trainer and teacher received a copy of the PRT manual 1 month before baseline assessments began. The PRT manual was adapted from How to Teach Pivotal Behaviors to Children with Autism: A Training Manual (Koegel et al., 1989) by replacing examples of parents working with their children with examples of teachers working with their students. All other aspects of the manual remained the same. The baseline condition also involved weekly trainer observation and feedback to the teachers, and they were instructed to provide feedback using their own judgment and clinical skill (naïve of any specific trainer training for PRT). Although trainers reported that observing and providing feedback to teachers was one aspect of their job, they typically focused on individual students within a classroom context to reduce problem behavior or increase rate of learning. The trainers reported that they did not typically observe the same teachers each week for the purpose of providing feedback on teacher use of a new intervention strategy. Trainers were also provided with the documents necessary for assessing teachers’ implementation of PRT during baseline, although they did not receive training on how to use these documents until the trainer training.

Classroom observations occurred once each week throughout baseline, trainer training and teacher-training phases, and again after a 3-month follow-up period. During each classroom observation, each teacher was instructed to use PRT for 10 min with each of the two participating students in her class. The trainer was instructed to observe both 10-min teacher/student interactions for a total of 20 min of observation and assess the teacher’s use of PRT. At the completion of the 20 min of observation, the trainer was instructed to provide feedback to the teacher based on the teacher/student interactions using the PRT feedback form. A trained research assistant videotaped the teacher/student interactions and the trainer/teacher interaction. Videotaped classroom observations were used to assess teacher implementation of PRT, and trainer ability to assess teachers’ use of PRT and provide accurate feedback.

Trainer training

Each trainer participated in 15 total hours of individual training over the course of 3 weeks. The length of individual training sessions was determined by each participating trainer based on her preference, and ranged from 2 to 4 h, with 5–7 total sessions per trainer. All trainers received permission from their employing school district to complete the training during their regular work hours. Trainer training was conducted individually and was provided by the author, who had 5 years of experience implementing and training others in PRT. Initially, the expert presented didactic instruction on the history and components of PRT, how to use PRT in classroom settings, and how to assess implementation of PRT. Video examples of PRT being implemented with children were used to illustrate correct and incorrect implementation of each PRT component. Trainers practiced implementing PRT with children with autism in the laboratory and received feedback from the expert. Trainers learned to assess implementation of PRT by observing research assistants use PRT with children in the laboratory. The expert explained and modeled the following process to trainers: observe the research assistant and complete the PRT assessment form based on the behavioral definitions, complete the PRT feedback form based upon the PRT assessment, and provide verbal feedback to the research assistant based upon the PRT feedback form. Based on their assessment of implementation of PRT, trainers completed the PRT feedback form and provided verbal feedback. Finally, the expert reviewed the procedures and materials for the teacher-training workshop, and answered any questions. Overall, trainers received instruction and were assessed in four areas: (a) how to implement PRT, (b) how to assess implementation of PRT, (c) how to provide feedback based on Parsons and Reid’s (1995) model for training supervisors to provide feedback to direct care staff, and (d) how to conduct the PRT workshop with their teachers. Each trainer was required to meet an implementation criterion of 80% correct use of all PRT components, a reliability criterion of 80% agreement with the expert for assessing implementation of PRT, and an implementation criterion of 80% correct completion of the PRT feedback form before trainer training was complete.

Teacher training

After completing trainer training, trainers conducted all aspects of teacher training. Each teacher participated in 10 h of training over the course of 9 weeks. The first 6 h of training were conducted as a group in-service workshop scheduled during the teachers’ regular work hours. The workshop included the following components: didactic instruction on the history and elements of PRT, video modeling of correct and incorrect examples of PRT, didactic instruction and discussion of how PRT may be implemented in classroom environments, observation of PRT being implemented by the trainer, practice implementing PRT with feedback from the trainer, and a brief introduction to how PRT FI is assessed. The remaining hours of training took place as 30-min classroom coaching sessions, once a week for 7 weeks.

Assessments

Trainers

The trainers were assessed in four areas: PRT implementation, assessment of PRT implementation, implementation of feedback procedures, and implementation of the PRT teacher-training workshop. To assess trainers’ implementation of PRT, probes were conducted before and after the trainer training, during the teacher-training workshop, and again at the follow-up classroom observation. During each probe, the trainer was instructed to use PRT with one student for 10 min.

Trainers completed the PRT Assessment form during each teacher/student classroom observation. The trainers’ ability to assess PRT implementation was measured by calculating the percent of agreement between how trainers scored their teachers’ implementation of PRT in vivo and how trained research assistants scored the same teacher/student interactions via videotape. Additionally, percent of agreement was calculated between the verbal feedback trainers provided to their teachers about components of PRT that were not implemented correctly and the trained research assistants’ scoring of components of PRT that were not implemented correctly. The percent of agreement between the trainer feedback and research-assistant scored fidelity was reported as trainer’s feedback reliability.

Teachers

Each teacher’s ability to implement PRT was assessed using 10-min videorecordings from classroom observations.

PRT implementation

Trainer and teacher implementation of PRT was evaluated from videorecordings. The trainer/teacher was instructed to use PRT with one student for 10 min during instructional time. Trained research assistants coded the videorecordings in 1-min segments using behavioral definitions for the following nine components of PRT: student attention, clarity of cues, developmental appropriateness of cues, interspersal of maintenance and acquisition tasks, child choice of activities, contingent consequences, direct reinforcement, reinforcement of attempts, turn taking. The behavioral definitions followed a four-point scale, with “1” indicating the component is implemented poorly during a majority of the segment, “2” indicating the component is implemented correctly about half of the segment, “3” indicating the component is implemented correctly during a majority of the segment, and “4” indicating the component is implemented correctly throughout the entire segment (please contact the author for complete coding guide and full behavioral definitions). An average score was calculated for each component across all 1-min segments for the full 10-min observation. Research assistants were blind to the research questions addressed by the investigation and were trained to reliability (80% agreement on practice videos) prior to coding independently. Interobserver agreement, or reliability, was calculated on coding of the trainers’ and teachers’ PRT implementation and on student behavior. Interobserver agreement was calculated across all participants and for each behavior separately. Reliability was defined as 80% agreement, or no more than .8 point difference. Reliability was calculated on 28% of PRT FI assessments and was 100% for Attention, 100% for Clarity, 100% for Appropriateness, 96% for Maintenance/Acquisition, 96% for Child Choice, 100% for Contingent, 98% for Direct Reinforcement, 92% for Reinforcement of Attempts, and 96% for Turn Taking.

Data analysis

Fidelity data were initially analyzed for each teacher/student interaction. Fidelity scores were similar across the two students for each teacher on a majority of observation days, so scores were averaged across students, resulting in one fidelity score for each observation day. Analysis of the data gathered during trainer training, the teacher-training workshop, and classroom observations was conducted using visual analysis, as is customary in studies employing a multiple-baseline design (Gliner et al., 2001; Kratochwill et al., 2010).

Satisfaction

A consumer satisfaction questionnaire was distributed to participants to assess the trainers’ and teachers’ satisfaction with the intervention and the training they received. The questionnaire was mailed to all trainers and teachers with a stamped return envelope. Participants were encouraged to complete and return the form anonymously. Questions assessed attitudes toward PRT as well as the usefulness and difficulty of specific components of the training.

Results

Trainers

All three trainers demonstrated a high level of competency in using the components of PRT at the start of trainer training and implementation of PRT remained high, with all trainers meeting fidelity of implementation criteria throughout the rest of the study (see Figure 1). During the trainer training, trainers correctly assessed implementation of PRT and provided accurate feedback. Percent agreement for the PRT implementation assessment was measured during each practice session. Each trainer surpassed a reliability criterion of 80% agreement with the expert by the end of training. All trainers also demonstrated feedback procedures (i.e. completing the feedback questionnaire based on the PRT Assessment form) with 100% accuracy throughout all training sessions. Thus, all trainers demonstrated proficiency in all training procedures during the trainer training.

Figure 1.

Trainer PRT implementation during trainer training, teacher training, and at follow-up.

Implementation of teacher-training workshop procedures varied by trainers. Trainers A and B completed 100% of procedures correctly. Trainer C completed 91% of procedures correctly, failing to properly prepare toys and training materials and model PRT during one of three scheduled times during the teacher training. When trainers’ implementation of PRT was assessed during the teacher-training workshop, Trainer A implemented all components of PRT correctly, whereas Trainers B and C both implemented 89% of components correctly—failing to accurately use turn-taking procedures (see Figure 1).

During weekly classroom coaching sessions, all trainers demonstrated marked improvement in ability to assess teachers’ correct implementation of PRT from baseline to treatment conditions. Difference scores of reliability for assessment of PRT implementation from baseline to treatment showed 40% improvement averaged across all trainers. Specifically, Trainer A averaged 47% reliability during baseline and 70% reliability during treatment (23% improvement), Trainer B averaged 21% reliability during baseline and 83% during treatment (62% improvement), and Trainer C averaged 41% reliability during baseline and 74% during treatment (33% improvement).

The verbal feedback trainers provided to teachers after each observation also improved, meaning their feedback became more reliable with feedback the expert would have given. Difference scores from baseline to treatment showed 13% improvement averaged across all trainers. Specifically, Trainer A averaged 49% reliability during baseline and 67% reliability during treatment (18% improvement), Trainer B averaged 55% reliability during baseline and 64% during treatment (9% improvement), and Trainer C averaged 52% reliability during baseline and 64% during treatment (12% improvement).

Trainers showed variability in the accuracy of their assessment of PRT implementation and feedback provided to teachers from day to day during both baseline and treatment phases (see Figure 2). Furthermore, trainers demonstrated different patterns of learning and maintenance of these abilities following initial training. Trainer A’s accuracy of both assessing PRT implementation and providing feedback increased gradually. In contrast, Trainers B and C show substantial improvement in accuracy of assessment of PRT implementation, but very limited gains in the accuracy of the feedback they provided to teachers, from baseline to treatment.

Figure 2.

Percent agreement between research assistants’ scoring of the PRT Assessment form and each trainer’s scoring of both the PRT Assessment form and verbal feedback to teacher.

Teachers

The majority of teachers learned to correctly implement all components of PRT, with six teachers implementing 100% of components correctly and three teachers implementing 89% of components correctly (see Figure 3). The majority of teachers showed improvements throughout the course of the study. Group A implemented 54% of components on average during baseline (r = 44%–67%), 76% of components on average during treatment (r = 50%–100%), and 73% of components during follow-up (r = 67%–78%). Group B implemented 65% of components on average during baseline (r = 50%–78%), 81% of components on average during treatment (r = 67%–100%), and 56% of components during follow-up (r = 22%–89%). Group C implemented 68% of components during baseline (r = 33%–78%), 76% of components during treatment (r = 56%–89%), and 78% of components during follow-up. However, there was variability in how and what the teachers learned (see Figure 4).

Figure 3.

Teachers’ average percent of PRT components correctly implemented during baseline and treatment. Error bars represent plus or minus one standard error of the mean.

Figure 4.

Teachers’ correct implementation of PRT.

Each teacher correctly demonstrated some components of PRT during baseline, so percent of skills learned was initially determined by dividing the number of components learned during treatment by the number of components unknown at the start of treatment (or those possible to learn). Six of the nine teachers learned 100% of the previously unknown components as demonstrated with one or both of their participating students. One teacher (B2) learned 50% of previously unknown components as demonstrated with each of her students. Another teacher (C3) correctly used 33% of previously unknown components with one of her students, but demonstrated no learning with her second student. The final teacher (C1) demonstrated no improvement with either student.

Teacher learning varied across components. All teachers correctly implemented three skills during a majority of baseline sessions: Attention, Clarity, and Appropriateness. None of the teachers implemented Child Choice or Turn Taking correctly during the majority of baseline sessions, and only two teachers (A1, C3) implemented Maintenance/Acquisition correctly during a majority of the baseline sessions.

Follow-up

A final classroom observation was completed 3 months after completion of the training. Trainers maintained their ability to implement PRT, with Trainer A implementing all skills correctly and Trainers B and C implementing 89% of skills correctly (see Figure 1). Trainers also demonstrated maintenance of PRT FI assessment skills from treatment to follow-up (see Figure 2). Trainers showed variability in their maintenance of the accuracy of feedback provided to teachers (see Figure 2). Trainer C showed a slight improvement (3%) in accuracy of feedback from treatment to follow-up. Trainer A had a slight loss (5%) in accuracy of feedback, but maintained improved accuracy over feedback provided during the baseline condition. Trainer B had a more substantial loss (13%) in accuracy of feedback but was also highly variable in her accuracy between teachers (range = 22%–78% accuracy).

During the follow-up assessment, five teachers maintained their ability (within 5% change from average implementation of PRT skills during treatment). One teacher improved implementation of PRT skills by 7%, and one teacher regressed in implementation of PRT skills by 55% (see Figure 4).

Satisfaction

Trainer rate of response for the consumer satisfaction questionnaire was 67% (two of the three trainers) and teacher rate of response was 86% (six of the seven teachers who participated in the follow-up condition).

Trainer response was very positive. Trainers were either satisfied or very satisfied with overall quality of training received and would recommend this training to another person in their position. Practice with feedback and direct training were rated as the most helpful parts of the training. Other questions related to the trainers’ perception of their own skills after completing the training. Trainers rated themselves to be “knowledgeable” or “very knowledgeable” about PRT and “experienced and comfortable” in implementing PRT. Trainers were either neutral or satisfied with their own ability to train others to use PRT. Both trainers believe in this approach and consider it “highly successful” for children with autism. Trainers considered the most difficult parts of PRT to implement to be responding to Multiple Cues, Turn Taking, and Reinforcement of Attempts.

Teachers’ responses were very positive. Teachers were either very satisfied (50%) or satisfied (50%) with the overall quality of training they received, and 100% of respondents would recommend this training to another person in their position. Feedback (50%), the 6-h workshop (16%), practice (16%), and turn taking (16%) were rated as the most helpful parts of the training. Teachers were either very satisfied (50%) or satisfied (50%) with their own ability to use PRT. Teachers responded that PRT was “successful” (86%) or “highly successful” (14%) for educating children with autism in their class. In all, 86% of teachers reported that they believe in this approach. Teachers considered the most difficult parts of PRT to implement to be responding to Multiple Cues (50%), Turn Taking (50%), and Reinforcement of Attempts (50%). Teachers reported using the entire intervention (50%), the entire intervention mixed with other interventions (33%), and parts of the intervention (16%). Teachers reported using PRT with a student one-on-one (66%), with two students (33%), with a small group of students (50%), with a large group (16%), and in multiple settings (16%).

Discussion

Overall, the TTT model shows promise as an effective method for training Autism or Behavior Specialists to train teachers in PRT. Trainers and teachers benefitted from participation in the study, with the majority of participants learning new skills and maintaining these skills over time. These findings indicate initial support to the usefulness and effectiveness of the TTT model and expand the body of literature on training teachers to use PRT, a naturalistic behavioral intervention.

Trainers learned the skills necessary to train teachers to implement PRT and maintained the learned skills over a 3-month follow-up period. Trainers effectively demonstrated the ability to assess implementation of PRT in classroom environments with their own teachers; however, reliability of their assessment of PRT and of the feedback they provided to teachers was variable. These findings suggest that providing additional training to trainers in their work environment may be beneficial. Additionally, ongoing support in the form of the expert accompanying the trainer during several classroom observations or providing feedback via phone or email may also be useful for trainers. Overall, these findings suggest that in only 15 h of instruction, school staff members can be trained to provide comprehensive training in PRT to teachers.

Varied accuracy of PRT implementation assessment may also be explained, in part, by the complicated assessment methods used in this study. Trainers would likely benefit from a less complex assessment tool for assessing PRT implementation. Additionally, trainers showed only modest improvement in the accuracy of the verbal feedback they gave to teachers after training. This suggests that despite their ability to accurately assess the teachers’ implementation of PRT, critical information on ways to improve was not always accurately communicated to teachers. Trainers may benefit from more specialized instruction on how to provide accurate feedback in a sensitive and efficient manner.

All trainers implemented PRT as well at follow-up as they did during the treatment phase and maintained their ability to assess implementation of PRT at follow-up, performing as well as they did during the treatment phase. Maintenance of feedback accuracy was slightly more varied. Overall, however, these findings suggest that the initial trainer training was sufficient to provide trainers with the skills necessary to train a majority of their teachers, and the skills were maintained at a high level 3 months after treatment was completed.

The majority of teachers learned to implement PRT during the course of the study. However, teachers’ skill level at the start of the study varied, with some teachers demonstrating more PRT components correctly than other teachers before receiving training. For example, all teachers correctly implemented some skills during a majority of the baseline sessions. Gaining the child’s attention before presenting a cue, use clear and developmentally appropriate language when presenting a cue, and providing cues that are related to the activity are skills all the teachers could perform before any formal training. In addition, over half of the teachers correctly implemented three additional skills through the majority of baseline sessions: providing consequences contingent upon the child’s response, providing reinforcement that is directly related to the child’s behavior, and reinforcing the child’s attempts. Only two of the nine teachers interspersed maintenance (those already mastered) and acquisition (those not yet mastered) tasks during the lesson. Finally, none of the teachers shared control with their students by following the child’s choice of activity or taking turns. Teachers had a broad range of experience in the field and specifically teaching children with autism. However, teachers were consistent in implementing some components of PRT correctly prior to any specific training. This suggests that some components of PRT may be considered “good teaching” and may be acquired during teacher education and certification programs or on the job. These findings have implications for how training time is allocated in future research studies and in clinical training programs. For example, less time may be needed when introducing and modeling the components that teachers knew before training: Attention, Clarity, and Appropriateness. Conversely, more time may need to be dedicated to training skills that are unique to PRT, such as the consequence strategies, Child Choice, and Turn Taking. Additionally, some components of PRT may not fit the context of some classroom environments. For example, teachers working in group environments or with older, more academically advanced children, may find PRT in need of modification to fit within their classroom structure.

Teachers also demonstrated variability in how skills were acquired. Some teachers showed improvement in implementation of certain skills during the baseline condition. This indicates some components of PRT may have been acquired prior to participation in the study or that exposure to PRT, via reading a brief manual and receiving feedback from an untrained supervisor, can lead to some improvement for some teachers. These findings are encouraging because in the educational community, it is common for teachers to attempt to learn new teaching strategies by reading a manual or talking with a supervisor or specialist. It is less common for a teacher to receive weekly feedback from a highly trained supervisor. However, after being observed by and receiving feedback from the trainer for only 30 min each week, some teachers were able to improve implementation of one or two components of PRT. This suggests that incorporating regular times for observation and feedback may improve the quality of teaching for some teachers. During baseline, the trainers were proficient in implementing PRT themselves, but had no specialized knowledge in training others to implement the techniques. Therefore, it is likely that school staff who are knowledgeable, but untrained in specific training techniques will have a positive, but limited, impact on teachers by observing them and providing feedback on a regular basis.

Most teachers showed improvement in implementation of PRT components during the treatment phase. However, teachers demonstrated gains both immediately following the teacher-training workshop and in a more gradual way over the course of the treatment classroom observations. Learning patterns varied, but not systematically by teacher or component. That is, some teachers improved in all components immediately following the teacher-training workshop, others improved in some skills following the workshop and in other skills toward the end of treatment, and still others showed improvement only toward the end of treatment. These results suggest that both the training workshop and the ongoing feedback in the form of classroom observations were essential parts of the training.

A majority of teachers maintained their skills at the follow-up assessment. This is encouraging because the follow-up period fell during an extended summer vacation. Overall, one teacher slightly improved in PRT implementation, and one teacher showed a dramatic loss of skill.

The results of this study add to the literature supporting verbal feedback, or coaching, as an important part of training in new teaching strategies. None of the teachers implemented all the components of PRT correctly immediately following the lecture portion of the teacher-training workshop. Rather, the teachers benefitted from continued feedback from the trainer. Some teachers did not master certain PRT components until the final session of treatment. This suggests that observation and feedback after a training workshop improves upon teachers’ ability to learn new teaching strategies.

There are some caveats one must consider when drawing conclusions from this research. First, due to the nature of applied research, there was limited control over the research environment and participants. The purpose of the baseline condition was to replicate training resources that are commonly available to teachers—the PRT manual, observation from a district staff member, and documents for assessing implementation of PRT. To allow time for reading, each trainer and teacher received a copy of the PRT manual 1 month before baseline assessments began. The baseline condition also introduced regular trainer observation and feedback to the teachers. Trainers were provided with the documents necessary for assessing teachers’ implementation of PRT during baseline, although they did not receive training on how to use these documents until the trainer training. Under these conditions, some teachers showed improvements in one or two of the components of PRT, and allowing for a longer baseline period would have been helpful to see if a more stable trend would have been established prior to the start of the intervention. However, although trainers reported that observing and providing feedback to teachers was a part of their job, they typically did not observe the same teachers each week. Thus, the baseline condition introduced in this study does not reflect “treatment as usual” for the participating trainers and teachers. This suggests that although school staff and teachers may have access to the skills and information necessary to help teachers make some improvements in ability to implement PRT, it is unlikely that they would typically dedicate regular time for training in this manner.

Another limitation is the small number of participants. This study included a small sample of school staff, teachers, and students in southern California. The small sample size limits the possibility of evaluating trainer, teacher, and student variables that may contribute to the effectiveness of this specific training model. For example, this study included trainers who had previous experience with PRT and two who had advanced degrees in special education with a specialization in autism. All trainers had a good understanding of PRT at the start of the study as demonstrated by their ability to implement a high percentage of PRT components correctly before receiving any explicit training. Additionally, very little information was available about the children participating in the study. Future research should evaluate the effectiveness of trainers with a broad range of professional experience and the learning outcomes for children with varied characteristics. The training model should be revised based on findings from this study, and the research should be replicated with more training groups and in other geographic areas.

The results of this study lend support to previous findings that most teachers can learn to implement PRT correctly with 10 h of direct instruction and personalized feedback. These findings also provide a new contribution to the literature on the TTT model in a school setting. School staff members, with only 15 h of direct training from the expert, were able to successfully train the majority of the teachers to implement PRT.

In conclusion, this research shows preliminary support for the TTT model as an effective method for disseminating EBP to classrooms. If school staff can conduct effective training, the cost of sending teachers to workshops and paying outside consultants can be dramatically reduced. School districts are under increased strain to provide high-quality services for children with autism, and application of the TTT model may be one strategy for increasing the number of qualified teachers without additional training costs.

Footnotes

Acknowledgements

Many thanks to Drs Laura Schreibman, Aubyn Stahmer, and my colleagues in the UCSD Autism Intervention Research Program who helped conduct this research and edit previous drafts of this manuscript.

Funding

This research was supported by the National Foundation for Autism Research and the San Diego branch of the Autism Society of America.

References

Bailey

Simeonsson

Yoder

. (1990) Preparing professionals to serve infants and toddlers with handicaps and their families: an integrative analysis across eight disciplines. Exceptional Children 57(1): 26–35.

California Department of Developmental Services (2003) Autistic Spectrum Disorders, Changes in the California Caseload: An Update. Sacramento, CA: California Health and Human Services Agency.

Fremouw

Harmatz

(1975) A helper model for behavioral treatment of speech anxiety. Journal of Consulting and Clinical Psychology 43: 652–660.

Gliner

Morgan

Harmon

(2001) Measurement reliability. Journal of American Academy of Child & Adolescent Psychiatry 40(4): 486–488.

Hess

Morrier

Heflin

. (2008) Autism treatment survey: services received by children with autism spectrum disorders in public school classrooms. Journal of Autism and Developmental Disorders 38(5): 961–971.

Hester

Kaiser

Alpert

. (1995) The generalized effects of training trainers to teach parents to implement milieu teaching. Journal of Early Intervention 20(1): 30–51.

Humphries

(2003) Effectiveness of pivotal response training as a behavioral intervention for young children with autism spectrum disorders. Bridges: Practice-Based Research Syntheses 2(4): 1–9.

Jones

Fremouw

Carples

(1977) Pyramid training of elementary school teachers to use a classroom management “skills package.” Journal of Applied Behavior Analysis 2: 239–253.

Kazdin

(1982) Single-Case Research Designs: Methods for Clinical and Applied Settings. New York: Oxford University Press.

10.

Koegel

Camarata

Valdez-Menchaca

. (1998) Setting generalization of question-asking by children with autism. American Journal on Mental Retardation 102(4): 346–357.

11.

Koegel

Surratt

(1992) Language intervention and disruptive behavior in preschool children with autism. Journal of Autism and Developmental Disorders 22(2): 141–153.

12.

Koegel

O’Dell

Koegel

(1987) A natural language teaching paradigm for nonverbal autistic children. Journal of Autism and Developmental Disorders 17(2): 187–200.

13.

Koegel

Schreibman

Good

. (eds) (1989) How to Teach Pivotal Behaviors to Children with Autism: A Training Manual. Santa Barbara, CA; San Diego, CA: University of California.

14.

Kratochwill

Hitchcock

Horner

. (2010) Single case designs technical documentation. Available at: http://ies.ed.gov/ncee/wwc/pdf/wwc_procedures_v2_standards_handbook.pdf

15.

Kuhn

Lerman

Vorndran

(2003) Pyramidal training for families of children with problem behavior. Journal of Applied Behavior Analysis 36(1): 77–88.

16.

Laski

Charlop

Schreibman

(1988) Training parents to use the natural language paradigm to increase their autistic children’s speech. Journal of Applied Behavior Analysis 21(4): 391–400.

17.

LaVigna

Christian

Willis

(2005) Developing behavioural services to meet defined standards within a national system of specialist education services. Developmental Neurorehabilitation 8(2): 144–155.

18.

Lucile Packard Foundation for Children’s Health (2011) Local Prevalence of Disabilities. Palo Alto, CA: Lucile Packard Foundation for Children’s Health.

19.

Morrier

Hess

Heflin

(2011) Teacher training for implementation of teacher strategies for students with autism spectrum disorders. Teacher Education and Special Education 34: 119–132.

20.

National Advisory Mental Health Council (2001) Blueprint for Change: Research on Child and Adolescent Mental Health. Bethesda, MD: National Advisory Mental Health Council.

21.

National Standards Project (2009) National Standards Report. Randolph, MA: National Autism Center.

22.

Neef

(1995) Research on training trainers in program implementation: an introduction and future directions. Journal of Applied Behavior Analysis 28(3): 297–299.

23.

Odom

(2009) The tie that binds: evidence-based practice, implementation science, and outcomes for children. Topics in Early Childhood Special Education 29(1): 53–61.

24.

Odom

Collet-Klingenberg

Rogers

. (2010) Evidence-based practices in interventions for children and youth with autism spectrum disorders. Preventing School Failure: Alternative Education for Children and Youth 54(4): 275–282.

25.

Page

Iwata

Reid

(1982) Pyramidal training: a large scale application with institutional staff. Journal of Applied Behavior Analysis 15(3): 335–351.

26.

Parsons

Reid

(1995) Training residential supervisors to provide feedback for maintaining staff teaching skills with people who have severe disabilities. Journal of Applied Behavior Analysis 28(3): 317–322.

27.

Pierce

Schreibman

(1997) Multiple peer use of pivotal response training to increase social behaviors of classmates with autism: results from trained and untrained peers. Journal of Applied Behavior Analysis 30(1): 157–160.

28.

Reid

Parsons

Green

(1989) Staff Management in Human Services: Behavioral Research and Application. Springfield, IL: Charles C Thomas Publisher.

29.

Rocha

Schreibman

Stahmer

(2007) Effectiveness of training parents to teach joint attention in children with autism. Journal of Early Intervention 29(2): 154–172.

30.

Scheuermann

Webber

Boutot

. (2003) Problems with personnel preparation in autism spectrum disorders. Focus on Autism and Other Developmental Disabilities 18(3): 197–206.

31.

Scull

Winkler

(2011) Shifting Trends in Special Education. Washington, DC: Thomas B. Fordham Institute.

32.

Shore

Iwata

Vollmer

. (1995) Pyramidal staff training in the extension of treatment for severe behavior disorders. Journal of Applied Behavior Analysis 28(3): 323–332.

33.

Simpson

de Boer-Ott

Smith-Myles

(2003) Inclusion of learners with autism spectrum disorders in general education settings. Topics in Language Disorders 23(2): 116–133.

34.

Stahmer

(2005) Teaching professionals and paraprofessionals to use pivotal response training: differences in training methods. Paper presented at the Annual Meeting of the California Association for Behavior Analysis, Dana Point, CA, February.

35.

Stahmer

Ingersoll

(2004) Inclusive programming for toddlers with autism spectrum disorders: outcomes from the Children’s Toddler School. Journal of Positive Behavior Interventions 6(2): 67–82.

36.

Suhrheinrich

(2011) Training teachers to use pivotal response training with children with autism: coaching as a critical component. Teacher Education and Special Education 34(4): 339–349.

37.

Suhrheinrich

Stahmer

Schreibman

(2007) A preliminary assessment of teachers’ implementation of pivotal response training. Journal of Speech: Language Pathology and Applied Behavior Analysis 2(1): 8–20.

38.

Suhrheinrich

Stahmer

Reed

. (2013) Implementation challenges in translating pivotal response training into community settings. Journal of Autism and Developmental Disorders 43(12): 2970–2976.

39.

Sze

Koegel

Brookman

. (2003) Rapid acquisition of speech in nonverbal children with autism: developing typical, social, and communicative interactions in children with autism using pivotal response training and self-management. Paper presented at the Association for Behavior Analysis, San Francisco, CA, November.

40.

Thorp

Stahmer

Schreibman

(1995) Effects of sociodramatic play training on children with autism. Journal of Autism and Developmental Disorders 25(3): 265–282.

41.

Whalen

Schreibman

(2003) Joint attention training for children with autism using behavior modification procedures. Journal of Childhood Psychology and Psychiatry 44(3): 456–468.

42.

Whalen

Henker

(1971) Pyramid therapy in a hospital for the retarded: methods, program evaluation, and long-term effects. American Journal of Mental Deficiency 75: 414–434.

43.

Wong

Odom

Hume

. (2013) Evidence-based practices for children, youth, and young adults with Autism Spectrum Disorder. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institute, Autism Evidence-Based Practice Review Group.