Educators’ Perspectives on Training Mechanisms That Facilitate Evidence-Based Practice Use for Autistic Students in General Education Settings: A Mixed-Methods Analysis

Recruitment

Following approval from the university and school district Institutional Review Boards, we used district-provided lists of elementary schools. We contacted 358 principals from schools identified in our selection process (see Participants) with study information and a request to forward study screeners to educators at their schools. Interested educators completed a screener and interest form (N = 139 screener responses). They were screened for meeting the inclusion criteria of serving at least one autistic student in an elementary general education setting in the prior 2 school years. Eligible educators who consented (N = 104) were contacted with a unique link to complete the survey via Qualtrics, which included a modified Autism-Treatment Survey measure and a demographic questionnaire. For both phases, participants were instructed to respond to their EBP use with autistic students in a general education classroom. After survey completion, participants (N = 86) were invited to a recorded interview (30–45 minutes) at a convenient time (via Zoom), and 81 (94.2%) completed an interview. Participants received a $40 gift card for the interview, and their schools received a resource kit (e.g., visual supports, books).

Modified Autism Treatment Survey

The Autism Treatment Survey (ATS; Hess et al., 2008) was designed to measure teachers’ frequency of use and training experiences with a comprehensive list of EBPs for autistic children (Hess et al., 2008). We updated the ATS as follows, we (a) included practices identified as EBPs for autistic students according to the NPDC (Steinbrenner et al., 2020), (b) used updated NPDC practice definitions validated via cognitive interviewing with two former educators (Hugh, 2020; Hugh et al., 2021), and (c) separated training type (e.g., conference) and provider. To develop a set of focused EBPs inclusive of both multicomponent practices and strategies that are used to support meaningful inclusion of autistic children in general education settings, we used explicit criteria for exclusion of practices from the NPDC review to focus the set of practice options on EBPs that teachers could access training from a variety of pathways (e.g., schools and online modules) and reduce the potential impact of resource barriers (e.g., cost of certification). The first and final authors and an external autism intervention researcher independently coded practices for inclusion or exclusion with no disagreements. Practices could not: (a) be assessments (e.g., functional behavior assessment), (b) be implemented only outside of schools (e.g., parent-implemented), or (c) require EBP-specific certification (e.g., Sensory Integration Therapy; see supplementary materials for a complete list of practices and definitions). We included 21 EBPs: social narratives, direct instruction, antecedent-based intervention, social skills training, self-management, functional communication training, discrete trial training, peer-mediated instruction and intervention, naturalistic intervention, reinforcement, modeling, task analysis, visual supports, prompting procedures, augmentative/alternative communication, extinction, response interruption/ redirection, time delay, technology-aided instruction and intervention, video modeling, and behavioral momentum intervention. For each of these, participants were asked to indicate if they (a) used the practice and (b) received training in the practice. If they indicated “yes,” they were given a set of training types and instructions to select all training types they had received in each practice, including teacher or educator preparation or certification program, school-based in-service or workshop, District/ESD in-service or workshop, coaching, consultation, webinar, conference, and self-taught with descriptions.

Phase 1. Quantitative Data

RQ1 was analyzed by computing frequencies and proportions of educators who reported receiving training by each training type. Missing data were not possible because of forced choice responding. For RQ1 and RQ2, we combined “coaching” with “consultation,” and we combined “school-based in-service or workshop” with “district workshop.” These were combined to improve model parsimony, and because of conceptual similarity (e.g., qualitative interviews indicated educators discussed these interchangeably “we had a coach who came in and talked about a specific student and watched us practice,” which met our definitions of both coaching and consultation) and strong statistical association of these training types, as each pair had tetrachoric correlations above r > .40 (tetrachoric because each training response was yes/no.)

RQ2 was analyzed using a generalized linear model predicting EBP use, with a logit link for the binomial distribution using a compound symmetry variance structure and robust estimator. To test the effects of training type on EBP use, we predicted each EBP by several dichotomous variables for attended/did not attend for each training type for that specific EBP. Therefore, we used mixed-effects models with all EBPs (N = 21, Level 1) nested within participants (N = 86, Level 2). Dichotomous attendance for each EBP training type was included as a Level 1 variable. Random effects were specified to adjust standard errors and confidence intervals due to the nesting of EBPs within participants. Separate single-variable predictor models were computed for predicting EBP use, including each training type as a predictor. A final model was computed predicting EBP use, with all training types included as predictor variables (excluding “other,” as it was infrequently reported and highly variable). For descriptive purposes, we ran separate combined models stratifying by participant role; however, our sample size did not permit valid statistical testing among roles. For these analyses, webinars were excluded for general education teachers as the model would not converge due to small cell sizes.

Phase 2. Qualitative Data

We developed a stable codebook using six randomly sampled transcripts (two per role) by (a) applying deductive codes from exploratory and explanatory implementation theories (Locke et al., 2021), (b) using inductive coding for concept development or emergent themes, and (c) continuing this process until data saturation (Saunders et al., 2018). We double-coded 20% of randomly selected transcripts and met weekly to discuss disagreements and establish consensus (DeSantis & Ugarriza, 2000; Hill et al., 1997, 2005). We selected every excerpt coded as “training” (defined: “Any description of formal or informal training they or others did/did not receive from any (a) source including colleagues, parents, pre-service program, coaches, consultants, other (b) description of being self-taught and accessing resources (e.g., websites) to learn about practices, (c) what was or was not helpful in their training, and (d) how they heard about a practice.”) for further subcode analysis. In alignment with effective PD features (Desimone, 2009), we developed and reviewed a sub-codebook with examples and non-examples. The first author sorted all excerpts for features and helpfulness of training (Desimone, 2009), and another author independently sorted a random subset of 20% (agreement = 86.7%). All disagreements were consensus coded. We calculated percentages of features in each excerpt (sum of PD features divided by all training excerpts multiplied by 100).

Active Learning

Active learning was the most frequently mentioned feature (49.2%); in more than half of those excerpts, it was described as helpful (57%). Participants experienced active learning both within contrived situations in traditional PD workshops and “on the job” through formal and informal mechanisms. Within PD workshops, trainers set up conditions in which participants observed videos and engaged in discussion, with one noting, “see[ing] teachers in action and then hav[ing] that time to notice and wonder what is happening and being reflective of your own practice is a great strategy.” Other workshops used to role-play and case studies, which most educators experienced as beneficial if applied to real-world contexts. For example, one participant described that being given a “fake student. . .doesn’t make a lot of sense until you have to put it in place.” Educators also valued making intervention materials (e.g., social stories) within workshops because it helped them “break apart” and modify the strategy. Those who observed live demonstrations from autistic children within workshops noted that it was helpful to observe trainers use practices with students. They appreciated practicing with the student before having to “juggle it all” in their classroom with many students at once.

Active learning components most often described as helpful were frequently in the context of direct coaching experiences in the educators’ classroom. Educators described that though they may have heard about or learned about practices in training or teacher preparation, observing and using practices with autistic students outside of coaching contexts and within natural interactions “on the job” was their formative learning experience. As described by one participant, “For me, I don’t learn it until I have to do it and put it into practice.” Participants shared that coaching as an active learning method was specifically beneficial because the coaches individualized to the context and student, saying, “This is the practice I’ve heard of in the book. Now let’s put it in person. Let’s apply it to your kids.” For example, one educator said, “In my higher ed program, it was more generalized, that if you used this with a student, this may be an outcome. But when I worked with the therapist in my classroom, we actually saw the gains.” Learning within their classrooms helped educators see what was “doable within [the] confines of the classroom, within the parameters that there is 22 other students and there’s all the other things.” One educator emphasized that knowing each autistic student was fundamental and only possible through experience, so the “specific training with real students” was essential.

Relatedly, the importance of repeated practice in the EBP and seeing the student response in context was emphasized in a few ways. First, several educators noted “trial and error,” with or without support from other educators or PD providers, as their primary learning mechanism in figuring out whether a practice “works” or does not for a specific student. Second, participants benefited from observing and working with their students and the coach via practice and feedback: “Not only has [the coach] said what it’s supposed to look like, now I’m going to get to watch her do it. Then the following week, she’s going to watch me teach one of my groups. Then. . . we’re going to debrief.” Finally, many participants shared that repeated observations, feedback, dialogue, and reflection were “invaluable” in helping them learn to use EBPs.

Educators described observing paraeducators, mentor teachers, special education teachers, speech-language pathologists, behavior specialists (e.g., Board Certified Behavior Analysts, BCBAs), and even principals use the EBP with their students as pivotal learning experiences. These professionals were lauded for having “led by example.” One teacher described observing a BCBA with their student saying, “But when you see it in action in real life with your students, that’s when you really go, ‘Aha, I get it. I want to use this.’” Following the observation, participants would reverse roles and receive formative feedback on how the educator and student did. Many teachers shared that they “felt like [they] learned more just being in [the] classroom every day” than from formalized PD efforts.

Collective Participation

Aspects of collective participation (i.e., participating in all or part of the training with class, team, or school members) also often arose (34.2% of excerpts, and helpful in nearly half of the excerpts, 44.9%). This feature was part of formal workshops and learning EBPs “on the job” through systematically designed training opportunities at the regional, district, school, classroom, and child levels. District and school-level policies that allowed for collective participation included: (a) regular early release to attend training, (b) competency-based programming for paraeducators, (c) required mentorship for new teachers, (d) “lesson study” with reciprocal observation and feedback for a classroom team, and (e) “book studies” in which teachers reviewed the manual, tried the EBP in their class, and held review, reflection, and problem-solving meetings. One educator noted that collective participation enabled all educators to be on board and as a group of “like-minded people . . . I could learn from them, and I could teach them my tricks, and maybe we could figure this all out together.”

Participants engaged in collective participation also at the classroom and child levels through consultation and coaching. Many educators described the benefits of these partnerships with colleagues in other disciplines, such as speech-language pathologists, behavior specialists, school psychologists, and principals. Many were described as “resources,” having very specialized training as BCBAs or in autism-specific classroom models that they brought to their school and informally taught to their classroom team. The different perspectives on their classroom team contributed significantly to participants’ learning, with one teacher explaining that the SLP worked with a “variety of kids and has great ideas, and you can kind of bounce off of each other with ideas when you’re brainstorming about a kid, I think that’s so valuable.” In the context of classroom-specific collective participation, paraeducators often described learning directly from their mentor teacher “through practice,” noting that, “You got to live it and have someone train [you].” Likewise, one teacher described a co-teaching model, in which they and the general educator would switch roles partway through the day to learn the strategies, as extremely beneficial. Collective participation within roles also was critical. A few paraeducators described learning directly from other paraeducators, saying, “that’s probably the strongest bond you get is with your other paras because you’re with them and they get it. They understand it.” Through “shadowing” one another, they would “pass down the knowledge.” One teacher reflected that collaborating with other teachers helped them learn how to use a curriculum; “we started getting things out, and then I started seeing how effective they were and that I didn’t have to ask anybody else to do it or wait for anybody else to do it for me. I could do it myself.”

Coherence

Educators brought up a training’s coherence, or alignment of the EBP and training content with the district, school, or program’s practices and policies, teacher beliefs, and needs of students across a range of connectedness from simple district payment for training to focused training related explicitly to curriculum and student needs (18.9%). For example, one educator said, “if there’s a need, [the district tries] to put together professional development for it.” Coherence was helpful in about half of the mentioned instances, primarily when the connections were stronger to roles, responsibilities, and students (53.0%). Participants emphasized that districts sent educators in groups to training and then created space for these educators to return to school to instruct each other on the EBPs learned. Experiences differed for “brand new special education teachers” or paraeducators, with one paraeducator describing, “Well, I had one training which was absolutely wonderful and it was at the [school district] . . . and paid for by [district]. It was specifically for sped paras.” In contrast, one educator described the district giving them “a spiral-bound book and said, ‘here, do this’” without follow-up or training.

More specific coherence was part of a school-wide initiative (e.g., Positive Behavior Intervention Supports [PBIS], Conscious Discipline, explicit reading interventions, and particular autism or inclusion curriculum or classroom models) or student-specific needs. For example, one educator explained that “a school-wide support system and having it be PBIS” was helpful for their EBP learning and use because it focused on positive reinforcement. Educators mentioned revisiting content from some training and initiatives at regular team or school meetings as “refreshers,” which benefited the whole school team. Educators also underscored the importance of all school personnel being on the “same page” at each school because of training to support an EBP being “absorbed school-wide,” which makes the “biggest difference for the students.”

District-provided training that matched student needs were applauded as incredibly valuable; “I think just the idea that the district is recognizing [that] we are put in these situations, and we need to be educated and that it’s important for everyone and every student and every adult to just be aware of these things.” Other responsive approaches coherent with school policies included having autism and behavior consultants who would come in as needed, or when educators “hit a wall.” Some participants, however, noted that those resources seemed more often used in self-contained than in general education settings. While educators wanted training coherent with their needs, some said they may have only accessed this specific training because they had a “rough classroom” or “autistic student [with] pretty severe behavior.”

Content Focus

Although mentioned minimally (17.7%), focusing on the content of learning the skill or how autistic students specifically learn skills often was described as helpful (65.2% of excerpts). Participants emphasized the importance of knowing why a practice may work for an autistic student/s or skill area—specifically, understanding autism and how autistic students’ brains work was a recurring theme. One educator described the most beneficial aspect of training as learning, “The reason behind it. Like, why is it important?” They connected the understanding of autistic students’ learning and brains often with one participant stating, “From the outside looking in, we might look at them and say, ‘They look totally typical.’ But really, their brain, there’s so much happening in there.” Participants described content focus on individualizing interventions for their specific students and considering the “whole child.”

Educators also described two primary skill areas for which content focus was beneficial and included in their training: social communication skills and behavior. Participants described the helpfulness of understanding students’ behavior, the “function of their behavior” as communication, and “how to communicate to autistic students” precisely. One described, “So I liked how they taught that a student just not being naughty . . . I liked that it was put in that context of: children aren’t just screaming to scream, they’re actually trying to tell you something.” Those who reflected on learning about behavioral function stated the impact on their thinking and perception of the students helping them better address the students’ needs.

Sustained Duration

The sustained duration was the least often described PD feature (13.9%), although often associated with helpfulness (44.0%). Many educators noted having multiple courses within a pre-service program that touched on EBP use (e.g., modeling) or specific practices being embedded in many PD workshops to the point that they could not recall when they began learning about it (e.g., reinforcement). In contrast, instances of repeated learning experiences or extended duration within a particular training type or experience (e.g., workshop) were rare. Structured learning opportunities that participants experienced as helpful often were in the context of collective training and groups throughout the school (e.g., book studies) during the school year for cyclical practice that incorporated new EBP components.