Abstract
Children with autism spectrum disorder (ASD) in public education settings experience difficulties with transitions during classroom routines, which can result in challenging behavior. Single-subject research supports techniques for transitions, but school-based approaches often require resources and training unavailable in low-resource districts, limiting implementation. We developed and evaluated the Schedules, Tools, and Activities for Transitions (STAT) program, a short-term, manualized intervention of behavioral supports to support daily routine transitions for students with ASD (K-5) in underresourced districts. We utilized a multisite, cluster-randomized, group comparison design (immediate treatment versus waitlist) with matched pairs (n = 150 students, 57 educators). Data indicated (a) no group differences for academic engagement or classroom independence, and (b) an advantage for STAT in reducing challenging behavior and increasing teacher fidelity. Results show preliminary support for an intervention that is feasible and perceived as sustainable in real-world settings.
Children with autism spectrum disorder (ASD) are being included in public education settings at swiftly increasing rates (National Center for Education Statistics). Students in these settings frequently experience difficulties with transitions during their daily routines (APA, 2013; Richler, Bishop, Kleinke, & Lord, 2007), such as between academic activities, from recess to the classroom, or from specials to classroom work. Transition-related difficulties range from active noncompliance to prompt dependence (Schreibman, Whalen, & Stahmer, 2000) and often result in challenging behavior, including disruption, noncompliance, tantrums, aggression, and self-injury. Such behaviors decrease learning opportunities and social interactions, impeding academic progress (Schreibman et al., 2000). Students who resist transitions can become dependent upon teacher support and less likely to develop independent, functional skills (Giangreco & Broer, 2005; Giangreco, Edelman, Luiselli, & MacFarland, 1997). Many students with ASD have strong cognitive and academic skills, particularly for procedural and nonverbal tasks (Mayes, Calhoun, Murray, Ahuja, & Smith, 2011; Minshew, Goldstein, Taylor, & Siegel, 1994), yet still struggle with transitions without adequate support.
Identifying ways for classroom staff to promote efficient transitions is critical to helping students succeed in classroom settings. Specific behavioral techniques to support transitions have been identified through single-subject research studies. Most of these techniques involve environmental modifications to prevent disruptive behaviors from occurring, such as warnings, visual supports, auditory cues (Goodman & Williams, 2007; Sterling-Turner & Jordan, 2007), and visual schedules (Cihak, 2011; Dooley, Wilczenski, & Torem, 2001; McClannahan & Krantz, 2010). Modeling (Hume, Loftin, & Lantz, 2009), reinforcement systems, and self-monitoring (Hume et al., 2009; Machalicek, O’Reilly, Beretvas, Sigafoos, & Lancioni, 2007) effectively help students learn, maintain new skills, and engage in less-challenging behavior.
Single-subject studies indicate that behavioral strategies in isolation (or combined with one or two other techniques) can improve one or two specific target behaviors (e.g., challenging behavior). These studies provide initial support for use of behavioral strategies, but have important limitations. First, teachers usually need to implement multiple strategies that are chosen to address the individual needs of the student (Goodman & Williams, 2007). However, implementing strategies “à la carte,” without evidence of their efficacy in combination, may not lead to optimal child outcomes (Chasson, Harris & Neely, 2007) as it places a high burden on teachers to select, individualize, implement, and evaluate strategies for the student (Smith, 2013). Factors related to implementation, such as teacher buy-in, treatment preferences, and perceptions of important behavioral targets, are also relevant (Forman, Olin, Hoagwood, Crowe, & Saka, 2009), but notably understudied in the context of intervention research.
Second, implementing research-based behavioral strategies effectively and sustainably in schools poses considerable challenges. Strategies have usually been studied using highly trained interventionists (Stahmer, Collings, & Palinkas, 2005), which is costly, in highly controlled environments that may be difficult or complicated to arrange in classrooms (Kasari & Smith, 2013; Rogers, 2000; Stahmer, 2007). Moreover, school settings differ from research settings, such as in the range of student needs, diversity in backgrounds, preexisting theoretical allegiances of providers, resources, and institution-specific regulations. These potential barriers should be taken into account to support effective dissemination and generalization (Dingfelder & Mandell, 2011; Lord et al., 2005), particularly in light of the educational disparities in ASD related to race, culture, and language (Tincani, Travers, & Boutot, 2009). For school-aged children with ASD, public schools are often the main service setting, and the largest school districts are in major urban centers that tend to be underresourced, with limited capacity to deliver behavioral strategies as designed.
Arick and colleagues were the first to package intervention strategies for transition difficulties in students with ASD. Their standardized intervention manual, The STAR Program: Strategies for Teaching Based on Autism Research (Arick, 2004), describes a comprehensive applied behavior analytic (ABA) treatment program to support successful transitions. The intervention was implemented with fidelity in preschool classes and was associated with improvements in academic instruction (Arick et al., 2003). A later field trial comparing the STAR Program, implemented by teachers who received 28+ hr of training, to a structured teaching program in Philadelphia public schools yielded some positive child outcomes, despite low treatment fidelity for both approaches (Mandell et al., 2013). However, the authors highlighted the importance of identifying the minimum components of ABA-based intervention for instruction to be effective (Mandell et al., 2013).
In sum, several gaps in the literature warrant attention. Transition-related difficulties can greatly disrupt a classroom, but available strategies are fragmentary and require substantial individualization. From a research standpoint, it is unclear how to best combine these strategies (Iovannone, Dunlap, Huber, & Kincaid, 2003), and how effective such combinations are in overcoming transition difficulties. Finally, while available strategies appear effective when employed by specialists in highly controlled settings, little research is available on their use in real-world settings. Accordingly, there is a need to integrate and standardize strategies into a package and then evaluate the package on a larger scale in authentic educational settings with a range of outcome measures, multiple teachers, and diverse samples of children with ASD.
We built upon the foundational work by Arick and colleagues by developing and evaluating a new protocol composed of the critical elements of behavioral intervention needed to support transitions within the daily school routine—the Schedules, Tools, and Activities for Transitions (STAT) Program—and carried out a random control trial (RCT) of STAT for students with ASD in self-contained classrooms in underresourced schools (i.e., schools that receive Title 1 funding from the U.S. federal government to improve performance of economically disadvantaged students). The participants were racially and ethnically diverse, and many lived in low-income households. This project expanded upon previous research in three main ways: (a) developing a manual that combines common ABA-based techniques into a brief, packaged intervention to support transitions; (b) examining whether the intervention could be delivered with fidelity by nonspecialists in real-world settings; and (c) evaluating implementation and intervention effectiveness in an RCT. Our hypotheses were threefold:
Our goal was to package an intervention to address important student outcomes for students with ASD that could reasonably be implemented in classrooms.
Method
This study was conducted as part of a parent community-partnered participatory research project across the sites of the Autism Intervention Research Network on Behavioral Health (AIR-B): Los Angeles, Philadelphia, and Rochester, NY. Sites formed partnerships with local school districts (Los Angeles Unified School District, School District of Philadelphia, Rochester City School District); the partnership included the research team and local stakeholders (school administration, educators, parents of children with ASD). Partnership discussions and focus groups yielded two intervention priorities for students with ASD: facilitating transitions within the daily routine and increasing peer engagement (Iadarola et al., 2014). This article reports on the transition intervention.
Participants
Participants were 150 students in 56 classrooms and 57 teachers, including 73 students and 28 teachers in the treatment group and 77 students and 29 teachers in the waitlist group (see Table 1 for participant characteristics), recruited over 2 academic years (2011-2012, 2012-2013).
Participant Characteristics.
Note. STAT = Schedules, Tools, and Activities for Transitions Program; IQ = intelligence quotient; DAS-II = Differential Abilities Scale, 2nd Edition.
Teachers
We recruited lead teachers in self-contained special education classrooms who worked with at least one student with ASD. Teachers were mostly female (84%), White (70%) special educators (95%).
Students
Inclusion criteria included a diagnosis of ASD, enrollment in kindergarten through fifth grade, and placement for 50% or more of the school day in a self-contained special education classroom. ASD diagnoses were confirmed using the Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2; Lord et al., 2012); the Differential Ability Scales, 2nd Edition (DAS-II) was administered to characterize students’ cognitive functioning (Elliott, 2007). Exclusion criteria included known genetic disorders (e.g., Down syndrome, Fragile X), profound vision or hearing loss, or severe motor disabilities. No exclusions were made based upon comorbid diagnoses or concomitant treatments.
Students were primarily male (75%) from diverse racial and ethnic backgrounds (36% African American, 5% multiracial, 4% Asian/Pacific Islander, 39% Hispanic/Latino/Spanish). The average participant intelligence quotient (IQ) was 67 (SD = 23; range = 25-119), indicating a high proportion of student participants with comorbid intellectual disability.
Recruitment/randomization
District administration at each site identified potential schools and classrooms for STAT. Then, a hierarchical consent process was initiated: Initial approval was obtained from each school’s principal. Administration suggested classrooms that were likely to contain students who met the inclusion criteria. Study staff next approached lead teachers and other adult educators (i.e., assistant teachers and paraprofessionals) in individual classrooms for consent. Teachers sent home consent forms to the parents of students, and research staff were available to speak with parents over the phone to review the procedures and answer questions. Classrooms in which at least one parent consented were considered enrolled and eligible for randomization (see Figure 1). Due to staffing constraints and potential teacher burden, we set a maximum of three students enrolled per classroom. Educators received a $25 gift certificate for completion of forms at each of the three time points.
CONSORT chart for student and classroom participants.
Classroom randomization was conducted separately by site, stratified by grade (i.e., Grades K-2; Grades 3-5), and controlled for classroom and treatment group differences. Randomization was conducted by the study statistician, with concealed allocation. Randomization by classroom was chosen to ensure that all students within one classroom were assigned to the same condition.
The immediate treatment group received six to 12 sessions of classroom-based intervention (STAT). Session number was determined by multiple factors, including rapidity of teacher skill acquisition and level of comfort with the strategies. It was also common for sessions to be cut short due to changes in classroom schedule, which required the coach to return for additional meetings. Regardless of the number of sessions required, teachers and coaches met until all material covered in the manual was discussed. Overall, most objectives were completed within two to three sessions. The total amount of coaching time for teachers was approximately 5 to 10 direct hr. Outcome measures were collected at baseline (T1) and immediately post-treatment (T2).
Primary Outcome Measure
We distinguished between a primary outcome measure (the outcome we considered most important) and secondary outcome measures (other outcomes of interest). We also conceptualized outcomes as proximal (i.e., directly related to intervention, which involved coaching teachers) and distal (i.e., affected in an indirect way by intervention, such as child behaviors that might be affected by teachers’ use of new strategies).
The Academic Engagement (AE) Observation was the primary outcome measure; this was considered a distal outcome and was selected based on our hypothesis that successful transitions result in high engagement during the activity following transitions (i.e., during learning opportunities). The measure was adapted by Mandell and colleagues (2013) from the Systematic Screening for Behavior Disorders (Walker & Severson, 1992). The AE Observation addresses critical components of teacher and student behavior in the classroom. For each 10-min AE Observation, a blind rater coded 20-s partial interval data, alternating between student behavior (e.g., on-task versus off-task) and teacher behavior (e.g., academic praise, behavioral correction), including instructional methods and student-directed communication. See online supplemental materials for a full list of behavior codes and their operational definitions.
Secondary Outcome Measures—Child
Adaptive Behavior Assessment System, Second Edition (ABAS-2)—Self-Direction Subscale
Teachers completed the Self-Direction Subscale of the ABAS-2 (Harrison & Oakland, 2003) to assess the student’s independence related to academic engagement and social behaviors, which we hypothesized would be positively influenced by more successful student transitions throughout the day (i.e., a distal outcome). Teachers rated each school-related behavior on a 4-point Likert-type scale from 0 = not able to 3 = always. The ABAS has acceptable internal consistency (.98) and construct validity (.78-.93 for the subscales).
School Situations Questionnaire (SSQ)
The teacher-completed SSQ is a 9-item questionnaire that assesses severity of challenging behavior in school situations (e.g., individual deskwork, small group instruction, lunch) on a 9-point Likert-type scale. It has adequate-to-high reliability (.84-.92) and consistency (.77-.82) (Altepeter & Breen, 1989). We chose the SSQ to determine whether improved transitions affected behavior across the school day (i.e., proximal outcome).
Teacher-nominated target problems
Teachers nominated the three most pressing target problems for the student at baseline to measure the potential influence of more successful transitions on the primary problems identified by teachers (i.e., proximal outcome). The frequency/constancy, intensity, and impact of target problems on the classroom were documented in a brief narrative, which was reviewed and updated at post-treatment with the teacher. This method was modified from a version used with parents in the Research Units on Pediatric Psychopharmacology (RUPP) Autism Network and shown to be reliable and valid in a placebo-controlled study (Arnold et al., 2003). Operational definitions were developed collaboratively by the coach and the teacher, with the coach providing guidance to ensure adequate specificity across measureable dimensions (e.g., frequency, severity). Once definitions were created, members of the research team categorized the behavior. Commonly reported target problems included noncompliance (30%), tantrums (25%), prompt dependence (26%), and aggression/self-injury (18%; See Table 4).
Secondary Outcome Measures—Educator
Buy-In Measure
Educators completed a 14-item questionnaire regarding their intentions, beliefs, and attitudes about participating in components of the intervention to capture buy-in and intervention acceptability. Item stems were taken from standardized measures based on the Theory of Planned Behavior (Fishbein, 2008), and psychometric properties are not yet available. Items related to teacher willingness (“During the next two weeks, I will use a visual or written schedule every time a transition occurs from a preferred activity to a non-preferred activity”) and perceived control over implementing the intervention (“It is up to me whether I reinforce the child’s behavior every time a successful transition occurs”). Questionnaires were administered at T2.
Teacher treatment fidelity
Blind observers rated teachers at T1 and T2 for adherence and quality of implementation of the STAT steps. Fidelity was assessed at specific time points and in the absence of the coach. Adherence was rated 0 (step not completed) or 1 (step completed). For example, the teacher would be rated on whether he or she “Provides a warning” prior to a transition (step 1 in STAT). Quality was rated 0 (step completed inaccurately) or 1 (step completed accurately), for a total score of 0 to 2 for each step. Quality ratings were coded independently of adherence ratings. For example, for “Provides a warning,” a teacher could be scored a “1” for adherence and also a “1” for quality (e.g., supplemented verbal warnings with visuals, gave gradual warnings) or a “0” for quality (e.g., quickly said “5 minutes left” without using visuals, only giving a warning once). To guide observers, examples of high- and low-quality ratings were provided on the data sheets. Scores were expressed as a percentage (sum of scores across all applicable steps divided by total possible score × 100). Prior to intervention, blind raters were required to complete a cross-site training, code a series of training videos, and achieve a kappa of 0.65.
Intervention procedures
The main intervention goal was to train teachers to systematically implement a package of techniques that have shown promise in single-subject studies.
Content
The aim of STAT is to support transitions within the daily routine (manual available at http://airbnetwork.org/tools_guidelines.asp). The intervention begins by collaboratively identifying transitions within the school routine that require improvement and then breaking down these transitions into nine specific steps (see Table 2). Through collaborative discussions, each step is individualized to meet student and classroom needs. The intervention includes seven core objectives and two optional objectives that were implemented upon teacher request or based upon the coach’s observations.
STAT Program Steps.
Note. STAT = Schedules, Tools, and Activities for Transitions Program.
Training and interventionists
Intervention procedures were implemented in a hierarchical training model. Teachers were taught to implement STAT by study staff, who acted as “coaches.” All coaches were bachelor’s, master’s, or doctoral level clinicians with experience in behavioral intervention. A cross-site training was conducted at the onset of the study to train coaches, which included review of the manual, role-plays, in-vivo feedback, and when possible, opportunities to observe supervisors implementing STAT in classrooms. In addition, interventionists were required to achieve a criterion (kappa = 0.65) for accurately identifying each of the STAT steps. This was followed by weekly cross-site calls for group supervision by a licensed psychologist who was also a behavior analyst. For blind observers, the team that adapted the current version of the AE measure conducted trainings across sites until observers met a criterion of kappa = 0.65 and interobserver agreement (IOA) = 80%.
Structure
STAT was conducted in classrooms, and most coaching sessions were conducted during ongoing daily instruction. Coaches and teachers met for 30 to 45 min, approximately 6 to 12 times over 6 to 12 weeks. All coaches and teachers completed the same number of manual objectives; the length and frequency of sessions depended upon teacher availability. If teachers had to cut a training session short, the objective was completed at a later date. Embedding the coaching into the classroom routine ensured that participation did not require staff coverage, extension of the teacher’s work day, or removal of students from the classroom, and it allowed for in-vivo practice and feedback. Coaches also provided teachers with transition-related materials (e.g., activity schedules, timers, reinforcer boards) at the beginning of the intervention and as needed thereafter. Teachers were encouraged to continue using STAT strategies between coaching sessions.
STAT emphasized a highly collaborative coach-teacher relationship that included (a) an initial classroom observation for establishing rapport and learning about the students and classroom culture; (b) discussion of the background and rationale for strategies while encouraging teacher questions/concerns; (c) input from the teacher on how to carry out each step; (d) when applicable, teachers and coaches modeling, practicing, and role-playing with in-vivo feedback; and (e) individualized program goals around students’ strengths and needs.
Coach implementation fidelity
For each student participant, one STAT session was randomly selected for a coach fidelity observation. Coaches were observed in-vivo by blind raters for the duration of one transition. Blind observers were trained to criterion (kappa > 0.65) on the STAT coding system, but they were not trained in the full STAT program. Coach behavior was rated from 0 to 2 on adherence (covering material presented in the manual) and quality (e.g., presents materials at an appropriate pace, praises teacher, assesses for understanding, encourages questions).
Statistical Analyses
Student t tests, Wilcoxon tests, χ2 tests, and Fisher exact tests were used for baseline comparisons between treatment groups. Generalized linear mixed models (GLMM) with main effects of treatment (group assignment and time) and time, treatment by time interactions, and subject-level random intercepts were constructed to model the longitudinal trajectories of the outcomes, using an intent-to-treat approach. Primary models for child outcomes covaried IQ (DAS-II) to take into account the relationship between cognitive ability and academic engagement/classroom independence) and the main effect of site. Treatment effect was defined as a significant interaction between group assignment and time (entry to end of treatment). Ordinal logistic regression was utilized to evaluate the effect of treatment on teachers-nominated target problems, controlling for site and IQ. Interactions between site by time, and site by time by group assignment allocation were also evaluated in post hoc analyses for the primary outcome models. Treatment moderators were assessed in secondary analyses by including interactions between moderators of interest with treatment group as predictors in the models. No site differences were found. Cohen’s f was reported as the effect size (ES), unless otherwise specified.
Results
Table 3 reports means and standard deviations for all child and teacher outcome measures at pre-intervention (T1) and post-intervention (T2) for the treatment (TX) and waitlist (WL) groups, along with p values and ES’s for the Intervention × Time interaction.
Means, Standard Deviations, and Effect Sizes of Both Groups for Primary and Secondary Outcomes.
Note. STAT = Schedules, Tools, and Activities for Transitions Program; ABAS = Adaptive Behavior Assessment System; SSQ = School Situations Questionnaire.
Cohen’s f.
Odds ratio.
Number and Percentage of Students for Whom Challenging Behavior Was Identified via the Teacher Nominated Target Problems Questionnaire.
Child Outcomes
On average, students spent most of their time following their teachers’ instructions at T1 and remained highly academically engaged at T2. There was no significant main effect of time, F(1, 240) = 0.29, p = .593, ES = 0.035 or interaction between group assignment and time (Table 3). On average, ABAS scores were stable from T1 to T2: F(1, 97) = 1.91, p = .1705, ES = 0.14. There was no significant interaction between group assignment and time. A significant group by time interaction occurred on the SSQ, F(1, 127) = 5.88, p = .017, with greater reduction in challenging behavior severity for students in TX than WL. Behavior severity remained stable in WL from T1 and T2, F(1, 127) = 0.0036, p = .950, whereas severity decreased from T1 to T2 within the TX group, F(1, 127) = 12.83, p < .001. Teachers’ ratings of target problems at T2 were collapsed into four categories: much improved, minimally improved, not changed, and worsened. Students in TX group had higher odds of being rated as improved than did students in WL, χ2(1) = 10.84, p = .001, odds ratio (OR) = 3.377 at exit.
Teacher Outcomes
Teachers in TX showed significant gains in fidelity from T1 to T2, F(1, 52) = 28.42, p < .001; teachers in WL also improved, but not significantly so, F(1, 52) = 2.54, p = .12. There was a significant treatment groups by time interaction in which teacher fidelity increased more in TX than WL, F(1, 52) = 7.148, p = .01, ES = 0.37. We assessed the impact of treatment fidelity on child outcomes. Teacher fidelity was not associated with changes in students’ severity of challenging behaviors on the SSQ, F(1, 105) = 2.13, p = .148, and was not a significant moderator of these changes, F(1, 104) = 0.70, p = .399. Similarly, teacher fidelity was not associated with students’ improvements in teacher nominated target problems, χ2(1) = 1.21, p = .272, nor did it moderate this effect of treatment, χ2(1) = 0.395, p = 0.530. Overall, teachers in the immediate treatment group reported high buy-in, M = 6.07; range = 3.55 to 7.00.
Discussion
There have been few successful demonstrations of implementations of ASD interventions by nonspecialists in community settings. The current study shows progress, although not a complete solution, with changes on proximal outcomes (assessed by secondary measures) but not more distal outcomes (assessed by primary measures). Teachers reported strong intervention buy-in and clear increases in treatment fidelity, despite overall fidelity rates still having room for improvement.
High-baseline academic engagement (above the widely accepted 80% benchmark) was observed in both groups and across all three sites, perhaps suggesting a ceiling effect. The presence of such high-baseline engagement was surprising, given previous reports of disengagement in students with developmental disabilities (Carter, Sisco, Brown, Brickham, & Al-Khabbaz, 2008) and ASD (Krueger, Goods, Mucchetti, & Kasari, 2012; Neely, Rispoli, Camargo, Davis, & Boles, 2013; O’Reilly, Sigafoos, Lancioni, Edrisinha, & Andrews, 2005; Rispoli et al., 2011). It is possible that educators who consented to the research were already highly invested in their classrooms and had set up successful engagement strategies prior to enrollment. High-baseline engagement makes it difficult to gauge the potential of STAT for improving the academic engagement of students with ASD, and this measure may not have been the best index of change in our sample. The program might show more benefit for students with low-to-moderate engagement, and stratifying samples based on baseline engagement rates may help address this question. Regardless, these findings are encouraging in that they descriptively suggest that educators in self-contained settings are effectively promoting academic engagement, even with the challenges of teaching in underresourced settings.
The STAT group surpassed the waitlist group on secondary outcome measures, particularly teacher-reported, school-based challenging behaviors. This is an important, clinically relevant finding, given that these were the behaviors identified as most interfering to academic and social functioning within the classroom, and given that substantial improvements were noted after a relatively short-term intervention. No improvements on a standardized assessment of classroom independence were noted; this may relate to our choice to use the self-direction domain only, which focuses on independent academic behaviors (not directly targeted by STAT), as opposed to transition-related independence and challenging behavior. The generalizability of improvements in target problems to additional behaviors and situations remains an open question, as we did not include measures that captured such information in the current study. A fruitful next step might involve coaching teachers to use STAT with other behaviors and in additional situations identified as problematic.
Teachers receiving intervention improved significantly in their use of transition supports; this contrasts with findings from previous studies indicating that packaged school-based interventions can be difficult to roll out with high and consistent fidelity even with much more teacher coaching than we provided (Mandell et al., 2013). The success in this study may be attributable to three factors. First, transitions have the advantages of being discrete, fairly time-limited, and frequent—characteristics that may have helped the teachers learn the specific strategies. Second, STAT was designed to include essential components to address target behaviors, but otherwise to be simple, straightforward, and easily implemented. Teachers reported finding the limited number of steps reasonable to carry out within the daily routine, as indicated by high scores on the buy-in measure. Third, coaching sessions were embedded within ongoing classroom activities, rather than provided in a separate venue, such as in-services or workshops. In addition to making it easier for the teacher’s schedule, this model allowed for ongoing collaboration and direct application of the program in the natural environment. Qualitative feedback from teachers will inform refinements of the program, with the goals of increasing acceptability and facilitating implementation. In a future paper, we plan to examine whether teacher buy-in related to teacher adoption of transition strategies and child outcome. Fidelity in the WL group also increased, although not significantly so; this suggests that teachers’ use of transition supports may improve during the school year (perhaps because of increased familiarity between teachers and students) even without intervention. Alternatively, this could represent carryover effects from the intervention, particularly if there was communication among teachers who were randomized within the same school.
Despite significant increases in treatment fidelity, post-intervention teacher fidelity in the STAT group (approximately 70%) fell below the typical benchmark (i.e., 80% fidelity) for behavioral interventions (National Autism Center, 2015). The optimal fidelity threshold is not yet known, particularly for large-scale classroom-based interventions; our data suggest that lower fidelity can still result in positive child outcomes. As packaged, school-based interventions are tested more rigorously, elucidating the relationship between fidelity and outcomes will be an important additional dimension.
Limitations and Future Directions
Study results must be considered in light of several limitations. First, using teacher report may have introduced bias, and blind observations specific to student challenging behavior would enhance future work. A second, broader issue relates to the chosen outcome measures. In a recent review on treatments for children with ASD under 5, Smith and Iadarola (Smith & Iadarola, 2015) highlighted how variability in outcome measures in research fuels difficulties with consistently evaluating the efficacy and clinical significance of treatments. This observation extends to the literature on school-age children and is a relevant limitation to the current study. The academic engagement and challenging behavior measures were chosen because they were conceptually linked to the intervention procedures, and they did appear to be sensitive to change in our sample. However, given the lack of consensus in the field regarding which measures most effectively capture improvement in children with ASD, it is difficult to interpret degrees of clinical significance from the findings (particularly in light of modest effect sizes) and to compare findings across intervention trials. Finally, due to the challenges scheduling consistent sessions in classrooms, there was more variability in dose (i.e., total amount of coaching time) than we originally anticipated, and it is possible that dose influenced student and teacher outcomes. Systematically tracking dose will be a critical component to future classroom-based intervention research, so that this relationship can be better understood. We also cannot assume that the reported improvements will maintain over time, and analysis of follow-up data will be an important extension of these findings. Other planned analyses include evaluating the relationship among teacher buy-in, fidelity, and child outcomes.
This study examined STAT within the context of kindergarten and elementary school classrooms. The strategies employed have been shown to be effective with younger and older children as well. However, because STAT involves a specific set of procedures to train teachers to implement these strategies, evaluating the effectiveness of STAT with other age groups, perhaps with modifications to meet the needs of these students, would add to our understanding of how to best support students and teachers around daily transitions.
Our previous research suggests that teachers in underresourced schools are eager for training in strategies for students with ASD (Iadarola et al., 2014). Our current research suggests that, with coaching, they can implement new techniques, despite logistical challenges. Indeed, our teachers faced many barriers; some were specific to teaching in an underresourced school (e.g., staffing issues, limited materials available), and some were more pervasive (e.g., pressure to modify state testing materials, adapting to the first year of common core). Within this context, our findings have important implications for teachers and students with ASD in underresourced schools.
While the results are preliminary, STAT illustrates a potential bridge for the research to practice gap. The program was carried out collaboratively within multiple underresourced communities and schools, and against the backdrop of those challenges, treatment effects were obtained. Our sample had high recruitment and retention (93%), and teacher fidelity and teacher report suggest that STAT can yield positive child and teacher outcomes. The staff and time resources associated with training have significant implications for a school’s ability to initiate and sustain interventions, particularly in districts with limited resources. Advantages of STAT include its short duration and circumscribed teacher training, both of which can facilitate successful implementation in schools (Han & Weiss, 2005) because they conserve resources in terms of time, staff, and financial expenditures (Bettini, Crockett, Brownell, & Merrill, 2016). In combination, this information provides initial evidence that time-limited packaged interventions developed with community partners, such as STAT, may be a feasible way to effect change.
Footnotes
Ethical Approval
Approval for all procedures performed in studies involving human participants was obtained by each institution’s Institutional Review Board.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.