Applying an Implementation Science Framework for Adoption of a Comprehensive Program for High School Students With Autism Spectrum Disorder

Adolescence

Peterson (1988) noted that adolescence is a period of human development beginning in biology and ending in society. Puberty brings changes in body form and chemistry, and cognitive abilities continue to advance toward adult functioning levels (Lerner, 1998). Interests in independence and self-direction, sexuality, and moral values become major themes of life, as do concerns about future and life transitions (e.g., attending college) (American Academy of Child and Adolescent Psychiatry, 2010). Family relations shift as adolescents strive for increased autonomy and parents adjust to inevitable changes in the parent–adolescent relationship (Steinberg & Morris, 2001). Moreover, peer relationships become predominant for most adolescents during this period (Parker, Rubin, Erath, Wojslawowicz, & Buskirk, 2006). These factors make adolescence one of the most active periods of developmental change.

ASD in Adolescence

The phenotype of ASD changes as individuals age (Howlin, 2005). Modest improvements in autism symptoms, as reported by parents, are found during adolescence and adulthood (Taylor & Seltzer, 2011). For example, the deficits in communication that are strikingly expressed in young children with ASD often improve as children grow into adolescence and adulthood (Seltzer et al., 2003), although the majority still face challenges (Howlin, Goode, Hutton, & Rutter, 2004). In contrast, the difficulty with social competence and formation of social relationships, perhaps the most defining quality of ASD manifested in early childhood, continues as a limitation and challenge for the majority of adolescents with ASD (Carter et al., in this special issue). The qualitative nature of repetitive behavior, a hallmark of ASD, also often changes from early childhood to adolescence (Seltzer, Shattuck, Abbeduto, & Greenberg, 2004). Shattuck et al. (2007) reported that externalized maladaptive behavior such as aggression tend to be less severe during adolescence (than during earlier years), though it is still described as a significant problem by parents (Fong, Wilgosh, & Sobsey, 1993). Co-morbid mental health conditions increase during adolescence, with depression (Ghaziuddin, Ghaziuddin, & Greden, 2002) and social anxiety (Bellini, 2006) being among the most common. For many teenagers with ASD in school settings, there are also continued learning needs related to academics (Fleury et al., in this special issue), independence and self-management (Hume, Boyd, Hamm, & Kucharczyk, in this special issue), and preparing for transition (Test et al., in this special issue).

Also, adolescence can be a period of particularly high stress for families of individuals with ASD as the normative challenges associated with the transition to adulthood are compounded with multiple difficulties unique to the disorder (Smith & Anderson, in this special issue). Not surprisingly, anxiety is high for mothers of children with ASD during the adolescent period as they anticipate their child’s transition from the school system and worry about the future (Lounds, Seltzer, Greenberg, & Shattuck, 2007). Many parents of older individuals with ASD experience “burnout” from the caregiving burden, and when compared with parents of adolescents with Down syndrome, parents of adolescents with ASD were more likely to report feeling like they were “walking on eggshells” (Seltzer, Greenberg, Floyd, Pettee, & Hong, 2001, p. 285).

High Schools

A change in expectations of students and interactions with teachers occurs when students leave middle school and enter high school. Relative to middle school, there is greater expectation of personal responsibility and independence (B. K. Barber & Olsen, 2004) and intensified emphasis on academic instruction reflected in the current adoption of the Common Core (http://www.corestandards.org/) and 21st Century School Initiative (http://www.p21.org/). The size of high school may be physically larger and contain more students than middle schools. Bullying exists as an issue, and students with ASD are particularly susceptible as victims (Kowalski & Fedina, 2011). In many high schools, personnel tend to focus on isolated and independent teaching, which limits collaborative learning among professionals and a shared vision for change (Corcoran & Silander, 2009).

Foundations

The professional development model established by the National Professional Development Center on Autism Spectrum Disorders (NPDC) is the procedural foundation for the CSESA program. The NPDC model consists of several components. First, teachers and service providers use the Autism Program Environment Rating Scale (APERS) to assess and improve basic program quality. Second, service providers specify measurable goals/objectives and assess progress using Goal Attainment Scaling (Ruble, McGrew, & Toland, 2012). Third, students’ goals are linked to specific evidence-based, focused intervention practices identified through the NPDC work (Odom, Collet-Klingenberg, Rogers, & Hatton, 2010). Cox et al. (2013) provided a detailed description of the NPDC model and Odom, Cox, and Brock (2013) reported the impact of the model on program quality, teachers’ use of EBPs, and the accomplishment of goals by students.

Peer and social competence (PASC)

Limited social interaction with peers and adults, absent social relationships, and difficulties in social problem solving are defining characteristics of ASD (Carter et al., in this special issue). The PASC feature of the CSESA model utilizes a peer-mediated approach developed by Carter and colleagues (Carter et al., 2013; Carter, Moss, Hoffman, Chung, & Sisco, 2011), which builds social networks as structures for social interaction with peers or social support in which peers without disabilities provide assistance to students with ASD in learning or social tasks. In addition, for students with ASD who have sufficient communication and intellectual abilities, the CSESA model employs the Social Competence Intervention–High School (SCI-HS), a social skills training program developed by Stichter and colleagues (2010) that focuses on understanding emotions, friendship development, social problem solving, and executive function.

Academics-literacy

For many students with ASD in high school, performance on academics and state achievement tests can be limited by literacy skills, especially comprehension. Because many subjects in high school require literacy skills to access the content of coursework (e.g., history, English, even hard sciences like chemistry and physics), limitations in literacy skills affect most areas of the curriculum. For students accessing the statewide assessments, the Collaborative Strategic Reading approach originally developed by Vaughn and colleagues (2011), for students with learning disabilities, has been adapted for use with students with ASD. This work focuses on reading comprehension, group peer support, and application of literacy skills in literacy-rich content subjects (e.g., English, history, biology). For children with ASD accessing the alternate assessment, CSESA has drawn from the alternative achievement literacy approach established by Browder and colleagues (2009). This approach focuses on listening comprehension, read-aloud techniques, vocabulary development, and simplified text writing.

Promoting responsibility, independence, and self-management (PRISM)

The logistic, organizational, and social characteristics of high school environments often pose challenges for students with ASD. Students may have problems getting from class to class, arriving with the proper materials, accessing help from teachers in appropriate ways, or engaging in challenging behavior (e.g., stereotypy, echolalia; Hume et al., in this special issue). The PRISM feature of the CSESA program is designed to increase students’ independence, decrease support from peer and teachers, promote the basic survival skills’ tasks so necessary in high school, and address challenging behavior. Using the Secondary School Success Checklist, members of the autism team in a school follow a process that establishes goals related to independence and function of challenging behavior (when necessary), links goals to EBPs (Odom, Collet-Klingenberg, et al., 2010), and identifies team members who will use them. CSESA investigators have developed the process to allow maximum participation of students with ASD to determine the prioritization of goals and outcomes.

Transition and families

The fourth feature of CSESA focuses on preparation of students for life after high school and in either post-secondary education or employment. Test and colleagues (in this special issue) have collaborated in adapting the features identified in previous work as being predictive of successful transition to post-secondary education or community employment (Test et al., 2009). School team members create maps of school and community resources related to students need for transition, include family members and students with ASD in planning work-based learning experiences, and draw on EBPs related to transition to support transition goals. This feature of CSESA ensures clear, measurable transition goals that are included in the Individualized Education Program (IEP; that is, addressing Indicator 13 of Individuals With Disabilities Education Act [IDEA]). Family members, who are integral to the transition process, often feel unprepared to assist their adolescent children in their movement out of the high school setting and into meaningful post-school lives (Smith & Anderson, in this special issue). To support families, Smith, Greenberg, and Mailick (2012) developed a psychoeducational program, called Transitioning Together, which is incorporated into the CSESA model. This weekly, 8-week program covers content such as autism in adulthood, transition planning, problem solving, and risks to adult independence.

As described thus far, the CSESA program is multi-dimensional, comprehensive, and complex. During program development, stakeholders have provided advice and input concerning all key program features. It is designed for teachers to use with students across the spectrum of autism and to be situated in public high schools. To move such a comprehensive program from the drawing board to actual use in high schools, we drew from the principles and practices of implementation science. In the next section, the principles of implementation science that can lead to adoption, utilization, and sustainability of complex programs such as CSESA are identified.

Clearly Articulated and Developed Intervention Model

Implementation, by necessity, requires a clearly articulated program model. This is the “it” to be implemented. In their discussions of factors that produce positive outcomes for students or other recipients of programs, Fixsen et al. (2013) proposed a simple formula: Positive Outcomes for Students = Effective Interventions X Effective Implementation Methods. Implementation science very much focuses on the second variable in this equation, but the first variable is essential as well. Blase and Fixsen (2013) established criteria for defining programs: (a) clear description of program features, (b) clear description of essential functions (i.e., fidelity), (c) operational definitions of essential functions, and (d) practical assessment of practitioners using the program.

The work of the larger CSESA project, in its first 2 years, has been to establish and articulate practices for each program feature, described previously. This program development followed a design experimentation model (Penuel, Fishman, & Cheng, 2011), which incorporated focus group information from constituents, pilot studies of individual features, and subsequent pilot studies of combinations of features. Information from these studies led to the specification of practices in procedural guides, fidelity measures for each feature, specification of roles and responsibilities, and a summative implementation index. This level of operationalization is a characteristic of well-designed comprehensive treatment programs for children with ASD (Odom et al., in press) and is necessary for supporting schools in adopting, implementing, and sustaining a CSI model like CSESA.

Theory of Change Implementation Model and Plan

The second feature of the successful outcomes formula noted previously is “effective implementation methods.” Implementation methods should be reflected in a conceptual model that can be organized around five applied frameworks (Duda, Fixsen, & Blase, 2013), which will be described later. For these implementation frameworks to be applied, an enabling context needs to be created and nurtured. Fixsen et al. (2013) proposed creating an intentional system for aligning policies, mandates to be facilitative for supporting implementation of EBP in school settings (see Figure 1). In this model, support from the administration (i.e., executive management team) is essential for establishing the policies that enable change to occur. In addition, there needs to be a mechanism for practitioners (e.g., teachers, staff) to be able to provide feedback and data on the effectiveness of policies on the practice itself. This cyclical communication method allows for consistent feedback and modification based on actual use of an EBP. A consistent theme in the implementation science literature is the necessity of administrative support (Fixsen et al., 2010; Pinto & Slevin, 1988). Also important, however, is the need for external support for systems change, because when in the midst of the everyday workings of a system like a high school or a school district, it is difficult to create conditions necessary for change (M. Barber et al., 2009). External implementation support may be provided by original program developers or technical assistance personnel with program model expertise (i.e., called purveyors from this point forward). Purveyors work with leadership Implementation Teams at the school building level to support systems change and implementation, and this team approach will be described in a subsequent section. The Implementation Team is most directly responsible for working with students, and the information they generate about the use of the CSI is communicated back to the administrative leadership, creating a feedback loop that is likely to support further adoption and sustainability.

OPEN IN VIEWER

Figure 1.

Fixsen, Blase, Metz, and Van Dyke (2013) conceptual model.

The CSESA program utilizes this implementation model when working with school systems to adopt the program. The CSESA team first meets with the administrative team in a district to marshal administrative support necessary for adoption and implementation of the program in high schools. To move forward with the mutual selection process, the administrative team must approve policies that allow the CSESA personnel to assist schools in creating autism teams (A-Teams), to work with the teams and to facilitate the collection of information about program implementation, program acceptability, and student outcomes. This information is shared with the A-team members and also district administrators, creating the feedback loop noted previously.

Stages of Implementation

Researchers and implementation scientists commonly acknowledge that full implementation of a CSI is a process that occurs in identifiable stages. Fixsen and colleagues (2010) proposed that implementation of programs occurs across four stages, each having different function. The implementation process begins with exploration in which program providers are actively considering adoption of a new program or practice. Activities to determine need and capacity include identifying whether staff has the interest, time, and expertise; learning about existing teaming structures; determining the alignment of the program with school or district improvement plans; and gathering information from stakeholders and engaging in leadership at the school and district levels (Saldana, Chamberlain, Wang, & Brown, 2011). This stage typically “ends” with a decision to move forward or not.

The next stage is installation in which school personnel and purveyors form local implementation teams; team members participate in training; implementation team and purveyors plan onset of implementation; and teams establish communication protocols. The work of the teams is to identify strengths of the organizations, pull resources and materials together needed to launch the new way of work, and develop an implementation action plan. During initial implementation, the next stage, team members start using CSI practices in their schools, with assistance from purveyors and possibly other team members. Vital at this stage is an implementation action plan with timelines and a clear strategy for using data to make decisions and modify the strategy if necessary. Full implementation occurs when the CSI practices become a part of the standard operating procedure for schools and may be utilized with minimal or no assistance from external coaches or other technical assistance providers.

CSESA is unique in that we made strong efforts to include best practices and stakeholder advice from a broad range of perspectives (e.g., individuals with lived experience, school personnel, technical assistance providers, service providers) to develop a model that can align with needs of high school students with ASD and the high school environment. For CSESA, the exploration stage of the implementation process begins with meeting district administrators, sharing thoroughly the features of the CSESA program and the roles, mutual expectations and responsibilities of all who would be involved, and intended outcomes. If district administrators approve, CSESA staff then meet with principals and school staff to again describe thoroughly the CSESA model and its possible use in their school. School teams have the opportunity to consider and “explore” the need, fit, and feasibility of how this model can be beneficial to their student body.

Only after the local school staff indicates interest in implementing the program does the process continue with establishing an A-Team at a school and writing a memorandum of understanding. Agreement by local school personnel and administration signals the beginning of the installation stage in which plans for training are established. Initial training usually will begin before the school year starts. Training will provide essential knowledge and skills related to assessing program quality, establishing measurable IEP goals, and matching IEP goals with EBPs.

In the fall of the school year, A-Teams begin using CSESA procedures, which marks the Initial Implementation stage. At this stage, CSESA staff and select A-Team members (if appropriate) provide concentrated coaching to assist school personnel with use of the CSESA procedures in the school and progressive rollout of program features (e.g., PASC, PRISM). Because CSESA is a complex program with multiple features, implementation of the entire model will occur over a 2-year period, and the initial implementation stage may last that long. Full implementation will occur when A-Teams and school personnel use the CSESA model independently, with CSESA staff no longer providing support, and the practices have become a part of the schools general operating procedures. In addition, a goal of CSESA is to build the internal capacity for the local school district to provide support to staff at other schools, through training and coaching, to employ the CSESA model. This elaboration across schools will also be an indicator of full implementation.

Teaming as an Essential Element

Implementation of CSIs requires a team approach (Fixsen et al., 2013). Pinto and Slevin (1988) noted that recruitment, selection, and training of the necessary personnel for a program team are critical success factors for program implementation. As noted, high schools themselves are complex systems. Implementing a CSI in an organization in which complexity already exists requires a team effort. The team should consist of a leader or co-leaders, some organization members involved in direct implementation of the program with the recipient, and a member(s) from the school administration. Team members and program support personnel should be directly involved in planning the actions and activities in the school context that will lead to implementation. For example, in their study of preschool inclusion of children having different disabilities, Lieber and colleagues (1997) found that collaboration among team members was a prime predictor of successful preschool programs. In a separate study of curriculum implementation in Head Start and state prekindergarten programs, Lieber et al. (2009) found that staff “buy-in” to the curriculum model was a key indicator that differentiated high and low implementers of the curriculum.

With the CSESA model, as noted, formation of A-Teams at the high school implementing the program is essential and a criterion for participating in the program. A-Teams may consist of a special education teacher as a leader or co-leader of the team, a related service provider, a general education teacher(s), a community-vocational transition liaison/teacher(s), and a member(s) of school administration. Other individuals may participate depending on the local school context. Attempts will be made to recruit a member or members of the team who has/have high social status within the school (e.g., a coach of a basketball team, a particularly popular general educator) to enhance the status and acceptability of the CSESA program (Saldana & Chamberlain, 2012). An essential feature is that all team members feel ownership in the CSESA program and process for implementation (M. Barber & Mourshed, 2007). Ownership is enhanced by the purveyor providing detailed information about the features of the program and creating opportunities for service provider to participate in decisions about its future implementation. Although CSESA implementation always starts with the CSESA foundation, A-Team members, in collaboration with CSESA staff, will plan the order and timing of the implementation of individual features (e.g., social competence, academic-literacy).

Coaching as a Key Feature

The features of a CSI program are often introduced through workshops, procedural manuals, and online modules, but a key finding of implementation science is that when purveyors only present information about the content with no follow-up, subsequent implementation in classes or other service contexts usually does not occur (Fixsen et al., 2010). In their study of the Learning Experiences—An Alternative Program for Preschoolers and Parents (LEAP) comprehensive treatment program, Strain and Bovey (2011) provided initial training and project materials to two groups of randomly assigned preschool teachers of children with ASD. With one group, they provided ongoing coaching and technical assistance for 2 years, and for the other, they only provided initial training and the program materials. The group receiving only the initial training and materials implemented the model at a significantly lower level of fidelity as compared with the group receiving ongoing coaching and feedback. This study is one of the few to experimentally demonstrate the important effect of coaching.

The CSESA program has adopted the coaching model established through NPDC (Kucharczyk et al., 2012). Coaching includes a cyclical, three-step process of pre-observation, observation/action, and post-observation. Throughout this cycle, coaches use fidelity tools to collect data on implementation, model implementation and reflection on practice, and plan for ways to enhance implementation if necessary. Initially coaches are CSESA staff members, with the responsibility shifting to the local school district personnel when feasible, and also the level of coaching decreasing as A-Team members and other school staff become independent implementers.

Sufficient Time for Implementation to Occur

As noted earlier and reiterated here, implementation of CSIs sometimes requires significant systems changes, and such changes require time to put into place. Fixsen et al. (2005) estimated that complex human service programs may take as long as 5 to 7 years to move to full implementation. In a recent treatment comparison study of two comprehensive treatment models programs for young children with ASD, the purveyors of both programs indicated that the training and feedback necessary for acceptable levels of fidelity and implementation would take at least 2 years (Boyd et al., 2013).

Given the complexity of the CSESA model, at least 2½ years is planned for a school to reach full implementation. The first ½ year focuses on exploration (i.e., meeting with high school staff, establishing memoranda of understanding, securing resources) and planning for training. The summer before implementation begins, CSESA staff provides initial training to A-Team and perhaps some school staff (the Installation phase). Initial implementation begins in the fall of the first year, with relatively heavy levels of coaching and involvement with A-Team and other school staff. CSESA staff also provide ongoing training and coaching as new features of the program are implemented (e.g., academic or social competence feature). Again, the goal is to have all the features of the CSESA program implemented by the end of the 2½-year period.

Readiness for Change

Not all organizations have the capacity to implement complex programs with fidelity. For such programs, there may be prerequisite requirements necessary for implementation, or the organizations themselves may not provide “enabling environments” (Duda et al., 2013). For example, the cultural norms of the school, the CSI’s alignment with teaching practices, or systemic variables (e.g., teachers not having planning or meeting time) may be incompatible with basic features of the CSI. In addition, some school personnel and faculty may feel they are already adequately accomplishing the goals that the CSI has established for itself and are in no need of the program. In such situations, individual schools or school districts may not be “ready” for attempts at implementation of a CSI program (Fixsen et al., 2010). Although many of the adaptive challenges are expected and tended to during the exploration stage, the work of the Implementation Team (or A-Team) is to build readiness. This can be achieved by working directly with leadership, getting a deeper understanding regarding the concerns, and providing transparent information about the intended outcomes.

To determine school programs’ readiness for change and adoption of the CSESA program, CSESA staff identify the goals of the CSESA program, the resources it can commit to implementation, the resources the school system and individual school would need to commit, and the alignment of CSESA features with the characteristics and logics of the school setting (e.g., block scheduling, availability of a consistent time during the day or week for SCI-HS training). During the exploration stage of implementation when CSESA staff meets with school administrators and principals, CSESA and school staff gauges their readiness as well as try to anticipate any potential barriers. For example, after learning about the CSESA program, a special education teacher might be motivated to become the A-Team leader and major advocate in the school, but she may not have any time in her schedule that she can commit to the project.