Making a collective impact: A School-to-Work Collaborative model

Abstract

BACKGROUND:

A seamless system of transition requires linkages and collaboration between schools and adult service agencies. A shared vision for change is needed including a common understanding of the issues and a joint approach to developing solutions.

OBJECTIVE:

The Indiana School-to-Work Collaborative developed a multi-component intervention package as a collective impact approach. The School-to-Work Collaborative interagency transition model was designed by using authentic stakeholder engagement throughout the process.

METHODS:

This study compares the effects of a school-to-work collaborative transition model to improve employment outcomes and agency connections for transition-age youth with disabilities where a community provider employment specialist (e.g., Career Coach) was embedded in the schools.

RESULTS:

Embedding a Career Coach from an adult employment provider in schools resulted in more work-based learning experiences, better employment outcomes, and more connections to adult service providers compared to schools without a Career Coach. Implementing policy changes from the federal and state levels without preparation time had an impact at the local level that ultimately impacted students and families.

CONCLUSION:

Strong, effective interagency collaboration can result in a collective impact. Bringing together key stakeholders to design, monitor, and evaluate the model, as well as intended and unintended consequences, can result in policy and procedural changes.

Keywords

Transition employment career coach

1. Introduction

Transition planning has been described as a road map, a blueprint or pathway to post-school participation and outcomes. Working together on behalf of transition-age youth and their families has been a cornerstone of transition planning and services since the 90s (IDEA, 1990, 1997, Rehabilitation Act Reauthorizations, 1992, 1997). Through a coordinated set of activities in a results-oriented process, the Individuals with Disabilities Education Improvement Act (IDEIA) of 2004 emphasized postschool outcomes for youth with disabilities in the areas of postsecondary education and training, employment, and independent living, if needed. The intention of the coordinated set of activities provisions in IDEA is to include outside agencies to support the student after high school in meeting their postsecondary goals, requiring collaboration, cooperation, coordination and communication (Trach, 2012) to foster a seamless system of transition. IDEIA also requires schools to identify transition services needs within the Transition Individualized Education Program (TIEP), and if appropriate, a statement of the interagency linkages and responsibilities.

The collaboration and coordination of federal core programs, e.g., Vocational Rehabilitation, Workforce Development and Departments of Education, is intended to be demonstrated through the Workforce Innovation and Opportunities Act (WIOA) (2014). To access high quality workforce services and preparation for competitive integrated employment for individuals with disabilities, WIOA 2014 requires that state vocational rehabilitation agencies commit at least 15 percent of their funding to provide transition services to youth through the pre-employment transition services (Pre-ETS). This requires collaboration and coordination with schools, Vocational Rehabilitation and community providers.

It is the act of working together at state and local levels where the policies, procedures and practices get implemented that leads to shared responsibility of effective transition service delivery (Noonan, McCall, Zheng, & Erickson, 2012; Noonan, Morningstar, & Erickson, 2008; Plotner, Mazzotti, Rose, & Teasley, 2018). Key elements of the definition of interagency collaboration include “a process through which the whole is greater than the sum of the parts” (Povenmire-Kirk, Diegelmann, et al., 2014, p. 52); formal and informal relationships between schools and adult agencies in which resources are shared (Noonan, et al., 2008); and, “a clear, purposeful, carefully designed process that promotes cross agency, cross program, and cross disciplinary collaborative efforts leading to tangible transition outcomes for youth” (Rowe, Alverson, Unruh, Fowler, Kellems, & Test, 2014). Working together or interagency collaboration requires not only clearly defined roles and responsibilities, but a process for communication, collaboration and coordination of resources.

The purpose of this article is to describe a five-year research study funded by the National Institute on Disability, Independent Living and Rehabilitation Research (NIDILRR) to determine the effects of embedding staff focused on providing employment resources in schools to improve employment outcomes of transition-age youth. The Indiana School-to-Work Collaborative developed a multi-component intervention package as a collective impact approach (Hanleybrown, Kania, & Kramer, 2012). A Career Coach from an adult agency employment provider focused on agency connections, work experiences and employment outcomes while overlapping services to promote a seamless system of transition services. Initial implementation of this model began prior to WIOA 2014. Results of the study describe how policy changes at the federal and state level impacted local level procedures and practices. Data from family and provider focus groups guided recommendations for future research.

1.1. Interagency collaboration research

Strong, effective interagency collaboration can result in a collective impact. The concept of collective impact was introduced by Kania and Kramer (2011) by describing several examples of highly structured cross-sector collaborative efforts that had achieved substantial impact on a larger scale social problem (e.g., transition outcomes). Cross-agency perspectives can improve the collective understanding of the issue or problem and create a sense of mutual accountability (Kania, Hanleybrown, & Juster, 2014). Five conditions of collective impact are described in Table 1 (Hanleybrown, Kania, & Kramer, 2012). In order for collective impact to be effective, teams must consider “who is engaged, how they work together, and how progress happens” (Kania, et al., p. 2).

Table 1
Conditions of Collective Impact

Conditions Conditions of Collective Impact Collective Impact

Common agenda All participants have a shared vision for change including a common understanding of the problem and a joint approach to solving it through agreed-upon actions. Stakeholders designed the STW model based on the shared understanding of the problem, and the solutions arose from the interaction of many organizations.

Shared measurement Collecting data and measuring results consistently across all participants ensures efforts remain aligned and participants hold each other accountable. All stakeholders agreed upon the same goal and key data elements to be measured.

Mutually reinforcing activities Participant activities must be differentiated while still being coordinated through a mutually reinforcing action plan. Larger impact depends on increasing cross-sector alignment and learning among many organizations. The Statewide Transition Advisory work group identified learnings and policy changes.

Continuous communication Consistent and open communication is needed across the many players to build trust, assure mutual objectives, and create common motivation. Government sectors were essential partners and the Statewide Transition Policy work group represented many, if not all, the state agencies, funders, and advocates serving transition-age youth.

Backbone support Creating and managing collective impact requires a separate organization(s) with staff and a specific set of skills to serve as the backbone for the entire initiative and coordinate participants and organizations and agencies. The Indiana Institute on Disability and Community coordinated the sites, provided TA, and shared lessons learned. Both DOE and VR funded professional development and technical assistance to move transition services forward.

Conditions	Conditions of Collective Impact	Collective Impact
Common agenda	All participants have a shared vision for change including a common understanding of the problem and a joint approach to solving it through agreed-upon actions.	Stakeholders designed the STW model based on the shared understanding of the problem, and the solutions arose from the interaction of many organizations.
Shared measurement	Collecting data and measuring results consistently across all participants ensures efforts remain aligned and participants hold each other accountable.	All stakeholders agreed upon the same goal and key data elements to be measured.
Mutually reinforcing activities	Participant activities must be differentiated while still being coordinated through a mutually reinforcing action plan.	Larger impact depends on increasing cross-sector alignment and learning among many organizations. The Statewide Transition Advisory work group identified learnings and policy changes.
Continuous communication	Consistent and open communication is needed across the many players to build trust, assure mutual objectives, and create common motivation.	Government sectors were essential partners and the Statewide Transition Policy work group represented many, if not all, the state agencies, funders, and advocates serving transition-age youth.
Backbone support	Creating and managing collective impact requires a separate organization(s) with staff and a specific set of skills to serve as the backbone for the entire initiative and coordinate participants and organizations and agencies.	The Indiana Institute on Disability and Community coordinated the sites, provided TA, and shared lessons learned. Both DOE and VR funded professional development and technical assistance to move transition services forward.

Adapted from Hanleybrown, F., Kania, J., & Kramer, M. (2012). Channeling change: Making collective impact work. Leland Stanford Jr. University: Stanford Social Innovation Review.

A seamless system of transition requires linkages and collaboration with adult service agencies. Interagency collaboration has been researched over the years to determine best practices for service delivery (Certo, Mautz, et al., 2003; Fabian, Simonsen, Deschamps, Dong, & Luecking, 2016; Luecking & Luecking, 2015; Noonan, et al., 2008; Povenmire-Kirk et al., 2014; Trach, 2012). Analysis of state-level transition teams (Noonan, et al., 2012), perceptions of understanding of stakeholders’ roles (Plotner, Mazzotti, Rose & Teasley, 2018), and strategies used by high performing local districts and communities (Noonan, et al., 2012) focused on improving interagency collaboration and ultimately, postschool outcomes for transition-age youth.

In a review of 10 studies on interagency collaboration, Fleming, Del Valle, Kim, and Leahy (2012) noted the advantages of interagency collaboration including pooling of funding, sharing of staff expertise, access to resources and services, and coordination of service delivery. On a systems-level, interagency collaboration offers consistent service delivery to achieve employment outcomes as students transition from high school to adult services.

One of the key predictors of postschool success is the linkages with adult support systems to obtain post-secondary outcomes (Test, Mazzotti, et al., 2009). According to the Division of Career Development and Transition position paper (Mazzotti, Rowe, Cameto, Test, & Morningstar, 2013), interagency collaboration is an additional area of research necessary to identify evidence-based interventions and practices that support effective interagency collaboration in districts and schools.

Demonstrated practices and models of interagency collaboration during the transition process emerged over the last 20 years (Landmark, Ju, & Zhang, 2010). Research, albeit limited, emerged on a few models of interagency collaboration such as the Service Integration Model (Certo & Luecking, 2006; Certo, et al., 2003) the Maryland Seamless Transition Collaborative (Luecking & Luecking, 2015), and the Communicating Interagency Relationships and Collaborative Linkages for Exceptional Students (CIRCLES) model (Povenmire-Kirk et al., 2014). The Service Integration Model (Certo et al., 2003) focused on students with severe intellectual disabilities during their last year of school (e.g., 21 years old). In this model, the service delivery model for transition shifts from the public school as sole provider to one based on service integration across multiple systems. School districts integrate their staff and resources with those of a community rehabilitation agency, state rehabilitation system, and state developmental disabilities system before graduation to jointly produce meaningful work outcomes and secure ongoing support after high school.

Povenmire-Kirk and colleagues (2014) implemented the Communicating Interagency Relationships and Collaborative Linkages for Exceptional Students (CIRCLES) three-tiered model of service delivery based on the theory of change through Community-level Teams, School-level Teams and IEP teams. CIRCLES allow agencies to provide services directly to the students and families who need involvement from multiple adult service providers while students are still in high school to improve interagency collaboration among teachers, adult service agencies, and community members and increase student’s self-determination and participation in IEP meetings (Flowers, Test, et al., 2018). Focus groups conducted after CIRCLES implementation were able to identify challenges and barriers (e.g., staff awareness on available services, student and family preparation, provider agencies’ needs for student information, practical issues) and successes (e.g., networking and collaboration, improved communication, student empowerment).

The Maryland Seamless Transition Collaborative (MSTC) is a model designed to improve the postsecondary outcomes of students with disabilities through a sequential delivery of specific transition service components beginning in early high school (Luecking & Luecking, 2015). Key intervention components include: Discovery, individualized work-based experiences, paid inclusive employment, family supports, early Vocational Rehabilitation (VR) agency case initiation, systems linkages and collaboration, and coordination with teachers and school staff. Results indicated that the model was able to address the known barriers for students to seamlessly transition to postschool settings.

These models begin to address a few of the major barriers to the effective delivery of services – the fragmentation, duplication, and inadequacy of these services and programs to promote outcomes. The ever-changing demands and expectations of school personnel to meet state proficiency testing, ensure students are college and career ready, improve graduation rates, decrease drop-out rates, and perform in a result-driven accountability system with limited resources often results in a lack of communication, coordination, and collaboration to implement an effective seamless transition process. School personnel are not usually equipped with the skill-set to address the needs of employers and navigate adult service systems.

1.2. Using action research to design the School-to-Work Collaborative service delivery model

Building and expanding upon the current models and research, the Indiana School-to-Work Collaborative used authentic stakeholder engagement and action research (Patton, 2015) to address the lack of collaboration among schools and adult service providers that impacted transition outcomes. Indiana has a long history of collaboration and coordination; however, due to a number of factors (e.g., state leadership change, budget constraints), relationships needed to be revitalized between local schools, VR and adult employment providers. Using the action or participatory research approach, the first author facilitated a number of meetings at the request of both schools and adult employment providers or Community Rehabilitation Programs (CRPs). Multiple issues emerged that led to the need for a new design for collaboration: 1). Schools expressed frustration about the number of students not connected to VR or other adult agencies; 2). Schools were frustrated that if school personnel (e.g., work study coordinator or transition coordinator) found a student a job prior to exiting school, CRPs were not providing the support needed for the student to keep the job; 3). Schools acknowledged the importance of collaboration, but were overwhelmed with the number of adult agencies wanting to come into the school, requesting their time, while teachers were trying to balance their teaching and student responsibilities; 4). Schools expressed the lack of Vocational Rehabilitation Counselors (VRCs) attending Transition IEP meetings (after sending several notifications) and acknowledged the lack of understanding of the ever-changing policies and procedures by VR, Medicaid Waiver, and other adult agencies; 5). VRCs often complained about being invited to Transition IEP meetings without sufficient notice, students not keeping appointments, and/or, not getting the necessary information for the referral. 6). CRPs were frustrated over the lack of good job matches (including knowing the employer and the business culture) for those students who obtained competitive jobs through the schools and/or the jobs were so part-time that students needed another job; and finally, 7). CRPs were frustrated with the lack of transition-age youth referrals and collaboration with schools, often due to VR not serving students until their last year or even last semester.

Key representatives from the schools (e.g., Department of Education, Special Education Directors, transition personnel), the Indiana Association of Rehabilitation Facilities (INARF) employment providers, Vocational Rehabilitation (VR), The Arc of Indiana, Indiana Association of People Supporting Employment First (INAPSE), and the Indiana Special Education Parent Center (IN*SOURCE) met regularly for months to design the model and obtain feedback from other key partners.

As the authors stated (Hanleybrown, Kania, & Kramer, 2012), three preconditions must be in place before launching a collective impact initiative:

Influential champion (or small group of champions). These champions command the respect necessary to bring cross-sector leaders together and keep their active engagement over time. Our key stakeholders became our champions because of the diminished collaboration that had occurred over time. Many of the stakeholders also served on the Statewide Transition Policy workgroup which is described below.

Financial resources. A sustainability plan was developed at the onset of the model design and coordinated with Vocational Rehabilitation Services and DOE.

Urgency for change. The lack of access and connections to services for transition-age youth and their families brought together key stakeholders to develop and implement a new school-to-work model for obtaining employment and connections with adult agencies.

1.3. The Indiana School-to-Work Collaborative framework

The premise for this collaborative model was to serve as an interdependent system that addresses issues and opportunities for employment for young adults with disabilities. The underlying rationale for the framework was to make a collective impact that implemented evidence-based practices and resulted in improved work experiences, employment outcomes, and adult agency connections that were aligned with the needs of each student and family. The approach, depicted in Fig. 1, identified strategies to address four major areas: Policy, Procedures, Practice, and People. These strategies were intended to help youth transition from students to employees. For best practices to occur, the local Collaboratives must encourage and support implementation of those practices. A system of professional development, including technical assistance, must be in place and designed to teach those practices to new and current practitioners using a variety of teaching modes. The state infrastructure needs policies to guide implementation of those procedures and practices as well as a quality assurance system based on accountability and data-based decision making. Because the components of a state system are interrelated, a change in one component is not likely to be sustained unless accompanied by changes in related components. Since the components are interrelated, inclusion of all stakeholders at the state and local levels ensured a more sustainable system.

Fig. 1

Conceptual Framework for the Indiana School-to-Work Collaborative.

1.3.1 The State Transition Policy Workgroup

The State Transition Policy Workgroup, representing over 20 different state agencies, INSOURCE, advocacy groups, associations, Independent Living Centers, disability-specific groups, community rehabilitation providers, and universities served as the oversight of the project in determining policy barriers, procedural guidelines, and sustainability efforts. Specifically, the State Director of Vocational Rehabilitation Services, the Associate Director of Special Education, Department of Education (IDOE), and other state agencies (e.g., Developmental Disabilities and Mental Health) were key partners in determining policy changes and sustainability. The Indiana Institute on Disability and Community (IIDC), Indiana University’s Center for Excellence on Disabilities, provided the technical assistance and professional development to the local collaboratives, conducted data collection, and provided data analysis.

The State Transition Policy Workgroup has operated for almost 20 years in various capacities by meeting quarterly to discuss a variety of statewide transition issues, legislation, policy updates, and other initiatives. At least once a year, the School-to-Work Collaborative project provided updates, shared preliminary results and successes, obtained feedback and addressed issues and/or barriers.

1.3.2 Key elements of the local Collaborative

Two aspects of the School-to-Work Collaborative were unique. First, an employment specialist (e.g., Career Coach) from the community rehabilitation provider was embedded in the school focusing on the work experiences and employment outcomes of students with disabilities. The Career Coach worked closely with school personnel to reduce or eliminate duplication of services. School personnel were able to rely on an entity with expertise in working with businesses to provide information, connect students to other adult services, and overlap services to promote a seamless transition. Second, a coalition of providers working together and serving on the local collaborative reduced the number of entities entering the school and reduced the confusion (and meeting time) for schools. The Career Coach represented the local Collaborative rather than an individual provider.

Each local Collaborative included key stakeholders: local VRC and/or VR supervisor; community employment/rehabilitation provider(s); school district(s) representatives such as a special education supervisor, director, transition coordinator or high school department chair; state parent center representative; and other representatives that were locally determined (e.g., workforce development, developmental disabilities, mental health, etc.). The local stakeholders served as the management team for the local Collaborative through consensus-building, shared data-based decision-making and problem-solving. Elements of the School-to-Work Collaborative model are described below.

1.3.2.1 Community rehabilitation providers.

Both CRPs and mental health agencies that provided employment services were part of the collaborative. The coalition of providers had to “opt-in” to be part of the School-to-Work Collaborative by signing a Memorandum of Understanding (MOU) to define expectations, roles and responsibilities, and procedures for addressing issues or conflict. The MOU was the same for all treatment sites, including marketing materials.

1.3.2.2 Career Coach as single point of contact.

A Career Coach, embedded in the school, served as a single point of contact and was employed by one of the community rehabilitation providers. The Career Coach represented the Collaborative to focus on employment services and help students/families access Vocational Rehabilitation and other services. By being immersed in the school culture, the Career Coach was able to get to know the students, communicate regularly with designated school staff, provide outreach to potential VR-eligible consumers, identify agencies that may need to be at the Transition IEP meeting, connect students and families to employment and other agencies, partner with school staff on self-determination or soft skills training, and provide consultation on quality work experiences that would possibly lead to an employment outcome. The Career Coach usually served more than one high school depending on size of the high school(s) and location (e.g., rural). The Career Coach supported VR in helping students and families choose an agency to provide the employment services (e.g., for work-based learning or work experience support, job development, job placement). The Career Coach was required to have completed the state-approved and Association of Community Rehabilitation Educators (ACRE) certified employment specialist training and have gained experience as an employment specialist.

1.3.2.3 Discovery profile.

The Discovery profile (Callahan & Condon, 2007; Condon, 2008) was a process used by the Indiana VR system as a way to collect information on the student’s attributes, strengths, preferences, interests, and support needs and to build upon the Transition IEP and transition assessments. A number of activities occurred throughout the Discovery process related to getting to know the students including informational interviews, job sampling, tours of businesses, work-based learning, and a participation in a variety of activities to learn about the student’s skills (e.g., cooking together) with the ultimate goal of a good job match.

1.3.2.4 Work experiences.

Work experience while in high school is one of most compelling predictors of post-school success (Test et al., 2009). Prior to the student’s last year (summer before their last year in high school or end of junior year), the Career Coach provided consultation on paid and nonpaid work experiences. These experiences were varied and could start with in-school jobs that led to out-of-school jobs. In the last year of high school, the focus was providing students with 1-3 work experiences funded by Vocational Rehabilitation. The VR Counselor and Career Coach would work together to assist students and families in choosing an employment provider from the provider list (those that opted into the Collaborative) to develop work experiences and provide the onsite support, with the intention of the work experiences leading to a permanent paid job. Therefore, the student (and family) began working with the employment provider and job coach prior to leaving school.

1.3.2.5 Student determination skills training.

Evidence continues to emerge that students with higher self-determination skills are more likely to be engaged in postschool employment. Research shows having higher levels of self-determination correlates with improved school and post-school outcomes (Cobb, Lehmann, Newman-Gonchar, & Alwell, 2009) and is a predictor of better post-school outcomes (Test, et al., 2009). Additionally, student involvement in IEPs indicates students exhibiting greater self-determination (Williams-Diehm, Wehmeyer, Palmer, Soukup, & Garner, 2008) are more informed about the process, experience greater efficacy in planning for their future, and are better able to articulate their goals (Arndt, Konrad, & Test, 2006; Martin et al., 2006).

Students in the School-to-Work (STW) project participated in a structured training to increase their self-determination, leadership skills, self-advocacy, and soft skill development using evidence-based curricula (e.g., Self-directed IEP, Whose Future Is It Anyway?, I’m Determined, Skills to Pay the Bills). Teachers expanded on the scope and sequence of the curricula using their own materials. Instruction occurred based on the school schedule and needs (e.g., some schools provided instruction each semester of the last two years; others infused the curricula into current courses; some provided instruction during resource time). Application of the learned skills included interviewing employment providers to select a provider (e.g., develop their work experiences and job placement), leading their Transition IEP meetings, or practicing the skills on the job with the support of the employment specialist. Many STW sites began implementing the curriculum during the sophomore and junior years leading up to the last year where students participated in work experiences.

1.3.2.6 Benefits planning for youth.

For those students on SSI or SSDI, a referral was made to the Indiana Benefits Information Network (BIN), to enable a benefits plan to be developed prior to the final paid job (after work experiences). VR paid for a benefits analysis with a certified benefit counselor for each student, as appropriate.

1.3.2.7 Family training and support.

Each local Collaborative included a parent representative from the state Parent Center, INSOURCE. The parent representative served as a mentor to families and coordinated trainings for families. Trainings included information to families about the Collaborative, benefits planning and use of work incentives, support for how to access services, expectations for employment, specifics about some adult support agencies, and other locally-determined topics.

2. Methods

Using both quantitative and qualitative methods, this study evaluated the effects of a school-to-work collaborative model to improve employment outcomes and agency connections for transition-age youth with disabilities where a community provider employment specialist (e.g., Career Coach) was embedded in the schools. The Collaborative partners included: Vocational Rehabilitation (VR); school district(s); community rehabilitation provider(s); representatives from the state parent center; and other relevant community partners (e.g., developmental disabilities, mental health, independent living centers, one-stop centers). The design of this study was based on the following research questions:

Research Question 1. Do schools with embedded employment resources (e.g., Career Coach as a single point of contact and Collaborative support) have more students who leave school with a job and work experiences than those from schools with no embedded resources?

Research Question 2. Do schools with embedded employment resources have more students leaving school with connections to adult support agencies, including Vocational Rehabilitation, than schools with no embedded resources?

Research Question 3. To what extent, if any, did implementing the local Collaborative result in policy or procedure changes?

2.1. Participants/Sample

The Collaborative selection procedure involved several steps. The sample for this study was comprised of high schools within six different school districts (some districts have more than one high school, depending on size; two districts chose to collaborate together due to proximity and past coordination and were targeted as one site). First, the authors and representatives from the State Transition Policy Workgroup selected five sites for the study through a Request for Participation (RFP) that met general selection criteria (e.g., willingness to work as Collaborative; willingness to use project-identified measures; willingness to follow research protocols, and support from the local Director of Special Education, high school principal, and school personnel to implement self-determination curriculum). Funding was provided to each of the sites for the five years of implementation, with the last year transitioning to the sustainability plan.

Second, five additional sites were selected as the control group with similar demographics (e.g., geographical location, percent of special education students, economic data, free and reduced lunch, size of high school). Each of the control sites performed their transition planning and services as typically done or a business-as-usual approach; however they were paid a stipend to collect data on self-determination and employment outcomes. Each treatment and control site were selected to represent the diversity of the state and comprised a heterogeneous group of large urban, suburban, and rural high schools.

2.2. Student characteristics

Two targeted groups of students participated in the project. The primary group were students enrolled in the course of study to obtain a diploma, who at some period in their high school academic career were struggling to maintain the requirements and/or moved to a certificate of completion. Typically, this includes students considered in the high incidence categories. The secondary targeted group were students receiving a diploma with a postsecondary goal of going directly into the workforce or who were uncertain of their future rather than planning on entering higher education or postschool training. Table 2 describes the primary disability for students enrolled in both the STW collaboratives and the control sites. There were 410 students served through the STW collaborative, freshmen through seniors, where the self-determination curriculum was implemented with a focus on the work experiences during the senior year; 118 students, primarily seniors, were served through the control sites.

Table 2
Student Characteristics

Disability Category School-to-Work Sites N = 410 Control Sites N = 118

Autism 13% 9%

Emotional Disability 14% 8%

Learning Disability 33% 35%

Mild Intellectual 19% 25%

Moderate Intellectual 3% 1%

Multiple Disabilities 2% 1%

Other Health Impairments 12% 6%

Orthopedic 0 1%

Traumatic Brain 0.2% 1%

Deaf/Blind 0.5% 0

Hard of Hearing/Deaf 2% 0

Unknown 0.8% 12%

Disability Category	School-to-Work Sites N = 410	Control Sites N = 118
Autism	13%	9%
Emotional Disability	14%	8%
Learning Disability	33%	35%
Mild Intellectual	19%	25%
Moderate Intellectual	3%	1%
Multiple Disabilities	2%	1%
Other Health Impairments	12%	6%
Orthopedic	0	1%
Traumatic Brain	0.2%	1%
Deaf/Blind	0.5%	0
Hard of Hearing/Deaf	2%	0
Unknown	0.8%	12%

2.3. Procedures

This research was reviewed and approved by the Indiana University Institutional Review Board (IRB). School personnel and Career Coaches were trained to obtain informed consent from all participants. Career Coaches were trained on data collection and entry methods. Monthly conference calls were held with the Career Coaches to share questions and problem-solve concerns. Teachers were trained on the self-determination framework and curricula. All local sites were provided training and technical assistance as needed.

2.4. Intervention package

Two aspects of this model are unique. First, a community provider employment specialist (Career Coach) was embedded in the school focusing on making connections to adult agencies and work experiences that lead to employment outcomes when students leave high school. Second, the Career Coach served as a single point of contact for the schools, students/families and the community rehabilitation providers (e.g., communities that have more than one community rehabilitation provider). The Career Coach represented a “coalition” of employment providers serving on the Collaborative, overlapping supports with schools to ensure a coordinated and seamless system of transition. This eliminated the number of meetings for schools to conference with each group.

3. Results

Using authentic stakeholder engagement and action research throughout the study, the conceptual framework was intended to address the known barriers impacting a seamless transition system. Results of this study addressed the three key overarching research questions; however, other categories emerged during the course of the implementation and are described below.

3.1. Employment upon leaving high school

Students in the STW collaborative participated in over 188 internships or work experiences, averaging nine hours a week (range of one to 40 hours per week) with a length of four weeks (range of one day to 27 weeks) during their last year of school. This did not preclude them from having earlier experiences; however, VR only paid for the last year of work experiences. Over the course of the four years of implementation, 91 students or 35% exited high school with a job based on their work experiences or internships. Students left school earning an average of $9.23 per hour (range of $7.25 to $22.00 per hour) and worked an average of 24 hours per week (range of four to 40 hours).

Three months after exiting high school, 85% (77) of the 91students were still employed, with three students having a career advancement. The other students either quit, moved or were fired. For the control sites, school personnel did not follow students or keep track of outcomes.

3.2. Agency connections

The key to transition is connections to adult agencies to help students reach their post-secondary goals, specifically employment. Table 3 shows the comparison of agency connections with the STW collaborative sites and the control sites. Over all four years, the STW sites had more students connected to either Vocational Rehabilitation and employment providers than the control sites. Students could not receive services from an employment provider unless they were enrolled in VR first, which students often were enrolled by the end of their junior year. The Career Coach and VR would assist the student in choosing the employment provider to best meet their employment support needs. As with employment outcomes, the school personnel at control sites often made VR referrals but without any follow-up. As shown in Table 3, a steady decrease occurred in student connection and/or referral to VR, especially the last year.

Table 3
Percent of Students Connected^* to Agencies

2014-2015 2015-2016 2016-2017 2017-2018

STW Control STW Control STW Control STW Control

Case Manager 6 0 6 0 1 4 2 0

Day Program 8 0 6 0 2 0 4 0

Developmental Disability 5 8 6 6 2 4 5 0

Employment Provider 67 0 65 0 57 0 45 0

Family &Children 1 8 3 0 0 4 2 0

Juvenile Justice 8 3 0 7 1 0 2 0

Mental Health Centers 4 0 17 19 1 0 3 0

Vocational Rehabilitation 90 15 88 0 76 0 53 0

Workforce Development 0 2 0 13 0 0 0 0

	2014-2015	2015-2016	2016-2017	2017-2018
Case Manager	6	0	6	0	1	4	2
Day Program	8	0	6	0	2	0	4
Developmental Disability	5	8	6	6	2	4	5
Employment Provider	67	0	65	0	57	0	45
Family &Children	1	8	3	0	0	4	2
Juvenile Justice	8	3	0	7	1	0	2
Mental Health Centers	4	0	17	19	1	0	3
Vocational Rehabilitation	90	15	88	0	76	0	53
Workforce Development	0	2	0	13	0	0	0

^*= students can be involved or connected to more than one agency.

3.3. Policy and/or procedure changes impacting local practice

3.3.1 Statewide implementation of self-determination training

Each of the STW sites were trained on various evidence-based curricula. Teachers came together each summer to identify lessons learned, share strategies and resources, and develop a scope and sequence. In the third year of implementation, the IDOE funded a statewide self-determination training for teachers impacting the results of the sites. Results of the pre-and post-tests given to students for both STW sites and control sites showed no difference; however, STW sites were more structured and used more evidence-based curricula and material.

3.3.2 Federal and state policy changes

Three key federal and state policies that impacted procedures and local practice were implemented during this study. First, as the Workforce Innovation and Opportunity Act of 2014 was passed and regulations and guidance developed, Indiana implemented a new employment service model with a different rate structure during the first year of implementation. The impact of this new system benefited transition-age youth by funding work experiences within the student’s last year. This allowed for an early development of a sustainability plan, and the data influenced how Pre-Employment Transition Services (Pre-ETS) would be designed and implemented statewide. Second, as Pre-ETS began, Indiana VR used the STW model where a Career Coach would be embedded in the schools, and providers would have to work together as designed within the Collaborative. Pre-ETS was included as part of the sustainability plan with its services and funding. Finally, at the beginning of the last year of implementation, Indiana VR executed an Order of Selection due to the lack of sufficient funds and lack of staff to serve all eligible individuals.

3.3.3 Vocational Rehabilitation procedural changes

Because VR only paid for work experiences in the students’ last year, enrollment for VR services usually began no later than the end of first semester or beginning of second semester of the students’ junior year. This allowed time to gather all needed collateral information and to attend VR meetings, and for the student and family to choose the employment provider. Therefore, some students began work experiences or internships the summer prior to their senior or last year.

3.4. What families told us

Through an online survey and structured interviewing process (Brantlinger, Jimenex, Klinger, Pugach & Richardson, 2005), parent representatives serving on the Collaborative and/or families of students receiving services provided feedback about their experience to inform policies and procedures. Key findings included: families learned more about the transition process through school personnel (e.g., teacher of record); the Career Coach was the predominate source from whom they learned about VR and received help to understand how to apply for services; the Career Coach was the central person for communication to keep things moving; families appreciated the earlier agency connections; families still struggle with how schools, VR and employment providers all work together and/or have the same goal; and families expressed frustration over the discontinuity of services due to staff turnover (e.g., the starts and stops).

4. Discussion

4.1. Limitations and implications for future research

The STW Collaborative model described here shows an authentic stakeholder engagement, design, and ownership while applying it across geographically and demographically diverse school systems and student populations. Several limitations are worth noting. First, the study was only able to follow employment outcomes for three months due to limited time and resources. Second, each STW site implemented each component of the model and received technical assistance. The fidelity of the implementation across sites varied based on staff and collaborating entities. Specifically, the self-determination curricula implementation had to fit into the existing school structure (e.g., block schedules, resource room, study skills). All required adapting content and lesson planning. Third, a larger percentage of STW site students were connected to agencies, specifically VR and employment providers. The prospects of eventual employment obtainment were favorable, but unknown, especially if a student went on to postsecondary education. More detailed VR closure data on these students may provide more information. Fourth, some of the challenges and barriers addressed below need further exploration to truly examine an STW model. Finally, future research is needed to examine the various components of the model individually versus as an intervention package.

4.2. Challenges, barriers and solutions

A number of challenges emerged through the implementation of the STW model that were unplanned from the onset of implementation. These challenges or barriers include: staff turnover, lack of quality work experiences, and the impact of policy changes on the local level.

4.2.1 Staff turnover

Workforce issues have plagued a number of businesses. The impact has been observed in both schools for recruiting teachers as well as community agencies hiring direct support professionals. What we did not expect was the depth and breadth of the issue. During the four years of implementation, the staff turnover included: 19 VR counselors, 16 Career Coaches, eight special education teachers, three special education department chairs, and two Special Education Directors. We do not know how many employment specialists or job coaches turned over from the employment providers supporting work experiences and/or internships.

The “start and stop” of services impacted primarily students and families, as well as the school and agency partnership as shown in Table 3 over the years. The loss of the momentum of services reduced the trust and complicated communication, especially for students and families. Families also expressed frustration with the lack of transfer of information or documentation, resulting in an expressed feeling that they were starting over with each new staff.

While students were waiting for services, school personnel with Career Coaches developed strategies to keep students engaged by having business or company tours, conducting transportation training, working with VR to choose an employment provider through speed dating formats, arranging mock interviews, and supporting students to participate in a fashion show (what to wear and not wear during a job interview) at a job fair in collaboration with cosmetology students, local thrift stores and employers.

4.2.2 Quality work experiences and internships

Many work experiences or internships allowed students to explore occupations that impacted their decision to enter directly into the workforce, attend postsecondary education, or enroll in an apprenticeship program. However, a lack of diversity of work experiences existed. Many of the work experiences were very basic for a student’s skill set and/or others seemed to be the same type of experience for all students.

Students who were working in traditional “first jobs” such as fast food or grocery stores were challenged to find time between school and work to explore other options. VR was very flexible ensuring these “first jobs” were part of Discovery process and not the final employment outcome. Partnerships were developed with Career Tech programs and Work Ones (Workforce One Stop Centers) to enhance coordination with businesses and opportunities for employment. Additional targeted professional development allowed for staff to look beyond the basic experiences.

For the control sites, a single person was not designated to be responsible for ensuring follow-up to agency referrals. Unless school personnel had contact with the student, no information was available to determine outcomes of the experiences and/or jobs.

4.2.3 Impact of policy changes

As new policies were implemented throughout the project (e.g., WIOA, new VR employment model, Pre-ETS, Order of Selection), local staff became confused and somewhat paralyzed. Some of these policy changes attributed to the high staff turnover of VR counselors and Career Coaches. Others just waited for guidance and direction. As a result, communication and clarification were more labor intensive and ongoing between the state agencies and local sites. Table 3 shows the decrease in STW students accessing VR and other services as a result of both the policy changes and staff turnover.

The implementation of Pre-ETS was integral to the sustainability of the STW model. Yet, it was viewed differently by each school district with some schools welcoming the services and others rejecting services they viewed as a duplication of their established programs. The implementation of VR Order of Selection seemed to cause the most confusion. Indiana VR had decided that they would only serve students who were eligible as Most Severe Disability (MSD). Because this model was serving students with high incidence disabilities and those directly entering the workforce, it was unclear to some VRC if the STW students could be served. VR utilized ongoing feedback to modify procedures to ensure students had access to services. Regardless of the specific policy change, the Collaboratives’ experience demonstrated the necessity of thorough communication and planning prior to implementing any changes to manage the process and expectations. While these changes were confusing to state agency staff, it was even more challenging for students and families to understand. One STW site even created a flow-chart of roles, responsibilities and timelines to share with students, families, teachers and agencies to help with clarification and expectations.

4.3. Implications for practice

Using the collective impact conditions and approach, we learned that the cross-sector perspectives from various stakeholders (e.g., students, families, community providers, state agencies, advocates), created ownership and buy-in of the model, established a collaborative culture for reviewing local transition data and sharing of ideas, problem-solving, and developing solutions at the local and state-level to inform policy and procedure changes. A number of areas emerged from this study that have implications for practice.

4.3.1 Authentic engagement

Key stakeholders designed the model, and the state transition policy workgroup with the local STW Collaboratives took ownership for this model by implementing the components, reviewing the data regularly, obtaining feedback on procedures, and using the data to inform policies and practices. We found: teachers sharing materials, lessons and activities; Career Coaches sharing strategies for student engagement and ideas for working with schools; and VR becoming flexible in their procedures (taking referrals earlier, changing procedures to ensure student access, and ensuring the model was fiscally viable for employment providers). VR used the collaborative approach as they implemented Pre-ETS statewide. Using the collective impact approach (see Table 1), this model can be adapted by other states and local teams by involving key stakeholders with a shared vision of improving interagency collaboration by embedding employment resources (e.g., Career Coach) from an employment provider in schools to enhance a seamless system of transition, improve communication, and improve agency connections and employment outcomes.

4.3.2 Planning for turnover

People leaving positions is not new to the industry. However, due to the lack of an adequate workforce, how we plan for and manage the expectations of students and families must be addressed at the onset of the process. Because the Collaboratives involved a variety of local stakeholder members, more than one member had information about and for the student (e.g., school, VR, employment provider). Identifying activities when the turnover occurs helps ensure the student to remain engaged. As mentioned above, other members stepped up and provided business tours, transportation training, and other work-related activities.

4.3.3 Empowering students and families with their information

As a result of being taught self-determination skills, students led their Transition IEP meetings, learned to become more self-determined by choosing their employment providers, and advocated on their behalf during meetings and in the employment process. Because of staff turnover, the need for good documentation is important so students and families do not have to “tell their story” numerous times. Families need to know the questions to ask to obtain copies of information and understand the process. Building trust and ongoing communication is key to any relationship. New staff members have to understand what has occurred, know the status of the process, and keep the process moving for students.

4.3.4 Ongoing professional development

We provided technical assistance and training to the Career Coaches and helped with onboarding new staff. We worked closely with adult agency providers on communicating with students (e.g., texting, appropriate communication); working with schools to understand the school culture (e.g., not requesting to see students during content area courses, learning the school’s protocol for security, observing communication approaches); and teaching school personnel how to use evidence-based self-determination curricula. Ongoing and future needs for professional development emerged in the areas of working with businesses, developing varied work experiences, and using the Discovery process to inform work experiences and the job development process, especially with the high turnover rate of staff essential to the success of the intervention. Both the Office of Special Education, IDOE and VR committed funding for professional development activities to enhance skill sets of local practitioners.

4.3.5 New policy changes

Within the four years of the project, three policy changes were implemented that impacted local implementation. As we observed, some of the local sites became paralyzed. Planning for policy or procedural changes is needed with guidance, communication, and training to all stakeholders. Policy changes at the federal and state levels must also allow time for understanding the implications, logistics, and intended and unintended consequences.

5. Conclusion

We believed involving stakeholders from the onset and embedding a Career Coach from an adult service provider in schools would assist students and families in making more connections and enhance the seamless system of transition. The model and results have potential to be applied across geographically and demographically diverse communities.

Conflict of interest

The authors declare they have no conflict of interest.

Footnotes

Acknowledgments

The Indiana School-to-Work Collaborative was funded by a grant from the National Institute on Disability, Independent Living, Rehabilitation Research, Disability and Rehabilitation Research Programs (DRRP). Grant number #H133A130028. A special thank you goes to Theresa Koleszar, Director, Indiana Vocational Rehabilitation and Nancy Zemaitis, Assistant Director of Special Education, Indiana Department of Education, for their ongoing support and flexibility to improve transition outcomes.

References

Brantlinger, E., Jimenez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative studies in special education. Exceptional Children, 71, 195–207.

Callahan, M., & Condon, E. (2007). Discovery: The Foundation of Job Development. In

Griffin ,

Hammis , &

Geary . The job developer’s handbook: Practical tactics for customized employment. Baltimore, MD: Paul H. Brookes Publishing Co.

Certo, N. J., Mautz, D., Pumpian, I., Sax, C., Smalley, K., Wade, H. A., & Noyes, D. (2003). Review and discussion of a model for seamless transition to adulthood. Education and Training in Developmental Disabilities, 38, 3–17.

Certo, N., & Luecking, R. (2006). Service integration and school to work transition: Customized employment as an outcome for youth with significant disabilities. Journal of Applied Rehabilitation Counseling, 37, 29–35.

Condon, E. (2008). Steps to Discovery. Retrieved from http://ruralinstitute.umt.edu/transition/Discovery.asp

Fabian, E., Simonsen, M., Deschamps, A., Dong, S., & Luecking, D. (2016). Service system collaboration in transition: An empirical exploration of its effects on rehabilitation outcomes for students with disabilities. Journal of Rehabilitation, 82(3), 3–10.

Fleming, A., Del Valle, R., Kim, M., & Leahy, M. J. (2012). Best practice models of effective vocational rehabilitation services delivery in the public rehabilitation program: A review and synthesis of the empirical literature. Rehabilitation Counseling Bulletin, 56(3), 146–159.

Flowers, C., Test, D. W., Povenmire-Kirk, T. C., Diegelmann, K. M., Bunch-Crump, K. R., Kemp-Inman, A., & Goodnight, C. I. (2018). A demonstration model of interagency collaboration for students with disabilities: A multilevel approach. The Journal of Special Education, 51(4), 211–221.

Hanleybrown, F., Kania, J., & Kramer, M. (2012). Channeling change: Making collective impact work. Stanford Social Innovation Review, 1–8.

10.

Individuals with Disabilities Education Act of 1990, 20 E.S.C. §1400 et seq. (1990) (amended 1997).

11.

Individuals with Disabilities Improvement Act of 2004. 20 E.S.C. §1400 et seq. (2004).

12.

Kania, J., Hanelybrown, F., & Splansky Juster, J. (2014). Essential mindset shifts for collective impact. Stanford Social Innovation Review, 1–5.

13.

Kania, J., & Kramer, M. (2011). Collective impact. Stanford Social Innovation Review, 35–41.

14.

Landmark, L. J., Ju, S., & Zhang, D. (2010). Substantiated best practices in transition. Fifteen plus years later. Career Development and Transition for Exceptional Individuals, 33, 165–176.

15.

Luecking

, & Luecking, R. (2015). Translating research into a seamless transition model. (2015). Career Development and Transition for Exceptional Individuals, 38(1), 4–13.

16.

Mazzotti, V. L., Rowe, D. A., Cameto, R., Test, D. W., & Morningstar, M. E. (2013). Identifying and promoting evidence-based practices and predictor success. A position paper of the Division of Career Development and Transition. Career Development and Transition for Exceptional Individuals, 36, 140–151.

17.

Mazzotti, V. L., Test, D. W., & Wood, C. L. (2012). Effects of multimedia goal setting instruction on students’ knowledge of the self-determined learning model of instruction and disruptive behavior. Journal of Positive Behavior Interventions, 15, 90–102.

18.

Noonan, P. M., Morningstar, M. E., & Gaumer Erickson, A. (2008). Improving interagency collaboration. Effective strategies used by high-performing local districts and communities. Career Development Exceptional Individuals, 35, 143–154.

19.

Noonan, P. M., McCall, Z. A., Zheng, C., & Gaumer Erickson, A. S. (2012). An analysis of collaboration in a state-level interagency transition team. Career Development Exceptional Individuals, 31, 132–143.

20.

Patton, M. Q. (2015). Qualitative research and evaluation methods (4th Ed.). Thousand Oaks, CA: Sage Publications.

21.

Plotner, A. J., Mazzotti, V. L., Rose, C. A., & Teasley, K. (2018). Perceptions of interagency collaboration: Relationships between secondary transition roles, communication and collaboration. Remedial and Special Education, 1–12.

22.

Povenmire-Kirk, T., Diegelmann, K., Crump, K., Schnorr, C., Test, D., Flowers, C., & Aspel, N. (2015). Implementing CIRCLES: A new model for interagency collaboration in transition planning. Journal of Vocational Rehabilitation, 42, 51–65.

23.

Rehabilitation Act 1973 amendments and reauthorizations (1992, 1997), 29, U.S.C.794.

24.

Rowe, D. A., Alverson, C. Y., Unruh, D. K., Fowler, C. H., Kellems, R, & Test, D. W. (2014). A Delphi study to operationalize evidence-based predictors in secondary transition. Career Development and Transition for Exceptional Individuals, 38, 113–126.

25.

Test, D. W., Mazzotti, V. L., Mustian, A. L., Fowler, C. H., Kortering, L., & Kohler, P. (2009). Evidence-based secondary transition predictors for improving postschool outcomes for students with disabilities. Career Development for Exceptional Individuals, 32, 160–181.

26.

Trach, J. S. (2012). Degree of collaboration for successful transition outcomes. Journal of Rehabilitation, 78(2) 39–48.

27.

Wehmeyer, M., Lawrence, M., Garner, N., Soukup, N., & Palmer, S. (2004). Whose Future is it Anyway? A student-directed transition planning process. Retrieved from http://www.ou.edu/education/centers-and-partnerships/zarrow/transition-educationmaterials/whos-future-is-it-anyway.

28.

Wehmeyer, M. L., Shogren, K. A., Palmer, S. B., Williams-Diehm, K. L., Little, T. D., & Boulton, A. (2013). The impact of the self-determined learning model of instruction on student self-determination. Exceptional Children, 78, 135–153.

29.

Workforce Innovation and Opportunities Act (2014).