The essential characteristics of successful organizational transformation: Findings from a Delphi panel of experts

Abstract

BACKGROUND:

Federal legislation has called for the phasing out of sheltered workshops and the transition to integrated employment, causing providers to struggle with how to adapt their model towards providing community integration services.

OBJECTIVE:

Our purpose was to identify the essential characteristics of successful organizational transformation for providers serving individuals with intellectual and developmental disabilities.

METHODS:

A Delphi panel consisting of 36 experts in the field of organizational transformation underwent an iterative process to respond to previously identified characteristics of successful organizational transformation, develop new characteristics, and then rank the final characteristics in order of importance.

RESULTS:

The identified essential characteristics to successful organizational transformation in ranked order were: clear and consistent goals; an agency culture that values inclusion; an active, person-centered job placement process; a strong internal and external communication plan; reallocated and restructured resources; an ongoing investment in professional staff development; a focus on customer engagement; effective employment performance measurement, quality assurance, and program oversight; a holistic approach; and multiple and diverse community partnerships.

CONCLUSIONS:

The Delphi panel confirmed the six characteristics identified in previous research, and added four new characteristics that reflected recent changes in the field of employment and the understanding of what creates lasting organizational change.

Keywords

Organizational transformation intellectual and developmental disabilities Delphi integrated employment

1 Introduction

Recent legislation and regulation governing Medicaid Home and Community Based Services (HCBS), the Workforce Innovation and Opportunity Act of 2014 (WIOA), and settlement agreements between states and the Department of Justice have clarified federal intent to support individuals with intellectual and developmental disabilities to have meaningful employment in their communities. HCBS guidance in 2011 and in the 2015 1915(c) Technical Guide make it clear that individual competitive employment is the preferred outcome of employment-related supports, defining it as “paid employment at or above the minimum wage in an integrated setting in the general workforce, in a job that meets personal and career goals” (Centers for Medicare and Medicaid Services, 2015; 2011). HCBS rules governing community settings were issued in 2014, and support “full access of individuals receiving Medicaid HCBS to the greater community, including opportunities to seek employment and work in competitive integrated settings” (Center for Medicare and Medicaid Services, 2014, p. 249).

In addition to the HCBS guidance, WIOA defines competitive integrated employment as full-time or part-time work at minimum wage or higher, with wages and benefits similar to those without disabilities performing the same work, and fully integrated with coworkers without disabilities, and establishes it as the optimum outcome. The legislation dramatically expands the role of state vocational rehabilitation (VR) services in supporting transition-age youth, and places new restrictions on the use of sub-minimum wage under Section 511. The new section requires as of July 2016a series of steps before an individual can be placed in a job paying less than minimum wage, prohibits schools from contracting with subminimum wage providers, and requires that all subminimum wage recipients receive annual employment counseling from the designated state unit, typically the state VR agency.

Finally, in recent years the Department of Justice has initiated legal actions in states related to access to integrated employment. Settlement agreements with Rhode Island in 2014 and Oregon in 2015 have extended enforcement of the Americans with Disabilities Act and the Olmstead decision regarding access to integrated community employment supports. Both settlements require that the state take action to ensure that employment is offered as a priority outcome, and that both participation in integrated employment and the quality of employment outcomes be improved.

One result of these settlements, regulations, and legislation is the ongoing phasing out of sheltered workshops and the transition to integrated employment and day services. While this has opened up community employment opportunities for thousands of people, providers have faced challenges in adapting their model towards providing effective community integration services (Butterworth, Fesko, & Ma, 2000; Rogan & Rinne, 2011). As more states come under investigation for potential violations of regulations that emphasize community integration, the need for providers to create an organizational transformation plan is greater than ever.

1.1 Sheltered workshop conversion

Community providers are the primary source of employment support for individuals with intellectual and developmental disabilities (IDD) in the United States, with over 8000 providers nationwide (Butterworth et al., 2016). The majority (over 70%) of those served by these providers are people with IDD, and over two thirds of providers offer both work and non-work services (Domin & Butterworth, 2012). Historically, the vast majority of providers predominantly offered sheltered or facility-based employment services with limited community-based or integrated employment options.

Recent national and state-level policy changes, along with a growing dissatisfaction with segregated work and non-work services among individuals with IDD and their family members (Migliore, Mank, Grossi, & Rogan, 2007), have created an imperative for organizations to change their organizational structures and service delivery models from primarily sheltered work to community-based work. While certain providers have successfully transformed their services (Brooks-Lane, Hutcheson, & Revell, 2005; Brown, Shiraga, & Kessler, 2006), many have either not begun, or have struggled to do so, despite growing demand for integrated work opportunities (Martinez, 2013).

1.2 Barriers to change

For many providers, the organizational change process is a challenge. Beyond societal barriers such as low expectations and the belief that people must be “job ready” before integrated employment, there are issues surrounding funding responsibilities, transportation, confusing definitions of employment models, and lack of training on understanding the business world (Rosenthal et al., 2012). Rogan and Rinne state that “moving to integrated community services necessitates a complete rethinking of mission, vision, values, and practices” (Rogan & Rinne, 2011, p. 250). At the same time, many organizations face myriad external and internal barriers to change, and often lack the strategic planning needed to complete the process successfully.

Research also suggests continued service and philosophical variation within the provider community, making the creation of a unified vision for service delivery difficult (Office of Disability Employment Policy, 2014). Inge et al. (2009) found that almost 89% of respondents to a national survey of provider administrators believe that facility-based programs are essential for individuals with disabilities who are having difficulty getting or maintaining real work in the labor force, and only 47% had a formal plan to expand integrated employment. Front-line staff also experience confusion about job development responsibilities, do not feel prepared to engage the mainstream business community, and have little training in providing appropriate supports to individuals with IDD in community settings (Migliore, Butterworth, Nord, & Gelb, 2011; Rosenthal et al., 2012; West & Patton, 2010).

The challenge to successful organizational transformation most cited was financial (Office of Disability Employment Policy, 2014; Rosenthal et al., 2012; West & Patton, 2010). Whether it was securing stable long-term funding for individuals, or balancing contractual service hours with being an employment service, separating from Medicaid systems left some providers exhausted from having to hunt for the same funds that once came so easily in the sheltered workshops (Butterworth, Fesko, & Ma, 2000; Rogan & Rinne, 2011).

Lack of planning, leadership, and communication was another major barrier faced by providers, as agency leaders had limited experience with organizational transformation and now had to lead a staff scattered all over the community. Resistance was also met from stakeholders, including family members, regarding the change process (Rogan, Held, & Rinne, 2001; Rogan & Rinne, 2011). In fact, families were found to be the stakeholder group most resistant to transformation (West, Revell, & Wehman, 1998). Rogan and Rinne (2011) found that families worried about people being out in the community rather than in one place, and that shutting down their building caused some family members to believe the provider was going out of business.

Lastly, several providers reported difficulties placing individuals with intellectual and developmental disabilities in community employment, particularly those individuals who had high support needs (Butterworth, Fesko, & Ma, 2000; Rogan & Rinne, 2011).

In the IDD system, national estimates suggest that there has been only modest growth in the number of individuals in integrated employment services since 1988 (Butterworth et al., 2016). At the same time, participation in facility-based and non-work services has grown steadily, suggesting that employment services continue to be viewed as an add-on service by providers (Butterworth et al., 2016; Domin & Butterworth, 2012; Mank, Cioffi, & Yovanoff, 2003; Nord et al., 2016). Murphy, Easterbrook, Bendetson, and Lieberman (2014) argue that, notwithstanding the widespread awareness of the value of integrated work in individual development, many organizations continue to allocate resources to program services focused on community outings and socialization rather than employment.

1.3 Strategies towards success

In 2002, the Institute for Community Inclusion and Virginia Commonwealth University collaborated to launch the Training and Technical Assistance for Providers (T-TAP) project (Butterworth, Gandolfo, Revell, & Inge, 2007). The goal of the T-TAP project was to assist community rehabilitation providers (CRPs) in facilitating integrated employment/customized employment outcomes Over the five-year course of the project, intensive work with 14 CRPs and 10 mentor organizations resulted in the identification of six characteristics of successful organizational transformation. These characteristics were:

Setting clear and uncompromising goals

Reallocating and restructuring resources

Facilitating rapid job placement, one person at a time

Developing partnerships

Considering the whole person

Communicating expectations to everyone, often

These characteristics are supported by recent literature on organizational change. Forward-thinking agencies emphasize maintaining a culture with a shared philosophical belief that integrated employment should be the preferred outcome, and that opportunities for employment should be available to everyone interested in working (Boeltzig-Brown, 2017; Institute for Community Inclusion, 2016). An investment in staff training with an emphasis on job development and other effective practices including customized employment can prove fruitful as individuals move out of sheltered workshops (Citron et al., 2008; Harvey, Henderson, & Wilson, 2016; Migliore et al., 2011). While families may resist engagement out of fear for the safety of their family member, they are often against sheltering their family member once having experienced the change to community employment (Dague, 2012). Finally, advice from providers who have successfully transformed away from sheltered settings explains the critical need for a combination of multi-level commitment, a comprehensive strategic plan to guide implementation, and the engagement of the full range of stakeholders (Lulinski, Timmons, & LeBlois, 2017).

This article will present the findings from an iterative Delphi panel consisting of experts in the organizational transformation process. Delphi members convened to evaluate the utility of the six previously identified T-TAP elements, as well as to identify any potential new characteristics that could further facilitate the transformation process. While the field is beginning to understand what is necessary to encourage a smooth transformation, providers often lack a road map for implementation. Through a ranking of the most important elements by the Delphi panel, the current research attempts to fill that gap.

The paper answers the following research questions:

What are the reactions of the Delphi panel to ICI’s six original characteristics identified in 2007 (clear goals, resource restructuring, efficient job placement, developing partnerships, considering the whole person, and consistent communication of expectations)?

What additional organizational elements are necessary, and how should they be defined?

Which elements are most important to the organizational transformation process?

2 Methods

The following methods section explains a) the research design, b) recruitment methods, c) the description of the sample, d) data collection techniques, and e) data analysis techniques employed.

2.1 Research design

The Delphi method is a “social research technique whose aim is to obtain the most reliable group consensus of a group of experts” (Linstone & Turoff, 1975 p.10). This research technique “permits researchers to combine the reports or testimony of a group of experts into one, useful statement” (Stitt-Gohdes & Crews, 2004, p. 56). It differs from other qualitative group-based research methods in four distinct ways: (1) the process ensures anonymity for all respondents; (2) the interview process is iterative, which provides opportunity for continuous and controlled feedback; (3) the method allows researchers to capture data that is statistically interpretable; (4) the possibility of using email or online surveys as a means to communicate and gather information allows for participants to be geographically distributed (Lindqvist & Närdanger, 2007). Because the Delphi method does not rely on creating a physical group setting for research, theprocess can involve more individuals than can effectively interact in a face-to-face group setting, which allows for an increased sample size and heterogeneity in the respondent group.

Another distinct advantage of the Delphi research method is that it minimizes the more undesirable aspects of group interaction. Other qualitative research methods that use group responses, such as focus groups and observational research, create the potential for individual respondents to be impacted by social pressures such as majority opinion, forceful persuasion, and a desire to stand by a publicly expressed opinion. The Delphi method allows for individual responses to be made without these pressures, which allows for higher quality responses from all participants. Direct debate that may take place in other forms of group process is replaced by a carefully crafted process of continual individual interview, along with feedback and synthesis of responses (Rowe & Wright, 1999).

2.2 Participants and recruitment methods

The research team implemented a variety of recruitment methods to establish a broad, multi-stakeholder group of experts in the field of organizational transformation. The researchers began by contacting individuals within their professional networks with experience in organizational transformation. Advisory board panel members of the Rehabilitation Research and Training Center on Advancing Employment for Individuals with Intellectual and Developmental Disabilities (www.thinkwork.org/rrtc) were invited to participate in the study, as many of them have decades of experience in provider-level conversion efforts. The team also identified a number of other published academics in the subject of organizational transformation based on their own knowledge of the field and on available literature. ICI’s internal technical assistance professionals were also consulted to identify direct support professionals to participate on the panel. In addition to their participation, the panel members were asked to nominate other academics and practitioners who had experience in organizational transformation.

Table 1
Demographic characteristics of the Delphi panel

% n

Gender

Male 33% 12

Female 61% 22

Not Answered 6% 2

Total 100% 36

Race

White 86 31

Black or African American 6% 2

Asian 0% 0

Other 0% 0

Not Answered 6% 2

Total 100% 36

Ethnicity

Hispanic or Latino 0% 0

Not Hispanic, Not Latino 100% 36

Total 100% 36

Highest Degree Obtained

High School or Equivalent 17% 6

College (2 or 4 year) 14% 5

Masters 44% 16

Doctorate 22% 8

Not Answered 3% 1

Total 100% 36

Age

21–30 8% 3

31–40 8% 3

41–50 6% 2

51–60 33% 12

61–70 36% 13

70+ 6% 2

Not Answered 3% 1

Total 100% 36

Geographic Distribution

New England (CT, ME, MA, NH, RI, VT) 24% 8

Midwest (IL, IN, MI, MN, OH, WI, MN, IO, MO, ND, SD, NE, KS, CO) 38% 13

South (AL, FL, GA, KY, MS, NC, SC, TN, AK, OK, LA, TX) 29% 9

Southwest (AZ, CA, HI, NV, UT, NM) 9% 3

Northwest (AK, ID, OR, WA, MT, WY) 18% 6

Not Answered 3% 1

	%	n
Gender
Male	33%	12
Female	61%	22
Not Answered	6%	2
Total	100%	36
Race
White	86	31
Black or African American	6%	2
Asian	0%	0
Other	0%	0
Not Answered	6%	2
Total	100%	36
Ethnicity
Hispanic or Latino	0%	0
Not Hispanic, Not Latino	100%	36
Total	100%	36
Highest Degree Obtained
High School or Equivalent	17%	6
College (2 or 4 year)	14%	5
Masters	44%	16
Doctorate	22%	8
Not Answered	3%	1
Total	100%	36
Age
21–30	8%	3
31–40	8%	3
41–50	6%	2
51–60	33%	12
61–70	36%	13
70+	6%	2
Not Answered	3%	1
Total	100%	36
Geographic Distribution
New England (CT, ME, MA, NH, RI, VT)	24%	8
Midwest (IL, IN, MI, MN, OH, WI, MN, IO, MO, ND, SD, NE, KS, CO)	38%	13
South (AL, FL, GA, KY, MS, NC, SC, TN, AK, OK, LA, TX)	29%	9
Southwest (AZ, CA, HI, NV, UT, NM)	9%	3
Northwest (AK, ID, OR, WA, MT, WY)	18%	6
Not Answered	3%	1

The recruitment process for the self-advocate respondents and family members was done through an intermediary organization, Self-Advocates Becoming Empowered (SABE, www.sabeusa.org). SABE is a national network of regional and state-level advocacy organizations, and is a key partner on the RRTC. The researchers asked colleagues within SABE to contact individuals or families of individuals with intellectual and developmental disabilities who had either been served by a provider undergoing transformation or who had taken part in a transformation process.

Once an initial sample of participants was selected, the researchers emailed an invitation to participate with detailed information on the background and objectives of the research, participant expectations, and information about the stipend being offered. Those recipients who agreed to participate were placed in a database. The recruitment process yielded a final list of 44 candidates. Of the 44, 36 individuals responded to both rounds of surveys (described below).

The majority of the Delphi panel was female, Caucasian, between the ages of 51 and 70, and reported having earned a master’s or doctoral degree. Their work was geographically distributed across 44 states, with many working in more than one region of the country (see Table 1). Furthermore, panelists were primarily individuals who are or who had been in leadership or administration roles within provider organizations (50%) or providers of training and technical assistance (41%). The other Delphi panelists were evenly distributed across the range of stakeholder groups, with 13 panelists identifying as part of more than one stakeholder group (see Table 2).

Table 2

Delphi panel stakeholder groups

	%	n
State agency staff (IDD/VR state agency administrator, DD council staff, member of other state-level organization)	15%	5
Community rehabilitation provider leadership/administrator (current or former)	50%	17
Community rehabilitation staff other than direct support professional (see definition in the line below)	6%	2
Direct support professional (Staff who help people with IDD find jobs. Also known as a job developer or an employment consultant.)	18%	6
Provider of training or technical assistance	41%	14
Researcher	18%	6
Family member of a person with an intellectual disability	24%	8
Individual with a disability/self-advocate	12%	4
Other	15%	5

In addition to representing a range of stakeholder groups, the panelists also displayed a vast amount of knowledge and experience in the organizational transformation process. Most had been working in the field for over 20 years and had experience in either participating or leading organizational transformation efforts in their past or present positions (68%). Twenty-six panelists (77%) reported being part of an organizational change process, with 20 panelists (59%) having served in a leadership role during a transformation process and 28 (82%) participating in strategic planning related to organizational change. Finally, 82% considered themselves advocates for integrated and competitive employment for individuals with IDD (see Table 3).

Table 3

Delphi panel organizational transformation experience

	%	n
I’ve been a member of a steering committee/advisory group	71%	24
I’ve participated in strategic planning related to organizational change	82%	28
I have conducted research on the topic of organizational change	27%	9
I am a member of a board of directors	41%	14
I have led an organizational change process	59%	20
I have been part of an organizational change process	77%	26
I am an advocate for integrated and competitive employment for people with IDD	82%	28
Other	9%	3

2.3 Data collection procedures

Data collection began by emailing a link to a SurveyGizmo survey containing the original six T-TAP characteristics. In this survey, panelists were asked to review the six characteristics and add any they identified as missing (Round 1). This resulted in the Delphi panel identifying ten characteristics necessary for successful organizational transformation. After this analysis, panelists were emailed a link to a second SurveyGizmo survey containing the ten characteristics, which they were asked to rank according to their importance (Round 2). The data collection procedure for each round is described in more detail below.

2.3.1 Round 1 survey

Panelists were prompted to review the six T-TAP characteristics and definitions, and then rate each characteristic as “essential” or “not essential” to the organizational transformation process. Participants were then asked whether they would change the original definitions in any way. They were then asked an open-ended question: “What would you change/add/delete in the characteristic’s definition?” Finally, panelists were also allowed to submit any additional characteristics and definitions they felt were essential to the process that the initial six did not capture.

2.3.2 Round 2 survey

The round two survey consisted of presenting the panelists with the list of characteristics and definitions that emerged from round one. The main goals were (1) to give participants a chance to review the final characteristics and definitions that emerged as a result of round one to ensure their accuracy, and (2) to ask panelists to rank the characteristics in their order of importance to organizational transformation, from most essential to least essential. The researchers presented each characteristic and definition on separate pages, which allowed the panelists to review each characteristic independently and thoroughly prior to ranking.

The panelists were given three weeks to submit their responses from the time each survey round was sent. The researchers administered the surveys over the telephone to three of the self-advocates for accessibility purposes. That process involved reading to the self-advocate “plain language” versions of characteristics and definitions that had been developed internally at ICI, clarifying any questions, then noting their responses in SurveyGizmo, along with any comments.

2.4 Data analysis techniques

2.4.1 Round 1 survey

Round one of the survey yielded two separate datasets: (1) revisions and responses to the existing six characteristics, and (2) a set of new characteristics suggested by the panelists. A comprehensive review of the first dataset revealed a list of recommended amendments to the six existing characteristic definitions. The proposed amendments were considered on an individual basis to determine their potential contribution to the characteristics’ definitions. Repeated amendments were noted and later incorporated into the redrafting of the initial six characteristics.

The second dataset was comprised of the panelists’ recommendations for new characteristics to add to the existing list of six. The researchers aggregated responses and created a list of codes based on common themes. The codes were agency culture, communication, financing, performance measurement and evaluation, staff qualifications, stakeholder engagement, and technology. The researchers then used the coding list to categorize the dataset into newly recommended characteristics. These newly-formed characteristics would contain language and definitions drawn from text provided by the panelists. The final product of this synthesis was a list of ten organizational characteristics, four of which were new additions synthesized from the panelists’ recommendations.

This list of ten characteristics was sent back to the panelists via email for verification of researcher interpretation. Panelists were asked if the characteristics and their definitions were accurate and appropriate, and were asked to confirm that the newly-added characteristics were representative of their combined suggestions. During this verification process, only two panelists submitted edits, which were integrated in the final version of characteristics.

2.4.2 Round 2 survey

Once the researchers had a verified list of ten characteristics along with their definitions, round two asked the panelists to rank the characteristics in order of essentialness to organizational change, with one being the most important and ten being the least important. The Round 2 survey was sent out to the 36 panelists who fully completed the Round 1survey. Thirty panelists (83%) completed the Round 2 survey.

A Freidman analysis of the results was conducted using SPSS Statistics 20, and there was statistically significant difference in perceived importance to organizational transformation depending on the characteristic, X²(9) = 71.42, p = 0.000. Mean ranks were used to establish order of importance. The following section presents the ten characteristics essential to successful organizational transformation in their final ranked order.

3 Results

The final ranked characteristics from the Delphi panel were as follows:

1. Clear and consistent goals: This means establishing an explicit commitment to increase integrated and competitive employment. The provider defines goals that are: (1) measurable, (2) flexible to the needs of individuals served, (3) compelling and easy to grasp, (4) directly reflective of the core mission, (5) modifiable, and (6) specific to an established time frame. Data is gathered and analyzed to demonstrate the organization’s commitment to change. There is visible support of the organization’s leadership. This includes strong and authentic guidance from its board of directors, executive officers, and management team. All are committed to organizational change objectives and dedicate robust energy toward achieving these objectives.

2. An agency culture that values inclusion: This belief guides what the agency will do and how it will do it (support people to work in the community) and what it will no longer support (sheltered work/subminimum wage). The agency culture values positive thinking, learning, creativity, innovation, and continuous quality improvement. This culture is transmitted through values-based training, ongoing technical assistance, and mentoring of staff who are encouraged to take calculated risks to support positive outcomes for individuals. Values are demonstrated by providing support to people in their home communities, rather than aggregating services at the agency.

3. An active, person-centered job placement process: Successful community rehabilitation providers are proactive in finding jobs for one person at a time. This “just do it” approach creates momentum and enthusiasm as successful employment outcomes are achieved and celebrated. This includes: (1) Implementing a well-defined, comprehensive career planning and a job placement process; (2) A significant time investment in discovery with an end product of defined standards for a good job match. The match should reflect the individual’s interests, preferences, and support needs; (3) Engaging individuals, families, and others in a team-based approach to support integrated and competitive employment; (4) Being assertive about networking for job leads, and using a placement plan that includes a series of small, doable tasks, due dates, and names of people who are responsible. Job seekers direct or contribute to the direction of the plan, giving them a sense of empowerment and accomplishment.

4. A strong internal and external communications plan: Internal to the organization, intent is best communicated by simple, visible practices and decisions made each day, with the expectations for integrated and competitive employment stated in a clear and authentic way. Expectations can be communicated to an organization’s stakeholders in a variety of ways, including 1) how money is spent, 2) goals-based data tracking, 3) the development of policy initiatives, 4) simple outreach activities such as newsletters, and 5) during initial intake meetings, annual reviews, and other family meetings. These activities can gradually shift attitudes toward integrated employment by highlighting successes and maintaining visibility of employment options. In addition, celebratory actions such as the use of banners, ringing a bell, and creating a “hall of fame” for employment success stories contribute to morale. Developing a communication plan that outlines and tracks practices that recognize each new job and accomplishment can help raise awareness and develop a strong base of support for employment.

External to the organization, successful providers make themselves and their services known in the community. They integrate themselves into business, into schools, and within community networks as a resource. The provider has an active onlinepresence where they list their success in job placement and celebrate community partnerships. Through their logo, marketing materials, website, and especially staff training, the goal of the provider is clearly communicated to the public, and they make themselves accessible to the community.

5. Reallocated and restructured resources: Analyzing, reallocating, and restructuring all dedicated resources to community employment is a central part of the change process. Successful organizations reinvent job positions and expectations to clearly focus on integrated employment outcomes and provide continuing opportunities for staff development. How organizations invest their resources is a direct reflection of their priorities, and it has a significant influence on outcomes. Active and ongoing investment in realigning all fiscal, material, and staff resources puts into place the supports and services needed for successfully increasing integrated, competitive employment. Opportunities to negotiate transitional funding are explored, while simultaneously developing a financial model for the agency post-transformation.

6. An ongoing investment in staff professional development: An engaged and educated workforce is key to providing sustained, high-quality job development and coaching supports. Frequent and ongoing training, continuing education, conference participation, and mentorship opportunities are critical to maintain core competencies and implement best practices. Successful organizations support employees at all levels to meaningfully contribute their ideas and energy to the mission.

7. A focus on customer engagement: Successful organizations engage and partner with stakeholders including self-advocates, families, existing customers, and the business community in an effort to meet both individual and market needs. Thriving organizations identify and solve workforce issues by matching the business need to an individual’s interests and skill set in a way that is mutually beneficial.

8. Effective employment performance measurement, quality assurance, and program oversight: The organizational structure fosters shared accountability across all staff. The agency has a clear framework for implementing administrative, management, and program strategies over defined periods of time to determine the impact. There is an understanding of baseline data and a point from which to chart desired outcomes. The successful provider has technology-enabled systems for tracking data. Data systems not only help internal provider staff communicate, but are also accessible to individuals, families, and other stakeholders to assess the effectiveness of the organization in supporting people to find jobs. Data collection includes outcomes of job seekers such as wages, hours, benefits, occupational classification and any other important information connected to getting people good jobs with a living wage, as well as activities of job development staff. As the saying has it, “What gets measured gets done.”

9. A holistic approach: Planning should consider the whole person with wrap-around life supports as necessary, and should use a career planning process that involves staff, parents, and friends. These stakeholders assist in defining a vision for employment with the individual and involving others in finding and supporting employment. Involvement of stakeholders can help to support variations in the individual’s schedule, transportation, workplace problem-solving, the use of assistive technology, and balancing the individual’s work and non-work activities schedule. Attention must be paid to supporting the maintenance of personal relationships as well as the development of new ones.

10. Multiple and diverse community partnerships: Supporting change requires engaging organizations and state systems and creating buy-in to the change process. Community partnerships should include local businesses, school districts, state agency offices, faith-based organizations, and other community-based organizations such as transportation resources. Partnering with other organizations that provide employment services promotes collaborative job development and innovative idea exchange. Effective partnerships promote actions that improve personal outcomes for those receiving services, foster positive change in the systems influencing these services, and meet the economic needs within local business communities.

4 Discussion

The purpose of this study was to uncover the necessary characteristics for organizational transformation. The Delphi panel agreed that the original six characteristics of successful organizational transformation still applied after nearly ten years (Butterworth et al., 2007). Those six characteristics included 1) having clear and uncompromising goals, 2) reallocating and restructuring resources, 3) focusing on active job placement one person at a time, 4) developing partnerships, 5) considering the whole person, and 6) communicating expectations to everyone, often. The panel suggested four additional characteristics in the areas of staff qualifications, agency culture, stakeholder engagement, and performance measurement and evaluation.

Panel members recommended certain modifications to the original definitions. For example, the definition of “a holistic approach” was expanded to include multiple stakeholders around the individual, as well as technological accommodations. “Developing partnerships” was adjusted to include a wider diversity of partnerships, such as with schools, transportation entities, and faith-based organizations. This represents a growing recognition in the field that different types of collaboration may be necessary to ensure meaningful community employment opportunities for all (U.S. Department of Labor, 2009). While the original six elements held true, the panelists added four new ones: 1) creating an agency culture that values inclusion, 2) maintaining an ongoing investment in staff professional development, 3) focusing on customer engagement, and 4) concentrating on effective employment performance measurement, quality assurance, and program oversight.

The panel’s four additional elements might reflect new priorities in the field, a focus on sustainability of transformation efforts, and ongoing improvement of previous efforts. For example, an “agency culture that values inclusion” represents an organizational commitment to helping job seekers find work in the community, which must be shared by all staff and stakeholders. This may be a necessary addition to the development of consistent goals and for sustainability of that mission. An “ongoing investment in staff professional development” allows the provider to stay competitive through trainings, mentorship, continuing education, and conference participation, promoting the mission and vision all the way down to the front line. A “focus on customer engagement” emphasizes the explicit linking of job seekers with employers, recognizing the employer as a dual customer, as well as the need for the mutual satisfaction of both parties (Migliore, Nye-Lengerman, Jordan, & Butterworth, 2017; Simonsen, Fabian, Buchanan, & Luecking, 2011). Finally, the panelists’ addition of “effective employment performance measurement, quality assurance, and program oversight” reflects the field’s emphasis on collection and use of both outcome data and process data to continually improve (Hall, Butterworth, & Winsor, 2014).

The findings presented the ten essential elements in ranked order of importance. Panel members agreed that all ten characteristics were worthy of inclusion, indicating continuing support for the six T-TAP characteristics as well an understanding that the organizational transformation process needs to reflect advancements and evolution in the field. The rankings provide a roadmap for providers, and can help providers identify priorities when challenged by limited resources.

4.1 Limitations

Literature identifies some drawbacks to the Delphi panel technique. These include 1) the assumption that all panels have the same level of expertise in the topic (which is typically not the case), potentially resulting in more general perspectives as opposed to an in-depth investigation of the topic, and 2) the potential for researchers to mold the opinions of respondents based on the iterative nature of the process (Hsu & Sandford, 2007).

This study also employed a convenience sample consisting mainly of individuals already known to the ICI, many of whom still work closely with the organization. So while the Delphi panel’s experiences with the transformation process were diverse enough to have the ten characteristics be generalizable to a variety of providers, the panel may have been less critical about the characteristics and definitions given their relationship with some members of the ICI staff (the panel did not reject any previously established or newly proposed characteristic).

5 Conclusions and directions for future research

This research provides the foundation for future work around organizational transformation. The research team is analyzing qualitative data from case study research that focuses on the implementation of the ten characteristics presented here. This case study data will add a richness to the description of the ten characteristics, and can help to translate the current findings of the Delphi panel from research to practice. In addition, the research team is developing a practical toolkit and technical assistance package that incorporates these strategies and will be used in an upcoming intervention study that will test the utility of these elements as a framework or roadmap for organizational change.

A growing number of providers are increasingly focused on full community inclusion andachieving improved competitive integrated employment outcomes. Our understanding of the transformation process continues to evolve as agencies close their workshops and support employment and community participation. The findings and updated recommendations from the Delphi panel reflect the changes that are underway in the field of employment for individuals with intellectual and developmental disabilities, and our evolving understanding of what it takes to successfully transform services and to create lasting organizational change.

Conflict of interest

None to report.

Footnotes

Acknowledgments

This research was supported under a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR Grant # 90RT5028-01-00). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents do not necessarily represent the policy of NIDILRR, ACL, HHS, and you should not assume endorsement by the Federal Government.

References

Boeltzig-Brown, H. (2017) KFI: Flexible scheduling and creative staffing. Promising practices in community life engagement series, Guidepost #1. Boston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

Butterworth, J., Gandolfo, C., Revell, W. G., & Inge, K. J. (2007) Community rehabilitation programs and organizational change: A mentor guide to increase customized employment outcomes. Training and Technical Assistance for Providers. Retrieved from www.t-taorg/documents/mentor_guide.pdf.

Butterworth, J., Fesko, S. L., & Ma, V. (2000) Because it was the right thing to do: Changeover from facility based services to community employment. Journal of Vocational Rehabilitation, 14(1), 23–35.

Butterworth, J., Smith, F. A., Winsor, J., Ciulla Timmons

, Migliore, A., & Domin, D. (2016) StateData: The national report on employment services and outcomesBoston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

Butterworth, J., Smith, F., Winsor, J., Migliore, A., Domin, D., Timmons, J., & Hall, A.C. (2015). StateData: The national report on employment services and outcomes. Boston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

Brooks-Lane, N., Hutcheson, S., & Revell, G. (2005). Supporting consumer directed employment outcomes. Journal of Vocational Rehabilitation, 23(2), 123–134.

Brown, L., Shiraga, B., & Kessler, K. (2006). The quest for ordinary lives: The integrated post-school vocational functioning of workers with significant disabilities. Research and Practice for Persons with Severe Disabilities, 31, 93–121.

Center for Medicare and Medicaid Services. (2015). Medicaid Employment: Options and Incentives. 2015 HCBS Conference, September 3, 2015.

Centers for Medicare and Medicaid Services. (2014) Medicaid Program; State Plan Home and Community-Based Services, 5-Year Period for Waivers, Provider Payment Reassignment, and Home and Community-Based Setting Requirements for Community First Choice (Section 1951(k) of the Act) and Home and Community Based Services (HCBS) Waivers (Section 1915(c) of the Act). Retrieved from https://www.medicaid.gov/medicaid-chip-program-information/by-topics/long-term-services-and-supports/home-and-communitybased-services/downloads/final-rule-slides-01292014.pdf

10.

Centers for Medicare and Medicaid Services. (2011). CMCS Informational Bulletin: Updates to the §1915 (c) Waiver Instructions and Technical Guide regarding employment and employment related services Retrieved from http://downloads.cms.gov/cmsgov/archived-downloads/CMCSBulletins/downloads/CIB-9.6.1.pdf.

11.

Citron, T., Brooks-Lane, N., Crandell, D., Brady, K., Cooper, M., & Revell, G. (2008) A revolution in the employment process of individuals with disabilities: Customized employment as the catalyst for system change. Journal of Vocational Rehabilitation, 28, 169–179.

12.

Dague, B. (2012). Sheltered employment, sheltered lives: Family perspectives of conversion to community-based employment, Journal of Vocational Rehabilitation, 37, 1–11.

13.

Domin, D., & Butterworth, J. (2012). The 2010-2011 national survey of community rehabilitation providers. Report1: Overview of services, trends and provider characteristics. Research to Practice Brief. Boston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

14.

Hall, A.C., Butterworth

, & Winsor, J. (2014). Data systems and decision-making: State IDD agencies and their employment data collection systems. Journal of Vocational Rehabilitation, 41(3), 179–191.

15.

Harvey, S., Henderson, L., & Wilson, J. (2016) Guide for provider transformation to an Employment First service model. Retrieved from http://www.tennesseeworks.org/wp-content/uploads/Guide-for-Provider-Transformation-to-an-Employment-First-Service-Model.pdf

16.

Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment Research & Evaluation, 12(10), 1–8.

17.

Inge, K. J., Wehman, P., Revell, G., Erickson, D., Butterworth, J., & Gilmore, D. S. (2009). Survey results from a national survey of community rehabilitation providers holding special wage certificates. Journal of Vocational Rehabilitation, 30(2), 67–85.

18.

Institute for Community Inclusion.(2016). New England Business Associates (MA): Everyone is job-ready. Community Reha bilitation Provider (CRP) Promising Practices. Boston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

19.

Lindqvist, P., & Nordänger, U. K. (2007). (Mis- ?) using the e-delphi method: An attempt to articulate the practical knowledge of teaching. Journal of Research Methods and Methodological Issues, 1(1), 1–13.

20.

Linstone, H. A., & Turoff, M. (1975). The Delphi method: Techniques and applications. London, U.K.: Addison-Wesley Publishing Company.

21.

Lulinski, A., Timmons, J. C., & Leblois, S. (2017). From sheltered work to competitive integrated employment: Lessons from the field. Bringing Employment First to Scale, Issue 8. Boston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

22.

Mank, D., Cioffi, A., & Yovanoff, P. (2003). Supported employment outcomes across a decade: Is there evidence of improvement in the quality of implementation?Mental Retardation, 41(3), 188–197.

23.

Martinez, K. (2013). Integrated employment, Employment First, and U.S. federal policy. Journal of Vocational Rehabilitation, 38, 165–168.

24.

Migliore, A., Butterworth, J., Nord, D., & Gelb, A. (2011) Improving job development through training and mentorship. Research to Practice Brief, Issue No. 51. Boston, MA: University of Massachusetts Boston, Institute for Community Inclusion.

25.

Migliore, A., Mank, D., Grossi, T., & Rogan, P. (2007) Integrated employment or sheltered workshops: Preferences of adults with intellectual disabilities, their families, and staff. Journal of Vocational Rehabilitation, 26(1), 5–19.

26.

Migliore, A., Nye-Lengerman, K., Jordan, M., & Butterworth, B. (2017). Raw data from qualitative interviews with employment consultants. Unpublished raw data.

27.

Murphy, S., Easterbrook, E, Bendetson, S., & Lieberman, S. (2014). TransCen, Inc.’s WorkLink program: A new day for day services. Journal of Vocational Rehabilitation, 40, 125–130.

28.

Nord, D., Butterworth, J., Carlson, D., Grossi, T., Hall, A., & Nye-Lengerman, K. (2016). Employment of people with IDD: What do we know and where are we going? In National goals for research for people with intellectual and developmental disabilities. Washington, DC: American Association on Intellectual and Developmental Disabilities.

29.

Office of Disability Employment Policy.(2014). Evaluation of the Employment First state leadership mentoring program. Unpublished raw data.

30.

Rogan, P., Held, M., & Rinne, S. (2001). A national study of organizational change from facility-based to community-based employment services. The Advance, 12(1), 1–4.

31.

Rogan, P., & Rinne, S. (2011). National call for organizational change from sheltered to integrated employment. Intellectual and Developmental Disabilities, 49(4), 248–260.

32.

Rosenthal, D. A., Hiatt, E. K., Anderson, C. A., Brooks, J., Hartman, E. C., Wilson, M. T., & Fujikawa, M. (2012). Facilitators and barriers to integrated employment: Results of focus group analysis. Journal Of Vocational Rehabilitation, 36(2), 73–86.

33.

Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15(4), 353–375.

34.

Simonsen, M., Fabian, E. S., Buchanan, L., & Luecking, R. G. (2011). Technical report: Strategies used by employment service providers in the job development process: Are they consistent with what employers want? Retrieved from https://www.dol.gov/odep/ietoolkit/publications/500.pdf

35.

Stitt–Gohdes, W. L., & Crews, T. B. (2004). The Delphi technique: A research strategy for career and technical education. Journal of Career and Technical Education, 20(2), 55–67.

36.

United States Department of Labor.(2009). Partnership in action: Examples of employer/faith-based and community organization partnerships. Retrieved from http://digitalcommons.ilr.cornell.edu/key_workplace/589

37.

West, E. A., & Patton, H. A. (2010). Case report: Positive behavior support and supported employment for adults with severe disability. Journal of Vocational Rehabilitation, 35(2), 104–111.

38.

West, M., Revell, G., & Wehman, P. (1998). Conversion from segregated services to supported employment: A continuing challenge to the VR service system. Education and Training in Mental Retardation and Developmental Disabilities, 33(3)239–247.