Successful vocational rehabilitation innovations: Building a better toolbox

1 Why this Special Issue?

This Special Issue of the Journal of Vocational Rehabilitation tells the story of a Rehabilitation Services Administration Model Demonstration entitled the “SGA Project” testing an intervention to improve earnings outcomes of Social Security Disability Insurance (SSDI) beneficiaries who were clients of Kentucky and Minnesota state vocational rehabilitation (VR) agencies. Hereafter, and throughout the papers, we will refer to SSDI beneficiaries receiving VR services as SSDI clients. The title SGA was purposefully chosen to keep the goal clear and to advance earnings outcomes above substantial gainful activity (SGA) which has implications for economic independence. For this project, SGA refers to the monthly earnings amounts as targets for earnings at VR closure rather than as a determination of eligibility for Social Security Administration Benefits. Monthly earnings amounts between 2015 and 2017 (the testing phase) were $1,090 to $1,170 for non-blind SSDI beneficiaries (SSA, 2020a).

Early in the design phase it was clear that some VR agencies were reaching high levels of performance on earnings outcomes even in states that had high unemployment rates. Efforts such as building business relations functions, direct provision of work incentive counseling services, and reducing bureaucratic inefficiencies that slowed down the pace of services were observed in high performing VR agencies. Many of these innovations were written into the Workforce Innovation and Opportunity Act of 2014 and are maturing in VR agencies all over the country. This bodes well for SSDI clients seeking higher earnings.

June 2020 marked the 100th birthday of the public VR program, a program that evolved through multiple eras of economic distress. The SGA Project operated shortly after the Great Recession. VR agencies are innovating again to consider more remote and virtual ways of connecting, expanding dual customer approaches to serve businesses and workers as clients (Moore, Haines, Bradshaw et al., 2018), and looking for more tools to put in a practitioner’s toolbox. What we hope to relay in these papers is that participation in rigorous research may expand options practitioners have to make a difference in the lives of SSDI clients. Participating in a randomized controlled study is a major undertaking. We owe a debt of gratitude to the Kentucky Office of Vocational Rehabilitation (OVR) and the Minnesota Vocational Rehabilitation Services (VRS), their personnel and partners for engaging with us. Our ultimate goals are the same: to improve employment outcomes of clients and to build a better toolbox.

2 The SGA Model demonstration

RSA funded the SGA Project to identify VR agency practices that lead to improved earnings outcomes above substantial gainful activity at VR closure for SSDI clients. They narrowed the population to people by including only those receiving SSDI benefits as an adult, were not receiving Supplemental Security Income (SSI), and were not eligible for SSDI because of blindness. Those that receive SSDI due to blindness receive a different package of services from VR agencies and follow different guidance from SSA. RSA awarded a cooperative agreement to the Institute for Community Inclusion at the University of Massachusetts Boston and Mathematica Policy Research. Phase I activities sought to identify high performing agencies, key practices that might explain that performance and include practitioner and expert opinion in assessment of those practices. The SGA Project worked with more than 100 practitioners representing more than 20 VR agencies to define the intervention. The SGA Project partners analyzed administrative data, hosted Delphi Panels, interviewed VR agency leadership and conducted eight VR agency case studies (Foley, Haines, and Mock, 2020a; Foley, Haines, and Mock, 2020b) to define a rapid coordinated team approach. ICI Senior Policy Fellows and consultants provided intensive technical assistance to support Kentucky and Minnesota to adopt and mold the intervention into their operations, policies, procedures, and practices (Marrone, Thelin, and Mock, 2020b). Mathematica Policy Research assessed earnings outcomes as described in Martin and Sevak (2020) and Kehn and Honeycutt (2020). Taylor and Blackburn (2020) interviewed and followed SSDI clients who participated to understand why people seek work, what influences their choices about earnings levels, and how VR services played a role in their decisions.

The intervention was a rapid coordinated team approach that had very specific pacing measures and required a rehabilitation counselor, financial specialist, and job placement specialist to work together prior to individual plan for employment (IPE) development. All of the elements were derived from practices in operation in at least one VR agency. In many ways, the intervention was a simple and concrete set of steps to modify the rehabilitation process. In application, significant shifts in culture, norms, philosophy, contracting, partnership, operations, eligibility determination, relationships with the Social Security Administration (SSA), data systems, and counselor tasks and skills were required. Once Kentucky OVR and Minnesota VRS agreed to participate, each required eighteen months of start-up and significant funds to proceed. Both participated in a randomized controlled trial, participated in intensive training and technical assistance activities and dedicated senior staff to champion implementation.

3 Why RCTs in VR agencies matter

Historically, once individuals enter the SSDI program, the likelihood is low that they will ever leave because of work and earnings (SSA, 2020b). Though relatively few SSDI beneficiaries leave cash benefits for a job, many have employment goals and engage in employment or job search and preparation activities. Surveys sponsored by SSA indicate that a large share of SSDI-only beneficiaries are interested in returning to work and that this share has grown over time; the most recent data suggest that more than 40 percent of SSDI-only have work goals and expectations (Livermore et al., 2019). At the time the SGA Project was being implemented, the SSDI program was experiencing tremendous growth. Some of the growth was attributed to the 2007–2009 economic recession, but it was also due to other factors, including the aging of the baby boom generation and increased labor force participation by women (Ruffing, 2014). Finding ways to better support the return-to-work efforts of SSDI beneficiaries and increase their SGA-level employment can help them become more independent and successful economically and can also help curb future growth in the SSDI program. Effective work supports, if provided soon after individuals experience employment-related consequences of disability onset, might also prevent them from having to leave the labor force and enter the disability programs.

State VR agencies are the primary source of employment-related services for individuals with significant disabilities. VR agencies serve over a million individuals with disabilities each year (U.S. Department of Education, 2018), and a large share of these individuals are receiving SSA disability benefits. In June 2018, VR agencies were serving over 344,000 SSA beneficiaries (SSA, 2018). Although providing services to many SSA disability beneficiaries, the federal-state VR program has been criticized for its failure to help SSA disability beneficiaries obtain jobs with substantial earnings. The U.S. General Accountability Office (GAO) found that although SSA beneficiaries who received VR services increased their earnings, only a small share of them had earnings that were high enough for them to leave the SSA disability rolls (GAO, 2007a). GAO recommended that the Secretary of Education identify and promote promising VR agency practices that improve the employment of SSA disability beneficiaries (GAO, 2007b).

Identifying a “promising practice” is much easier than proving a practice is actually effective. Descriptive evidence that a practice appears to be effective is usually sufficient to label it as promising. More rigorous evidence is needed to consider a practice as “evidence-based.” The National Technical Assistance Center on Transition (NTACT) funded by the U.S. Department of Education identifies effective transition services for youth with disabilities, including practices used by VR agencies, and assesses the level of evidence that supports each practice’s effectiveness. None of the effective VR practices identified by NTACT, which include practices used with both transition-age youth and adults, have the strongest level of evidence needed to be considered as evidence-based (NTACT, 2018). The National Institute on Disability Independent Living and Rehabilitation Research funded a Vocational Rehabilitation Research and Training Center at ICI to conduct a systematic review of VR research literature. Boeltzig-Brown et al. (2017) reviewed more than 550 studies finding very few experimental studies. Other researchers have confirmed the finding that very few VR practices have been studied using rigorous methods (Fleming et al., 2013; Leahy et al., 2014). Experimental evaluation methods, when implemented appropriately, are considered the most rigorous methods for testing the impact of a service innovation or practice. They involve randomly assigning individuals (or groups of individuals defined by service areas or other criteria) to receive a new service or practice of interest (treatment) or receive either the usual services or no services (control). Because individuals are randomly assigned to the groups, they should be similar with respect to their characteristics and other factors that might affect their outcomes, aside from the treatment. Thus, any differences in their outcomes can be attributed to the treatment.

Ethical concerns about ensuring that all VR clients have access to the same set of services or have informed choice about those services and practical challenges associated with implementing random assignment have likely limited VR agencies’ use of experimental evaluation methods. Participating in experimental designs comes with financial costs including staff training, investment in data systems, contracting modifications, union negotiations, and salaries of dedicated personnel. It also comes with political risks such as appearing to favor a select population, a particular intervention, or a specific geographic area. Despite this, VR leadership demonstrate a strong interest and willingness to engage in research. Connelly and Wooderson (2020) of the Council of State Administrators of Vocational Rehabilitation (CSAVR) report in this Special Issue that VR agencies consider whether participating in research builds the capabilities of VR personnel to fully use the practices available to them. Recent RSA model demonstrations have required VR agencies to be the lead with evaluators as partners. Five VR agencies operate the Work Based Learning Model Demonstrations and four lead the Career Pathways for Individuals with Disabilities Model Demonstrations. These efforts raise the possibility that VR practices will generate a more rigorous level of evidence.

The Kentucky OVR and Minnesota VRS implementing the SGA Project demonstrations described in this Special Issue successfully used an office-level random assignment design, which permitted a rigorous test of the intervention. The office-level random assignment addressed many practical issues associated with implementing an experimental study in the service delivery context. Other experimental or rigorous nonexperimental research designs are also feasible for VR agencies to implement with enough forethought and a strong desire to know if something works. Helping people with disabilities find and maintain employment can be challenging because so many factors influence the outcomes.

To that end, the Special Issue contains two companion articles to further understand who applies for SSDI benefits. Contreary and Honeycutt (2020) use three panels of the Survey of Income and Program Participation data matched to SSA data to help build evidence of which individuals are at risk of applying for SSDI benefits. Stapleton and Martin (2020) use longitudinal statistics to identify who enters SSDI and the timing of that entry relative to VR application. Innovation and experimentation can help VR agencies learn about what works for whom and under what circumstances, and ultimately, make VR services more effective.