Vocational rehabilitation applicants,the services they receive,and their employment outcomes

1.1 Background on vocational rehabilitation program

People with disabilities in the United States who are unemployed, underemployed, or at risk of losing employment may access the Vocational Rehabilitation (VR) program, which is jointly-funded by the federal government and by the states and territories. The process begins when a person with a disability applies, hereby defining them as an applicant and a non-participant. An applicant is accepted if they have a verified disability impeding employment who requires VR services to prepare for, secure, retain, advance in, or regain employment that is consistent with their unique strengths, resources, priorities, concerns, abilities, capabilities, interests, and informed choice ( State Vocational Rehabilitation Program, Assessment for Determining Eligibility and Priority for Services, 2021). If the program lacks resources to serve the eligible applicant, they may be placed on an order of selection (OOS) waiting list (this applies to approximately half the programs). The criteria associated with placement on the list varies from program to program, however people with significant disability are always served with higher priority than people with non-significant disability.

Once accepted (or admitted from the OOS waiting list into the program), the applicant works with their Vocational Rehabilitation Counselor (VRC). Together, they develop an individualized plan for employment (IPE) which includes the employment goal, services needed to attain it, providers of these services (and who is fiscally responsible for them), plan for progress evaluation, and the specific responsibilities of the participant and counselor. Upon signing this first IPE, the non-participant applicant transitions into what we define as a participant. Note, this is different from the United States Department of Education’s Rehabilitation Services Administration’s definition 1 This plan is amended as necessary. The employment goal must be an employment outcome to enter, advance, or retain employment in alignment with the individual’s unique strengths, resources, priorities, concerns, abilities, capabilities, interests, and informed choice. It may be either a type of competitive integrated employment (CIE) or supported employment. CIE is any employment where compensation, job duties, and the environment are the same for the person with disability as they are for people without disability. This includes self-employment, customized employment, telecommuting, and business ownership. Supported employment is for someone who either historically has not had CIE or had CIE on an intermittent basis due to the severity of their disability. Individuals have supported employment if they have CIE that is individualized and customized for their unique strengths, abilities, interests, and informed choice including the provision of ongoing support services (or if they work in an integrated environment, without competitive compensation, for a short-term basis while working towards CIE). It is expected they will need these ongoing services after they transition out of the VR and state-supported employment programs. (State Vocational Rehabilitation Services Program, Applicable Definitions, 2021, paras. 9, 53).

Thirty-one services are available to adult VR participants. These fall into three categories—12 education and training services (e.g., college and university training, technical education, apprenticeship training), 11 career services (e.g., assessments, job placement, short-term job supports), and eight other services (e.g., transportation, rehabilitation technology, maintenance) ( Rehabilitation Services Administration, 2017). Participants are successfully rehabilitated when they maintain employment aligning with their IPE goal for 90 consecutive days concluding with exit from the program. A case could also close if the participant is unable to continue to work towards getting a job. A conceptual timeline of the VR process is presented in Figure 1.

OPEN IN VIEWER

Fig. 1

Conceptual timeline of the VR process with process milestones and achievement time benchmarks. VR has established goals for quality assurance, including a maximum time between application and eligibility (60 days) and between eligibility and development of the initial IPE (90 days). Employment must be maintained for at least 90 days prior to exit and VR services can be provided any time between the IPE and exit.

Previous research has investigated specific diagnoses or impairment groups within the VR population. Investigations have considered race, ethnicity, age, and employment status at the time of application and how they were associated with exiting VR with CIE ( Ahonle et al., 2019; Dutta et al., 2008; Glynn & Schaller, 2017; Lee et al., 2020; Sung et al., 2014; Ufere et al., 2019). However, most of these studies focused on specific diagnoses instead of functional impairments, which limits the generalizability and applicability of findings to others who also participated in the VR system.

For example, Native Americans with mental impairments were half as likely, and African Americans with sensory impairments were 80%as likely to find successful employment than European Americans (Dutta et al., 2008). In one (unspecified) state, people who were employed at application had 130%higher likelihood of exiting with CIE than those who started VR unemployed (Ufere et al., 2019). Nationally, having employment at application found people with mental impairments to be 1.5 times as likely, people with sensory impairments to be 4 times as likely, and people with physical impairments to be nearly 3 times as likely to exit with CIE compared to those who were unemployed at program start (Dutta et al., 2008). Last, higher education levels were shown to be associated with increased CIE rates for people with attention deficit hyperactive disorder (ADHD) (Glynn & Schaller, 2017), traumatic brain injury (TBI) (Ahonle et al., 2019), cerebral palsy (CP) ( Huang et al., 2013), youth with intellectual disabilities (ID) ( Kaya, 2018), and youth with autism spectrum disorder (ASD) ( Kaya et al., 2016).

In addition to investigating demographics and basic personal characteristics, the previous research has also investigated relationships between VR services and CIE at exit, again for specific diagnoses or impairment groups in the population. While there are some services with common associations for many of the diagnoses, the specific services listed for each group vary because each investigation focused on a specific diagnosis or subset of impairments. Their findings imply the efficacy of specific services for each subpopulation. These studies only compare participants receiving the service with participants not receiving the service in the specific diagnosis’ subgroup. They neither capture, nor compare, the outcomes of non-participants in order to more fully understand the VR program outcomes.

For example, job placement, short-term job supports, and job search assistance were associated with increased CIE rates for people with affective disorders (S a ´ nchez, 2018). Whereas, short-term job supports, job placement, and on-the-job training were associated with increased CIE rates for participants with TBI (Ahonle et al., 2019). For the Deaf-Blind, job search assistance, job placement assistance, short-term job supports, and vocational rehabilitation counseling guidance were all associated with increased CIE rates ( McDonnall & Cmar, 2019). Receipt of short-term job supports, job placement, on-the-job training, maintenance services, and rehabilitation technology were associated with increased CIE rates for people with CP (Huang et al., 2013). For people with orthopedic disabilities, job placement, rehabilitation technology, and vocational rehabilitation counseling guidance were associated with increased rates of CIE achievement ( Chan et al., 2006). Job-related services were associated with increased CIE rates for people with epilepsy (Sung et al., 2014). Job placement, physical restoration, vocational rehabilitation counseling guidance, and rehabilitation technology were associated with increased CIE rates among people with spinal cord injuries (SCI) ( Marini et al., 2008). For youth with ID, a combination of short-term job supports, on-the-job training, and maintenance services increased CIE rates (Kaya, 2018). Increased CIE rates for youth with ASD were associated with receipt of short-term job supports, job placement services, vocational rehabilitation counseling guidance, and job readiness training (Kaya et al., 2016).

Program time factors also have been found to be associated with CIE. People with MS, SCI, and cancer survivors who achieved CIE had shorter periods between application and eligibility determination ( Chiu et al., 2013, 2014; Marini et al., 2008). The less time spent receiving VR services was associated with higher odds of achieving CIE for people with MS (Chiu et al., 2013) and cancer survivors (who were unemployed at the time of application) ( Chan et al., 2008). The period between eligibility and exit was shorter for those with SCI who received VR services (Marini et al., 2008). While the generalizability of these findings is limited, they suggest that time in the program may be a valuable program outcome to investigate.

There also is some evidence that public monetary or health benefits may serve as work disincentives to decrease CIE rates, which has also been investigated for specific diagnoses, including ADHD, ASD, epilepsy, learning disabilities (LD), orthopedic impairments, and psychiatric disabilities. The most common benefits investigated have been the association of Social Security Administration (SSA) benefits with CIE (Chan et al., 2006; Ditchman et al., 2018; Kaya, 2018; Kaya et al., 2016; Rumrill et al., 2017). However, some have investigated the effects of broader sets of benefits on CIE rates, such as disability-related medical or cash benefits (Chiu et al., 2013), or a combined pool of public benefits expanded beyond SSA benefits to also include Temporary Assistance for Needy Families (TANF) and State General Assistance (Glynn & Schaller, 2017). These studies indicate a desire to reduce public benefit dependency, motivating understanding how reliance on public benefits changes between application and exit to the program.

1.3 Publicly available VR data –The RSA-911 case service report

The Rehabilitation Services Administration (RSA) collects data to evaluate VR programs via the United States RSA-911 Case Service Report. The recorded variables include individual characteristics (including age, sex, race, ethnicity, and living arrangements), supports (i.e., student supports, public financial benefits, and medical insurance), disability information, employment status, Workforce Innovation Opportunity Act (WIOA) core program involvement, employment barriers, education information, program information (e.g., if an applicant was determined eligible, directly admitted to the VR program, placed on an OOS waiting list, time on the OOS waiting list before starting or exiting the program), program services (e.g., pre-employment transition services, VR training services, VR career services, VR other services), employment outcomes with specific employment data (i.e., job standard occupational classification (SOC) code, weekly hours, and weekly wages), exit outcomes with specific employment data, and post-exit employment data. Periodically, the RSA updates their data collection practices adding new variables, removing some variables, and clarifying the definitions of the variables. A new set of data collection practices went into effect on July 1, 2017.

1.4 Our gap in knowledge

While we have some understanding of how demographic characteristics and VR services relate to CIE achievement for specific diagnoses, it is not generalizable to types of impairments, nor to the VR population as a whole. We do not know how employment outcomes compare between participants and non-participants. We have limited insight to other program outcomes beyond whether or not CIE is achieved at exit. We do not yet have a broad understanding of the entire VR population, the types of services applicants receive, and their impact on program outcomes such as type of employment at exit, the reason for exit, change in source of primary income, and change in reliance on public benefits.

Since the 2017 data collection practices went into effect, to our knowledge nobody has yet characterized the VR population using the new information included in this dataset. Furthermore, reports to the U.S. Congress, such as the Rehabilitation Services Administration Report for Fiscal Year 2016 Report on Federal Activities Under the Rehabilitation Act of 1973, as amended (2019), summarize and provide basic data analysis, but they do not describe detailed personal characteristics of VR applicants (let alone participants and non-participants) related to receipt of public benefits, education level, employment status at program entry, referral sources, and types of residence, all of which might also influence reaching employment goals. These reports do not describe information about the specific receipt of services or program outcomes related to time spent in the program, and changes in primary financial support. They do not discuss any program outcomes of non-participants. These shortcomings make it difficult to fully understand the impacts of the VR program on employment and other program outcomes for the population.

1.5 Objective

The objective of this paper is to chronicle the characteristics of VR applicants, the services they received, and their program outcomes as reported in the United States RSA-911 Case Service Report. This will provide researchers and VR program administrators with fundamental information about the use and efficacy of the VR program, beyond what is typically reported to the U.S. Congress. Our intent is to inform future research and hypotheses about the distribution of variables that may influence program outcomes or may identify gaps in VR services. The specific research questions answered in this study include:

What is the distribution of personal characteristics, VR service receipts, and program outcomes for VR applicants and how do these vary between participants and non-participants?

2.1 Data sets and cleaning process

The United States RSA-911 Case Service Report data for program years 2017 and 2018 was cleaned and organized using rules developed from the data reporting guidelines, publicly available information on public benefit limits (e.g., SSI has supplemental benefits in some states, TANF rules vary widely by location), and United States Bureau of Labor Statistics national wage data for SOC Codes ( Skowronski, 2018; U.S. Department of Veterans Affairs, 2020b, 2020c; U.S. Social Security Administration, n.d., 2020b; National Academy of Social Insurance, n.d.; Rehabilitation Services Administration, 2017; Schott, 2020, 2020; U.S. Administration for Children and Families & Office of Family Assistance, 2018; U.S. Bureau of Labor Statistics, n.d.; U.S. Department of Veterans Affairs, 2020a; U.S. Social Security Administration, 2020a; U.S. Administration for Children and Families & Office of Family Assistance, 2019; U.S. Bureau of Labor Statistics, 2010; Congressional Research Service, 2012). We defined our population of interest as applicants 25 to 80 years-old who were not enrolled in a secondary education program. This allowed us to study those who were likely to be focused on obtaining employment as opposed to going to school or pursuing other non-employment activities. For example, within those excluded, some may have been seeking employment to supplement college expenses, others likely sought employment to live independently, and yet others may have been focused on developing skills that could someday lead to living without their parents. We chose 25 years as the separating age to align with the minimum age in the complementary United States labor force data used to answer our second research question. This age restriction resulted in a focus on adult VR services, therefore pre-employment transition services were excluded from the analysis because (although available to everyone) those are designed for transition-age youth. Two subsets of VR applicants were also created—participants, i.e., applicants who went through the eligibility process, developed, and signed their individualized plan for employment (IPE) and non-participants, i.e., applicants who exited prior to developing their IPE. Participants and non-participants were included in the analysis to identify similarities and differences between those eligible for VR services who exited early and those who established an IPE and gained access to a more complete set of services.

Data files received from the RSA were dated March 23, 2021. The initial dataset included 1,010,456 cases closed between July 1, 2017 and June 30, 2019. After filtering out students and applicants under 25 years old and over 80 years old, it was reduced to 572,490 applicants. Altogether, 358,231 of these cases were participants (63%) and 211,105 cases were non-participants (37%). The remaining 3,154 applicants (0.6%) were excluded from the participant and non-participant groups due to missing exit type data (i.e., these people were only analyzed in the applicant group). A descriptive statistical analysis of personal characteristics, receipt of VR services, and program outcomes was conducted. When relevant, the descriptive statistics were compared to similar populations in the United States labor force which include all the people in the U.S., in this case 25 years-old and older (i.e., no maximum age was specified), who were employed or actively seeking employment. All statistics were calculated using IBM SPSS Statistics 26.

2.2 Managing employment outcome variables

Within the data set, there was a specific variable called employment outcome. This was captured twice, once when the first employment outcome in alignment with the IPE was attained and a second time at exit. The United States Department of Education, which houses the RSA, defines employment outcome as employment “consistent with an individual’s unique strengths, resources, priorities, concerns, abilities, capabilities, interests, and informed choice” (State Vocational Rehabilitation Services Program, Applicable Definitions, 2021, para. 15). Someone who does not achieve that, is said to have no outcome. The dataset categorized anyone with no outcome as a missing data point in the employment outcome variables. Insights into those with no outcomes are valuable to objectives related to improving program retention and program outcomes. Therefore, during data cleaning, the researchers assigned an outcome to VR applicants with no outcome according to their exit type. These were: exit pre-program, exit ineligible for the program, and exit unemployed after entering the program. The actual employment status at exit was not captured for those who exited pre-program and were ineligible for the program. It is possible they gained employment on their own prior to the IPE and withdrew from the program; it is also possible they decided not to pursue employment.

3.1 Distributions of personal characteristics

3.1.1 Age, sex, race and ethnicity and their comparison to the labor force

The age distribution of VR applicants (i.e., at application) was largely between 25 and 50 years. It decreased from 25 years (approximately 3%), with a valley around 42 to 44 years (approximately 2.2%), a second peak around 54 years (approximately 2.7%), and then quickly declined after about 55 years. Figure 2 presents the distribution of participant and non-participant ages at application and exit, which follow a similar shape to the distribution of applicants at application. For all stages in the process, the median age for VR participants was between 45 and 47 years, and for non-participants, it was between 42 and 43 years.

OPEN IN VIEWER

Fig. 2

Distribution of age at application and exit for participants and non-participants.

Table 1 compares distributions for sex, race, ethnicity, and education between the VR population and the U.S. labor force. There were similar proportions of men and women 2 in VR as were in the 2019 U.S. labor force. However, there was a higher proportion of Black individuals in the VR populations (26–30%) than there were in the 2019 U.S. labor force (13%). There were also fewer Hispanic individuals in the VR population (10–11%) than the U.S. labor force (18%) ( U.S. Bureau of Labor Statistics & Current Population Survey, 2020).

OPEN IN VIEWER

Table 1

Frequency proportion of personal characteristics for VR populations and the U.S. labor force

Characteristic	2019 estimated U.S. labor force	Applicants	Non-participants	Participants
Number of cases		572,4901	211,105	358,231
Proportion of population		100%	37%	63%
Sex
Female	47%	47%	46%	47%
Male	53%	53%	54%	53%
Race2
White	77%	69%	65%	71%
Black	13%	27%	30%	26%
Asian	6%	2%	2%	2%
American Indian/Indigenous	1%	3%	3%	2%
Hawaiian	< 1%	1%	1%	1%
Number of races
No race identified	Unknown	1%	3%	1%
One race identified	Unknown	96%	95%	97%
Two or more races	2%	2%	3%	2%
Ethnicity
Hispanic	18%	11%	10%	11%
Education level at program entry3
Did not complete high school or equivalent	6%	18%	24%	18%
High school diploma	25%	39%	19%	39%
GED	9%	28%	9%
Certificate of completion of IEP	3%	1%	3%
Post-secondary ED certificate (non-degree)	4%	5%	4%
At least 1 year of post-secondary ED	15%	11%	8%	11%
Associate’s degree	11%	6%	7%	6%
Bachelor’s degree	27%	8%	7%	8%
Higher than bachelor’s	16%	3%	2%	3%

1. Due to missing exit type information necessary to classify applicants as participants or non-participants, 3,154 applicants were excluded from the participant and non-participant groups and only analyzed in the applicant group. 2. Estimates for race in the U.S. labor force are for those who identify with a single race. 3. Education level at program entry is compared to the 2020 U.S. labor force. The 2020 U.S. labor force provided statistics for high school diploma and equivalent (this is assumed to include GED, Certificate of Completion of Individualized Education Plan (IEP) and some post-secondary education without a degree).

4.1 Implications for future research

Not surprisingly, the reported impairments within the VR population are quite heterogeneous. Seven of the 20 impairment categories encompass 78%of applicants. If impairment is ignored, the results of an investigation will likely be skewed towards these seven most prevalent groups. Dutta et al. (2008) is perhaps the broadest study on the VR population, where they separated the participants into three groups: sensory impairments, physical impairments, and mental impairments. With the heterogeneity of the population, it is important to consider impairments more explicitly in research designs. Many diagnoses share impairments, so researchers may not be serving VR participants to the best of their capabilities when studies are limited to a specific diagnosis. Diagnoses that are commonly labeled as disabilities often encompass a group of impairments. Additionally, it is the impairment/health condition’s interaction with the environment that creates a disability (World Report on Disability, 2011). It is challenging to make recommendations and changes to a program that will have a strong impact if the research applies to a small subset of people based on their diagnosis.

As other reports and research have not focused on non-participants, it is noteworthy that 43%of non-participants received at least one VR service, and 65%of non-participants exited between being determined eligible and developing their IPE. Like participants who do not receive services, these are reportable individuals in official reports and their employment outcomes (i.e., employment at exit) are not recorded. From an economic perspective, they consume resources with minimal outcome tracking. While we do not have detailed information regarding non-participant employment outcomes, which limits our understanding of the efficacy of the VR program, we do know the most likely exit reasons were that the applicant lost interest and VR lost the ability to contact the applicant (Table 4b). We do not know if receipt of VR services as a non-participant alter the likelihood of CIE achievement. If these non-participants did not achieve CIE, then what unseen variables are impacting their ability or willingness to stay in the program? Could these potential unseen variables be related to the OOS in some states and the time spent on the waiting list? What role does OOS have on participant and non-participant outcomes? Future investigations answering these questions could lead to improved understanding of the efficacy of VR and program retention needs stimulating innovative improvements to the program.

This study also suggests that there are large percentages of VR participants who are unable to reach their employment goals by the end of their program. It is possible that some popular practices help and others do not (and some may even hurt goal achievement). Future investigations need to determine which services fall in each category. For services that have a negative association with CIE at exit, we need to understand why that is, and if pairing them with another service can moderate the effect to create a positive outcome. The most common reasons for exit for participants, behind achieving employment, are participant lost interest and VR lost the ability to contact the participant. To improve program outcomes, research should focus on program retention and the relationships between VR services and program outcomes. With 31 types of services offered, there are opportunities to strategically overcome barriers associated with reduced likelihood of both finishing the program and exiting with CIE.

It is also notable that there are differences in representation of racial and ethnic minorities between the VR population and the U.S. labor force. Additionally, education levels are much lower in the VR population than in the U.S. labor force. An explanation of why these differences occur could not be ascertained with the data available for this study. However, it is conceivable that cultural differences associated with race, ethnicity, and education may influence the perceived usefulness of a service to a participant in a specific subgroup or it could reflect differences in the availability of services to different demographics. Future research could further explore how services provided to VR applicants differ across demographic variables to better understand who is using VR services and potentially identify areas where services are not being deployed equitably. Further, it might also identify situations where services are designed for a specific group and if the service design needs to be modified to be relevant/have increased perceived usefulness to other groups.

Whether or not an applicant is employed at program start, and whether or not they have a supported employment goal, suggests a difference in their program outcome goals and the services needed to meet them. Surprisingly, 17%of participants with supported employment goals instead exited with CIE. It would be worthwhile to understand what is different with those who had a supported employment goal and exited with CIE compared to both those who exit unemployed or with supported employment. Future investigations into the association of services and CIE might consider investigating groups separately based on the goals and/or starting employment conditions.

At the time of application, 48%of participants did not receive any public benefits, 42%received one benefit and about 10%received two to five benefits. There was a small reduction in receipt of each of the public benefits (except SSDI which had a 10%increase) as a program outcome. While some previous studies considered a negative association between receipt of benefits and exiting with CIE to be work disincentive, that would imply the receipt of the benefits deters the participant from gaining work. The evaluation of additional variables in tandem with benefit receipt could determine if the relationship is truly a work disincentive, or if there is a mediating variable associated with benefit receipt. This is important to differentiate because work disincentive may connotate a deterrent to motivation when receipt of benefits could simply imply a more complex situation. Additionally, it is important to understand the association of benefit receipt with employment achievement and other program outcomes for both the participant and non-participant. If this was investigated for the full population instead of by specific impairments or diagnoses, it could lead to insights important to participation and program outcomes.

5.1 Limitations

Perhaps the most important limitation of this study is that the dataset was designed for administrative purposes instead of research. The dataset was not structured to facilitate a large-scale longitudinal analysis of VR program outcomes.

For future research, it is important to note many of the variables in the dataset are correlated (reporting them was beyond the scope of this study). Correlations between variables may make it difficult to establish which variables are the most important determinants of unsuccessful or successful employment outcomes. For example, the most common employment barriers are related to poverty and skill deficits, which intuitively are connected to the receipt of public financial benefits and low education levels. We also find many variables that could indicate important latent variables. For example, both the referral source and type of residence could indicate latent variables related to social supports. Another example (a combination of potential correlated variables and indications of latent variables) is WIOA program involvement and services. It was noted that the services available through WIOA’s Wagner-Peyser Employment Program may also be provided by vendors that are not part of the VR program. So, is it employment services that are important, or is it more specifically how employment services are delivered as part of the particular WIOA program? This can also occur with program outcomes. For example, is a change in health insurance related to another outcome such as the change in receipt of SSA benefits, change in employment, or another latent variable? This study motivates untangling the relationships between the variables, selecting the most appropriate ones for deeper analysis, and identifying latent variables with multivariate statistical methodologies.

Overall, this study highlights the importance of conducting VR research on larger impairment groups rather than specific disability diagnoses, consideration of IPE goals, how service delivery varies with different demographic groups, and the need to investigate program retention for both participants and non-participants. While access to more detailed data regarding prescription of services, timelines of service delivery, and participant goals would be useful, the current dataset still provides a rich opportunity for further investigation. In particular, future research should identify services or sets of services that are most impactful to achieving CIE upon exiting the program. This knowledge would help future investigations streamline their designs related to demographics and equity, specific impairments, and employment goals to maximize impact.