Vocational rehabilitation service usage and outcomes for transition-age youth with serious mental health conditions

Data include demographic characteristics and disability information at application, and service descriptions and employment outcomes for those who receive services.

Youth characteristics: Age, race, Hispanic ethnicity, sex, educational attainment, and employment status at the time of application. Age of clients was categorized as 14–17 years, 18–21 years, 22–24 years. These age groups reflect youth who are minors (ages 14–17), the youngest adults (ages 18–21) who may still be enrolled in secondary school, and youth beyond the transition ages eligible for Pre-ETS (ages 22–24). For multivariate statistics, educational status and employment status at application were combined to form 6 categories; Unemployed with Less than High School Degree, Unemployed with High School Degree, Unemployed with Greater than High School Education, Employed with Less than High School Degree, Employed with High School Degree, and Employed with Greater than High School Education.

SVRA characteristics: The proportion of adults with successful closure, proportion of adult dropouts, percentage of youth clients, and duration from application to IPE. These variables were chosen to reflect when agencies ranked high or low on basic SVRA service provision and outcomes (Foley et al., 2020) and inclusion of youth (Honeycutt et al., 2015). The two adult variables were based on clients over age 24 at application. The proportion of adults with successful closure (in 2015) = # adults closed with an employment outcome/# adult clients. The proportion of adult dropouts (from 2015–2017) = the # eligible adults closed before IPE/# adult clients. The percentage of youth clients (from 2015–2017) = # youth clients/# all clients. Each of these three variables was first ranked, then categorized as being high if they were among the 12.5% of states (n = 6) with the highest rates, low if they fell within the bottom 12.5% of states, and medium if they fell within the middle 75%. Duration from eligibility determination to IPE was calculated for all clients with an IPE, which was then ranked by quartile (1st quartile = shortest duration, 4th quartile = longest duration).

Service use: For each youth, we examined receipt (yes/no) of a variety of SVRA services (college/graduate school, occupational/vocational training, on-the-job training/apprenticeship, other training, job search/job placement, supported employment/customized employment/short-term employment support). We also examined their receipt of an IPE from the RSA-911 closure status variable. An IPE is required to access most SVRA services in most states. A client may access only limited services, such as vocational counseling, prior to developing an IPE, therefore the IPE is an indicator of progressing towards receiving significant SVRA services, including education or training services and supports related to employment goals.

Service outcome: We examined one service outcome of interest: successful closure. Successful closure was defined as closure with employment and was examined among those with an IPE. This includes any kind of employment tracked by SVRAs; competitive integrated employment, Randolph-Sheppard Business Enterprise Program, state agency-managed Business Enterprise Program, supported employment in integrated employment, supported employment on short-term basis, and uncompensated employment.

Research questions:

How are youth with a primary MHD different from youth with other disabilities in their characteristics at application, and their use and outcomes of SVRA services?

2.2 Analytic approach

To answer research question one, we used descriptive statistics. Because of the large sample size, only differences between groups of at least 5% were considered meaningful. To answer research question two, we examined completion of an IPE for youth with MHD who were eligible for services, and successful closure for youth with MHD with an IPE. The recorded employment status at closure for youth without an IPE contained no successful closures, thus this analysis was restricted to those with an IPE. To identify youth with MHD with different likelihoods of having an IPE/successful closure, two multivariate statistical approaches were used to characterize the relationship between youth/SVRA characteristics and IPE/successful closure. The first approach was logistic regression to estimate the odds of having an IPE/successful closure, and we present odds ratios for the estimates. Statistically significant odds ratio values that are greater than 1.0 indicate increased odds of having an IPE/successful closure, whereas statistically significant odds ratio values less than 1.0 indicate decreased odds. The second approach was classification and regression tree (CART) analysis, using SPSS version 2X, to display population subgroups and their relative risks of having an IPE/successful closure. CART (Breiman et al., 1984) is a machine learning approach that relies on recursive partitioning of the data. CART analysis creates statistically distinct subgroups based on sequential, hierarchical splits in the population that yield the strongest between-subgroup differences regarding a selected outcome (Breiman, et al., 1984; Toschke et al., 2005; Steadman et al., 2000). This tree-growing method maximizes within-group homogeneity, and splits in the data were found based on squared probabilities of membership in each outcome category (using the Gini calculation) with a minimum change improvement of 0.0001. Only splits that produced final groups that contained at least 5% of the sample were considered. Thus, CART analysis identifies subgroups, often with multiple concurrent risk factors (e.g. Black males with low educational attainment) with high and low risk for having an IPE/successful closure. The CART analysis provides a graphic that is useful for contrasting the relative risk between subgroups. However, CART analysis does not yield point estimates that simultaneously adjust for all variables in the model. Therefore, using CART analysis with logistic regression identifies how the combination of specific risk factors contribute to high and low risk subgroups and specific point estimates for each risk factor.

As can be seen in Table 3, the proportion of youth that were not found eligible was comparable across disability groups. The MHD group was less likely (64.5%) to have an IPE than those in other disability groups (>71.0%). In examining the status of youth when their case was closed, two differences with other disability groups stand out; more in the MHD group (28.7% vs. <21.8%) exited after being determined eligible but before receiving an IPE and fewer (60.7% vs. >68.1%) exited after receiving services. Examining employment status at exit among those that received services revealed that exiting with employment was also lower (44.0%) in the MHD group than other disability groups (≥52.0%; Table 3).

Having an IPE: Logistic regression analysis revealed the largest individual-level effect on the likelihood of having an IPE was for youth who were ages 22–24, who were 34.3% less likely to have an IPE than 14–17 year-olds. Youth ages 18–21 years were also 28.9% less likely than 14–17 year-olds to have an IPE (See Table 4). Another large effect was observed based on employment and education status at application. Relative to those who were unemployed without a high school certificate, the likelihood of having an IPE was greater for those who had some postsecondary education/training and were either employed (31.2% greater) or unemployed (22.6% greater). The odds of having an IPE were also significantly lower for those who were Black, American Indian/Native American, or Multiracial compared to White youth, lower for Hispanic than non-Hispanic youth, and lower for female than male youth.

All SVRA characteristics were significant in both the high and low ranked states relative to medium ranked states. The greatest differences were for those in SVRAs with low adult dropout rates, which increased the likelihood of having an IPE by 65.9%, whereas high adult dropout rates reduced the likelihood by 43.4%. High rates of successful adult closures also increased the likelihood, by 31.2%. Overall, the model correctly predicted 68.7% of cases, but only accounted for 2.3% (Cox & Snell R²) of the variance in receiving services.

CART analysis provided a picture of how these variables combined to form subgroups that have greater or lesser likelihood of having an IPE (See Fig. 1). Consistent with the logistic regression findings, the first and thus the most differentiating split was by the SVRA characteristic of adult dropout rate, with more youth in states with low adult dropout (79.0%; Node 1) and fewer youth in high ranked states (51.9%, Node 2) having an IPE. These were the two groups in the analysis that had the highest and lowest rates of having IPEs. No other individual- or state-level characteristic significantly discriminated between those that did and did not have an IPE in these states. Most youth (80%) were in medium-ranked states on adult dropout rate (Node 3). Within these youth, age was the most differentiating split, with 74.6% of youth ages 14–17 having an IPE (Node 6), compared to 66.3% and 67.6% of youth over age 17 (Nodes 4 & 5, respectively). These findings suggest that the impact of age group on having an IPE is primarily seen in states with medium ranking adult dropout, and not a significant factor in high and low ranking states. This model was significant, χ² (df = 20) = 1662.198, p < 0.001, and correctly classified 68.7% of the sample.

OPEN IN VIEWER

Fig. 1

Classification and regression tree analysis of having received an Individual Plan for Employment (IPE) among youth with a mental health disability.

Employment at closure: Logistic regression predicting successful case closure (i.e., with employment) among youth with an IPE revealed several strong individual-level predictors (See Table 5). Very large effects were observed among youth who were employed at application regardless of educational attainment. These youth were 105–298% more likely to exit with employment than those who were unemployed with less than a high school diploma. It is also noteworthy that among those who were unemployed at application, those with some postsecondary education/training also had substantially enhanced odds of employment at exit (OR = 1.467). Successful closure was less likely in Non-White, Non-Asian youth, youth who were Hispanic, or female (Table 5).

SVRA characteristics generally had smaller odds ratios than many individual-level factors. The largest effects were for SVRAs with low ranking for successful adult closure rates (OR = 0.624) and low adult dropout rates (OR = 1.237), though in opposite directions. The duration from eligibility to IPE was also significant. For every quartile longer the performance was in duration, there was a 5.2% reduction in the likelihood of successful employment closure. This highlights the importance of the pacing of services. The impact of other SVRA characteristics varied (see Table 5). This model was significant, χ² (df = 21) = 2292.525, p < 0.001, and correctly classified 57.6% of the sample.

CART analysis identified how these individual and SVRA characteristics combined to form subgroups that had higher or lower likelihood of successful closure (Fig. 2). Consistent with results of the logistic regression, employment/education status at application was the most differentiating split. The single highest rate of successful closure (58.3%; Node 4) was among youth with employment and a high school or greater education. The second highest rate of successful closure (50.2%; Node 3) was among youth either with employment but less than high school education or unemployed but with more than a high school education. No other variables significantly discriminated between subgroups within these two groups. Those who were unemployed without a high school education had exceptionally low rates of successful closure (33.7%; Node 2), particularly for Black, Native American, or multiracial youth who had the lowest rates of successful closure (29.8%; Node 9). This model correctly classified 60.3% of successful closures.

OPEN IN VIEWER

Fig. 2

Classification and regression tree analysis of exiting SVRA services with employment (i.e., a successful closure) among youth in the Primary Mental Health group who had an Individual Plan for Employment.

4.1 Under representation

While the recorded disability category for services in the RSA-911 database may mask the actual or complete picture of impairments of each youth, the distribution of youth with MHD versus those with COG disabilities suggests a large under-representation of youth with MHD in SVRA services. Estimates of the prevalence of MHD in adolescents range from 12% (Forness et al., 2012) to 22% (Merikangas et al., 2010). The prevalence of all developmental disabilities (i.e., intellectual disability, autism spectrum disorder, developmental delay) in 3–17 year olds was 7.4% in 2019 (Zablotsky et al., 2023). The prevalence of specific learning disabilities is 4% to 9% (Landerl & Moll 2010). That there are almost 4 times as many youth with cognitive disabilities as youth with MHD in the RSA-911 database strongly suggests a substantial underrepresentation of youth with MHD.

4.2 Disability differences in age at application

Age at application was the most striking difference in characteristics between youth with MHD and youth with other disabilities. Reasons why youth with MHD were older at application were not apparent from these data. One possible contributing factor is that MHD often have a later age of onset than other childhood disabilities. Almost half of individuals who will develop a psychiatric disorder do so by age 14, but another 24% have onset at ages 15–24 (Kessler et al., 2005). The peak ages of onset for psychotic disorders are 22 and 25 years for males and females, respectively (Malla & McGorry, 2019). This may account for their SVRA applications at older ages. However, the size of the age disparity suggests that connections to SVRAs for these youth are not occurring as they should during high school or through the transition planning process mandated by the Individuals with Disabilities Education Act (IDEA; 2004). IDEA mandates transition planning that requires postsecondary goals and service planning for all transition-age special education students. This is a common pathway to SVRA applications. However, Forness and colleagues (2012) found that less than 10% of students with serious mental health conditions receive special education services. This limits the effectiveness of IDEA transition planning for connecting most students with MHD to SVRA services and may contribute to fewer applying at high school ages. Both mental health services and SVRA workers assume that youth with MHD will be connected to SVRA services through secondary schools (Gatesy-Davis et al., 2021). Hence, while the WIOA-mandated youth services should enhance the transition to employment, those with MHD may be additionally disadvantaged by their underrepresentation within the special education population. The low rate of applying for SVRA services at high school ages might be remedied by a combination of better identification of MHD by schools and referral to SVRAs by other systems, such as child mental health, juvenile justice or child welfare.

One disadvantage of being in the oldest age group at application was a reduced odds of having an IPE, particularly among those in SVRAs with medium-ranking adult dropout rates. Access to robust SVRA employment services requires an IPE. The oldest youth are also not eligible for Pre-ETS. While the impact of Pre-ETS on employment outcomes for youth with disabilities is yet to be rigorously examined (Rooney-Kron et al., 2024), being excluded from accessing these potentially beneficial services (Ford et al., 2019; Lau & McKelvey, 2023; Salon et al., 2019) through delayed referral to SVRAs is of concern. The current findings provide compelling evidence that call for children’s services, schools, and SVRAs to identify youth with MHD in high school, regardless of special education status, and connect them to SVRA services and to Pre-ETS.

4.3 Exiting SVRA services early

Several findings suggest that SVRAs are not supporting youth with MHD as well as other youth, including that SVRA-eligible youth with MHD were more likely to leave services after being determined eligible but before completing the IPE.

Individuals may exit SVRA services for a variety of reasons. Ipsen and Goe (2016) examined factors associated with premature exit from SVRA services among adults of all ages and across disabilities. They found that 30% reported that they left because they had a job and 34% reported that they left because they were dissatisfied with services. Some of the premature exit from services in youth with MHD may be due to dissatisfaction. Gatesy-Davis and colleagues (2022) found that mental health and SVRA system leaders both felt that SVRA service providers did not understand the particular needs of youth with MHD and MH system leaders indicated that slow SVRA case processing is a disincentive for this population. This is consistent with our finding that youth with MHD in states with slow pacing (i.e., duration from eligibility determination to IPE completion) were less likely to have successful closures. The problem associated with pacing has been cited by others (Ipsen & Goe, 2021; Foley et al., 2020; Honeycutt & Stapleton, 2013). Improvements in the pace with which youth with MHD are assessed, develop an IPE and move into employment services would likely improve retention. Youth with MHD may need tailored outreach and support, which might be better achieved through closer collaboration between SVRA and MH providers.

Within youth with MHD, the likelihood of youth exiting SVRA early followed the same pattern as the SVRA’s adult dropout rate. CART analysis suggests that the impact of SVRA adult dropout rate was so strong that it eclipsed all other factors in states with very high and very low rates. Taken together these findings suggest there may be common factors contributing to both adult and youth dropout rates. A greater understanding of these factors in high and low SVRAs may help identify remedies. At the individual level, in addition to age, youth having some postsecondary education or training at application was also a strong predictor of having an IPE. It may be that skills that contribute to postsecondary success may be similar to those that lead to developing an IPE. There may also be a tendency for SVRA workers to be more engaged with those that have demonstrated school success. Honeycutt and colleagues (2017) similarly found that educational attainment among youth with disabilities impacted receipt of services. This suggests a general pattern across youth with disabilities and argues for greater effort to engage and serve youth with lower educational attainment.

Our findings about the impact of race and ethnicity on having an IPE reflects inequities in society more broadly; youth who were African-American, Native American, Multiracial or Hispanic were less likely to have an IPE. Race and ethnicity were not, however, characteristics that maximally discriminated between youth having an IPE in the CART analysis. This suggests that further research on the interaction of race and ethnicity with other demographic variables as well as state-level characteristics is needed.

4.4 Engaged in employment at case closure

The current findings that youth with MHD were less likely than youth in other disability groups to have successful closures after receiving services are similar to and extend Honeycutt and colleagues’ (2017) findings to more recent cohorts. More research is needed to understand why SVRA services are less likely to produce successful closures in youth with MHD. These findings are also similar to those found in adults (Mann et al., 2017) suggesting a better understanding of the strengths and weaknesses of SVRA services for those with MHD is needed to reduce this inequity in the future.

Within youth with MHD, the finding that those most likely to close successfully had employment at application, regardless of educational attainment, is consistent with findings that employment experience is important to future employment success for youth with disabilities and those without (Baum & Ruhm, 2016; Mamun et al., 2018; Metcalfe et al., 2019). Having employment at application has also been reported as a strong predictor of successful closure cross-disability among SVRA clients (i.e., youth and adults) (Wang & Etheridge, 2022; Hill, Mann & Gellar, 2022). Thus, recent employment may be a key contributor to successful closures for all SVRA clients, which in youth, by definition, is early career employment. Services that help youth with MHD obtain early work experiences, such as Pre-ETS, or through career and technical education, may provide great advantage in obtaining subsequent employment. Conversely, SVRAs may need to offer different employment supports to youth without early employment experience to enhance their likelihood of success.

The analysis of racial and ethnic differences in the likelihood of successful closure within youth with MHD indicated that race only discriminated successful closures within youth who were unemployed and had less than a high school certificate at application. It may be that race is particularly powerful among youth already at risk of poor outcomes. These employment outcomes mirror inequities in society that SVRA services do not overcome. Other studies of employment outcomes in youth (Honeycutt et al., 2017; Sima et al., 2015; Gonzalez et al., 2011) and adults (McDonnall et al., 2020; Sannicandro et al., 2018; Alverson & Yamamoto, 2018) with disabilities have also found racial and ethnic inequities in employment outcomes.

4.5 Limitations

Several limitations of this study are important to consider when interpreting the findings. RSA-911 data are collected for administrative purposes and have the potential for errors due to staff input, or limitations in the measures they contain. The measures do not require staff to report reasons for exiting services prior to completing services, information that would help explain why eligible youth with MHD often exit before having an IPE and why they benefit less from SVRA services than youth with other disabilities. Further, SVRA programs only report known employment status at closure meaning that some youth recorded as without employment may in fact be employed or engaged in vocational activity. While many of the findings reported in this study are predictive (e.g. characteristics at baseline that predict employment at closure), they are correlational and provide no causal evidence on the relationship between individual or state-level characteristics and outcomes. Lastly, the percent correct classification and the proportion of variance explained by the regression models is moderate and indicates that many other variables are important.