Abstract
Introduction
Much has been written about the monetary benefits and costs of supported employment programs over the past three decades. Cost-efficiency and cost-effectiveness studies have been conducted on adult service programs (Lewis, Johnson, Bruininks, Kallsen, & Guillery, 1992), entire states (Rusch, Conley, & McCaughrin, 1993), the United States as a whole (Cimera, 2010) and numerous other countries (Jaeger et al., 2006; Lee, Yoo, & Peters, 2003; Tuckerman, Smith, & Borland, 1999). Studies have explored the monetary returns of working in the community from the perspectives of individuals with specific disabilities, such as intellectual disability (Hill et al., 1987), autism spectrum disorders (Cimera & Burgess, 2011), and psychiatric conditions (Bond et al., 1995), as well as to the taxpayer (Wehman et al., 1985) and society in general (Tines, Rusch, McCaughrin, & Conley, 1990). Additionally, studies have been conducted on the efficacy of training strategies, such as natural supports (Zivolich, Shueman, & Weiner, 1997), the impact of early transition services (Cimera, Burgess, & Bedsem, 2014), and the utilization of facility-based programs in the employment process (Cimera, 2011a).
In depth summaries of this extensive body of literature may be found elsewhere (cf. Cimera, 2012; Kregel et al., 2000); however, suffice to say that supported employment had been determined to be generally cost-efficient from both the worker’s and taxpayers’ perspectives as well as cost-effective when compared to facility-based programs, such as sheltered workshops. Yet, often lost within the discussions regarding supported employment’s monetary merits is the fact that supported employment is not always cost-efficient or cost-effective.
For instance, Rusch et al. (1993) found that, for every dollar invested in supported employment, supported employees in Illinois returned only $0.77 to taxpayers. Cimera (2010) substantiated these findings when he determined that supported employees with intellectual disabilities were not cost-efficient from the taxpayer’s perspective in Indiana (benefit-cost ratio of 0.99), Arizona (0.89), Hawaii (0.86), Washington (0.84), Wisconsin (0.79), California (0.78), Illinois (0.63), Puerto Rico (0.54), or the Virgin Islands (0.37).
Even when data suggest that supported employment is cost-efficient or cost-effective, it must be kept in mind these findings represent the average outcome for the participants being investigated. So when 214 supported employees with intellectual disabilities generate $1.97 in benefits for every dollar they lost as a result of working in their communities, as was found by Hill et al. (1987), this does not mean that all individuals were better off monetarily for being in supported employment. In fact, these findings do not even indicate that the majority of supported employees generated more benefits than costs. It might have been that a few extreme outliers raised the average benefit-cost ratio over 1.00, thus making supported employment programs cost-efficient.
But does this really matter? If 99% of supported employees are not cost-efficient or cost-effective, the program can still return an overall net benefit if the remaining 1% of its participants more than compensates for the net costs of the others. In other words, the benefits garnered by the minority could pay for the costs of the majority.
From a policy perspective, the percent of individuals who are, or are not, cost-effective or efficient does not matter – as long as the program as a whole generates more benefits as a cost. It does not matter if a handful of people balance out the costs of the rest or whether everybody produces a net benefit. The bottom line is whether the program, as a whole, is a good investment.
From a practitioner’s perspective, however, the individuals who are not cost-efficient or cost-effective to serve are of paramount importance. The only way supported employment can improve the quality of its services is to investigate those who are currently not being served efficiently. Only then can new strategies and policies be devised to help ensure that, someday, everybody will benefit from being competitively employment in their communities – if they so choose. In short, supported employment’s long-term future is dependent upon the examination of its current shortcomings.
The present study is the first in a series that will explore individuals for whom supported employment is not cost-effective – that is, the costs they generate exceeded their corresponding benefits. Specifically, this study examined the costs to state-federal vocational rehabilitation (VR) agencies accrued by 40,118 supported employees who had their cases closed throughout the United States and its territories in 2013 and compares them to estimated costs of sheltered workshops found in the literature. Costs-per-month of service and per-hour-worked were explored for each VR agency as well as across various demographic variables (e.g., gender, ethnicity, primary impairment) in the hopes of identifying areas in which supported employment may beimproved.
Methods
Source of data
Data for the present study were provided by the Rehabilitation Services Administration (RSA). RSA’s 911 database contains data on all individuals who apply for services from each state-federal VR program (i.e., general/combined agencies and agencies for the blind) in the United States and its territories. Data presented here are from all cases officially closed in FY 2013.
Participants
In 2013, 589,402 people applied for services from VR. The participants of this study include the 40,118 applicants to VR who selected supported employment as the vocational goal in their IPE. The majority of these individuals were white (70.6%) males (60.4%) who had at least a high school education (59.6%). “Self” was the most common source of referral to VR (25.0%); 53.0% were employed at the time their cases were officially closed. Their demographics can be found in Table 1.
Variables
State-federal VR agencies
There were 80 state-federal VR agencies throughout the United States and its territories in 2013. Twenty-four served individuals with vision impairments (i.e., agencies for the blind); the remaining 56 served individuals of all disabilities (i.e., general/combined agencies). Of these 80 agencies, 78 provided supported employment services to its applicants.
Supported employment’s costs
The costs of services contracted by state-federal VR agencies were included in RSA’s 911 database. These expenditures do not include services provided directly by rehabilitation counselors, such as assessment or counseling. To calculate cost-per-month served, the cumulative cost of services funded by VR was divided by the number of months from the time the participants signed their Individual Plans for Employment (IPE) to when their cases were officially closed. Cost-per-hour worked was calculated by dividing the cost-per-month of service by the average number of hours worked per month. It should be noted that cost-per-month of service included applicants to VR who did not become employed (i.e., individuals who generated cost, but produced no pecuniary benefits to themselves or taxpayers); while the costs-per-hour worked included only applicants who obtained employment within their community.
Sheltered workshop costs
The vocational rehabilitation literature was reviewed to estimate the cost of sheltered workshops. Six studies were identified that clearly reported unique data on the annual or monthly per capita costs of sheltered employees (i.e., Cimera, 2007, 2011; Conley et al., 1989; Lewis et al., 1992; Hill et al., 1987; Rogers et al., 1995). Data from these studies were converted to 2013 dollars by multiplying the dollar value by 2013’s consumers’ price index (CPI) and then dividing the resulting product by the CPI of the year that the dollar value was originally designated (Levin & McEwan, 2000). In the cases where only annual data were presented, costs were divided by twelve months.
To calculate cost-per-hour worked, monthly costs of sheltered workshops were divided the average number of hours sheltered employees worked per month. When studies did not present the average number of hours worked by sheltered employees, the figure of 74 hours per month was used. This figure was based upon multi-state data on facility-based programs presented by NASDDS and HSRI (2009).
From the six sets of data on sheltered workshops found within the literature, a range of costs was identified. For cost-per-month of service, the upper limit was $993.36 (Lewis et al., 1992) while the lower limit was $593.72 (Cimera, 2007), with a mean of $728.58 for all six studies combined. For cost-per-hour worked, the range was $13.42 (Lewis et al., 1992) and $6.73 (Cimera, 2007), with a mean of $9.94.
Impairment types and impairment causes
RSA’s 911 database also included data on each applicant’s primary and (if appropriate) secondary disabilities. Both primary and secondary disabilities were comprised of two variables: (a) impairment type, and (b) the cause of the impairment. There were 19 options for impairment type (e.g., blindness, neurological impairments, cognitive impairments) and 37 options for the cause of the impairment (e.g., cause unknown, autism, congenital condition, traumatic brain injury). For example, an individual might have a “cognitive impairment” caused by “mental retardation” or a “mobility orthopedic/neurological impairment” caused by a “stroke.” For the purposes of this study, costs are presented by both primary impairment and impairment cause. Secondary disabilities, however, are presented as either “had a secondary disability” or “did not have a secondary disability.”
Research questions
The present study investigated the percentage of supported employees served by state-federal VR agencies who generated costs higher than the identified range of costs of sheltered workshops (i.e., the percentage of individuals who were not cost-effective in supported employment compared to sheltered workshops). Costs were examined with regard to cost-per-month served and cost-per-hour worked. In each analysis, supported employment costs were compared to three “levels” of sheltered workshop costs: (a) the lowest identified cost in the literature (i.e., $593.72 per month served and $6.73 per hour worked), (b) the highest identified cost (i.e., $993.36 per month served and $13.42 per hour worked), and (c) the mean costs of all six data sources combined (i.e., $728.58 per month served and $9.94 per hour worked). Percent of cost-ineffectiveness was examined by individual demographics (e.g., gender, ethnicity, primary impairment, cause of primary impairment, and the presence of secondary conditions) and by the state-federal VR agency that provided the supported employment services.
Results
Cost-per-month served
By demographics
Supported employees whose cases were closed in 2013 averaged costing VR $342.07 per month of services (SD = $418.45). This is well below the lowest range of sheltered workshop costs identified in the literature (i.e., $593.72). However, nearly 1 out of 5 (18.0%) supported employees were not cost-effective when using this threshold; 12.8% were not cost-effective compared to the mean cost of sheltered workshops (i.e., $728.58), and 7.1% were not cost-effective compared to the highest cost of sheltered workshops (i.e., $993.36).
When costs-per-month of service were examined across demographic variables, very few differences were evident with regard to gender or ethnicity. The average per capita cost for all gender and ethnic groups examined were below the lowest estimate of sheltered workshop costs. Moreover, in each of these groups, the proportion of supported employees who were not cost-effective hovered around 18.0%. When costs-per-month of service were compared to the mean and high estimates of sheltered workshop costs, roughly 13.0% and 7.0% of each demographic group were found not to be cost-effective to serve via supported employment (see Table 2).
As with gender and ethnicity, very few differences were found when costs-per-month of service were examined across impairment type and cause of impairment. All 19 impairment, and all 37 cause, categories had average per capita cost-per-month served below the lowest range of sheltered workshop costs. However, slight variations were noted in the percentage of individuals who were not cost-effective.
For instance, 11.4% of supported employees who had “other visual impairments” were not cost effective relative to the lowest estimate of sheltered workshop costs. This is compared to 37.5% of supported employees with “other hearing impairments.” When the highest estimate of sheltered workshop costs was examined, these figures were 2.9% and 12.5%, respectively. Similar variations were noted for the cause categories (see Tables 3 and 4).
By agency
Unlike with the demographic variables, considerable disparity existed in the average cost-per-month of service generate by each state-federal VR agency. Five VR agencies had average per capita monthly costs of service that exceeded the lowest range of sheltered workshop costs: Agency 27(mean $646.86; SD = $545.39), Agency 37 (mean =$1,564.22; SD = $4,028.69), Agency 42 (mean =$602.38; SD = $503.30), Agency 52 (mean = $893.54; SD = $622.04), and Agency 60 (mean = $854.48; SD = $1,856.33). Three of these VR agencies (i.e., Agencies 37, 52, and 60) had average supported employee costs exceeding the mean sheltered workshop cost (i.e., $728.58) and one (Agency 37) exceeded the highest estimate of sheltered workshop costs (i.e., $993.36).
There were also considerable variations among the state-federal VR agencies regarding the percent of supported employees who were not cost-effective. For example, 62.6% of supported employees served by Agency 52 were not cost-effective relative to the lowest estimate of sheltered workshop costs, while 37.0% were not cost-effective compared to the highest estimate of sheltered workshop costs. Conversely, these figures were only 0.2% and 0.0% for supported employees in Agency 29 (see Table 5).
Cost-per-hour worked
By demographics
The average supported employee who had their case closed in 2013 cost VR $7.23 (SD = $15.46) per hour they worked in the community. This is well above the $6.73 lower limit of hourly sheltered workshop costs identified in the literature. In fact, nearly a third (32.8%) of all supported employees were not cost-effective when compared to this benchmark, while 13.2% were not cost-effective compared to the highest estimate of sheltered workshop costs.
When examined by demographics, supported employees of both genders, and nearly all ethnicities examined, also produced average per capita costs exceeding sheltered workshop’s lowest cost-point. Only African Americans (mean = $6.02; SD = $8.15) and Hispanics/Latinos/as (mean = $6.32; SD = $8.06) had average per capita costs below the $6.73 per hour worked threshold (see Table 6).
With regard to types and cause of impairments, the average hourly costs of supported employees in 10 out of the 19 impairment categories, and 16 out of 37 cause categories, were higher than sheltered workshop’s lowest cost-point. In fact, individuals with cerebral palsy (mean = $13.53; SD = $84.87), muscular dystrophy (mean = $15.79; SD = $19.05), and had both mobility and manipulation impairments (mean = $14.70; SD = $96.34) produced an average cost-per-hour worked exceeding even thehighest estimate of sheltered workshop’s costs ($13.42). (See Tables 7 and 8).
By agency
As with cost-per-month of service, there were significant differences in the degree of cost-effectiveness produced by each state-federal VR agency. For instance, supported employees served by Agency 52 cost an average of $29.22 (SD = $25.56) each hour they worked in the community; the majority of whom (69.3%) were not cost-effective even when compared to the highest estimate of sheltered workshop costs. Supported employees from Agency 23, on the other hand, cost an average of only $0.42 (SD = $1.53) per hour – 0.6% of whom were not cost-effective at the lowest estimate of sheltered workshop costs (See Table 9).
Discussion
Supported employment has been repeatedly found to be a good investment for both taxpayers and workers with disabilities when compared to facility-based programs (Cimera, 2012; Kregel et al., 2000); however, not all supported employees are cost-effective. Some are actually cheaper to serve in sheltered workshops. Unfortunately, there has been very little attention paid to this overlooked segment of the supported employment population.
Examining individuals who are not cost-effective to serve via supported employment provides critical information needed to improve the quality of supported employment services. For instance, if it were determined that females with autism were not cost-effective to serve via supported employment, better training techniques that match the unique needs of these individuals could be developed to more efficiently promote community-based employment. If supported employees in rural regions, or areas of low economic growth, were found to be not cost-effective, greater consideration could be given to job development and job carving techniques. And so forth. Quite simply, if the quality of supported employment program is to improve, we must first determine where individuals are obtaining effective and efficient services, and where they are not. We can then learn from what’s working – and try to change what is not.
The present study is the first in a series of research that investigates individuals who are not cost-effective to serve in supported employment. It examined rates of ineffectiveness in relation to individual demographics (i.e., gender, ethnicity, primary impairment, cause of primary impairment, and the presence of secondary conditions) as well as by each state-federal VR agency throughout the United States and its territories.
It did this by calculating the cost-per-month served and cost-per-hour worked for each of the 40,118 supported employees who had their cases closed by VR in 2013. These costs were then compared to the estimated costs of sheltered workshops found in six refereed, published studies. From these six unique data sources, three cost-points were identified – thelowest estimate of sheltered workshop costs ($593.72 per month of service; $6.73 per hour worked), the highest estimate of sheltered workshop costs ($993.36 per month of services; $13.42 per hour worked), and the average of all six data sets ($728.58 per month of service; $13.42 per hour worked). When the analyses were conducted, several critical findings for the future of supported employment were identified.
The first finding was that, on average, individuals were cheaper to serve per month in supported employment (mean = $342.07; SD = $418.45) than in sheltered workshops, even when the lowest monthly cost of sheltered workshops ($593.72) was used as a benchmark. This result supports numerous studies that had previously determined supported employment generally costs less per month than sheltered workshops (Kregel et al., 2000).
The second finding is that rates of cost-per-month ineffectiveness did not appear to be significantly influenced by the supported employee’s gender, ethnicity, or disability(-ies). On average, supported employees in all demographic categories (including individuals with multiple disabilities) had per capita costs below the lowest estimate of sheltered workshop’s monthly costs. Moreover, the rate of ineffectiveness remained relatively consistent across demographic variables – with roughly 18% of supported employees not being cost-effective relative to the lowest estimate of sheltered workshop’s monthly cost.
This finding is a little surprising given several studies have determined that the costs of supported employment varied according to the individual’s disabilities (Cimera, 2012). Then again, these studies also found that supported employment was cost-effective compared to sheltered workshops regardless of the supported employee’s condition or conditions – which was collaborated by the data from the present study.
The third finding is that, although nearly all state-federal VR agencies were cost-effective with regard to cost-per-month served, there was considerable variation in the rates of in-effectiveness among agencies. For instance, supported employees served in Agency 52 cost VR an average of $893.54 per month of service (SD = $622.04). Individuals from Agencies 23 and 29, on the other hand, generated average monthly costs of $28.19 (SD = $92.12) and $36.44 (SD = $69.15), respectively. Further, over a third (37.0%) of supported employees from Agency 52 were not cost-effective even compared to the highest cost estimate for sheltered workshops. Only 0.2% of supported employees from Agency 29 were not cost-effective compared to the lowest estimate of sheltered workshop costs. When costs-per-hour worked were examined, these disparities widenedconsiderably.
The 21,257 individuals who became successfully employed within their communities generated an average cost to VR of $7.23 (SD = $15.46) per hour worked – well above the lowest estimated cost of sheltered workshop (i.e., $6.73). Moreover, supported employees in nearly every demographic category, as well as in 10 out of 19 impairment and 16 out of 37 cause categories, accrued average hourly costs higher than this cost-point. In fact, approximately one-third of all supported employees were not cost-effective relative to the lowest estimate of sheltered workshop hourly cost, while more than 1 out of 10 were not cost-effective even at sheltered workshop’s highest estimated costs.
As with cost-per-month served, average cost-per-hour worked and rate of cost-ineffective varied dramatically among state-federal VR agencies. Eight out 78 state-federal VR agencies had average hourly costs that exceeded the highest estimate of sheltered workshop’s costs: Agency 16 (mean = $13.67; SD =$18.81), Agency 21 (mean = $15.47; SD = $21.65), Agency 37 (mean = $1,623.94; SD = n/a), Agency 40(mean = $15.45; SD = $21.45), Agency 46 (mean =$14.33; SD = $17.13), Agency 52 (mean = $29.22; SD = 25.56), Agency 53 (mean = $14.87; SD =$12.53), and Agency 78 (mean = $19.50; SD =$33.92). Moreover, significant proportions of supported employees were not cost-effective at this cost-point. For instance, 69.3% of Agency 52’s, 34.3% of Agency 16’s, and 33.3% of Agency 5’s supported employees were more costly to serve in the community relative to the most expensive estimate of sheltered workshop costs. Conversely, only 0.6% of supported employees funded by Agencies 23 and 29 were not cost-effective compared to the lowest estimate of sheltered workshop’s costs.
So what does all of this mean? Why are some state-federal VR agencies able to provide more cost-effective supported employment services than other agencies? Answering this question will not only make supported employment programs more cost-effective throughout the United States, it will also lower supported employment’s costs to taxpayers and enable more individuals with disabilities to become employed in the community without the need for additional resources.
One potential explanation for these disparities in cost-effectiveness is that each state-federal VR agency serves different populations. Perhaps, agencies with high rates of cost-ineffectiveness serve individuals with more numerous, or intensive, disabilities. This possibility certainly requires further investigation; however, given that cost-effectiveness did not vary substantially by gender, ethnicity, type or cause of impairment, or number of disabilities, this explanation seems unlikely to produce the dramatic differences in cost-effectiveness noted here.
Another potential explanation is that states with high rates of cost-effectiveness had low rates of success placing people in the community. That is, agencies had low costs because they did not place many people in the community and therefore did not incur the expenditures associated with job development and training. As with disparities in population served, this possibility merits further examination. However, upon initial examination, this does not appear to be answer either. States with low monthly and hourly supported employment costs, such as Agencies 23, 29, 41, and 44 had relatively high rates of successful employment (i.e., 63.2%, 58.1%, 62.8%, and 58.3%, respectively).
Perhaps other state factors, such as cost of living or degree of urbanization, are the cause of the disparities in effectiveness. It may be that agencies serving more urbanized areas have better public transportation and greater access to entry-level jobs than more rural areas, thus decreasing the costs of the services needed for successful employment. If so, future research will need to identify ways of job developing and providing transportation in rural and economically impoverished areas.
Still, it seems most likely that the greatest influences on cost-effectiveness are the policies that dictate the day-to-day practices of adult service programs as well as the training that adult service providers receive. For instance, it may be that states mandating more rigorous qualifications or training of job coaches experience higher rates of employment and lower costs than states where direct service professionals have high turnover and are frequently replaced with less experienced staff. The same may also be said for agencies focusing greater attention upon the fidelity of programmatic integrity. A great deal of research will need to be conducted to investigate these issues, yet the potential rewards of such research extend well beyond supported employment’s monetary benefits and costs.
In addition to extensive investigations into these areas, discussions among advocates need to address how supported employment programs should be evaluated. In the present study, I used cost-per-month of service and cost-per-hour worked. Other metrics, such as cumulative costs or cost-per-dollar earned might also be considered. Moreover, non-monetary outcomes must also be considered, such as the preferences of individuals with disabilities, their level of happiness, and degree of safety.
Finally, issues of effectiveness and efficiency must be balanced. Agency 52, for example, had an extremely high rate of successful employment (i.e., 100%), which should be applauded. However, they produce these stellar results with some of the highest costs in the country. When should costs trump successful outcomes? A policymaker, not a researcher, must answer this question.
As with any preliminary research, many weaknesses must be keep in mind when considering the data presented here. For instance, 2013 may not have been a typical year for adult service programs thorough the United States. It could be that certain agencies were in the middle of substantial transitions, such changes in leadership or method of funding programs. The results found here might have been influenced by such changes. Longitudinal analyses need to be conducted to insure the rates of cost-effectiveness presented here are part of a larger pattern, and not an aberration.
Second, although data from six sources were utilized, the estimates of sheltered workshop’s hourly and monthly costs used here may not be accurate. Unfortunately, without a national database on facility-based programs, or the willingness of a substantial number of facility-based programs to grant access to their fiscal records, this potential weakness cannot be addressed at this time.
Further, as stated before, this study only examined monetary costs. Non-monetary outcomes of employment (both community- and facility-based), such as increased self-esteem, family satisfaction, and working safe environments, are also critical to consider. Additional research will need to explore them as well.
Finally, this study did not factor out economic factors, such as cost-of-living, or other state characteristics, such as degree of urbanization, that could have influenced an agency’s cost-effectiveness. These issues will be addressed in the proceeding studies.
Conclusions
Glossing over a program’s shortcomings is a disservice to both the implementers of that program as well as the participants for whom the program was developed. Supported employment is a remarkable tool that can enable individuals who are traditionally un-, or under-, employed to experience the many benefits, and tribulations, associated with working in the community. But supported employment is not, nor will it ever be, perfect. Only by examining instances in which supported employment has produced less than stellar outcomes, can supported employment – and by extension, the people it serves – grow and prosper.
Conflict of interest
The author has no conflict of interest to report.
