Personal,environmental,and service-delivery determinants of employment quality for state vocational rehabilitation consumers: A multilevel analysis

2.1 Data sources

The RSA-911 datasets for fiscal year (FY) 2007, FY2008, and FY2009 were used to investigate any patterns underlying variations among state VR agencies in achieving higher employment quality for VR customers with successful closures. In addition, information used to construct state-level economic indicators and VR agency characteristics data were extracted from the national databases provided by RSA, the U.S. Department of Labor, and the U.S. Department of Commerce. The criteria for case selection were that the VR consumers (a) received services at general and combined state VR agencies in the United States (excluding the U.S. territories) and not state VR agencies for the blind; (b) had successful closures with competitive employment as an outcome after receiving VR services; (c) were of working age (between 16 and 64) at the time of application; and (d) had records with no missing data on all variables of interest.

2.2 Outcome variable

Employment quality was the outcome variable for this study and was operationalized based on work hours, earnings, and benefits. Specifically, the variable was constructed based on three standardized quality indicators: (a) hours worked per week (up to 40 hours), (b) weekly earnings, and (c) employer-based health insurance. To have an unbiased estimate of the outcome variable, weekly earnings were adjusted with the weight of state average salary to control for between-state differences in economic conditions and living expenses. Also, hours worked per week were limited to 40 hours to represent the general expectation of full-time employment. Employer-based health insurance was a ditchomous (Yes/No) variable. The value of employment quality was calculated by summing the three Z-scores of the adjusted number of working hours per week, adjusted weekly earnings, and employer-based health insurance. A higher score of employment quality refers to a better quality job that a consumer had after receiving VR services. Specifically, a positive value of employment quality indicates that someone has better employment quality than the average VR consumer with a successful closure, whereas a negative value indicates employment quality lower than the VR national average based on the three quality indicators used in this study. The employment quality scores in this study ranged from –5.2 to 41.4 with M = 0 and SD = 2.4 for all three fiscal years. Less than 1% of the participants had an employment quality score above 6.0. In terms of state variations in employment quality, the range of state average scores was from –1.1 to 1.5. The distributions ofmeans and standard deviations in state average employment quality are presented in Appendix A.

2.3 Predictor variables

This study used the ICF framework and assumed that the employment quality outcome variable can be attributed to two (or more) levels of predictor variables: individual (Level-1 or micro) and environment (Level-2 or macro). Individual factors consist of variables related to consumer characteristics and types of VR services received. Environmental factors include variables pertaining to state-level economic conditions or VR policy and agency characteristics. All Level-1 and Level-2 variables of interest were collected from 2007 to 2009 and describedbelow.

2.4 Data analysis

Having successful VR closure resulting in quality employment not only depends on an individual’s efforts but may also be affected by the macro environment (e.g., national economic development, VR policies, and societal attitudes toward disability). Therefore, people with disabilities will share the same organizational and environmental benefits and risks if they are served by the same state VR agency or live in the same state. Multilevel analysis or hierarchical linear modeling (HLM) is appropriate and suitable for research designs where the data for participants are organized (nested) at more than one level (Tabachnick & Fidell, 2007). With multilevel linear modeling, each of the levels in the structure is formally represented by its own sub-model. These sub-models express relationships among variables within a given level, and specify how variables at one level influence relations occurring at another (Raudenbush & Bryk, 2002). In this study, employment quality was measured for VR consumers at the individual level (Level-1), and they (VR consumers), in turn, were grouped within states (Level-2). Three stages of multilevel modeling were utilized to investigate how personal characteristics, VR services, and environmental factors, either separately or interactively, predict employment quality of VR consumers with successful closure (see Table 1).

Stage 1. To examine the “pure” state environmental effect (i.e., the general state effect), only the outcome variable was entered in Model 1 and all selected participants were clustered by their state identification to explore the within-state as well as between-state difference (or similarity) in employment quality. Between-state difference indicates the possibility of having different employment quality for an “average/typical” consumer receiving the same VR services but in different states. The intraclass coefficient ρ was computed to determine the amount of variance in employment quality that could be accounted for by between-state differences. For this estimate, the equation ρ=τ00²/(σ² +τ00²) was used, where σ² is the value for Level-1 (within-state) variance and τ00² is the value for Level-2 (between-state) variance. A strong pure environmental effect would suggest that VR employment quality can be largely determined by the state from which a person receives VR services.

Stage 2. At this stage, the impact of personal factors and state VR service factors on employment quality were sequentially tested. Model 2 estimated the extent to which individual characteristics (i.e., age, gender, race/ethnicity, types of disability, and pre-service education level) determine employment quality based on the decrease or increase in the percentage of Level-1 variance σ². To what extent personal characteristics influenced between-state variation can also be assessed (i.e., the decrease/increase in the percentage of Level-2 variance τ00²). Next, state VR service variables were added to Model 3 to test the influence of state VR services on employment quality after controlling for personal factors.

Stage 3. At this stage, state-level environmental variables pertaining to state economic environment and state VR agencies were separately added to the models to examine their ability to predict employment quality after controlling for the personal and VR service (Level-1) factors. The magnitude of the decrease in the percentage of Level-2 variance (τ00²) was used to evaluate how strongly a specific environmental factor explained between-state variation in employment quality. Before adding to the models, all Level-2 variables were centered with grand means to have better interpretability of results.

3.1 Sample characteristics

The total numbers of participants were 180,641 in FY2007; 179,692 in FY2008; and 157,330 in FY2009 (see Table 2).

Data show a substantial drop in the number of successful closures in 2009, whereas the numbers of applicants from 2007 to 2009 remained relatively unchanged. The three annual samples had similar distributions of sample characteristics, supporting the homogeneity assumption needed to make fair cross-year comparisons. The majority of participants were male, Caucasian, had a high school degree or higher, and had a significant disability. The sensory disability group was smaller than the other three disability groups. Participants aged 16–20 years old represented the largest age group (24–25%), followed by those 41–50 years old (20–23%); consumers 51–64 years old constituted the smallest of the age groups (16–18%). About two-thirds of participants did not receive any public monetary or medical benefits. In terms of VR services, the service patterns were generally stable (see Table 3); participants were likely to receive approximately two counseling/assessment-related services and one service from each of the other three service categories (i.e., training, job placement, and support services). The rehabilitation process took VR consumers on average two years and cost about $4,800 to achieve successful competitive employment.

3.2 Between-state differences

In Model 1 (see Table 4), the estimated pure state-environmental effect (i.e., between-state difference) accounts for 7% of the variance in employment quality [ρ=τ₀₀²/(σ²+τ₀₀²)  ₌  6.7% in 2007; 7.2% for 2008; and 7.1% for 2009]. Results suggest thatreceiving VR services in different states explains about 7% of the variation in employment quality for the average person with a disability.

3.3 Personal and service (Level-1) factors

Personal factors and VR service factors were sequentially added to Models 2 and 3. As can be observed from Table 4, all results across the threeyears were quite similar. Compared to Model 1, the inclusion of personal factors (Model 2) not only substantially decreased the within-state/individual variation (20% of Level-1 variance) but also decreased the between-state/environmental difference in employment quality (50% of Level-2 variance). These results suggest that the compositions of consumers with successful employment outcomes among all states varied in personal characteristics, which, in turn, strongly influenced their likelihood of having quality employment. After controlling for personal factors, VR service factors (Model 3) contributed to another 5% decrease in the within-state variation of employment quality but were unable to decrease any between-state variation. Therefore, the effectiveness of VR services has been supported at the individual level. The inability of VR services to influence between-state variation suggests that different state VR agencies are providing similar and tailored VR services to consumers with the same demographic and disability characteristics (Chiu et al., 2014; Strauser et al., 2010).

Specifically, disability characteristics and disability benefits were dominant predictors of employment quality among personal factors for Model 2 in Table 5. The significant predictors included gender (in favor of men), education, disability type, significance of disability (in favor of non-significant disability), and the receipt of cash or medical benefits. Rehabilitation consumers were more likely to find good jobs with benefits if they had higher educational levels. People with sensory impairments had the highest employment quality, followed by those with physical impairments and mental health impairments, whereas individuals with cognitive impairments had the lowest employment quality scores. Moreover, age was a statistically but not clinically significant predictor of employment quality due to the negligible effect size. African Americans and Asians had slightly lower employment quality outcomes compared to Caucasians; nonetheless, race/ethnicity, in general, was not a sufficient predictor of employment quality.

Model 3 demonstrates the predictive strengths of VR service variables (see Table 5). All six VR service-related variables were significant predictors of employment quality but only supported employment, change in educational attainment, and job placement services had effect sizes large enough to influence employment quality. Findings indicate that people receiving supported employment, job placement services, or both are more likely to have lower employment quality. In contrast, people with post-service higher levels of educational attainment tend to have better quality jobs. Additionally, the inclusion of VR service variables successfully decreased most of the effect sizes of those significant disability and benefits related predictors (through the comparison of the results in Model 2 and Model 3). VR services interacted with most disability types, some race/ethnic groups (i.e., Asian/Pacific Islanders), and the recipients of cash or medical benefits to affect employment quality.

3.4 State environmental (Level-2) factors

To find potentially useful environmental factors, each of the 11 state environmental variables was added to a new model to evaluate the change in the value of between-state variance (τ₀₀²), after controlling for both personal and VR service (Level-1) factors. As can be seen in Table 4, Social Security beneficiary rate was identified as the best environmental factor, with an additional 38% drop in between-state (Level-2) variance (τ₀₀²) in FY 2007, 19% for FY 2008, and 30% for FY 2009 (Model 7 vs. Model 3). Among other environmental factors, per capita income decreased the between-state variance in employment quality by 15–19% (Model 6 vs. Model 3) across FY2007–2009. Urban population rate was also able to explain about 10% of the between-state variation. State unemployment, significant disability, and administration cost rates also decreased the state-level variation in employment quality but the magnitudes were small.

Overall, the three state economic factors included in the study all helped to explain the between-state differences in employment quality across the three years, but some effects were small. Of the VR system factors, only Social Security beneficiary rate, significant disability rate, and administration cost contributed to the decreased between-state difference in employment quality.

3.5 Person-environment interaction model of employment quality: An example

To present the interaction of Level-1 and Level-2 factors on employment quality, Model 6 was used to demonstrate how state per capita income moderates the relationship between personal/VR service factors and employment quality. Across the three years, the state per capita income variable changed the extentto which employment quality was associated with a number of individual Level-1 variables, including: gender, pre-service educational level, Latino/Hispanic, mental illness, significant disability, cash benefits, support services, and educational change.

Specifically, people living in states with higher average personal income tend to experience less gender and educational disparity in employment quality and vice versa. The correlation coefficient of gender on employment quality would drop 0.01 value for every $1,000 increase in state’s average personal income above the national average. Likewise, the discrepancy in employment quality between physical impairments and mental impairments becomes smaller as a state’s average personal income exceeds the national average. Also, public cash beneficiaries living in states with better economic conditions were found to score higher on the employment quality indicator compared to those who living in poorer states. Lastly, all the χ² tests of the models in this study reached statistical significance, suggesting there is need to explore of other additional Level-1 andLevel-2 factors, especially the former.

4.1 Implications for VR practice

According to the WHO (1995), the greatest cause of ill-health, suffering, and mortality in the world is poverty. Since the Great Recession, the gulf between the poor and the rich of the world is widening. In the United States, incomes for the highest-earning one percent of Americans skyrocketed 31 percent between 2009 and 2012, while the rest of the country grew an average of just 0.4 percent (Saez, 2013). Poverty and income inequality have direct and indirect effects on the social, mental, and physical well-being of individuals with and without disabilities. The WIOA is a bipartisan, bicameral bill to improve America’s workforce development system and has an emphasis on job-driven training and helping Americans find good paying jobs with benefits as a pathway back to the middle class.

According to a study by the U.S. Census Bureau, the impact of education levels on work-life earnings surpasses that of all other demographic factors (Julian & Kominski, 2011). In addition, the median yearly earnings (in constant 2012 dollars) was $46,900 for young adults with a bachelor’s degree, $30,000 for those with a high school credential, and $22,900 for those lacking a high school credential (Kena et al., 2014). Given the demonstrated effect of postsecondary educational intervention in the present study, counselors in state VR agencies must pay attention to the benefits of education on improving the employment quality of their consumers who have the preparation and potential for higher education. However, postsecondary education represents an expensive investment. To assure return on this investment, counselors must make appropriate arrangement to provide postsecondary education interventions to help consumers with college life adjustment and college persistence. VR services were found to decrease the disadvantage experienced by consumers with mental illness and cognitive impairments. Counselors should continue to integrate evidence-based practices (e.g., transition services, supported employment, and postsecondary education) that could improve the employment outcomes and employment quality for these subpopulations of people with disabilities who had the lowest levels of work participation. Additionally, men were found to have higher employment quality outcomes after receiving VR services than their female counterparts. This finding may reflect the larger societal issue that women in general still make less than men after controlling for age, race, hours and education (Goldin, 2014). However, counselors may want to provide job search/job readiness interventions that can help improve women consumers’ confidence and negotiation skills in the job placement process. Finally, counselors, researchers, and policy makers should be aware of the potential impact macro-level variables may have on the relationships between employment quality and consumer and VR service characteristics. For example, findings from this study suggest that higher state per capita income can mitigate gender and disability disparities, whereas these disparities may be exacerbated in states with poorer overall economic conditions.

4.2 Limitations

There are several limitations of the present study that should be considered. First, missing data threaten the internal and external validity of results. The RSA-911 datasets are generated from information recorded by counselors, and it is possible for some unexpected manual or memory errors (i.e., recall bias) to occur during the retrospective data-entry procedure. The RSA has developed 18 cross-checks to reduce potential errors. However, some states (e.g., New Jersey, Arizona) still had large amounts of missing data for specific variables, such as employer-based health insurance, or Social Security disability beneficiary status. To guard against threats to internal validity, the missing data in employer-based health insurance were replaced with 0 (no insurance). Therefore, caution is required when interpreting the state-level comparisons reported in the results. Moreover, owing to the eligibility determination criteria used by state VR agencies and the cross-sectional nature of RSA-911 datasets, the findings cannot be generalized to all people with disabilities and causality cannot be inferred.

Another limitation of this study is that VR services were grouped into four broad categories in the present study and did not consider the association of individual services with employment quality. Therefore, further research is needed to identify evidence-based VR services that directly increase employment quality for VR consumers. In addition, this study was limited to the personal and environmental variables included in the datasets. Future research is needed to more comprehensively evaluate the impact of additional ICF-defined envirionmental variables, such as those relate to the local environment (e.g., county level) and counselor/service provider characteristics.

Finally, a rudimentary measurement model of employment quality was used for this study. The focus on full-time employment, earnings, and employer-based health insurance may ignore other health and mental health benefits of employment in integrated settings. This measure may not be optimal for all VR consumers. Therefore, future research should focus on developing and validating measurement models of employment quality integrating both objective indicators and subjective measures that can be tailored to different disability groups. Despite these limitations, the preliminary results provided by the present study provide insightful information to guide future study designs that may better capture the interaction of different combinations of Level-1 and Level-2 factors.