Abstract
Background
The Vocational Rehabilitation Technical Assistance Center for Quality Employment (VRTAC-QE; H264K200003) is a national technical assistance center funded, beginning in 2020, by the Rehabilitation Services Administration and intended to provide a sustainable national platform for state vocational rehabilitation agencies (SVRAs), their affiliates, persons with disabilities, employers, and policy groups to increase capacity in their ability to improve the quality and impact of vocational rehabilitation (VR) services leading to high quality competitive integrated employment (CIE).
Objective
This program description is designed to provide an overview of the VRTAC-QE and report activities and progress toward its goals and objectives.
Methods
At the conclusion of the fourth year of this five-year project, our purpose in this article is to describe the VRTAC-QE, to update the readership on our progress to-date in meeting our goals and objectives, and to share considerations for the final year of this national technical assistance center.
Results
In this article we describe the structure of the Center, the process of identifying and delivering training and technical assistance (TTA), and summarize the delivery of TTA to date, in terms of the project's scope and in the context of program evaluation metrics.
Conclusion
We conclude the article with a review of lessons learned and next steps as we enter the final year of this project.
Introduction
Participation in employment is a fundamental human right, associated with financial, psychological, social, and health benefits (Bishop, 2020; Chan et al., 2017; Dean et al., 2018; Reichard et al., 2019; Repke & Ipsen, 2020; Tansey et al., 2023a). Work provides opportunities for community participation, access to health care, income, social connection, and engagement in meaningful and productive activity (National Institute on Disability, Independent Living and Rehabilitation Research [NIDILRR], 2019; Tansey et al., 2023b). Conversely, unemployment, income inequality, and poverty, for which PWD are at significantly increased risk, are associated with poor physical and mental health, lower levels of self-esteem and life satisfaction, lack of access to health care, stigma, social isolation, and psychological distress (Bhattarai & Smedema, 2023; Chan et al., 2017; Tansey et al., 2023b). Americans with disabilities have historically and consistently faced substantial barriers to employment participation (Brehmer et al., 2022; Tansey et al., 2023b). Progress in reducing the historic employment gap has been evident since the Great Recession (2007–2009) and continues into the present COVID-19 era. In the past year, both the labor force participation rate and the employment-to-population ratio for Americans with disabilities have reached historic highs (Kessler Foundation and University of New Hampshire, Institute on Disability [NTIDE], 2023). Yet, the relative gap when compared with Americans without disabilities has remained essentially unchanged, with both the labor force participation rate and the employment-to-population ratio remaining roughly double those for PWD (Houtenville & Boege, 2023) and this disparity is greatest among those with the most significant disabilities from traditionally underserved populations (Bishop et al., 2022; USDOL, 2019).
For over a century, the state-federal vocational rehabilitation (VR) program has played an important role in increasing the employment opportunities of Americans with disabilities through the provision of services and supports designed to lead to high-quality employment, independence, self-sufficiency, and full integration (U.S. Department of Education [US DOE], 2020a). Serving more than one million individuals with disabilities each year, the VR program is administered by 78 state vocational rehabilitation agencies (SVRAs) across the 50 US States, the District of Columbia, the Commonwealth of Puerto Rico, American Samoa, Guam, the Commonwealth of the Northern Mariana Islands, and the U.S. Virgin Islands (US DOE, 2022). The SVRAs provide a wide range of VR services to individuals with disabilities in the service of achieving their employment goals. These include career counseling, work-based learning experiences (internships, apprenticeships, and short-term employment), postsecondary and pre-employment transition services, supported employment services, transportation services, and others (US DOE, 2022).
Over the past decade, highly significant health, economic, legislative, and disability and rehabilitation policy events have combined with the enduring fact of lower labor force participation by people with disabilities to challenge the capacity of the state-federal Vocational Rehabilitation Program to fulfill its mission. Among the most impactful were the enactment of the Workforce Innovation and Opportunity Act ([WIOA], 2014) and the global COVID-19 Pandemic.
The specific VR impacts of the former were described in detail by the Department of Education in 2020 (US DOE 2020a, 2020b) and summarized by Iwanaga et al. (2023). WIOA emphasized that addressing the chronic and persistent unemployment and underemployment of Americans with disabilities requires both the identification and the implementation by VR agencies of effective and innovative employment and rehabilitation practices, and evidence-based and research-supported interventions and practices. This need for such innovation was made even more critical in the context of the global COVID-19 pandemic, a public health crisis with an ongoing and still-fluid impact across all sectors of the U.S. labor market.
WIOA-related policy and budgetary changes to the Rehabilitation Act emphasized employer engagement, customized and supported employment, customized training, postsecondary education, pre-employment transition services (pre-ETS), youth with disabilities, interagency collaboration, and accountability (Castruita Rios et al., 2023; Iwanaga et al., 2023; Kim et al., 2023). These emphases coincided with fiscal challenges identified by the US Department of Education in 2020 (US DOE, 2020b), including the inability of some states to provide non-federal matching funds required for the VR program, preventing states from accessing the full federal VR program fund match, and the requirement that VR agencies reserve at least 15% of federal VR program funds for pre-ETS for eligible and potentially eligible students with disabilities. Lack of adequate funding has made it necessary for many states to restrict provision of VR services through the implementation and expansion of orders of selection, resulting in fewer eligible individuals are being served (Iwanaga et al., 2023). Other critical VR issues identified as challenges to the VR program included reductions in the number of applicants, the number of persons with disabilities being determined eligible for services, and the number of participants exiting with employment outcomes (US DOE, 2020b). Further, fewer than one-quarter of VR participants who were enrolled in an education or training program leading to a recognized postsecondary credential or employment were identified as having achieved a measurable skill gain (MSG) or, documented academic, technical, occupational, or other progress that participants have achieved toward a recognized postsecondary credential or employment (US DOE, 2020b).
In response to these and other challenges, in 2020 the US Department of Education posted a notice inviting applications for two technical assistance centers for vocational rehabilitation designed to increase the number and quality of employment outcomes for individuals with disabilities: the Vocational Rehabilitation Technical Assistance Center-Quality Management (VRTAC-QM) and the Vocational Rehabilitation Technical Assistance Center-Quality Employment (VRTAC-QE). Both technical assistance centers were envisioned as assisting SVRAs “in equipping, and increasing the number of, personnel with the necessary skills and training to implement [the] expanded provisions in the Rehabilitation Act made by WIOA” (USDOE, 2020b, p. 19,909). The intention for the VRTAC-QM was to provide training and technical assistance to SVRAs that would better enable VR personnel to manage resources, improve effective service delivery, and increase both the number and quality of employment outcomes for individuals with disabilities (US DOE, 2020a). The purpose of the VRTAC-QE was “to upgrade and increase the competencies, skills, and knowledge of VR personnel to implement and sustain employment strategies and supporting practices that enable individuals with disabilities to achieve quality employment and career advancement, particularly competitive integrated employment (CIE) as defined in the Rehabilitation Act (US DOE, 2020a, p. 46,605).
In 2020, the Rehabilitation Services Administration (RSA) awarded researchers at the University of Wisconsin-Madison, in collaboration with its partners, the contract to conduct the VRTAC-QE for a 5-year period. At the conclusion of its fourth year, given the national importance of the mission of the VRTAC-QE to the state-federal VR Program, the VRTAC-QE takes the opportunity of this milestone to present a summary of the progress made, lessons learned, and our plans going forward. Our purpose in this article is to describe the VRTAC-QE, to update the readership on our progress to-date in meeting our general and specific goals and objectives, and to share considerations for the final year of this national technical assistance center. Specifically, in this article we first describe the format and process of identifying and delivering training and technical assistance (TTA). We then summarize the delivery of TTA to date, in terms of the project's scope and in the context of program evaluation metrics. We conclude with lessons learned and next steps as we approach the final project year.
VRTAC-QE overview
The VRTAC-QE is operationally based at the University of Wisconsin-Madison, and operates in partnerships with several national TTA partners, each having expertise and extensive research, training, and/or service delivery experience in the content areas for which they have technical assistance and training responsibility. The Center employs several SVRA liaisons, each with advanced degrees and professional experience in vocational rehabilitation, who share responsibility for working with individual SVRAs in the development, coordination, and delivery of TTA. VRTAC-QE also has an evaluation team focused on conducting and reporting on the ongoing formative and summative program evaluation, consistent with the evaluation plan approved by the Rehabilitation Services Administration (RSA). Finally, critical to the implementation and continuing activities of the VRTAC-QE is the VRTAC-QE Steering Committee. This external committee serves to advance the conceptualization, implementation, and evaluation of the VRTAC-QE activities toward its overarching goal of increasing the capacity of SVRAs to support persons with disabilities in their efforts to obtain and retain high quality employment.
Project partners
VRTAC-QE partners with a wide range of institutions and organizations, each bringing specialized knowledge and experience to improve employment outcomes for individuals with disabilities. The following identifies the partners and their training foci, and the number of SVRA Intensive Training Agreements in which the listed training topics have been requested and included to date. Virginia Commonwealth University specializes in supported employment (15 ITAs) and customized employment (14 ITAs), subminimum wage issues (2 ITAs), and job coaching (2 ITAs). The University of Texas at El Paso focuses on outreach to Hispanic/Latinx communities (5 ITAs), veterans (2 ITAs), and Asian Americans (1 ITA), addressing the unique challenges these population face in accessing VR services. The University of Illinois Urbana-Champaign (UIUC) contributes expertise in working with persons with justice involvement (2 ITAs), apprenticeships/internships/work based learning (7 ITAs), on-the-job training (1 ITA), job placement (1 ITA), and foster care youth (2 ITAs). Florida Atlantic University provides TTA on self-employment (10 ITAs).
South Carolina State University is dedicated to outreach within African American population (5 ITAs) and faith-based communities (3 ITAs), as well as addressing disaster preparedness (2 ITAs) and counselor bias (2 ITAs), ensuring more equitable access to VR services. Autism Workforce focuses on autism (1 ITA) and business engagement (9 ITAs), bridging the gap between employers and individuals with disabilities. Western Washington University provides outreach to Native American populations (2 ITAs). The University of Wisconsin-Madison provides TTA on career pathways (6 ITAs), motivational interviewing (2 ITAs) and providing crucial support to underserved and rural communities (5 ITAs). The University of Kentucky-Human Development Institute addressed remote service provision (3 ITAs), customized training programs (4 ITAs), transportation issues (4 ITAs), and the impact of COVID-19 on VR services. YolBe and the Council of State Administrators of Vocational Rehabilitation (CSAVR) focus on business outreach and engagement (10 ITAs), ensuring strong employer partnerships to create inclusive workplaces. The University of Wisconsin-Stout Vocational Rehabilitation Institute (SVRI) works on eligibility determination for VR services (3 ITAs), streamlining access for individuals with disabilities. These partnerships play a pivotal role in advancing the quality and accessibility of employment services for individuals with disabilities across the nation.
Models of training and technical assistance
As stipulated by the RSA, the VRTAC-QE provides TTA to SVRAs and their affiliates, persons with disabilities, employers, and policy groups on a broad range of employment strategies through three models of delivery: intensive, targeted, and universal TA. These modes of TTA delivery are described below.
Intensive TA (ITA) is provided to specific SVRAs, primarily on-site, over an extended period, based on an ongoing relationship between the VRTAC-QE and the SVRA under the terms of a signed intensive training and technical assistance agreement. The process of establishing an ITA agreement typically involves the SVRA contacting VRTAC-QE to request TA, followed by a meeting between VRTAC-QE staff and the SVRA leadership to discuss the SVRA's specific training needs. ITA agreements typically involve training in multiple focus areas. The ITA agreement incorporates a VRTAC-QE review of Case Service Report (RSA 911) data and a summary of the SVRAs current status with respect to relevant RSA performance measures, a summary of TTA requested, a schedule for the provision of TTA, developed in coordination with our training partners, and a logic model and evaluation plan. Intensive TTA is provided over an extended period, depending on the needs of the SVRA.
The VRTAC-QE team offers tailored training programs, designed to address the specific operational needs of SVRAs, CRPs and other agencies, ensuring that VR professionals are equipped with the most current evidence-based practices. Through a combination of webinars, workshops, TTA, and individualized consultations, the team facilitates knowledge exchange and problem-solving, encouraging the adoption of innovative approaches that improve service delivery and employment outcomes for individuals with disabilities.
In addition, VRTAC-QE provides critical guidance on regulatory compliance and the integration of best practices, helping affiliate agencies align their service delivery models with federal requirements and emerging trends in the field of VR. The VRTAC-QE team also supports program development, outreach, and the establishment of partnerships with employers, thereby strengthening the capacity of CRPs and non-SVRAs to connect individuals with disabilities to meaningful, quality employment opportunities. By offering specialized TA and fostering cross-agency collaboration, VRTAC-QE enhances the effectiveness of professionals working in and with SVRAs, ensuring that they have the knowledge and resources necessary to deliver high-quality services.
Targeted TTA is provided based on an identified need common to one or more SVRAs, is provided on a time-limited basis and with a more limited commitment of TTA center resources. Targeted TTA is provided through virtual or in-person methods tailored to the identified needs of the participating SVRA and its personnel. The process for establishing and developing a Targeted TTA agreement is similar to that for the ITA agreement. In addition to providing targeted training events on specific topics identified by requesting SVRAs, the VRTAC-QE developed a series of Learning Communities (LCs. The LCs focus on specific topics in which we are providing TTA. The LCs are small (generally 12–24 members) working groups composed of engaged professionals from the general, combined, and blind SVRAs, including counselors, supervisors, and field service directors, as well as employers, members from related agencies, and individuals with disabilities and their family members and advocates. The LCs have been led and coordinated by subject matter experts within the VRTAC-QE training partners. The purpose of the LCs is to exchange ideas with professionals and consumers from around the country to develop effective policy and promote change designed to have a significant, long-term impact on the employment outcomes of individuals with disabilities. The topics of the LCs have included: Career Pathways, Customized Training, Apprenticeship, Self-Employment, Sub-minimum wage, Business Relationships, Military Veterans, and a newly-begun Justice-involved LC. To expand the engagement of SVRAs in targeted technical assistance, the VRTAC-QE continued to develop learning communities through Project Year 4. We anticipate one additional learning community to launch in year 5: Transportation. Participants in each LC work together to develop a Toolkit or similar product for national distribution to promote effective implementation. Each LC has been scheduled to begin and complete their work in a 12-month period. Implementation groups are currently recruiting members interested in learning and applying the steps in implementing the completed LC toolkits with their agencies.
Universal TA (UTA) is broadly available to SVRA personnel and other interested parties through their own initiative, resulting in minimal interaction with VRTAC-QE staff. Universal TTA includes generalized presentations, products, and related activities available through the VRTAC-QE website or through brief contacts with the TTA providers. Over the four years of the project the VRTAC-QE and its partners have developed an extensive library of UTA that is available on the website. This includes recorded national webinars, outreach guides, fact sheets, resource guides, research summaries, research reports, toolkits, and recorded professional conference presentations. These materials are publicly available to interested parties on the VRTAC-QE website (tacqe.com) through a searchable database organized in terms of topic, groups (e.g., VR Counselors, Community Providers, Employers, Veterans), type of resource, and source of content.
The VRTAC-QE UTA includes materials developed by project staff and partner content experts on a wide range of VR and QE-related topics in the form of factsheets, research summaries, and asynchronous 101 (introductory) and 102 (more advanced) video recorded topical trainings. VRTAC-QE provides a monthly live-webinar in which topic experts discuss topics relevant to SVRA personnel, rehabilitation counseling and related professionals, individuals with disabilities and their families, and the public, for which website members can register and attend in real-time and the recorded webinars are then posted on the website. As of this writing, 32 webinars have been provided and the recordings are posted on the VRTAC-QE website. Recent webinar topics include: “Updates to the CRCC Code of Professional Ethics”, “Building Business Partnerships”, “Subminimum Wage”, “Transportation”, “Customized Employment”, “Self-Employment”, and “Remote Career Guidance Services”.
VRTAC-QE has also developed multiple outreach guides describing effective practices in improving VR outreach to underserved communities, including, for example, formerly incarcerated VR clients, African American outreach, Hispanic/Latinx outreach, Native American outreach, Military Veterans outreach, and others. Other available UTA includes the series of QE-relevant articles written and published in a collection of special issues of leading rehabilitation professional journals, and recordings of presentations provided at the annual VRTAC-QE National Symposia on Employment. To date, the VRTAC-QE has developed several hundred UTA resources, and also incorporates resources from other technical assistance centers and relevant agencies on the website. Membership on the website is generally not required to access available UTA, however, membership is free and provides access to the VRTAC-QE weekly email newsletter and resource guide, “TACQE Tuesday” and additional resources. Much of the available UTA is approved for CRCC CEU credits.
Method
Training and technical assistance topic overview
The specific topics on which the VRTAC-QE provides TTA were, to a great extent, determined in collaboration with or through guidance from the RSA, which identified in both the Proposed Priorities, Requirements, and Definitions and Notice Inviting Applications several specific TTA topics in terms of employment strategies (e.g., career pathways; apprenticeships, pre-apprenticeships, and on-the-job training; supported employment, customized employment, customized training and credential programs, self-employment, and business engagement and employer supports), supporting practices (e.g., pre-employment transition services, career counseling techniques and resources including labor market information tools, strategies for involving workforce development partners, community rehabilitation programs, and other community-based organizations), and outreach to underserved populations (US DOE, 2020a, 2020b).
In order to effectively identify additional training and TA topic areas relevant to SVRAs and their affiliates, VRTAC-QE has taken formal steps, at the commencement of the project and in the years since, to identify additional TTA topics of relevance to and based on the perspectives of multiple stakeholders, including VR counselors and other personnel, consumers and potential consumers and their families, employers, cooperating rehabilitation professionals, relevant state and federal agencies, and others. This process has included several elements. First, as described in Bishop et al. (2022) and Tansey et al. (2023a, 2023b), in collaboration with the VRTAC-QM and NTACT-C, VRTAC-QE conducted a series of national surveys and needs assessments, with multiple stakeholder groups. In Project Year (PY) 1, VRTAC-QE also completed a Comprehensive Review focused on identifying the needs, priorities, innovative approaches to develop career readiness and quality employment outcomes, existing literature, emerging practices, and manualized interventions to inform and develop TA programming. Specifically, for this review VRTAC-QE collected RSA Monitoring Reports for SVRAs over the five-year period from 2016–2021. In addition, through Internet searchers and direct outreach to SVRAs, other records, such as comprehensive statewide needs assessments, statewide plans, technical assistance reports, and other state workforce development agency reports were collected. These records were evaluated using qualitative analysis to identify common themes regarding technical assistance needs, innovative practices, and other common factors related to quality employment. The comprehensive review included five years (2016–2021) of published RSA monitoring reports (42), RSA technical assistance reports (6), WIOA State Plans for 78 state VR agencies (84), and 46 Comprehensive Statewide Needs Assessments were also included in the record for a total of 178 reports reviewed to identify areas of technical assistance needed by the SVRAs. Through this process, the VRTAC-QE identified the most important needs for technical assistance in terms of outreach, VR services, and employment-related outcomes.
Currently, based on the procedures and guidance described here, the VRTAC-QE provides TTA across three categories: Employment Outcomes, Employment Services, and Outreach, on the topics identified in Table 1. In terms of frequency of requests for TTA, the most frequently requested employment outcome topics include (1) supported employment (included in 15 ITAs), (2) customized employment (14 ITAs), and (3) self-employment (10 ITAs). In the category of employment services, the most frequently requested TTA topics include (1) business engagement (included in 9 ITAs), apprenticeships/internships/work based learning (7 ITAs), and career pathways (6 ITAs). In terms of Outreach TTA, the most frequently requested topics include (1) business outreach and engagement (included in 10 ITAs), outreach to Hispanic/Latinx communities (5 ITAs), outreach to African American population (5 ITAs), outreach to rural communities (5 ITAs), and outreach to faith-based communities (3 ITAs).
VRTAC-QE Training and technical assistance: categories and topics.
Results
The four-year impact of VRTAC-QE: Program evaluation
In this remainder of this article, we provide information on the four-year impact of the VRTAC-QE on State Vocational Rehabilitation Agencies (SVRA's). The goal of the VRTAC-QE is to increase the number and quality of employment outcomes for individuals with disabilities through training and technical assistance to State VR agency personnel. To accomplish this goal, the project identified six core objectives, described below with a brief discussion of the current impact and status of each.
Objective 1: Development and maintenance of a state-of-the-art, accessible website
As a fundamental component of each element of the VRTAC-QE's mission and objectives, the development of a state-of-the-art website was a primary initial focus. The VRTAC-QE has collaborated with Employment Resource, Inc. (ERI) on the development of the website and ERI has maintained responsibility for ongoing website development and maintenance throughout the project. The website (www.tacqe.com) was launched on March 22, 2021. It provides a centralized space for accessing UTA, requesting intensive or targeted technical assistance, and as a hub for managing the ongoing training provided by the VRTAC-QE and collaborating entities to participating SVRAs and other learning communities. At the conclusion of Project Year 4 there were 12,838 members registered on the website. This includes members from each of the U.S. states and territories. Members can access UTA trainings that provide CRCC continuing education credits. At the time of writing this article, VRTAC-QE had issued 32,671 training certificates, and provided over 23,170 h of CRC Continuing Education to 3,696 users with CRCs. The VRTAC-QE continues to make updates and revisions as we identify areas for improvement and new content.
Objective 2: Conduct a multi-level comprehensive review to identify best practices
Multi-level comprehensive review has been an ongoing process. It was begun in PY1 and has continued through the current project year. This review has included the completion of comprehensive literature reviews and publication of topical summaries, the collection and review of multiple RSA and state reports for each SVRA, and the development and completion of national surveys of needs, barriers, and effective practices. Based on multiple sources and perspectives, this combination of activities was critical in informing and developing the VRTAC-QE TTA. Dissemination of the results across multiple professional and lay outlets has promoted increased awareness of effective VR practices leading to quality employment for individuals with disabilities. Peer-reviewed articles based on the comprehensive review have been published open-access, and links to these articles have been added to the comprehensive review document found on the VRTAC-QE website. In PY4, we have conducted a new needs assessment in collaboration with the National Technical Assistance Center on Transition-Collaborative (NTACT:C). The data have been collected for the needs assessment and are currently being analyzed toward identifying changes in technical assistance needs of SVRAs. A report and peer- reviewed submission on the 2024 Needs Assessment will be published in the coming year.
Objective 3: Provide intensive, targeted, and universal training and TA to state VR agencies, based on their specific needs
Since the initiation of VRTAC-QE, a total of 20 state VR agencies (SVRA) have signed Intensive TA (ITA) agreements. This includes three (3) SVRAs in PY1, 14 in PY2, and 3 in PY3. Another 4 states are in ITA plan development. Seven states have been closed with completed ITAs. ITA was provided to 16 SVRAs during Year 4 of the project through existing or new ITAs. Since the initiation of the project, 4 SVRAs have completed Targeted TA agreements (TTAA). In addition, multiple states have engaged in TTA through our Learning Communities, previously described in this article. Table 2 displays the number of states represented with participants in the completed and ongoing learning communities. In PY4 and 5 implementation groups based on these LCs will conclude, completing the circle of TA development, dissemination, and adoption and implementation.
Number of states represented by participants in VRTAC-QE learning communities.
Territories
VRTAC-QE has undertaken significant initiatives to enhance VR services in U.S. Territories, including American Samoa, Guam, and the Commonwealth of the Northern Mariana Island. A pivotal component of this effort is the establishment of a community of practice, which promotes collaboration and knowledge sharing among VR professionals across these regions. This platform enables participants to exchange best practices, address common challenges, and align local VR efforts with national standards, thereby fostering a more cohesive approach to service delivery. Additionally, VRTAC-QE has facilitated the organization of pre-conference events at the Pacific Rim International Conference on Disability and Diversity, which serves as critical forums for professionals to engage in focused discussion on the specific VR needs and priorities of the territories. To further support these regions, VRTAC-QE has conducted comprehensive policy reviews that ensure local VR agencies comply with federal regulations while begin responsive to the unique economic and cultural contexts of American Samoa, Guam, and CNMI. A particular emphasis has been placed on promoting self-employment opportunities acknowledging the potential for entrepreneurship to provide sustainable livelihoods for individuals with disabilities in these territories. Through tailored, individualized support, VRTAC-QE assists local agencies in developing self-employment programs and resources that are specifically designed to meet the needs of their communities. This strategic approach not only enhances the effectiveness of VR services in these territories but also empowers individuals with disabilities to pursue entrepreneurship as a viable and rewarding career path.
Objective 4: Coordinate training and TA with other TA centers
VRTAC-QE has, throughout the project, actively collaborated with colleagues across other RSA-funded technical assistance centers including (e.g., VRTAC-QM, NTACT:C, AIVRTAC, and the OIB-Training and Research Center for Blindness and Low Vision). VRTAC-QE actively participates in monthly TACC calls with RSA and the other centers. VRTAC-QE regularly collaborates with QM and NTACT:C to address SVRA-specific needs, has engaged the other TACs across multiple conferences and training events, and conducted cooperative needs assessments with VRTAC-QM and NTACT-C.
Objective 5: Disseminate VRTAC-QE summative findings, products (e.g., comprehensive review reports, training manuals), and project activities
VRTAC-QE has continually maximized the dissemination capacity of our primary dissemination source, the VRTAC-QE website through several initiatives. The TACQE Tuesday email series continues to disseminate new products, trainings, and resources available to SVRAs and their affiliates. As a result of these efforts, 2,076 individuals registered for our monthly webinar series in Year 4 alone. We anticipate continued growth in website utilization data as a function of upcoming webinars and the ongoing release of the toolkits created by the LCs and new UTA. We have developed and provided over 395 universal training materials and resources to SVRAs through webinars, training modules, publications and presentations.
The VRTAC-QE National Symposium on Quality Employment held in Madison, WI in 2023 and 2024 (and planned for May 21 and 22, 2025) provides a national opportunity for VRTAC-QE research sharing and dissemination. The VRTAC-QE 2024 Symposium on Quality Employment also served as a dissemination mechanism in Year 4. A total of 2,472 individuals registered for the symposium with 2,170 online participants and 302 in participants. Registrations at the symposium included 195 students. The symposium offered 72 concurrent sessions, 39 poster sessions, 8 roundtable sessions, and four plenary sessions. The VRTAC-QM and NTACT-C collaborated on this event through provision of timely information related to technical assistance activities provided by their respective centers. In short, the scope of the VRTAC-QE dissemination efforts has been extensive and increases annually.
Objective 6: Conduct a rigorous project evaluation that incorporates RSA 911 data to assess the effectiveness of VRTAC-QE employment strategies and supporting activities
The project evaluation team continues to monitor data toward responding to this objective. The team provides evaluation data on all webinar and asynchronous training modules. In addition to RSA 911 metrics, the evaluation team continues to evaluate progress through additional quantitative and qualitative methods on the quality, relevance, usefulness, and impact of TTA on SVRAs’ capacity to support individuals with disabilities as they seek quality employment. This evaluation includes and extends beyond the specific Government Performance Results Act of 1993 (GPRA) measures, performance, and project measures described below, by seeking information on the changes occurring across SVRAs in response to technical assistance and providing a context for the quantitative measures to promote greater understanding and progressive action in promoting sustainability of short-term changes and growth in capacity in these agencies.
Government Performance Results Act of 1993 (GPRA) measures
In addition to the above objectives, the VRTAC-QE responds to several Government Performance Results Act of 1993 (GPRA) measures. The GPRA was enacted to improve government performance management and accountability by requiring agencies to engage in the assessment of performance management. Federally-funded programs are required to define and report performance objectives (Darby & Kinnevy, 2010). The VRTAC-QE addresses several specific GPRA measures. The first four GPRA measures that VRTAC-QE addresses specify the number and percentage of participating SVRAs reporting: (1) improved coordination and collaboration with federal, state, and local organizations as a result of QE training and TA, (2) that the VRTAC-QE training and TA is high in quality, relevant, and useful to the work of the SVRAs, (3) progress in achieving their level of performance for measurable skill gains (MSG) indicators, and (4) the number and percentage change in consumers achieving competitive integrated employment (CIE) compared to the prior year. An additional program measure established specifically for the VRTAC-QE addresses the number and percentage of SVRAs that adopt quality employment strategies and practices as a result of training and technical assistance provided under this grant.
In addition to these GPRA measures, VRTAC-QE identifies several project measures as primary indicators of performance: (1) the number of Intensive TA Agreements signed and in progress during project year, (2) the number of ITA completed (completion of ITA and training provision per ITA agreements signed by the State VR agencies and VRTAC-QE) during the project year); (3) the number of targeted training and TA events on quality employment provided and the numbers of participants during the project year; (4) the number of universal training and TA materials and resources on QE developed by VRTAC-QE and/or its partners available to SVRA personnel through publications, webinars, and VRTAC-QE website during the project year, and (5) the number and percentage of SVRA personnel reporting that the training and TA is high in quality, relevant, and useful to their work. Finally, Project Measure 6 concerns the development, quality, and relevance of the VRTAC-QE website.
As is clear from this discussion, the VRTAC-QE addresses and evaluates multiple project objectives and measures in the process of meeting its broad primary goal of increasing the number and quality of employment outcomes for individuals with disabilities through training and technical assistance to State VR agency personnel. In this article, our objective is to summarize progress in key areas outlined above to provide an accurate perspective on our efforts and impact to date. A detailed, multi-year analysis of each of the measures identified is not possible in the space allotted, however, in the following section, we highlight progress made through Project Year 4. In the discussion thus far, the Performance Measures have been summarized.
In terms of GPRA measure 1, VRTAC-QE uses multiple measures to evaluate whether SVRAs experience improved coordination and collaboration with federal, state, and local organizations as a result of training and TA. First, VRTAC-QE interviews and surveys SVRA leadership and training personnel in SVRAs with which we are working. We also survey ITA and TTA participants on their experiences with collaboration. Using RSA 911 data, we also track the number of referral sources to SVRAs. This is an indirect measure of the target variable based on the premise that increased coordination and collaboration with federal, state, and local organizations will result in an increased number of such agencies making referrals to SVRAs. In PY 4, the vast majority (96.8%) of SVRA leadership and training personnel with active ITA Agreements indicated that their agency has experienced increased coordination or collaboration with other agencies as a result of training and TA through VRTAC-QE. A majority (approximately 60%) of survey respondents who had participated in ITA training indicated that their agency has experienced increased coordination or collaboration with other agencies as a result of VRTAC-QE training and TA. Further, over 80% of Learning Community members agreed that their participation had expanded their agency's collaborations with other agencies and service providers, and almost 80% reported being better able to develop productive collaborations because of their participation. Finally, based on available RSA-911 data from 2020–2023, the number of referrals from agencies coordinating or collaborating with SVRA agencies with ITA agreements uniformly increased in the second year of VRTAC- QE ITA (2022) relative to the prior year, and increased again in 2023. Collectively, these results reflect that the majority of participating SVRAs and trainees report and demonstrate improved coordination and collaboration with federal, state, or local organizations as a result of VRTAC-QE training and TA.
VRTAC-QE has, since the initiation of TTA, continuously evaluated the extent to which its training and TA is high in quality, relevant, and useful to the participants’ work (GPRA Measure 2) by way of participant surveys which include both quantitative and qualitative components. The quantitative component involves ratings on several questions concerning the quality, relevance, and utility, and other elements of the training and the trainers, based on a five-point Likert-type scale, ranging from 1 (“Strongly disagree”) to 5 (“Strongly agree”). Our goal throughout has been that at least 80% of participants agree that the TTA is high quality, relevant, and useful. Through the three years of ITA delivery, 90.94% of participants agreed or strongly agreed with the statement evaluating quality, 89.45% with the statement evaluating relevance, and 87.45% with the statement evaluating usefulness. For Targeted TA, the percentages are: 81.43%, 86.3%, and 87.5% respectively. For Universal TA, the percentages are: 94.5%, 93.74%, and 88.21% respectively. In summary, VRTAC-QE trainings are very positively rated in terms of quality, relevance, and usefulness, and the project is grateful for the excellent service and training and technical support provided by our SVRA liaisons and partners.
In terms of GPRA measure 3, intended to assess the increase in the number and percentage of SVRAs that receive training and TA who achieve their negotiated level of performance for the measurable skill gains (MSG), we considered SVRAs with which VRTAC-QE is engaged in ITA on Career Pathways and/or Customized Training as these trainings are closely associated with achievement of this outcome. Based on available RSA 911 data, four out of the ten (40%) states receiving relevant ITA achieved their Negotiated MSG Rate for PY 2022, and three additional states achieved 99% of the Negotiated Rate; or effectively 70% of participating SVRAs considered under this measure are within 1% of or have achieved their negotiated MSG rate in the most recent Project Year for which data were available.
With respect to the GPRA Measure 4, concerning the extent to which there has been an increase in the number and percentage of consumers achieving an employment outcome, to measure this outcome VRTAC-QE evaluates changes in employment rates based on RSA-911 data element 356 (Employment Outcome at Exit). VRTAC-QE provides RSA with SVRA comparisons of the most recent program year with both (a) the past program year and (b) a rolling average of the previous two-years in terms of both number and percentage. Based on RSA employment rate data for PY 2020–2022 and the available two quarters of 2023 for states with active or previously active ITA and TTA agreements, in the SVRAs with ITA agreements, since 2020 the total number exiting with an employment outcome has increased annually. The percentage exiting employed (relative to total number exiting employed or not employed) has varied, decreasing in 2021 and 2022 relative to the prior year. However, for 2023, based on available data through Q3, both the number and percentage exiting employed have increased. Comparisons of the most recent program year with both (a) the past program year and (b) a two- year rolling average in terms of both number and percentage suggest an overall increase.
A similar trend is evident in the states with TTAA in terms of increasing numbers of VR clients exiting employed, but the relative percentage varies. Although each of the TTAA SVRAs experienced an increase in the number exiting employed relative to the prior year and the average for the prior two-years, only 2 out of the 6 states (33.33%) had a higher employment rate relative to the prior year, and only 2 out of the 6 states (33.33%) had a higher employment rate relative to the two-year rolling average. As most SVRAs with ITA agreements began working with VRTAC-QE in 2021, the full data from FY 2023 and beyond will be more informative in terms of reflecting changes potentially associated with ITA and TTA.
Finally, with respect to the number and percentage of SVRA that adopt quality employment strategies and practices as a result of TTA provided under this grant, the evaluation results consistently support that participating SVRAs report adopting quality employment strategies and practices as a direct result of VRTAC-QE TTA. Specifically, based post-training evaluation surveys of TTA participants in PY4, across all UTA, TTA, and ITA activities, among 2,023 participants 1,777 (87.8%) intended to adopt quality employment strategies and practices as a direct result of training and TA. At 6-months following ITA/TTA, 71.4% of responding participants reported having adopted new strategies or ways of working in response to TTA provided by the VRTAC-QE.
Conclusion
VRTAC-QE lessons learned and next steps
The VRTAC-QE project has learned several key lessons through its work with VR agencies, professionals, and various stakeholders. One of the most significant lessons learned is the need for customized technical assistance to address the diverse needs of the agencies and the populations they serve. A standardized approach often fails to account for the unique challenges faced by different regions, sectors, and disability groups. Instead, targeted interventions, such as individualized training and context-specific policy support, have proven more effective in improving service delivery and employment outcomes for individuals with disabilities. Additionally, fostering strong partnerships, such as those facilitated through communities of practice, has been essential for promoting knowledge sharing and the adoption of innovative practices across the VR field.
The next steps for VRTAC-QE will focus on expanding its support to underserved populations and enhancing its engagement with non-traditional VR partners. This includes deepening collaboration with CRPs, community-based organizations, and other non-state agencies to ensure they are fully equipped to provide high-quality services. Another key area of focus will be strengthening business engagement to improve employment outcomes in emerging fields, such as STEM and remote work, which offer new opportunities for individuals with disabilities. By continuing to refine and tailor its technical assistance, VRTAC-QE aims to address evolving challenges within the VR system, fostering more inclusive and sustainable pathways to quality employment for individuals with disabilities across diverse contexts.
The VRTAC-QE impact and outcomes would not be possible without the contributions of its many partners, including Virginia Commonwealth University, University of Illinois, University of Kentucky, Florida Atlantic University, University of Texas at El Paso, South Carolina State University, the Council of State Administrators of Vocational Rehabilitation and its sister organization the National Employment Team, Autism Workforce, and Yolbe as well as the many individuals on our steering committee that provide critical feedback and direction for the project. The contributions of the experts from these universities have been essential to the implementation and outcomes of the VRTAC-QE project.
Footnotes
Acknowledgements
The authors would like to acknowledge the support and assistance of the RSA and its staff, without whose assistance the study could not have been completed.
Ethics statement
The University of Wisconsin-Madison Education and Social/Behavioral Science Institutional Review Board (IRB) was consulted concerning this project. Because the primary focus of our activities with human subjects involved program evaluation, the project did not require IRB approval.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The contents of this article were developed under a grant, the Vocational Rehabilitation Technical Assistance Center For Quality Employment, H264K200003, from the U.S. Department of Education.
Disclaimer
The contents of this article were developed under a grant, the Vocational Rehabilitation Technical Assistance Center For Quality Employment, H264K200003, from the U.S. Department of Education. However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal government.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.