The youth transition demonstration project in Miami,Florida: Design,implementation,and three-year impacts

2.1 The YTD program model at BHBF

The development of the YTD program model drew on the framework presented in Guideposts for Success, developed by the National Collaborative on Workforce and Disability for Youth (2005 & 2009). Guideposts is based on a comprehensive review of research, demonstration projects, and recognized effective practices. It is intended to help practitioners and policymakers conceptualize optimum service delivery for youth with disabilities.

The YTD program model had six components. Most of them were identified in Guideposts, although they were customized to meet the needs of the YTD target population (Luecking & Wittenburg, 2009). Foremost among them were individualized work-based experiences, including worksite tours, subsidized jobs, and, most notably, competitive paid employment in integrated settings. A youth empowerment component enabled youth to acquire the skills and knowledge they needed to chart their own courses and advocate for themselves. YTD fostered empowerment by engaging youth in intensive planning that focused on education, employment, health care, and independent living. Family supports included family-focused training, support for parent networking, and provision of transition-related information. YTD also facilitated system linkages, characterized by connections with service providers that youth may need to access health care, education programs, transportation, and accommodations and assistive technologies for education and employment. SSA’s waivers for YTD made paid employment more rewarding and provided for the temporary continuation of disability payments despite a negative continuing disability review or age-18 medical redetermination. Benefits counseling, which helped youth and parents understand the waivers and the standard SSA program rules, was also central to the YTD model. BHBF implemented all six components.

Two other noteworthy features of YTD were (1) intensive programmatic TA provided by an independent organization and (2) adoption of the Efforts-to-Outcomes management information system. TransCen delivered hands-on assistance and training that helped the staff of the YTD projects maintain the interventions’ fidelity to the program model. Within TransCen, a single individual was the primary point of contact with BHBF. Project staff used Efforts-to-Outcomes to record their delivery of services to individual youth and to record any key outcomes those youth achieved, such as completing an assessment or getting a job.

2.2 Evaluation enrollees

SSA contracted with Mathematica Policy Research to evaluate the YTD projects, including BHBF. At six-month intervals, SSA gave Mathematica lists of youth disability recipients in Miami-Dade County. On a rolling basis, Mathematica identified youth on the lists who were in BHBF’s target age range of 16 to 22 and randomly selected 5,573 of them for recruitment into the study (Table 1). The recruitment process began in March 2008 and ran through September 2010, when the target number of youth (880) had completed a baseline interview and provided written affirmative consent (parents or guardians provided consent for youth who were minors) to enroll in the evaluation.

Mathematica randomly assigned 460 of the 880 consenting youth to a treatment group whose members were eligible to enroll in BHBF, and randomly assigned 399 to a control group (Table 1). The remaining 21 youth had siblings who had enrolled in the evaluation already; Mathematica purposefully assigned them to the same groups as their siblings and excluded them from the research sample. Eighty-four percent (388) of the 460 randomly assigned treatment group members met with BHBF representatives, signed project enrollment forms, and thus became BHBF participants.

An examination of the baseline characteristics available in SSA administrative records revealed a number of differences between the 880 youth who enrolled in the evaluation of BHBF and the 4,693 youth who were recruited by Mathematica but did not enroll (Table 2). Relative to non-enrollees, youth who enrolled in the evaluation were 3 percentage points less likely to be male; two-tenths of a year younger, on average; 4 percentage points more likely to have a parent or other relative as a representative payee for disability payments; and 6 percentage points more likely to have a cognitive or developmental disability as the primary disabling condition. All of these differences are statistically significant (p < 0.10).

To ensure the intervention’s fidelity to the program model, designers incorporated into YTD the key feature of a continuum of TA and training intended to build the projects’ capacity to effectively implement the program model. The TA was intense, involved a range of delivery modes, and was informed by data from the projects’ management information systems and by emerging findings from the YTD evaluation. Each YTD project was assigned a TA liaison from TransCen. The liaisons were trained and experienced in delivering training and assistance in the implementation of evidence-based practices in secondary transitions. The TA liaison to BHBF had a doctorate in vocational special needs education and was a senior research associate at TransCen. The liaisons gave the YTD projects general support and helped them resolve specific implementationissues.

The design for BHBF’s TA had three basic approaches. First, the TA liaison worked closely with senior managers of the project to develop a service plan consistent with the YTD program model. The liaison visited the project several times during its first months of operation to ensure the plan was being implemented effectively. Second, the TA liaison delivered in-person TA and training to BHBF frontline staff. Early on, this consisted of group trainings on topics such as strategies for engaging families, coordinating with external service providers, and networking with employers. As the project matured, the in-person TA shifted to one-on-one troubleshooting by the liaison with individual frontline staff. For example, the TA liaison accompanied BHBF employment specialists while they were developing jobs with employers and meeting with youth to identify their employment goals. Third, the TA liaison provided just-in-time advice and guidance to BHBF frontline staff to help them quickly resolve issues that typically concerned individual project participants or specific employers.

The quality and intensity of the TA provided to the YTD projects would be difficult to replicate quickly on a broad scale, because there are few professionals with the training and experience to deliver this kind of TA. For this reason, SSA considered YTD a “proof of concept” demonstration. As such, its objectives were to (1) establish whether transition projects could be implemented with both a high degree of fidelity to best practices in serving youth with disabilities and a strong and consistent focus on services designed to promote their employment prospects; and (2) rigorously assess whether such projects could improve employment and other outcomes for those youth in the short run (one year) and medium run (three years). SSA did not expect that positive findings from the YTD evaluation would lead quickly to the wide-scale rollout of YTD-like interventions for youth with disabilities. Instead, the agency anticipated that positive findings would lead to follow-on demonstrations and research on whether effective intervention projects could be implemented on a broader scale, possibly with more limited TA resources.

2.4 Design for the implementation analysis

The implementation analysis component of the BHBF evaluation was intended to both measure specific facets of project implementation and gauge the need for and application of advice, assistance, and training to address fidelity to the program model. The analysis examined whether the BHBF intervention included all of the core components of the program model, particularly focusing on the work-basedexperiences. The study’s various sources of qualitative and quantitative data ensured that the implementation analysis would reflect multiple perspectives on these key issues.

The qualitative data for the implementation analysis consisted primarily of information collected on site visits to Miami-Dade County and in telephone calls with BHBF management staff. The evaluation team conducted four site visits: one to support an early assessment of the implementation of BHBF, one to underscore the need for the project’s management and frontline staff to emphasize employment services and outcomes, and two to systematically gather data for assessing project operations. The first two visits included reviews of case files for selected project participants and an assessment of how completely and accurately the services delivered to participating youth were recorded. The last two visits included interviews with BHBF staff at ServiceSource and representatives of partner organizations, observations of the delivery of project services, and focus group discussions with participating youth and their parents. The evaluation team also conducted biweekly conference calls with project management and reviewed project documents, such as monthly management reports.

The quantitative data for the implementation analysis came from BHBF’s management information system. We used those data to answer questions about the efforts of BHBF staff to engage youth in project services and the types and amounts of BHBF services delivered to project participants. Our analysis of service delivery focused on the 388 treatment group youth who had completed the paperwork necessary for them to be classified as BHBF participants; the analysis was based on data for the first 15 months after those youth enrolled in the evaluation. Four service categories are used in the analysis: (1) employment services; (2) benefits planning; (3) education services; and (4) case management, encompassing person-centered planning as well as general case management.

2.5 Design for the impact analysis

The impact analysis component of the BHBF evaluation was based on a randomized controlled trial design. Mathematica used a computer algorithm to randomly assign the youth who had agreed to be in the study to either a treatment or control group (Fraker & Rangarajan, 2009). The treatment group members were eligible for both BHBF services and SSA’s waivers for YTD. The control group members could access whatever non-BHBF services happened to be available in Miami-Dade County and were subject to SSA’s standard program rules; they were not eligible for the waivers. Because of random assignment, the two groups were expected to be equivalent, on average, at the beginning of the study. Consequently, any differences in their aggregate outcomes could be attributed to BHBF. The evaluation tracked employment, earnings, disability payments, and other outcomes for the youth for three years. It compared the aggregate outcomes of the treatment and control groups to assess whether BHBF helped youth secure paid employment, increase their incomes, and achieve other transition objectives.

Surveys conducted as part of the BHBF evaluation, along with SSA administrative records, were the data sources for the impact analysis. As part of the enrollment process, Mathematica conducted a baseline survey of youth, collecting data on demographic characteristics, personal and family background, work experience, and attitudes and expectations. Administrative files of the Internal Revenue Service and SSA provided additional baseline data on employer-reported earnings in the calendar year preceding enrollment and on monthly disability payment amounts. Mathematica also surveyed youth at one year and three years after enrollment, achieving response rates of 86.8 percent and 81.5 percent, respectively (Table 1). The follow-up surveys gathered information on the receipt of services, education, employment, earnings, and contact with the justice system. For all enrolled youth, not just those who responded to the follow-up surveys, SSA administrative files were the source of post-enrollment data on monthly disability paymentamounts.

Random assignment worked as expected—youth in the treatment and control groups had statistically equivalent baseline characteristics. We tested for treatment-control differences in 50 baseline characteristics, using chi-square tests for categorical measures and t-tests for continuous measures. In Table 3, we provide the test results for a subset of the baseline characteristics. For 44 of the characteristics, the tests revealed no statistically significant differences between the treatment and control groups. For example, the table shows that treatment group youth were 3.4 percentage points less likely to be male than control group youth were, and one-tenth of a year older, on average; however, neither of these differences is statistically significant at the 0.10 level. For six of the baseline characteristics (job training in the last year; volunteer work in the last year; mother is a high school graduate; father is employed; use of reading, hearing, speaking, or walking aids; and the disability payment amount in the previous 12 months), the tests revealed statistically significant differences (p < 0.10). This is about the number of significant differences we would expect on the basis of random chance in the absence of systematic differences between the two groups.

We used multivariate statistical models to estimate the impacts of BHBF at one year and three years after youth enrolled in the evaluation. Within the models, we used data from the baseline survey and administrative files to control for differences between the treatment and control groups that arose by chance during random assignment, and to improve the precision of the impact estimates.

3.1 Implementation challenges and a mid-course correction

TransCen’s TA liaison to BHBF visited the project several times during its first year of operation, and those visits, along with findings from tabulating data on services delivered during that year, revealed that the project’s frontline staff were devoting much of their effort to general case management instead of working with employers to develop jobs and helping participants prepare for and obtain work experiences. Many of the participants were not being placed in jobs, nor were they receiving services to quickly move them toward that goal.

Based on this information, the TA liaison concluded that the project’s frontline staff needed to refocus their effort on giving participants employment-related services and placing them in jobs. The TA liaison adjusted the approach to delivering TA to make it more employment-centered, field-based (as opposed to being delivered primarily in workshops and webinars), and delivered in real time directly to frontline staff. The BHBF management team supported these adjustments, thus expediting their implementation and enhancing their effectiveness. In some instances, the TA liaison accompanied project staff in making initial contacts with employers, meeting with participants to discover their skills and interests, and negotiating work experiences for participants with employers. This allowed the liaison to model best practices and observe staff interactions with employers and youth. The TA liaison gave immediate feedback to the staff so they could hone their techniques. Also, the liaison led biweekly case reviews focused on individual participants who were especially challenging to serve. Those reviews allowed the TA liaison to guide the staff in thinking about employment options for those youth and in exploring ways to address the challenges involved in serving them. In short, the TA liaison coached the BHBF staff to correctly and effectively deliver the key component of the program model, work experience.

While TA was being adjusted in the spring and early summer of 2009 (12 to 15 months after the start of project services), the management of BHBF established numeric goals for staff contacts with employers and placements of participants in jobs. These goals were specified both for the project as a whole and for individual frontline staff. Equally important was the systematic quantitative monitoring of progress toward the goals. The Mathematica evaluation team provided BHBF management and the TA liaison with monthly monitoring reports, based on data from the project’s management information system. These monthly reports, whose results were broken down by individual staff member, showed: (1) the employment status of each assigned BHBF participant, (2) the number of initial and follow-up employer contacts, and (3) the number of staff hours by service activity. These reports provided snapshots of how the project was performing relative to its established goals and documented the efforts and participant outcomes for individual frontline staff. The TA liaison and project managers reviewed these reports at a monthly meeting. The TA liaison could use their hard data to adjust training and support in response to emerging issues, and they helped guide project staff in focusing their efforts on job placements.

When the monthly monitoring reports revealed low numbers of employer contacts and job placements for certain staff members, the BHBF management team and the TA liaison worked with those staff to help them improve their delivery of the intervention. In some cases, project management had to replace staff with new employees who were more at ease networking with employers and helping youth find jobs.

3.2 Delivery of BHBF services

The BHBF program model emphasized rapidly engaging the treatment group youth who had agreed to participate in the project. Findings from the evaluation’s implementation analysis show that the project achieved this objective. Our analysis of data from the project’s management information system revealed that the median elapsed time between the onset of BHBF participation and the first contact by project staff for delivery of services was four days. Within 30 days, 98 percent of BHBF participants had received a first service contact, and 90 percent had received a second service contact. The services received most often at the first contact were benefits planning (58 percent) and case management (37 percent). In contrast, the services received most often during the last contact were employment services (48 percent) and benefits planning (38 percent).

The intensity of BHBF’s services (of any type) to participating youth was high, whether measured by the number of service contacts or their cumulative duration. These findings are based on data for each service contact, as recorded by project staff. On average, project staff made 48.5 service contacts per participant, lasting a total of 28.5 hours (Table 4). Some of those contacts were with employers and other individuals or organizations on behalf of specific participants. The average cumulative duration of service contacts that directly involved participants was 18 hours (results not shown in the table). The average duration of a single service contact was 26.8 minutes, whereas the median duration was 15.0 minutes, indicating that some contacts were lengthy but many were brief, such as a telephone call to remind a participant of a job interview. Only 17.6 percent of service contacts exceeded 30 minutes in length.

Consistent with the YTD and the BHBF program models, employment services (such as career exploration, job development, job search assistance, and job coaching) were delivered with more intensity than any other type of service. Virtually all BHBF participants received employment services (Table 4) and, on average, BHBF staff made 20 contacts per participant to deliver those services, for a cumulative duration of 13.9 hours. The median cumulative duration of the contacts was about half the average duration, indicating that a few participants received an especially large number of employment service hours. Benefits planning services were also relatively intense in BHBF. Again, virtually all participants received benefits planning services and, on average, project staff made 15.3 contacts per participant to deliver them, for a cumulative duration of 7.9 hours. In contrast, the intensity of case management services and education services was relatively low. Among the 95.9 percent of participants to whom project staff provided case management, the average number of service contacts was 9.4, and the average cumulative duration of those contacts was 3.5 hours. The corresponding statistics for the 83.5 percent of participants to whom BHBF staff delivered education services are 4.0 service contacts for a cumulative duration of 2.1 hours.

During the year after they enrolled in the evaluation, treatment group youth were more likely than their counterparts in the control group to receive services of any type from any source, including but not limited to BHBF. They were also more likely to receive services specifically designed to promote employment, such as career counseling, help preparing resumes, placement in apprenticeships, job search assistance, and counseling on SSA benefits and work incentives. Table 5 shows results from the evaluation’s one-year follow-up survey; 80.5 percent of treatment group youth reported they had received any services, and 58.2 percent reported they had received employment services. These rates reflect statistically significant (p < 0.01) BHBF impacts of 9.9 and 12.5 percentage points, respectively, relative to the rates for control group youth.

Note that the data on service receipt by evaluation enrollees (underlying Table 5) are fundamentally different from the data on service delivery by BHBF staff (underlying Table 4). The Table 4 data pertain only to the 388 treatment group youth who participated in BHBF. BHBF staff entered those data into the project’s management information system as they delivered services to those youth. The Table 5 data are based on the 738 treatment and control group youth who responded to the evaluation’s one-year follow-up survey (as documented in Table 1). The youth reported those data, and they included services they received from all sources, not just BHBF.

Because work-based experiences are a key component of the YTD program model, a critical short-term indicator of BHBF services’ strength is whether treatment group youth were more likely than control group youth to have been employed during the recall period for the one-year follow-up survey, which largely overlapped with the time when they were receiving project services. Table 5 shows that treatment group youth were 9.4 percentage points more likely than control group youth (p < 0.01) to have worked in paid or unpaid jobs at some time during the year after they enrolled in the evaluation. BHBF had an identical impact on employment for pay. Among the treatment group youth who worked for pay, about half had hourly earnings of between $7 and $9 in 2008 dollars, slightly more than one-quarter earned less than $7 per hour, and slightly fewer than one-quarter earned more than $9 per hour (results not shown in the table).

BHBF had statistically significant beneficial impacts on a number of outcomes for youth during the third year after they enrolled in the evaluation, which was after BHBF services ended. Most notably, the rate of paid employment at any time during the year was 7.8 percentage points higher (p < 0.05) for treatment group youth than for control group youth (Table 6). This was accompanied by an impact of $615 on mean earnings over the course of the year (p < 0.05). These findings are based on self-reports by the 685 treatment and control group members of the research sample who responded to the evaluation’s three-year follow-up survey (Table 1).

Even though treatment group youth had higher earnings than their control group counterparts, their disability payments were also higher; BHBF had a statistically significant (p < 0.01) impact of $598 on the mean yearly disability payment amount. This difference likely exists because of SSA’s waivers for YTD, which allowed some BHBF participants to continue receiving SSI payments that otherwise would have been discontinued, and to keep more of their payments than would otherwise have been possible given their earnings. Because BHBF increased both earnings and disability payments, it is not surprising that the project had a statistically significant (p < 0.01) impact of $1,266 on the mean income of youth during the third year after they enrolled in the evaluation, with income defined as the sum of earnings and disability payments.

BHBF had no impact on youth’s participation in education or training programs during the third year following their enrollment in the evaluation. However, it did increase participation in a broader set of productive activities that included paid and unpaid employment in addition to education and training. Treatment group youth were 8.4 percentage points more likely than control group youth to have participated in one or more of these productive activities during the third post-enrollment year (p < 0.05).

The project had a beneficial impact on contact by youth with the justice system. Less than 1 percent of treatment group youth reported in the evaluation’s three-year follow-up survey that they had been arrested or charged with delinquency or a criminal complaint during the previous year. This is a statistically significant (p < 0.05) 2.7 percentage points lower than the reported arrest/charge rate for control group youth.

4.1 Limitations and implications for future research

As we discussed in Section 2.3, BHBF was implemented with ample amounts of expert TA for frontline staff, and the experts who provided the TA concentrated on helping those staff deliver employment-focused transition services with a high degree of fidelity to the YTD program model. The beneficial impacts of BHBF as revealed by our evaluation, although modest in size, document what a program like this can accomplish under favorable circumstances—that is, when high-quality TA is readily available. These findings suggest a need for more investment in developing TA resources and in implementing and evaluating demonstrations that may be less intensive in their use of those resources than the YTD projects were. Indeed, motivated in part by early findings from the YTD evaluation, SSA undertook a nine-year evaluation of PROMISE demonstration projects in 2013. With funding from the U.S. Department of Education (2013a, b), these projects are providing education-and-employment-focused transition services to 6,000 SSI youth ages 14 through 16 in 11 states. These projects are, in general, less intensive in their use of expert TA resources than the YTD projects were.

Although we estimated that BHBF had beneficial impacts on a number of important outcome measures during the third year after youth enrolled in the evaluation, when expressed as standardized effect sizes, most of the impact estimates are small. For the statistically significant (p < 0.10) impact estimates reported in Table 6, the effect sizes range from 0.17 (paid employment and disability payment amount) to 0.39 (earnings from employment). All of these fall short of the conventional threshold for a medium effect size: 0.50 (Cohen, 1988). But because about 1.2 million adolescents and young adults ages 13–25 received federally administered SSI payments in 2015 with a total value of $8.7 billion (SSA, 2017), an intervention that achieves even modest increases in employment and earnings among those individuals, and carries the potential to lessen their dependency on disability payments in the future, could be a valuable policy tool for managing the size of the SSI program.

The youth who enrolled in the evaluation of BHBF were volunteers who responded to Mathematica’s outreach efforts. Sixteen percent of the youth recipients of disability payments who were targeted for recruitment completed a baseline survey and consented in writing to enroll in the evaluation. Some enrollees already possessed the characteristics that BHBF was designed to affect. For example, the baseline survey revealed that 19 percent had worked for pay in the previous year, and 90 percent expected to do so in the next five years. More intensive or alternative recruiting strategies might have yielded evaluation enrollees with fewer of these baseline characteristics, creating a greater opportunity for BHBF to influence treatment group members and possibly resulting in more and larger impacts on three-year outcomes.

Because the evaluation enrollees were volunteers, the evaluation’s impact estimates should not be extrapolated to the full population of youth disability recipients in Miami-Dade County. More specifically, they should not be used to inform policy decisions about the implementation of hypothetical, mandatory employment-focused interventions for youth disability recipients. However, future interventions for this population are likely to be voluntary, in which case the BHBF impact estimates could be instructive. For example, participation in the PROMISE demonstration projects, funded by the U.S. Department of Education (2013a, b), is voluntary.

SSA’s waivers for YTD participants, which remained in effect for four years after youth enrolled in the evaluation, virtually precluded BHBF from reducing the amount of disability payments the youth received during that period. However, under the terms of Mathematica’s evaluation contract with SSA, data on evaluation enrollees were collected and analyzed for just three years, thus effectively eliminating the possibility that the evaluation would find BHBF reduced disability payments, and increasing the likelihood that it would find the project increased them. SSA plans to conduct analyses of BHBF’s long-term impacts on disability payments and other outcomes using data that it acquires in administering its disability programs.

The randomized controlled trial design for the analysis of the impacts of BHBF was not a significant barrier to the recruitment of youth into the study, nor did it generate many questions or complaints from youth, parents, or other stakeholders. Therefore, designers of future studies should not allow anxiety about random assignment to deter them from specifying rigorous experimental evaluation designs to evaluate interventions for youth with disabilities.

4.2 Implications for policy and practice

The findings from this study have two important implications for policy and practice. First, transition interventions that include work experiences as a key service component can make significant differences in the later outcomes of youth with disabilities. The impact analysis of BHBF revealed that, well after the intervention ended, the employment rate of youth in the treatment group exceeded that of their counterparts in the control group. This parallels the results of other studies that suggest strong early work experience and employment-focused interventions positively influence employment outcomes in early adulthood for youth with disabilities (Carter, Austin, & Trainor, 2012; Wehman, Sima, Ketchum, West, Chan, & Luecking, 2014). In addition to improving directly targeted outcomes such as employment and earnings, these interventions may have indirect beneficial impacts, such as decreasing youth’s contacts with the justice system. Federal policy is already moving in the direction of promoting work experiences for youth with disabilities. As a prominent example, the Workforce Innovation and Opportunity Act of 2014 (U.S. Congress, 2014) highlights, among other things, the importance of work experience for youth as they prepare to leave high school and enter the world of work. This study’s findings reinforce the value of this type of intervention, especially when applied to a population with long-standing and seemingly intractable low employmentrates.

Second, project staff who implement employment-focused interventions should be carefully selected, and they will need professional development and support to acquire the skills they will need to help make employment possible for youth with disabilities. It is often challenging for interventions for youth with disabilities to establish and maintain a sharp focus on employment. As documented in this study, when such interventions are staffed with individuals from traditional social service backgrounds, general case management can divert staff from the challenging yet ultimately more productive activities of job development and job placement. The management of BHBF found that the skills and interests of some of its original frontline staff were not ideally matched to the employment focus of the YTD/BHBF program model, particularly with respect to job development. BHBF management adjusted staff retention and hiring policies to achieve a better match between project staff and project goals, ultimately establishing frontline staff who had interest and experience in serving youth with disabilities and conducting outreach to employers. This finding reinforces the need for these kinds of programs to be attentive to key characteristics and traits of potential staff in their recruitment and hiring policies and practices (Tilson & Simonsen, 2013).

This study supports the use of expert TA to ensure intervention fidelity in research projects on transition services. Focused TA can prevent the deviation from program designs and ideal protocols that often happens in such studies; this deviation can negatively influence findings (Duan, Braslow, Weisz, & Wells, 2001). And for BHBF, a critical complement to TA was quantitative monitoring by project managers to ensure frontline staff were devoting enough time to job development and job placement, and were achieving positive employment outcomes for participants. The monitoring was made easier by a management information system that tracked employer contacts, services, and job placements at the individual staff and participant levels. These granular data were the basis for monthly monitoring reports on efforts and outcomes at several different levels of aggregation across project staff and participants. These reports were a powerful performance management tool for BHBF and also informed the delivery of training and support to the project staff.

The field of transition and disability employment services has long recognized the importance of professional development for staff providing employment services. As new disability policies, such as the Workforce Innovation and Opportunity Act, are implemented, attention to professional skill development and support will be critical. To fulfill the mandates of evolving transition and employment policy, transition and employment service staff will need to develop the skills necessary to help youth pursue and succeed in employment. This study suggests that such skills can be reinforced by in-the-field TA and support as a supplement to traditional classroom training and online training. Professionals with the training and experience to deliver such field-based assistance are not widely available, so policymakers should consider ways to expand the field of subject matter experts who are qualified to provide this training and TA.