Insights about the transition system for SSI youth from the national evaluation of Promoting Readiness of Minors in SSI (PROMISE)

2.1 Transition system challenges when the PROMISE initiative was launched

Despite the number and variety of public programs available, youth with disabilities and their families face many challenges in accessing and using them (U.S. Government Accountability Office, 2012, 2016; Hirano, Rowe, Lindstrom, & Chan, 2018). These challenges include encountering different eligibility rules for programs, navigating a fragmented and uncoordinated transition system, facing limited or delayed access to services, and not having enough information about services and the transition process. Further, youth with disabilities might be inadequately prepared for postsecondary education and employment because many of them are not accessing career development learning and experiential activities despite the potential availability through federal and state programs (Carter et al., 2010a; Carter, Trainor, Cakiroglu, Swedeen, & Owens, 2010b). For example, despite a majority of special education students receiving support services at school, only 13 percent engaged in school-sponsored work activities in 2012, and the proportion with paid work experiences outside of school declined from 27 percent in 2003 to 19 percent in 2012 (Liu et al., 2018). These challenges might be more prevalent for SSI youth and their families because of their poverty, limited resources, and significant health conditions for both youth and other family members (Rupp et al., 2006).

Failure to overcome these challenges can limit the success of youth with significant disabilities as they become young adults, and creates concerns about the long-term fiscal burden on the federal government. The child SSI program is an important pathway to the adult SSI program. Individuals who enter the federal disability programs at a young age may go on to receive benefits for many decades. Although the eligibility rules for adults are more stringent than those for children, approximately 65 percent of youth SSI recipients go on to receive SSI as adults (Hemmeter & Bailey, 2015). Adults who receive SSI payments for decades incur large lifetime disability program and other expenditures. One study estimated that individuals who enter SSI or DI as adults before the age of 30 remain on benefits for an average of 33 years and incur average SSI, DI, Medicare, and Medicaid expenditures of about $600,000 (in 2012 dollars) during that period (Riley & Rupp, 2015). Others have estimated the lifetime expenditures of SSI youth who remain on benefits to be even greater (Enayati & Shaw, 2019). About one in four adult SSI recipients, and nearly two in three adult SSI recipients under the age of 30, first started receiving benefits as children (SSA, 2018c). These estimates suggest potentially large lifetime fiscal outlays associated with many youth SSI recipients.

Thus, before PROMISE, youth with disabilities, their families, as well as service providers and practitioners, faced a complex and fragmented transition service system that lacked coordination and comprehensiveness in meeting the needs of the youth and their families. The system also did little to mitigate the risks of lifetime dependency on public transfer payments for the youth and long-term fiscal burden on the federal government. It was in this environment that the federal partners implemented the PROMISE initiative to apply a service model that would address these challenges and rigorously test it in a variety of settings around the country.

2.2 The PROMISE approach to addressing the challenges

The federal agencies sponsoring PROMISE envisioned projects that would address many of the challenges cited above, including by providing services to both youth and their families, and improving partnerships among service-providing agencies at federal, state, and local levels. Based on evidence from the literature, input from the public, and consultation with subject matter experts, the federal partners postulated that two main features of the PROMISE projects would make them more effective: (1) strong partnerships between the agencies that provide services to SSI youth and their families, and (2) an individual- and family-centered approach to case management and service delivery. The federal partners also identified a set of services that could achieve the desired results and thus required the PROMISE projects to include the following core components (ED 2013): 2

Formal partnerships between state agencies that provide the following services: VR services, special education and related services, workforce development services, Medicaid services, income assistance from Temporary Assistance for Needy Families, and services provided by federally funded state developmental disability and mental health services programs

Case management to ensure that PROMISE services would be appropriately planned and coordinated, help participants navigate the broader service delivery system, and help with transition planning for post-school goals and services

Benefits counseling and financial education for youth and their families on SSA work incentives, eligibility requirements of various programs, rules governing earnings and assets, and topics promoting families’ financial stability

Career and work-based learning experiences, including paid and unpaid work experiences in an integrated setting while youth were in high school

Parent training and information in two areas: (1) the parents’ role in supporting and advocating for their youth to help them achieve their education and employment goals, and (2) resources for improving the education and employment outcomes of the parents and the economic self-sufficiency of the family

Each of the six PROMISE projects implemented the core services in ways they believed would be most effective in addressing the challenges faced by SSI youth in their states. They also offered a variety of services in addition to the core requirements. Some examples of these include the following: education services (such as supporting families at school IEP meetings and connecting youth to postsecondary education opportunities); offering youth self-determination training; and providing funds for individual development and Achieving Better Life Experience accounts. Beginning in September 2013, ED awarded a total of $230 million to six PROMISE projects, initially for five years, but later offered the option to extend the projects for up to one more year. Table 1 summarizes the location, enrollment period, service end date, and number of youth enrolled in the research sample of each PROMISE project. The projects enrolled youth in the PROMISE study from April 2014 through April 2016. From April 2014 through August 2019, the projects delivered services to youth for periods ranging from 53 to 58 months.

3.1 Study design and samples

The PROMISE evaluation is based on a random assignment design. PROMISE-eligible youth who agreed to participate in the study were randomly assigned to either a treatment group, which meant they were eligible to receive PROMISE services, or a control group, which meant they were not eligible for PROMISE services but could receive other services available in their communities independent of the PROMISE project. The evaluation design allowed an assessment of the extent to which PROMISE affected participation in youth transition and family support services while accounting for the fact that similar services were available to the control group from other sources. Random assignment is expected to lead to the creation of two groups of youth with similar pre-intervention experiences and characteristics, on average. As a result, we can attribute any observed differences in outcomes between the two groups to be an accurate estimate of the project’s impacts.

To be eligible for PROMISE, youth had to be age 14 through 16 at the time of enrollment, in SSI current pay status at some time during the PROMISE enrollment period (and not terminated from SSI before enrolling in the evaluation), living in a PROMISE project service delivery area, and not residing in an institution. Each PROMISE project was required to recruit at least 2,000 eligible youth into the study, using lists of eligible youth provided by SSA. A total of 12,584 randomly assigned treatment and control group youth participated in the PROMISE evaluation; as intended, the two groups were largely similar on demographic, disability, and family characteristics. These youth were statistically similar to SSI youth who did not enroll in PROMISE on nearly all observable characteristics (with differences varying by project), but likely differed in unobservable ways related to their own and their parents’ motivation for seeking the opportunity to obtain transition services and enrolling in the study.

3.2 Process analyses

The process analyses for the PROMISE evaluation documented the choices made by the projects to deliver services, along with the resultant experiences with respect to enrollment, service delivery, and agency partnerships. Specifically, the analysis addressed four research objectives: (1) document the project’s intended design and fidelity to the PROMISE model; (2) assess partner organization development, maintenance, and roles; (3) support the evaluation’s impact analysis; and (4) identify lessons and promising practices. The analysis achieved these objectives by collecting and analyzing data from multiple sources, including interviews with project staff, observations of service delivery, focus groups conducted with youth and families, management information systems developed by each project, and monthly calls with state and federal partners involved in each project. Details about the process analysis methods and findings can be found in each project’s process analysis report (Anderson et al., 2018; Honeycutt et al., 2018a; Kauff, Honeycutt, Katz, Mastrianni, & Rizzuto, 2018; Matulewicz et al., 2018; McCutcheon et al., 2018; Selekman et al., 2018).

3.3 Cost analyses

The PROMISE cost analyses produced estimates of the economic cost to implement each PROMISE project, including the costs not directly incurred by the project, such as volunteer labor and donated facilities or supplies. These cost estimates represent the resources needed to implement a similar project and may differ from the funding that each PROMISE project received. We followed a seven-step analytic framework to compute a project’s total annual costs, annual cost per treatment group enrollee regardless of participation in PROMISE services, and annual costs by project component. That framework involved collecting costs directly from each PROMISE project for a specified 12-month period after start-up and enrollment were completed, and separating the costs into various categories and components. Details of the cost analysis methods can be found in Mamun et al. (2019). An important caveat regarding the cost estimates is that they reflect the average service intensity observed during the targeted 12-month accounting period, though youth and families might not have received services at that same intensity throughout their involvement in the project.

3.4 Impact analyses

The impact analyses for the PROMISE evaluation relied on analyses of survey and administrative data (from SSA, VR, and state Medicaid programs) as of 18 months after youth and the families enrolled in PROMISE. The basic approach for estimating impacts was to compare average outcomes for the treatment and control groups while using a regression-based adjustment to account for any differences in baseline characteristics. Details about the impact analysis methods and findings can be found in Mamun et al. (2019). The results reflected in this article represent early or interim evidence from the PROMISE impact evaluation. The evaluation will follow youth and families for five years after they enrolled to observe longer-term impacts on employment, education, and program participation. The findings from the impact evaluation are limited by the fact that the youth who enrolled in PROMISE were volunteers who were not necessarily representative of all PROMISE-eligible youth in the areas served by each project. However, because participation in most transition-related services and efforts is voluntary rather than mandatory, the PROMISE impact findings are likely indicative of the transition-related activities in which SSI youth in the PROMISE states typically engage and the impacts PROMISE may have on youth outcomes.

4.1 Although many SSI youth received transition services in the absence of PROMISE, there was substantial room for PROMISE to improve on the use of existing services

The data on control group youth collected via the national evaluation’s 18-month follow-up survey provide a picture of the transition services that SSI youth typically accessed in the 11 states participating in PROMISE. These data suggest that the large majority of control group youth received at least some type of transition services during the first 18 months after they enrolled in the study (Table 2). Roughly 90 percent of control group youth in all PROMISE states received at least one transition service; the exception was Arkansas (82 percent). School transition planning was the most common, reported for at least 60 percent of control group youth in all states. This finding is not surprising, given the IDEA provisions that require schools to engage in transition planning with the families of youth with disabilities enrolled in special education. Most of the SSI youth enrolled in PROMISE were receiving special education (74 to 93 percent across the projects; not shown). Life skills training was also commonly reported; nearly half or more of control group youth in all states except Arkansas (42 percent) received this service.

Control group youth were generally less likely to have used the core PROMISE services offered to treatment group youth than other services. The share of control group youth receiving at least one of the core PROMISE services (case management, employment-promoting services, benefits counseling, and financial education) ranged from about 50 percent in Arkansas and California to 65 percent in Maryland.

Among the PROMISE core services, parents of control group youth most commonly reported their youth receiving employment-promoting services (such as training to learn new job skills or help finding or applying for a job); these services were used by 36 to 54 percent of control group youth in all PROMISE states. Interviews with staff of the PROMISE projects and other agencies in their states suggest that schools and the state VR agencies were primary sources for employment-promoting services under the status quo. In some of the states, local schools had relationships with the VR agencies even before implementation of WIOA to offer employment-promoting opportunities for youth.

However, administrative data from the state VR agencies suggest that the state VR agencies were not a significant source of employment services for control group youth during the period represented by the data— from October 2014 through October 2016 (see bottom panel of Table 2). In all programs except Wisconsin, less than 10 percent of control group youth had applied for VR services and less than 6 percent had received any services. These data might underrepresent VR’s involvement with younger SSI youth (PROMISE enrollees were under age 18 for all or most of the period covered by these data); also, during this period, most states had not yet begun to formally track the provision of pre-employment transition services required by WIOA for students with disabilities. Nonetheless, it appears that during the early years of WIOA implementation, VR agencies were not a significant source of employment-promoting services for SSI youth under age 18. Information learned from the process analysis interviews conducted for the national evaluation is consistent with these administrative data findings, underscoring that in many of the PROMISE states, the pre-employment transition services prompted by WIOA were slow to result in actual service delivery. In addition, during this period, some state VR agencies remained reluctant to serve youth until they were nearing the time they would leave high school, which could occur as late as age 21 for special education students in most of the PROMISE states.

Nontrivial shares of control group youth also received case management, another core PROMISE service (ranging from 27 to 44 percent across the projects). The PROMISE evaluation survey did not query the source of the case management services, but the interviews conducted for the process evaluation suggest they were offered through Medicaid, developmental disability, independent living, and VR programs. Case management offered by these entities typically focused on specific services— for example, employment and training or health conditions and medical care. The process evaluation interviews identified very few instances in which more broad-based case management of the variety that the PROMISE projects offered was available to control group youth and their families under the status quo.

Control group youth were unlikely to have received the other two core PROMISE services: financial education and benefits counseling. Parents of about 15 to 20 percent of control group youth reported their children receiving financial education. Financial education services were inconsistently a part of the counterfactual service environment in PROMISE states. Although these services might have been offered by some schools or community providers, their availability or consistency varied by geographic location within states or might not have been tailored either to youth or youth with disabilities. Less than 10 percent of control group youth in all states received benefits counseling. These services were available through providers funded by the SSA Work Incentives Planning and Assistance (WIPA) grants. Other entities also provided these services using funding through other avenues, with state VR agencies being a primary source.

Although financial education and benefits counseling services were widely available in the absence of PROMISE, SSI youth under age 18 in general do not routinely access them for a variety of reasons. First, parents may be unaware of them or not view them as necessary because their youth are still young. Findings from the process analysis interviews suggest that this latter reason contributed to relatively low take-up of benefits counseling services among PROMISE treatment group youth who were offered these services. Second, WIPA-funded providers are supposed to prioritize services to SSA disability beneficiaries who are employed or soon to be employed, which might exclude many in-school youth not yet working or not planning to work extensively. The process analysis interviews with WIPA staff suggest that outside of PROMISE, they typically did not serve many SSI recipients under age 18 and generally did not conduct any particular outreach to this group. Finally, as noted above, few control group youth interacted with the state VR programs, and these programs are a primary source of referrals for and provision of benefits counseling services.

The differences in the control group youth experiences in receiving transition services across the PROMISE states suggest there are state differences in their transition service systems, which, in turn, influenced the PROMISE project’s ability to engage the system in serving PROMISE treatment group youth. As described in the projects’ process analysis reports, each project encountered different challenges and experiences while implementing aspects of PROMISE, some of which were unique to their service environments. For example, whether a state VR agency was in order of selection (and thus limiting its services only to those with the most significant disabilities) or its philosophy about serving youth as young as age 14 affected the ability of the PROMISE projects to successfully connect youth with VR services. Limited community capacity to provide services or a lack of any organization qualified to provide a particular service affected the PROMISE projects’ decisions about how to structure partnerships and whether to have their own staff deliver a particular service. Employment-promoting services, self-determination training, and benefits counseling are examples of services for which some of the projects encountered limited community capacity and had to develop ways to increase it.

PROMISE and other interventions that seek to improve access to and use of transition services by SSI youth have a greater potential for generating impacts for those services that relatively few youth access, whether because of limited community capacity, lack of family interest or awareness, or other reasons. Based on the experiences of the control group youth depicted in Table 2, it appears that although most SSI youth received some type of transition services, there was substantial room to improve their use of every type of service shown, especially those required under the PROMISE initiative.

PROMISE projects had positive impacts on most of the youth and family services measured by the national evaluation (Table 3). For most projects, this was true even for those services that were more common among youth in the control group, such as school transition planning and life skills training. However, among the greatest impacts of the projects were two specific services for youth—case management and employment-promoting services—both of which reflect the core PROMISE model and have been found to positively influence postsecondary outcomes (Enayati & Karpur, 2019; Haber et al., 2016; Mazzotti & Plotner, 2016). In addition, the projects had positive impacts on service use by other family members. Below, we summarize findings from the PROMISE evaluation for these three innovations.

4.3 Formal partnerships can enhance interagency collaboration around transition services but require time to develop, service benchmarks, and regular communication

Interagency collaboration is a key component of promoting transition success for youth with disabilities (Kohler et al., 2016; Oertle & Seader, 2015; ED, 2017). The characteristics of collaboration for transition include formal relationships among agencies via agreements and designated staff; a shared understanding of the goals, activities, and challenges; and coordinated service delivery, planning, and information for youth and families. Because of its importance, such collaboration was a feature of the PROMISE projects. ED required applicants to document their collaborations and the involvement of certain state agencies as project partners. As a result, all or almost all PROMISE projects included state representatives from education, labor, VR, health, human or social services, and developmental disability agencies, as well as nonprofit service providers, on their state-level steering committees. These committees advised the projects on service development; addressed implementation challenges; and offered support, information, and connections throughout the projects’ service areas.

For interagency collaboration to result in enhanced services on the ground, state-level partnerships must carry over to the local level where staff interact directly with youth. Experiences from the PROMISE projects offer several lessons about how to promote successful local-level collaboration.

4.4 Targeting services to SSI youth is difficult without knowing who they are

Low rates for the use of certain services among control group youth, along with the demand for transition services by SSI youth and families as illustrated by their enrollment in PROMISE, suggest that state and local programs might reach more SSI youth if the programs could identify them. The PROMISE projects enrolled 16 to 43 percent of the families with SSI youth they contacted. The sizeable enrollment rates in most of the projects reflect the interest in and demand for transition services by SSI youth and families. The PROMISE projects expended considerable effort on outreach attempts and benefited from the use of SSI family contact lists provided by SSA under its demonstration authority. Similar lists for targeting SSI youth generally cannot be used by community service providers because federal law limits how SSA may share information about its program participants.

Without access to SSA information to identify SSI youth, other methods for targeting them for transition services might be considered. State agencies involved with education, Medicaid, and VR could develop interagency agreements that would allow them to share Medicaid data to identify SSI youth for outreach and service delivery. Researchers affiliated with Wisconsin PROMISE proposed such a plan for the VR agency to use in a proposal to sustain features of PROMISE after the demonstration had ended (Anderson, Schlegelmilch, & Hartman, 2019). Programs serving youth with disabilities could also make more concerted efforts to identify families receiving SSI and refer them to relevant resources, such as the WIPA programs. American Job Centers (referred to as One-stop Career Centers at the time) faced a similar issue in attempting to identify job seekers with disabilities to serve under the Disability Program Navigator initiative (Livermore & Coleman, 2010). This issue was addressed under the U.S. Department of Labor’s subsequent Disability Employment Initiative (DEI) by requiring the centers implementing DEI innovations to participate as employment networks under SSA’s Ticket to Work program (Klayman, Di Biase, Searson, Hock, & Ketema, 2019). As employment networks, they had access to information about Ticket-eligible SSI and SSDI beneficiaries and thus could identify them among their customers if they chose to do so. However, in the case of SSI youth with disabilities, this mechanism would not be available because they are not Ticket eligible until after turning age 18 and found eligible for SSI or SSDI based on the criteria used for adults. In early 2018, SSA sought public input on strategies for improving the adult economic outcomes of SSI youth to inform its deliberations about potential policy changes and the design of future demonstration projects (SSA, 2018b). Expanding the Ticket to Work program to SSI youth was a specific area in which the agency sought detailed input regarding how such an expansion could be implemented.

4.5 Though PROMISE service costs varied across projects, they represented a relatively large investment to support SSI youth

The national evaluation analyzed the costs of PROMISE project services during a period when the projects were in a relatively steady state— neither ramping up nor winding down services. Based on this analysis, and on information collected about specific staff activities, the evaluation produced estimates of how project efforts were distributed across key activities and the average (per treatment group enrollee) annual costs of each project (Table 5). Direct services delivered to youth and their families (including services delivered by project staff, contractors, and through formal nonmonetary agreements with other organizations) accounted for the majority of costs for each PROMISE project. Among direct services, case management constituted the largest share of total costs in all projects, generally followed by career and work-based learning experiences. At the same time, the share of costs that direct services accounted for varied across PROMISE projects— ranging from 52 percent of the total costs for NYS PROMISE to 75 percent for Arkansas PROMISE, with the remaining share accounted for by administrative costs.

The national evaluation’s estimates of annual cost per enrollee ranged from $5,490 for ASPIRE to $9,148 for Arkansas PROMISE (Table 5). A key factor potentially explaining the variation in the per enrollee costs is the extent to which the project provided services directly as opposed to leveraging existing services available in the community. Arkansas PROMISE delivered or paid for most of its services directly; its cost per enrollee was high compared to the other projects. ASPIRE leveraged existing services to a relatively large extent; its cost per enrollee was low compared to the other projects. If the costs of services received from other agencies (that is, the cost of the existing services the projects leveraged) could be accounted for, all of the projects’ costs would be higher than the estimates shown.

To put these costs into context, in 2014 the federal government spent an estimated $5,000 per youth with disability (under age 18) on public programs and supports specific to them or that represented assistance programs used by many such youth (Shenk & Livermore, 2019). 3 Thus, the annual cost per enrollee across the PROMISE projects was roughly similar to the average annual cost of all federal programs currently available to youth with disabilities, and this investment was in addition to the existing federal outlays for youth with disabilities.

Intensive case management was the central feature of all of the PROMISE projects and represented the largest component of service costs. To enable case managers to provide broad-based case management that addressed a variety of youth and family needs, PROMISE case managers carried relatively small caseloads— about 30 or fewer families each in most projects. Although some PROMISE case managers were on the staff of education and VR agencies, they generally functioned independently of those agencies’ other programs— necessary to ensure that project demonstration funds were spent as intended and only treatment group members would receive PROMISE services. As of the writing of this article, none of the PROMISE states has continued to provide the kind of broad-based case management to SSI youth offered under PROMISE.

Certainly costs and limited resources are important reasons why intensive targeting and case management services for SSI youth have not been adopted outside of PROMISE. Moreover, state and local agencies have little incentive to invest significant resources in providing specialized services to this population that go beyond their usual services. For some providers, this population is relatively small and difficult to identify from among its service users overall, and there is little justification for shifting limited resources away from other users to serve SSI youth. Under the current system, SSA has the most to gain from the successful transition of SSI youth to adulthood. However, SSA currently lacks the resources and authority to provide case management services to its beneficiaries; its primary function is to provide income support. SSA does reimburse VR agencies and other providers when they help beneficiaries find and maintain employment, but currently does so only for adult beneficiaries. Expanding the Ticket to Work program to SSI youth might help address this but there are numerous complexities that SSA would need to address in designing and implementing such an expansion. VR agencies seem a logical place to house case management services tailored to SSI youth and families, especially if successful employment and independence are the primary goals of the transition services. However, other community organizations could function in this capacity, given appropriate funding and incentives to serve this population, which under the current system appear to be insufficient.