Abstract
Purpose:
This randomized controlled trial compared the efficacy between the Social Enterprise Intervention (SEI) and Individual Placement and Support (IPS) with homeless youth with mental illness to assess their impact on nonvocational outcomes.
Method:
Seventy-two homeless youth were recruited from one agency and randomized to the SEI (n = 36) or IPS (n = 36) conditions. Over 20 months, SEI participants received four SEI components, and IPS participants received eight IPS components. Data were collected on mental health, housing, and social support outcomes at baseline and one follow-up period.
Results:
Findings from mixed analysis of variance indicate that both SEI and IPS participants displayed statistically significant changes in their self-esteem, attention-deficit/hyperactivity disorder (ADHD) problems, inattention problems, living in a shelter, and living in a private residence.
Conclusions:
This is the first study to demonstrate the comparative efficacy and short-term nonvocational outcomes of SEI and IPS with homeless youth with mental illness. Assessment of long-term outcomes of both interventions is warranted.
Keywords
Over 2 million youth experience homelessness each year in the United States (Whitbeck, 2009). One quarter experience homelessness more than once in their lives, with each homeless episode lasting on average 10 months (United States Department of Health and Human Services, 2001). Patterns of recurrent homelessness can be both precipitated and prolonged by these youths’ low educational and employment skills combined with high rates of mental illness and substance abuse (Cauce et al., 2000; Whitbeck, 2009). Several studies document that over one third of homeless youth have dropped out of school, do not attend school regularly, or fail to earn a high school diploma by age 18 (Thompson, Pollio, Constantine, & Von Nebbitt, 2002; Whitbeck, 2009).
With respect to mental health, rates of depression in homeless youth range from 15% to 50% compared to 2–8% in the general youth population. The incidence of conduct disorders among homeless youth is between 21% and 93% compared to 1–4% in the general youth population. Likewise, attention-deficit/hyperactivity disorder (ADHD) is estimated to be as high as 33% in homeless youth compared to 4–5% in the general adolescent population (Cauce et al., 2000; Costello, Mustillo, Erkanli, Keeler, & Angold, 2003; Merscham, Van Leeuwen, & McGuire, 2009). Further, comorbidity is common among young people experiencing homelessness, with one study reporting that they had 6 times the odds of meeting criteria for lifetime comorbid mental disorders compared to their housed counterparts (Whitbeck, Johnson, Hoyt, & Cauce, 2004). Homeless youth are also more likely than their housed peers to display low self-esteem. In one study, only 50% of homeless youth reported feeling good about themselves compared to 66% of housed youth (Smart & Ogborne, 1994). As evidenced, rates of mental illness are high among this population, often contributing to their difficulty in securing employment, obtaining stable housing, and exiting homelessness.
The assumption that “work is therapeutic” in the general housed population has received considerable attention from policymakers and academics alike (Bond, Resnick, et al., 2001; Konczal, 2012; Mueser et al., 1997), and more recently from homeless youth researchers (Ferguson, Xie, & Glynn, 2012). That a job is considered by many to be the most viable path to worker well-being (defined as positive physical and mental health; Lambert, 2008) suggests that employment can benefit individuals in various nonvocational domains, such as increasing their self-esteem and satisfaction with life, helping them cope with or resolve mental illness and substance use, and facilitating other functional outcomes such as obtaining stable housing. Unfortunately, the opportunity to achieve positive nonvocational outcomes via employment for young people experiencing homelessness and mental illness remains distant. Unemployment rates among homeless young people range from 39% to 71% across various street- and shelter-living samples (Courtney, Piliavin, Grogan-Kaylor, & Nesmith, 2001; Ferguson & Xie, 2008; Lenz-Rashid, 2006; Whitbeck, 2009), which surpass unemployment estimates for former foster youth and nationally representative samples of housed young adults (Courtney & Dworsky, 2006; Pecora et al., 2006). Likewise, individuals experiencing mental illness continue to be disproportionately unemployed in comparison with the general housed population (Bond, Resnick, et al., 2001).
As such, homeless young people are missing an important opportunity to reap the well-being and quality-of-life benefits associated with employment. Their high unemployment rates in turn shift the responsibility to the mental health system to positively affect their nonvocational outcomes through various psychosocial, behavioral, and pharmacological interventions. This is problematic for at least two reasons. First, the mental health system is neither designed nor funded to be solely responsible for supporting individuals’ health, mental health, and quality-of-life outcomes (Osgood, 2005). For individuals with mental illness and substance use disorders, for instance, one major life domain is employment, which can facilitate recovery, in particular when people experience a strong fit with their position and receive integrated clinical services (Davidson, O’Connell, Tondora, Staeheli, & Evans, 2005; Drake, Bond, & Becker, 2012).
Second, homeless young people’s rates of service use in mental health and substance use services are extremely low, thus excluding them from benefiting from positive nonvocational outcomes associated with mental and behavioral health interventions. In a study of Los Angeles homeless youths’ service-use patterns, youth reported using health and mental health services at very low rates (e.g., medical services [28%], substance abuse treatment [10%], and mental health services [9%]; De Rosa et al., 1999). However, a more recent study on the use of employment services among homeless youth (ages 13–24) in Los Angeles found that nearly half of the sample (47%) participated in employment services in the month prior to the study (Barman-Adhikari & Rice, 2014). That homeless youth are engaging in employment services at 2–4 times the rate of their engagement in health, mental health, and substance use services indicates an important opportunity for researchers and service providers to target and measure nonvocational outcomes that result from employment services.
Existing Employment Interventions for Homeless Youth With Mental Illness
One key feature of employment programs with demonstrated efficacy in helping individuals with mental illness obtain and maintain employment and attain positive nonvocational outcomes is the integration of clinical and vocational services (Cook et al., 2005). Considerable evidence indicates that clients who participate in vocational programs with integrated and coordinated clinical services report improvements in nonvocational outcomes (e.g., social support, self-esteem, and life satisfaction) in addition to improvements in vocational outcomes (e.g., employability, work functioning, work hours, and income; Bond, Becker, et al., 2001; Bond, Resnick, et al., 2001; Cook et al., 2005; Mueser et al., 1997). Findings also reveal that clients who receive greater amounts of employment-specific vocational services and who remain for longer durations in vocational programs achieve statistically significantly better outcomes than those who receive fewer vocational services for shorter durations (Cook et al., 2005). Additionally, increased amounts of vocational services have been found to have a positive impact on employment outcomes, whereas greater amounts of clinical services are associated with poorer employment outcomes (Cook et al., 2005). These findings suggest that enhancing the amount of vocational services to clients with mental illness to complement or exceed their existing levels of clinical services may ultimately benefit them in obtaining and maintaining competitive employment, while concurrently addressing their mental health challenges.
Social enterprise intervention (SEI)
The principal investigator (PI), in collaboration with a homeless youth agency in Los Angeles, previously had developed and piloted an employment intervention for homeless youth, the SEI (Ferguson, 2007; Ferguson & Xie, 2008). The SEI trains homeless youth in vocational/technical and business skills and mentors them in developing a social enterprise that is housed within a host social service agency. Peer mentoring, clinical supports, and case-management services are integrated throughout the intervention by trained peer mentors and mental health and case-management staff.
The SEI is an asset-based model of youth development, tied to the youths’ clinical needs and treatment, which incorporates internal developmental assets including commitment to learning, social competencies, positive values, and positive identity. The assets framework proposes empirically grounded internal and external assets in youth that help improve positive outcomes and protect them from high-risk behaviors (Benson, 2003; Ferguson, 2007). The underlying philosophy of the SEI is that individuals’ economic well-being impacts all dimensions of their personal well-being. Through employment in a peer-based, team-oriented social business, individuals benefit from time structure, social contact, social context, and social identity, all of which positively influence their mental health (Harnois & Gabriel, 2000). Given homeless youths’ high levels of distrust toward helping professionals, the SEI combines several existing effective practices with this population including a peer-based, team-oriented approach, mentorship, individual counseling, and experiential therapy (Karabanow & Clement, 2004).
Individual placement and support (IPS)
The PI, also in collaboration with two homeless youth agencies in Los Angeles, previously had adapted the IPS model—originally designed for adults with severe mental illness (Drake et al., 2012)—to work with homeless youth with mental illness (Ferguson et al., 2012). The IPS model follows eight supported employment principles: (1) zero exclusion: All clients who want to participate are eligible, (2) integration of vocational and mental health treatment services: Vocational and mental health treatment staff are co-located and frequent communication between team members is essential, (3) competitive employment: Clients are assisted in obtaining community-based jobs at competitive wages, (4) benefits counseling: People who receive government benefits need personalized benefit planning when considering employment, (5) rapid job search: The job search process begins within 1 month of the client meeting with an employment specialist and beginning a career profile or vocational assessment, (6) follow-along supports: Individualized assistance to working clients is available for as long as needed, (7) preferences: Client preferences influence the type of job sought and the nature and type of support offered, and (8) systematic job development: employment specialists build an employer network based on clients’ interests, developing relationships with local employers by making systematic contacts (Drake et al., 2012).
Collectively, the eight IPS principles draw from theories of psychiatric rehabilitation and recovery with individuals with severe mental illness (Drake et al., 2012). The theory of psychiatric rehabilitation using supported employment is that individuals’ functional adjustment can be improved by creating a supportive environment and enhancing their skills or abilities (Anthony, Cohen, & Farkas, 1990). Likewise, the theory of recovery is that individuals can get better from their illness and pursue meaningful life goals such as employment (Deegan, 1988).
The feasibility of both the SEI and IPS models with homeless youth experiencing mental illness was established in two prior pilot studies, both of which used quasi-experimental designs (Ferguson & Xie, 2008; Ferguson et al., 2012). Findings suggest that both interventions are feasible and initially successful in helping this population achieve positive nonvocational outcomes (e.g., reduced depression, improved life satisfaction and social support, etc.). A logical next step would be the use of a randomized design to compare the efficacy and short-term nonvocational outcomes of both interventions with homeless youth experiencing mental illness.
Use of Randomized Controlled Trials (RCTs) in Homeless Youth Research
One major limitation to date in the literature with homeless youth is the lack of RCTs that test the efficacy and effectiveness of research-supported interventions. Recent systematic reviews of intervention studies using both randomized and nonrandomized designs with this population have assigned quality ratings of poor and fair to extant studies; notably, none of the previous studies reviewed received a quality rating of good (Altena, Brilleslijper-Kater, & Wolf, 2010; Xiang, 2012). This is due largely to the limitations of existing studies caused by lack of randomization, weak methodological designs, small samples, and high attrition rates (Altena et al., 2010; Xiang, 2012). Other systematic reviews have opted not to use a randomized design as a selection criterion at all since so few RCTs have been used to date (Slesnick, Dashora, Letcher, Erdem, & Serovich, 2009).
In response to the limited use of RCTs with homeless youth to test research-supported interventions, this study compares the efficacy and short-term outcomes of two interventions that combine employment and clinical services with homeless youth experiencing mental illness. The SEI is a research-supported intervention using a group approach that engages homeless youth in paid employment as well as case-management and mental health services through involvement in an agency-run social enterprise (Ferguson & Xie, 2008). The IPS model of supported employment is an individually focused, evidence-based intervention, which provides individuals with severe mental illness with customized, long-term, and integrated vocational, case-management, and clinical services to help them gain and maintain competitive employment (Drake et al., 2012; Marshall et al., 2014).
A randomized comparative efficacy trial of the short-term, nonvocational outcomes of the SEI and IPS was conducted over 20 months with 72 homeless youth at a homeless youth drop-in center in Los Angeles. This article reports the primary analysis of data from this RCT, which answered two research questions: (1) Do homeless youth with mental illness participating in an employment intervention integrated with clinical services (SEI or IPS) experience improvements in their (a) mental health status (self-esteem, life satisfaction, depression, attention-deficit problems, and antisocial personality problems), (b) housing stability, and (c) social support; and (2) what are the differences between the SEI and IPS groups on mental health, housing stability, and social support outcomes between baseline and follow-up?
Method
Project Setting and Staff
From November 2009 to June 2011, researchers conducted a randomized comparative efficacy study of the SEI and IPS models with 72 homeless youth (ages 16–24) at one drop-in center for homeless youth in Los Angeles. During this 20-month period, the host agency served roughly 1,400 young people (ages 16–24). On average, 80–90 youth visited the agency daily. The agency sponsored the project and provided space, partial staffing, and supervisory support for SEI and IPS staff. The SEI instructor and one clinical social worker as well as the IPS employment specialist were hired with grant funding. The SEI and IPS case managers and additional clinicians were existing agency staff, who partnered with the study by providing case-management services and mental health counseling to study participants as part of their existing agency caseloads. Two peer mentors (i.e., young adults who were formerly homeless and had achieved important life goals such as housing, employment, and recovery) were also hired through the grant for the SEI condition and trained and supervised by SEI staff. Additional collaborators included university master of business administration students, who conducted a feasibility study of the SEI business. The local chamber of commerce also provided support to the IPS employment specialist by hosting monthly networking events for local employers.
Sampling and Recruitment Procedures
The research team used nonprobability quota sampling to select the SEI and IPS youth. During a 4-month period, the PI and two trained research assistants (RAs) spent 10 hr per week in the host agency to recruit homeless youth into the study. The researchers approached the youth in the central dayroom in random order in three areas of the drop-in center where the youth commonly congregated (i.e., couch area, recreation tables, and outside) and asked them whether they wanted to participate in an employment intervention research study. Researchers attempted to recruit one third of the sample from each of the three areas. Researchers also attempted to recruit genders and races/ethnicities of youth in the proportion that they were represented in the agency. At the time of this study, the gender and racial/ethnic breakdown of the host-agency clientele was 65% male, 34% female, and 1% transgender and 35% African American/Black, 25% Latino/Hispanic, 24% Caucasian/White, and 16% other (i.e., Asian, American Indian, and mixed).
Youth were eligible to participate if they met four criteria: (a) age 16–24, (b) English speaking, (c) primary clinical diagnosis in the past year (with at least one symptom in the past 4 weeks) using the Diagnostic Interview Schedule for Children - Youth version (DISC-Y) interview for one of six mental illnesses (i.e., major depressive episode, mania/hypomania, generalized anxiety, post-traumatic stress disorder, conduct disorder, and alcohol/substance use disorders; Fisher, Shaffer, Lucas, & National Institute of Mental Health [NIMH] DISC Editorial Board, 2005), and (d) desire to work. Of the 75 youth approached during the engagement phase, 72 signed up to participate in the study (96% response rate).
Randomization and Treatment Allocation
A power analysis was calculated to estimate the actual power achieved based on an assumed 40% attrition rate, medium-to-large effect sizes (d = .6–.8, equivalent to f = .3–.4; obtained in our pilot study, see Ferguson & Xie, 2008), and the proposed sample size. The estimated power was .85–.97. The power analysis revealed that 44 subjects (22 per group) would be sufficient to achieve 80% power to detect a statistically significant treatment effect. Attrition analysis in the SEI pilot study revealed a 31 and 33% attrition rate in the SEI treatment and comparison group, respectively (Ferguson & Xie, 2008). With overrecruitment of participants at enrollment in anticipation of a 40% attrition rate in the current study, the planned sample size for each group was 36 youth per group (i.e., 72 in total).
Upon screening and having the youth sign the consent/assent form, researchers randomly assigned 72 youth with mental illness to one of two conditions: the SEI group or the IPS group. Youth selected a sealed envelope containing the name of a study group that had been generated via computer algorithm. Blocking was used to ensure an equal number of subjects between the two study groups (Kirk, 1995). Once randomized, a rolling start procedure was used since the SEI intervention was taught using a group format, whereas the IPS used an individual format. Once 24 youth were eligible and randomized (i.e., 12 to each group), SEI and IPS staff began their respective interventions.
SEI Condition
The SEI model was implemented in four stages: (1) vocational skill acquisition (4 months), (2) small business skill acquisition (4 months), (3) SEI formation and product distribution (12 months), and (4) clinical/case-management services (ongoing for 20 months). Table 1 outlines the four SEI stages with their time frame, characteristics, mechanisms of influence, and objectives.
Stages of the SEI Model.
Note. SEI = Social Enterprise Intervention; MBA = master of business administration; CM = case manager.
SEI vocational skill acquisition
The SEI participants received a 4-month vocational course to learn the technical skills necessary to form a social enterprise (vocational course syllabus available from author). The course was taught by the SEI instructor to small groups of youth (12 per group), who met 3 days a week for 1.5 hr each day. The SEI instructor conducted a focus group with the youth to identify their employment preferences and skill levels and to tailor the curriculum to their individual needs. Further, the SEI instructor sought feedback from the youth in creating the vocational course syllabus. Two products were determined from the focus group with SEI participants: silk screening and photographic art (i.e., silk screening onto apparel the photographs taken by the youth). The two peer mentors assisted the SEI instructor with classes and provided supportive mentoring to the youth. The youth also met with the SEI clinician and case manager outside of class weekly to prioritize goals, develop treatment plans, and receive mental health treatment and case-management services.
SEI small business skill acquisition
Over the next 4 months, the SEI youth then learned the skills needed to run the business (business course syllabus available from author). The course was developed by the SEI instructor with feedback from the youth and taught by the SEI instructor 3 days a week for 1.5 hr each day using both lecture and lab sessions. The mentors continued to assist with the course and the clinician and case manager continued meeting weekly with the youth.
SEI formation and product distribution
A social enterprise was developed in the host agency through which the youth could apply their vocational and business skills to create, market, and sell their apparel products. The SEI instructor continued to mentor the youth during this 12-month stage. The youth designed a website to advertise their products and participated in local flea markets to sell their products. The peer mentors assisted the youth in all business activities and continued in their mentoring role. The SEI clinician and case manager also continued to work with the youth to address their mental health and case-management needs.
Clinical and case-management services
During the 20-month study, the SEI clinician and case manager met with the youth individually on a weekly basis either in person at the agency or via phone or social media check-ins. The SEI youths’ DISC-Y diagnoses were used by the clinician to customize treatment, which was individually tailored depending on youths’ specific diagnoses. For youth with mental illness and substance abuse, the SEI clinician used cognitive behavioral therapy, referrals to psychiatrists for medication, motivational interviewing, and harm-reduction strategies. All clinical and case-management services were offered at the host agency, with the exception of referrals for medication, health care, and drug treatment, which were provided by specialized agencies in the community. Fidelity to the SEI curriculum was assessed via quarterly meetings with agency administrative and SEI project staff. Dosage of the SEI intervention was assessed using weekly attendance sheets for the SEI vocational and business courses, the social enterprise phase, and the clinical and case-management services.
IPS Condition
With the exception of one IPS employment specialist, who was hired through the grant, the researchers relied on host-agency staff, who were already known and trusted by the study participants, to implement the IPS model. The PI and a certified IPS trainer, also hired through a separate grant, trained host-agency IPS staff in the IPS model and provided technical assistance to IPS staff throughout the study. The eight IPS principles are outlined in Table 2 with their time frame, characteristics, mechanisms of influence, and objectives.
Principles of the IPS Model.
Note. IPS = Individual Placement and Support; CM = case manager.
To implement the IPS at the host agency, one employment specialist, two case managers, and two clinicians were assigned the 22 available IPS cases among them at baseline. The Supported Employment Fidelity Scale suggests a maximum caseload of 20 clients per employment specialist in order to achieve high fidelity (Swanson, Becker, Drake, & Merrens, 2008). Case managers and clinicians had smaller caseloads given the mental health and other life challenges common among homeless young people (Whitbeck, 2009). Over the 20 months, all IPS participants met individually with the employment specialist, one case manager, and one clinician at least weekly. Meetings took place either in person within the agency or community, by phone, or through social media check-ins. Regarding job development in the community, the IPS employment specialist also spent about 40% of each week out in the community building relationships with new and existing employers. IPS studies with adults, which have high reported fidelity, indicate that employment specialists should spend 60–70% of their time in job development in the community (Swanson et al., 2008). Fidelity to the IPS was assessed via quarterly meetings with agency administrative and IPS project staff. Dosage of the IPS intervention was assessed using weekly attendance sheets for the meetings between IPS staff (employment specialist, clinician, and case manager) and the youth participants.
Data Collection
Data were collected from the SEI and IPS groups within the host agency by the PI and RAs at two time points over a 20-month period: baseline (i.e., before entering the intervention, study months 1–4) and follow-up (i.e., study months 20–24). Interviews lasted 45–60 min and included standardized and researcher-developed measures for demographics, homelessness history, social support, mental health status, and housing status. All youth received gift cards to a local vendor in the amount of US $10 for the baseline interview and US $20 for the follow-up interview. Raw data from the interviews were entered by an RA into a database created in SPSS Version 23. All research procedures were approved by the institutional review board at the PI’s university.
Measures
Mental health
Mental health status was comprised of five variables. Self-esteem was measured using the Rosenberg Self-Esteem Scale (RSE; Rosenberg, 1979), a 10-item assessment of feelings about self-worth. Each item is rated using a 4-point Likert-type scale ranging from strongly disagree to strongly agree. Five items are recoded, so that higher scores indicate higher levels of self-esteem. The possible range of RSE raw scores is 0–30. Scores between 15 and 25 are within normal range, whereas scores below 15 suggest low self-esteem. The Cronbach’s α for the 10 RSE items was α = .85 at baseline and α = .87 at follow-up. Construct validity of the RSE was established using exploratory factor analysis (EFA). EFA loadings on one extracted factor for the RSE ranged from .20 to .83 at baseline and .20–.78 at follow-up (Tabachnik & Fidell, 2001). Only one factor loading was .20 at both baseline and follow-up, with the remaining loadings all above .55.
Satisfaction with life was assessed using the Satisfaction with Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985), a 5-item scale that measures the cognitive-judgmental aspects of general life satisfaction. Each item is scored from 1 to 7 on a Likert-type scale ranging from strongly disagree to strongly agree. Scores range from 5 to 35, with higher scores reflecting greater satisfaction. The Cronbach’s α for the 5 SWLS items was α = .80 at baseline and α = .84 at follow-up. EFA factor loadings on one extracted factor for the SWLS were .62–.83 at baseline and .65–.91 at follow-up.
Depression was assessed using the Adult Self-Report (ASR) Diagnostic and Statitstical Manual (DSM)-Oriented Scale for depressive problems, consisting of 14 items that measure related emotions and behaviors over the past 6 months (Achenbach, 1997). Scoring profiles used normed scales for adults ages 18–35. Higher scores reflected a higher presence of emotions and behaviors. For men, raw scores between 10 and 12 (11 and 13 for women) were within the borderline clinical range, whereas scores 13 and greater (14 and greater for women) were considered in the clinical range. The Cronbach’s α at baseline for depressive problems was α = .82 and at follow-up was α = .84. EFA factor loadings for one extracted factor, depressive problems, were .30–.78 at baseline and .35–.80 at follow-up. Three factor loadings at baseline were .30; one loading at follow-up was .35. All remaining loadings at baseline and follow-up were above .42.
Attention-deficit/hyperactivity disorder (ADHD) was assessed using the ASR DSM-Oriented Scale for AD/H problems, consisting of 7 items for inattention (AD) and 6 items for hyperactivity–impulsivity (H) that measure related emotions and behaviors over the past 6 months. Scoring profiles used normed scales for adults ages 18–35. Higher scores reflected a higher presence of emotions and behaviors. For men, raw scores between 13 and 16 (12 and 13 for women) were within the borderline clinical range, whereas scores 17 and greater (14 and greater for women) were in the clinical range. Cronbach’s αs at baseline and follow-up for ADHD problems were α = .73 and α = .78. EFA factor loadings for one extracted factor, ADHD problems, were .10–.70 at baseline and .20–.80 at follow-up. Two factor loadings at baseline were .10 and one at follow-up was .20; the remaining loadings were all above .40.
Antisocial personality disorder (ASPD) was assessed using the ASR DSM-Oriented Scale for antisocial personality problems, consisting of 20 items for related emotions and behaviors measured over the prior 6 months. Scoring profiles used normed scales for adults ages 18–35. Higher scores reflected a higher presence of emotions and behaviors. For men, raw scores between 11 and 12 (10 and 13 for women) were within the borderline clinical range, whereas scores 13 and greater (14 and greater for women) were considered in the clinical range. Cronbach’s αs at baseline and follow-up for ASPD problems were α = .75 and α = .86. EFA factor loadings for one extracted factor, ASPD problems, were .10–.70 at baseline and .26–.74 at follow-up. Three factor loadings at baseline ranged from .10 to .20, whereas one loading at follow-up was .26 and two loadings were .30. All remaining loadings were above .40.
Housing stability
Housing stability was assessed with three separate items. Youth were asked over the past 3 months whether they had lived on the streets; in a shelter or institution (i.e., youth or adult shelter, or detention facility, jail, or prison); or in a private residence with family (i.e., biological or foster), relatives, friends, a partner, roommate, or alone. Each item was dichotomous and scored as 0 = no or 1 = yes.
Social support
Social support was a composite-score variable of the sum of 4 items on the ASR Friends Subscale. The response categories range from none to 5 or more for questions including: “About how many close friends do you have?” The range of scores is from 0 to 12, with higher scores indicating greater social support for the youth. The Cronbach’s α for the 4 items of the Friends Subscale was α = .81 at baseline and α = .79 at follow-up. EFA factor loadings for one extracted factor, social support, were .54–.90 at baseline and .67–.88 at follow-up.
Data Analyses
Global empirical analyses including frequency distributions, means, and standard deviations (SDs) were used to describe characteristics and relationships among the variables. Paired-samples t tests (or McNemar’s tests) were conducted to examine differences in baseline characteristics between groups as well as to compare pre-intervention characteristics of participants available for analysis at post-intervention with those who were missing to evaluate potential sampling biases. Changes in mean scores between baseline and follow-up were also compared across both the SEI and IPS conditions. Subsequently, mixed analysis of variance analyses were used to identify differences over time between SEI and IPS participants on the study’s outcome variables. Lastly, the effect size using Cohen’s (1988) d of intervention effects for key outcome variables was calculated in an effort to better interpret the practical meaning of intervention effects. The criterion for statistical significance was at the .05 level.
Results
Sample Characteristics
Table 3 displays demographics between the SEI and IPS groups at baseline. Forty eight young people (ages 16–24) began the intervention at baseline (26 in the SEI and 22 in the IPS group). Collectively across both groups, participants were on average 22.17 years old (SD = 1.58). Forty youth (83.3%) were male, and 8 (16.7%) were female. Eighteen youth (37.5%) identified as Black/African American, 16 (33.3%) as mixed race/ethnicity, 8 (16.7%) as Hispanic/Latino, 5 (10.4%) as White/Caucasian/European American, and 1 (2.1%) as Asian/Pacific Islander. Regarding educational backgrounds, 29 youth (60.4%) had a high school diploma or General Educational Development (GED). At the time the study began, 14 (29.2%) youth were working. Of the 48 youth, 20 (41.7%) were living on the streets, 21 (43.8%) were staying in shelters, and 28 (38.9%) were living in a private residence. With respect to mental health, 27 (56.3%) met the criteria for major depressive episode, 21 (43.8%) met the criteria ASPD, and 12 (26.1%) met the criteria for ADHD.
Baseline Characteristics for SEI and IPS Groups.
Note. N = 48. SEI = Social Enterprise Intervention; ASR = Adult Self-Report; IPS = Individual Placement and Support.
aMean (M). bStandard deviation (SD). cCohen’s measure of effect size (d).
Sample characteristics were compared with agency-level demographic data to determine whether the 48 study youth were representative of the agency’s annual service population. Of the total population served by the agency, 35% were Black/African American, 29% White/Caucasian/European American, 22% Hispanic/Latino, 11% mixed race/ethnicity, 2% Asian American, and 1% unreported. Our sample had slightly fewer Hispanics/Latinos, one third the amount of Whites/Caucasians, and 3 times the amount of youth identifying as mixed race/ethnicity than the agency’s general population. Regarding the gender of agency clients served, 65% were male, 34% female, and 1% transgender. Our sample had a greater percentage of males and about half as many females as the agency’s population.
Baseline and Attrition Analyses
There were no statistically significant differences in baseline variables between the SEI and IPS groups with respect to age, gender, race/ethnicity, educational level, working status, housing status, mental health, or social support. In the SEI group, 26 youth began the SEI intervention and 21 were available at follow-up (81%). In the IPS group, 22 youth began the IPS intervention, whereas 17 could be located at follow-up (77%). Between baseline and follow-up, 5 youth discontinued participation in each intervention. Collectively, 72 youth were enrolled in the study and randomized; 48 were available for the baseline interview, and 38 completed the full intervention (SEI or IPS) and follow-up interview. The flow of participants through each stage of the intervention study is portrayed in Figure 1.
Flow of participants through study stages.
Attrition analysis was conducted to compare differences in baseline variables between the youth who dropped out of the study and those who completed the follow-up interview. Findings indicate that the groups differed in their statistical significance on hyperactivity problems (one of the two subscales of ADHD problems). Youth who dropped out of the study had a mean of 5.50 (SD = 1.63), whereas those who completed the follow-up had a mean of 3.77, SD = 2.45; t(44) = 54, p < .05. Cohen’s effect size value (d = .79) suggests large practical significance, 95% confidence interval (CI) [0.157, 1.412]. There was no statistically significant difference between the SEI and IPS groups on the rate of study attrition. There was also no statistically significant difference between groups in intervention dosage (i.e., number of youths’ weekly meetings with SEI or IPS employment, clinical, and case-management staff). Further, no adverse events were reported by participants in either group during or after this RCT.
Analysis of SEI and IPS Intervention Efficacy
Table 4 displays the baseline and follow-up mean score differences for the 38 (of 48 total) SEI and IPS youth who completed the full intervention and the follow-up interview.
Change in Outcomes Between Baseline and Follow-Up Across SEI and IPS Conditions.
Note. n = 38. CI = confidence interval.
aMean (M). bStandard deviation (SD). cCohen’s measure of effect size (d).
*p < .05. **p < .01. †p < .10.
Statistically significant changes were detected across all youth in either condition between baseline and follow-up in self-esteem (20.28 vs. 22.16), ADHD problems (8.43 vs. 6.40), inattention problems (4.67 vs. 3.27), living in a shelter (0.47 vs. 0.19), and living in a private residence (0.56 vs. 0.84). Although only statistically significant at the p < .10 level, changes were also detected in depressive problems (7.43 vs. 5.90) and hyperactivity problems (3.77 vs. 3.13). The largest effect sizes were for living in a private residence (d = .64), living in a shelter (d = −.62), inattention problems (d = −.54), and ADHD problems (d = −.46), each of which reflects moderate practical importance. Lower effect sizes were detected for self-esteem (d = .33), depressive problems (d = −.32), and hyperactivity problems (d = −.27), reflecting a small effect or practical importance.
Analysis of Treatment Differences Between SEI and IPS Groups Over Time
Within-subjects effects
Table 5 presents the baseline and follow-up mean score differences for the 21 SEI and 17 IPS participants who completed the intervention and follow-up interview.
Group Differences in Outcome Measures Between SEI and IPS Groups Over Time.
Note. n = 38. SEI = Social Enterprise Intervention; IPS = Individual Placement and Support; ASR = Adult Self-Report.
aMean (M). bStandard deviation (SD).
*p < .05. **p < .01. †p < .10.
Within-subjects tests indicate that there was a statistically significant time effect (from baseline to follow-up) for five outcome variables. Both the SEI and IPS groups experienced statistically significant changes between baseline and follow-up in self-esteem, F(1, 28) = 5.90, p < .05, ADHD problems, F(1, 28) = 8.86, p < .01, inattention problems, F(1, 28) = 11.13, p < .01, living in a shelter, F(1, 28) = 8.43, p < .01, and living in a private residence, F(1, 28) = 6.43, p < .05. Additionally, although only statistically significant at the p < .10 level, changes were also detected for both groups between baseline and follow-up in depressive problems, F(1, 28) = 3.51, p < .10, and hyperactivity problems, F(1, 28) = 3.29, p < .10.
For the SEI group, Cohen’s effect size values were d = .36 for self-esteem, 95% CI [−0.499, 1.226], d = −.51 for ADHD problems, 95% CI [−1.377, 0.361], d = −.55 for inattention problems, 95% CI [−1.420, 0.323], d = −.43 for living in a shelter, 95% CI [−1.294, 0.436], and d = .53 for living in a private residence, 95% CI [−0.339, 1.402]. The effect size values for depressive problems and hyperactivity problems were d = −.38, 95% CI [−1.239, 0.487] and d = −.31, 95% CI [−1.168, 0.553], respectively. The largest effect sizes for inattention problems (d = −.55), living in a private residence (d = .53), and ADHD problems (d = −.51) all reflect an intermediate effect or moderate practical importance.
For the IPS group, Cohen’s effect size values were d = .28 for self-esteem, 95% CI [−0.680, 1.231], d = −.41 for ADHD problems, 95% CI [−1.371, 0.550], d = −.53 for inattention problems, 95% CI [−1.492, 0.442], d = −.89 for living in a shelter, 95% CI [−1.891, 0.103], and d = .76 for living in a private residence, 95% CI [−0.222, 1.748]. The effect size values for depressive problems and hyperactivity problems were d = −.24, 95% CI [−1.191, 0.717] and d = −.21, 95% CI [−1.159, 0.748], respectively. The largest effect sizes for living in a shelter (d = −.89) and living in a private residence (d = .76) reflect a large effect or high practical importance, whereas the effect sizes for inattention problems (d = −.53) and ADHD problems (d = −.41) reflect moderate practical importance.
Conversely, there were no statistically significant interactions between condition (SEI vs. IPS) and time (baseline and follow-up). Despite both groups changing significantly over time, they were not changing in different ways over time on mental health and housing outcomes.
Between-subjects effects
Between-groups tests indicate that there was no statistically significant effect for condition (SEI vs. IPS), that is, neither group fared significantly better or worse than the other on the study’s mental health and housing stability outcomes.
Discussion and Applications to Practice
Findings from this comparative efficacy study offer valuable new information on the nonvocational (i.e., mental health and housing) outcomes associated with homeless youths’ participation in two employment interventions integrated with clinical services, whose feasibility previously had been tested. Prior studies with both interventions suggest that work is therapeutic for young people experiencing homelessness and mental illness: that is, involvement in social enterprises and competitive employment is associated with improvements in mental health, social support, and life satisfaction (Ferguson & Xie, 2008; Ferguson et al., 2012). Where the two interventions diverge is in their practice approach: The SEI is a team or group-based approach, whereas the IPS adopts an individualized, customized approach. This RCT was designed to further examine the efficacy and short-term nonvocational outcomes of both interventions with a population of homeless young people experiencing mental illness.
It is interesting to note that despite the theoretical and practical differences in the two employment approaches (e.g., the SEI adopts a team approach whereas the IPS is individually focused), neither intervention fared better or worse than the other in terms of statistical significance on the study’s mental health and housing stability outcomes. This finding supports the practice assumption common with populations experiencing homelessness and mental illness that there is no wrong path to employment (Shaheen & Rio, 2007). For individuals who encounter multiple life challenges including housing instability and mental illness, it is important to offer them an array of employment options that capitalize upon their strengths, experiences, and preferences. Both the SEI and IPS offered comparable clinical and case-management services in addition to employment services. The nature of the employment supports/services differed by intervention (agency-run social enterprise focus of the SEI vs. competitive employment focus of the IPS) as did the underlying intervention philosophy (team approach of the SEI vs. individual approach of the IPS) and intervention environment (internal agency setting of the SEI versus external community and employment-based settings of the IPS). This comparative efficacy trial is one of the first to examine two employment interventions for homeless youth with distinct natures, philosophies, and environments. Findings reveal that both types of interventions were associated with positive nonvocational outcomes related to mental health and housing. This suggests that the specific type of intervention (i.e., SEI vs. IPS) might not be as important as having an employment program integrated with clinical and case-management services, frequent contact with youth, and ongoing supports. As efficacy has been established, a subsequent comparative effectiveness trial is warranted to further examine nonvocational outcomes related to mental health and housing between the SEI and IPS employment interventions with young people experiencing homelessness and mental illness.
Participants in both the SEI and IPS conditions experienced statistically significant changes between baseline and follow-up on several nonvocational outcomes with medium-to-large effect sizes reflecting moderate-to-high practical importance. For instance, youth from both the SEI and IPS groups experienced clinically meaningful improvements in self-esteem (SEI d = .36 vs. IPS d = .28); reductions in depressive problems (SEI d = −.38 vs. IPS d = −.24), hyperactivity problems (SEI d = −.31 vs. IPS d = −.21), ADHD problems (SEI d = −.51 vs. IPS d = −.41), and inattention problems (SEI d = −.55 vs. IPS d = −.53); decreases in shelter use (SEI d = −.43 vs. IPS d = −.89); and increases in private residential living (SEI d = .53 vs. IPS d = .76). It is important to note that by follow-up, our sample size was small in both groups (SEI n = 21 vs. IPS n = 17), thus limiting our power. Nonetheless, we detected a variety of clinical effects ranging from small (self-esteem, depressive problems, and hyperactivity problems), to medium (ADHD problems, inattention problems), to large (shelter use and private residential living). This suggests that a relationship exists between the two employment interventions and these nonvocational outcomes and thus deserves further study with a larger sample and greater power.
Both the SEI and IPS influenced positive mental health outcomes among participants including statistically significant improvements in self-esteem and reductions in ADHD problems and inattention problems. The slightly stronger effect sizes for the SEI compared with the IPS on all mental health outcomes might be explained by the SEI’s use of a team approach to employment that relies on social support among SEI participants and from peer mentors throughout all phases of the intervention. Prior research reveals that social support is a critical factor on which many homeless youth rely to meet their basic needs, particularly those youth who do not frequently use agency services (Kipke, Unger, O’Connor, Palmer, & LaFrance, 1997; Unger et al., 1998). Social support is useful in helping homeless youth survive on the streets and endure daily stressors associated with homelessness. Extant findings suggest that involvement in a supportive peer group positively influences the mental health of homeless youth (Bao, Whitbeck, & Hoyt, 2000).
There are several theories about the mechanisms by which social support is effective in reducing stress, enhancing coping strategies, improving quality of life, and promoting prosocial behaviors. First, the buffer theory posits that social support serves as a mediating factor between psychosocial adversity and episodes of illness (Alloway & Bebbington, 1987). According to the buffering hypothesis, involvement in supportive networks provides coping skills, which can be used in stressful situations, thus increasing the likelihood of positive mental health outcomes. Second, the social integration perspective suggests that individuals who are embedded in supportive networks are more likely to demonstrate a positive mood, which is likely to contribute to positive mental health outcomes (Willis, 1991). The SEI, which is a team-based intervention that is taught in a group setting, relies on peers—both homeless peers and formerly homeless peer mentors—to model prosocial behavior as well as support and encourage participants in overcoming life challenges. It might be that through involvement in a peer-based supportive work environment, youth developed new or strengthened existing coping skills and optimism, which in turn facilitated more positive mental health outcomes. Future studies should test the buffering and social integration hypotheses with this population by examining the mediating role of coping skills and positive mood on mental health outcomes.
It is interesting to note that among the mental health outcomes, ADHD problems and inattention problems had the largest effect sizes, with SEI effect sizes slightly larger than those of the IPS. The impact of employment on ADHD symptoms in homeless youth and the general population has been understudied, as prior research largely has focused on pharmacological treatments and nonpharmacological approaches other than employment for ADHD (e.g., dietary interventions, psychological interventions, neurofeedback, and behavioral parent training; Faraone & Glatt, 2009; Lakhan & Kirchgessner, 2012; Sonuga-Barke et al., 2013). Understanding the role of employment as a complement or supplement to existing treatment approaches deserves greater attention for at least two reasons. First, ADHD is one of the most commonly diagnosed psychiatric conditions in the United States in both children and adults (Lakhan & Kirchgessner, 2012). In the general population, 4–5% of individuals are estimated to have ADHD, whereas as many as one third of all homeless youth meet the criteria for diagnosis (Cauce et al., 2000; Costello et al., 2003; Merscham et al., 2009). Diagnostic rates continue to rise as well. Rates in children aged 4–17 years have increased from 7.8% in 2003 to 9.5% in 2007, reflecting an increase of 21.8% in a 4-year period (Visser, Bitsko, Danielson, Perou, & Blumberg, 2010). Likewise, as many as half of all children diagnosed with ADHD under the age of 18 continue to present symptoms into adulthood (Wilens et al., 2009).
Understanding how employment functions as a feasible and effective treatment approach for ADHD deserves greater attention for a second reason as well. In this study, moderate effect sizes were detected with the SEI and IPS for ADHD problems (SEI d = −.51 vs. IPS d = −.41) and inattention problems (SEI d = −.55 vs. IPS d = −.53). These effect sizes for employment interventions surpass existing effect sizes from the literature for various other nonpharmacological treatments for ADHD including dietary interventions, psychological interventions, neurofeedback, and behavioral parent training (Sonuga-Barke et al., 2013). In light of these findings, future study of the effectiveness of employment-based interventions such as the SEI and IPS on reducing ADHD symptoms among homeless youth is a logical next step.
With respect to housing outcomes, both the SEI and IPS had a strong impact on the housing status of participants, supporting the importance of safe and stable housing to success in employment and mental health treatment (Padgett, Henwood, & Tsemberis, 2016). The stronger effect sizes for the IPS compared with the SEI on shelter living and private residential living might be explained by the need to be stably housed to be successful in obtaining and maintaining competitive employment, where traditional workplace norms often disadvantage homeless youth by requiring that they possess a mailing address, secure their own transportation to a work site, and prioritize their personal hygiene (e.g., daily showers, laundry, etc.; Ferguson, Bender, Thompson, Maccio, & Pollio, 2011). In contrast, working in a social enterprise hosted within a homeless youth organization might have made fewer demands on participants for having a personal mailing address, transportation, and a clean appearance, as these organizations typically permit youth to use the agency mailing addresses, offer them bus tokens/metro cards, and have on-site places to shower and wash laundry (De Rosa et al., 1999; Kipke et al., 1997). Although employment outcomes were outside the scope of interest in the current study, there were no statistically significant differences between the SEI and IPS groups in their weekly income, suggesting that neither group earned a significantly higher weekly pay to facilitate exiting the shelter system and financing private residential living.
We lost five participants in both groups to attrition once they began the intervention, which highlights that not every employment intervention (team or individually focused) works for everyone. Likewise, major life challenges can prompt individuals to abandon their work pursuits despite the on-site and integrated case-management and clinical services. In this study, the youth dropped out due to transience, disappearance, incarceration, and commitment to inpatient treatment for substance use disorder. Interestingly, youth who discontinued participation had higher scores on hyperactivity problems in terms of statistical significance. This finding could indicate that traditional employment (through a team environment in the SEI or individually focused in the IPS) might be more challenging for individuals struggling with hyperactivity symptoms; that is, traditional employment might not capture the interest and commitment of an overactive mind. This finding underscores the importance of providing screening, diagnosis, and treatment of ADHD symptoms for homeless youth (most of whom are out of school and not connected to school-based services for youth with psychiatric disorders) as well as customizing work supports to match clients’ preferences (Drake et al., 2012; Thompson et al., 2002).
This study had several limitations, which should be considered when interpreting findings. First, although we oversampled by 40% to factor in likely attrition, the sample size was modest, which contributes to limited statistical power. Small samples and high attrition rates are common limitations in intervention research with this population (Altena et al., 2010; Xiang, 2012). Despite limited power, this study attempted to address many of the limitations in prior intervention research with homeless youth, such as lack of randomization and weak methodological designs. This study also provides valuable information on SEI and IPS manual implementation processes, distributions of nonvocational outcome variables, comparative efficacy data on the SEI and IPS, and effect sizes for a future randomized effectiveness trial.
Similarly, the use of one agency as the study site limits the generalizability of findings to the larger population of homeless youth (i.e., homeless youth in shelters and those who do not use drop-in centers). It is possible that certain subgroups of youth (e.g., more service-engaged, more stable, less transient, higher functioning) chose to participate over others. To remedy this limitation, a randomized comparative effectiveness trial is needed to compare the SEI and IPS with a broader population of homeless youth. This would then provide a better understanding of which subpopulations of homeless youth are more likely to engage in different employment interventions and how different groups progress in their nonvocational outcomes.
Related, given that both study conditions (SEI and IPS) took place in the same agency, were administered by staff who commonly consulted with each other on cases (e.g., group supervision) and involved youth who commonly associated with peers and street families, it is possible that cross-group contamination occurred. In order to limit contamination of the intervention among staff in either group, we hired new employment staff for each condition and a clinical staff member to cover SEI clinical cases. To address possible contamination among SEI and IPS youth participants, we instructed youth of the importance of not sharing their training materials or experiences with other youth outside their program during the study period. At the follow-up interview, we asked SEI and IPS participants about their perceptions of any cross-condition diffusion or imitation, which were rarely reported.
Further, this study was a randomized comparative efficacy trial of two employment interventions (i.e., SEI and IPS), which previously had been compared with usual care (i.e., treatment as usual) comparison groups in separate quasi-experimental feasibility studies. We acknowledge that in the current study, we lacked a no-treatment or placebo control group as an additional study condition. As such, the lack of differences found between the SEI and IPS interventions could also be consistent with the hypothesis that neither treatment was better than no treatment or placebo. A future comparative effectiveness trial of the SEI and IPS interventions should include a third no-treatment or placebo group to test this hypothesis.
Finally, funding limitations precluded a longer-term follow-up of participants, with the only follow-up occurring during months 20–24 of the study. We are therefore unable to determine whether the positive short-term outcomes of changes in mental health status and housing stability were sustained over time. A future longitudinal study with short- and long-term follow-ups is needed to examine the sustainability of nonvocational outcomes over a longer period of time.
Conclusions
Ending homelessness among young people experiencing mental illness—and preventing their transition to chronic adult homelessness—requires a comprehensive approach that combines housing, employment, educational, mental health, and case-management services. Although there are myriad programs across the country to support homeless youth, intervention research to date with this population largely has been compromised by nonrandomized designs, small samples, and high attrition rates (Altena et al., 2010; Slesnick et al., 2009; Xiang, 2012).
In an effort to remedy some of the limitations in the extant research, this RCT compared the efficacy and short-term nonvocational outcomes of two employment interventions integrated with clinical and case-management supports. Results reveal that participants in both the SEI and IPS demonstrated statistically significant changes in their self-esteem, ADHD problems, inattention problems, and housing status. Effect sizes for mental health outcomes were slightly higher for the SEI condition, whereas effect sizes for housing outcomes were slightly higher for the IPS condition. Despite the differences in the two approaches, neither intervention fared better than the other in terms of statistical significance on mental health and housing stability outcomes. This finding suggests that the employment intervention type might not be as important as the particular intervention components (i.e., integrated clinical and case-management services, frequent contact, and ongoing mentoring and support), which were offered in both interventions. This finding also reaffirms that there is no wrong path to employment for individuals experiencing homelessness and mental illness (Shaheen & Rio, 2007). Employment interventions should therefore be flexible and client-driven to incorporate clients’ strengths and preferences (Drake et al., 2012) as well as offer both team-based and individually focused services. Future research is needed to test the comparative effectiveness of the SEI and IPS interventions with a broader population of homeless youth, in multiple agencies, and over a longer period of time.
Footnotes
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the National Institute of Mental Health (1R34MH082804-01A2) and the Columbia University Center for Homelessness Prevention Studies Scholars’ Program.