A Randomized Controlled Trial Study of Transforming Impossible into Possible (TIP) Policy Experiment in South Korea

Abstract

Purpose:

This randomized controlled trial (RCT) study examined the efficacy of the Transforming Impossible into Possible (TIP) program for low-income, low-skilled jobseekers in the Self-Sufficiency Program (SSP) in South Korea.

Method:

A total of 169 participants in SSP regional centers were randomly assigned to the intervention group (104 participants) and the wait-list control group (65 participants). Employment hope, perceived employment barriers, economic self-sufficiency (ESS), self-esteem and self-efficacy were measured at two time points, and data were analyzed using a paired sample t-test and an analysis of covariance (ANCOVA).

Results:

The TIP intervention group showed statistically significant increase in employment hope, ESS, self-esteem, and self-efficacy. The ANCOVA result indicated significant pre–post score differences in these variables between the TIP and the control groups.

Discussion:

TIP program can be effective in increasing psychological capital for low-income and low-skilled jobseekers. Further research ought to replicate the current RCT study to determine its application to other populations.

Keywords

Transforming Impossible into Possible (TIP)psychological self-sufficiency employment hope psychological capital workforce development poverty field of practice

Psychological Self-Sufficiency (PSS) Theory

PSS theory is a social work practice theory which illustrates how individuals continue to facilitate the dynamic process of goal-directed commitment and action when the structural conditions may not necessarily be favorable (P. Y. P. Hong, Choi, & Key, 2018; R. Hong, Northcut, Spira, & Hong, 2019). Initially, the core construct of PSS emerged from focus groups of participants and service providers in job training programs when exploring the participant-centered definition of self-sufficiency in workforce development (P. Y. P. Hong, 2013; P. Y. P. Hong, Sheriff, & Naeger, 2009). Employment hope and perceived employment barriers were conceived of as a process-based, bottom-up corollaries of the PSS theory (P. Y. P. Hong, 2013; P. Y. P. Hong, O'Brien, et al., 2019). Both conceptual pillars of PSS were qualitatively and quantitatively validated through a series of studies conducted at numerous local workforce development agencies in both St. Louis and Chicago from 2004 to 2018. Based on the data from more than 5,000 surveys, the Employment Hope Scale (EHS; P. Y. P. Hong & Choi, 2013; P. Y. P. Hong, Choi, & Polanin, 2014; P. Y. P. Hong, Polanin, & Pigott, 2012, Hong, Song, Choi, & Park, 2016) and the Perceived Employment Barrier Scale (PEBS; P. Y. P. Hong, Polanin, Key, & Choi, 2014; P. Y. P. Hong, Song, Choi, & Park, 2018) were validated and used for empirically testing of the theory using cross-sectional and longitudinal data.

As a goal-directed process, PSS is posited to represent the force within someone that activates interchangeable cognitive (explicit) and noncognitive (implicit) process, enabling them to alter the intrapsychic structures from negative and hopeless narratives to those that are positive and hopeful by focusing on the possibilities and opportunities (R. Hong et al., 2019). PSS is also a counternarrative to the top-down outcome-based definition of self-sufficiency typically understood as leaving welfare and finding employment under the provisions of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (U.S. Public Law 104-903) and Workforce Innovation and Opportunity Act (U.S. Public Law 113-128). Absent in these provisions are any reference to the integral steps needed as to “how” the self-sufficiency outcome can be attained (Bernstein, Brocht, & Spade-Aguilar, 2000; Daugherty & Baber, 2001; Gowdy & Pearmutter, 1993; P. Y. P. Hong, 2013). Limited by the market-driven workforce development policy provisions, community-based training programs have focused mainly on short-term outcome measures (i.e., employment, sustained employment, increased earnings, and/or welfare exit). Subjective experiences about successes or failures along the trajectory of seeking long-term employment outcomes among low-income jobseekers have not been attended to (P. Y. P. Hong, 2013). Responding to practitioners’ concerns about being confined to top-down labor market outcomes in human service programs (Harvey, Hong, & Kweli, 2010), understand of the PSS process has become all the more salient particularly in light of “how subjective contexts of individuals can either impede movement forward or facilitate that movement” (R. Hong et al., 2019, p. 3). By strengthening the internal locus of control for individuals to exercise against barriers, PSS represents one possible route of how individuals could reach economic self-sufficiency (ESS) or long-term employment outcomes (P. Y. P. Hong, 2013; P. Y. P. Hong, Choi, & Polanin, 2014).

Transforming Impossible into Possible (TIP) Group Model

Developed by Dr. Philip Hong and his research team at the Center for Research on Self-Sufficiency at Loyola University Chicago, TIP is a participant-centered group intervention model that was designed based on the PSS theory to increase and strengthen one’s PSS process (P. Y. P. Hong, 2016a, 2016b). The TIP program was introduced as a promising social work practice model that embodies the subjective contexts of participants to empower themselves and create an inside-out, bottom-up change strategy by enhancing one’s PSS (P. Y. P. Hong, 2016a, 2016b). Recently, it was featured in a U.S. Department of Health and Human Services report by Mathematica Policy Research as one of the top five evidence-informed social–emotional training programs in workforce development for low-income populations (Anderson, Brown, Cavadel, Derr, & Kauff, 2018). Findings from a 2- year TIP program implementation results from a job readiness training (JRT) at a vocational training organization on the West Side of Chicago consistently revealed that perceived employment barriers decreased and employment hope increased, demonstrating an overall increasing pattern of PSS, over time for six cohorts during approximately 12 weeks of program participation (P. Y. P. Hong & Fittin, 2017). This also was coupled by continued increase in ESS for program participants, which translated to objective labor market outcomes. Compared to agency program outcomes prior to TIP implementation, program completion rate stayed about the same from 80% to 81%. The job placement rate was 65% in 2 years prior to TIP implementation and 78% during the 2 years of TIP implementation—an increase of 13% points. The job retention rate was 60% before TIP and 71% during this period of integrating TIP into the agency’s JRT curriculum—an increase of 11% points.

The TIP program attempts to help individuals go through the recursive, integrative process that allows the “TIPPING” (being in the process of transforming) of barriers to goal-directed hope actions by discovering the source of strength that one can draw from in their lives (R. Hong et al., 2019). By engaging in praxis—“reflection and action upon the world in order to transform it” (Freire, 2000, 2014, p. 33)—along with various mindfulness and kinesthetic activities, the TIP program allows individuals to deftly weave the meanings of their difficulties, identify obscure strengths within and around themselves, and successfully transform their perceived barriers into hope driven actions (P. Y. P. Hong, 2016a, 2016b; R. Hong et al., 2019).

The recursively interrelated 15 group modules containing nine themes in the TIP program encourages individuals to begin with a transformative journey by seeking one’s existential meaning as it relates to pursuing a goal in life. The 9 themes and 15 modules are (1) identity and purpose (TIP 1: who am I/purpose in life); (2) forgiveness (TIP 2: self-compassion); (3) goal orientation (TIP 3: goal setting; TIP 4: improvement and pathways [1]); (4) barriers (TIP 5: barriers inventory); (5) source of strength (TIP 6: my strength comes from); (6) employment hope (TIP 7: love and self-worth; TIP 8: self-perceived capability; TIP 9: future and the possibilities; TIP 10: self-motivation; TIP 11: skills and resources inventory; TIP 12: renewed goal commitment/improvement and pathways [2]); (7) unresolved triggers of stress and anger (TIP 13: caring for stress and anger); (8) gratitude (TIP 14: finding meaning in life); and (9) social support and compassion (TIP 15: seeking help and helping others). The innovative quality of TIP is the nonlinear thematic matching of the outer rings—Themes 1 and 9, Themes 2 and 8, Themes 3 and 7, and Themes 4 and 6—surrounding the inner core of Theme 5 (see Figures 1 and 2).

Figure 1.
TIP program curricular content.

Figure 2.
TIP circle by nine themes. Source: Hong (2016a).

Since its inception in 2014, TIP program has been utilized in the context of engaging welfare recipients for employment stability and has been expanded to strengthen programs that serve the homelessness, returning citizens, and refugees for employment and financial literacy, fathers for family strengthening, and youth for education and employment. The comprehensive and multidimensional representation of the neurobiological core content in the TIP program has been theoretically shown to increase the brain’s integrative process by which it has been hypothesized to strengthen executive functioning and emotional regulation in the brain (R. Hong & P. Y. P. Hong, 2019).

The Self-Sufficiency Program (SSP) in South Korea

Over the last three decades, the market-based concept of self-sufficiency in the U.S. social welfare policy has impacted welfare reforms in other countries. The SSP in South Korea is one example of U.S. policy transfer (P. Y. P. Hong et al., 2016). Modeled on the U.S. welfare-to-work policy-, the SSP is South Korea’s version of the workforce development policy for welfare recipients (P. Y. P. Hong et al. 2016). Under the National Minimum Livelihood Security Act of 2001, which guarantees basic living standards for low-income individuals, the South Korean government has launched SSP nationally to provide low-income individuals with social service support and employment opportunities. Work abled individuals between ages 18 and 65 years who are living in households with incomes below the poverty line—approximately US$1,360 per month for a family of four or US$500 for a family of one—are required to participate in SSP to receive cash assistance (The Ministry of Health and Welfare, 2016; Yoo & Lee, 2011). The collaboration between two major government agencies—the Ministry of Health and Welfare and the Ministry of Employment and Labor—resulted in the establishment of many local self-sufficiency centers focused on helping low-income individuals to gain self-sufficiency by providing various services such as capability building, social services, job training, and employment support (Central Self-Sufficiency Center, n.d.). The self-sufficiency centers are currently widespread in South Korea under the auspices of the Central Self-Sufficiency Center that oversees 14 metropolitan centers and 249 local centers in South Korea (Ministry of Health and Welfare, 2016).

Unfortunately, the effectiveness of early SSP implementation measured by the labor market participation rate was minimal. The Department of Health and Welfare frontline staff voiced frustrations regarding the SSP’s limited impact and governmental adherence to the ineffective status quo. Kim and Zurlo (2007) found that of 3,907 SSP participants, after 500 participants (12.8%) dropped out, only 435 (11.1%) of the 2,972 (76.1%) remaining in the program found employment. They found that SSP participants who were younger and more educated and who have more additional family income sources were more likely to be self-sufficient (Kim & Zurlo, 2007). Yoo and Lee (2011) found that among a total of 265 individuals in the SSP as well as those who left during the 2-year study period, 68.2% (n = 180) had severe mental health issues, and individuals with mental health problems were 4 times as likely to leave the SSP as those without.

Acknowledging the need to provide more intensive and holistic programming that centers around emotional and psychological support for SSP participants, the PSS measures—EHS and PEBS—and the PSS theory that guides the implementation of the TIP program have garnered serious attention from the South Korean government. Informed by the emerging literature on improvement science (Bryk, Gomez, Grunow, & LeMahieu, 2015; Langley et al., 2009)—PSS in this context being a process-focused, participant-centered learning and evolving social work practice theory—South Korea’s Department of Health and Welfare incorporated the PSS measures and the TIP model into SSP as a national demonstration project in 2015–2017. This move of conducting a national policy demonstration using P. Y. P. Hong’s (2013) model of evaluating the PSS “process” in lieu of the employment outcomes—using the EHS and PEBS measures as a predictive proxy for long-term sustainable employment effects among low-income jobseekers—is a significant policy investment for South Korea. It represents the government’s recognition and political will to diverge from its narrowly restricted outcome-based evaluation of SSP to a more comprehensive and inclusive evaluation of the process that leads to the ESS outcome by focusing on the “psychological” or “emotional” factors (Song, Kwon, Kim, Lee, & Park, 2013; Um, 2010). Moreover, it symbolizes the government’s embracement of P. Y. P. Hong’s (2016a) TIP program because of its emphasis on a human-centered process of shifting individuals’ perceived barriers to goal-directed hope actions by which a system-level transformation emerges (P. Y. P. Hong et al., 2018; R. Hong et al., 2019).

While implications from the previous Korean studies (Song et al., 2013; Um, 2010) may only be particularistic rather than being generalizable due to the exploratory nature of evidence generated from one or two agencies, it is clear that psychological capital found in emotional and psychological interventions is central to helping SSP participants to achieve self-sufficiency. Hence, the purpose of this study is to test the efficacy of the TIP program among the SSP participants in South Korea in terms of its effects on psychological capital measures—that is, employment hope, self-esteem, and self-efficacy—and ESS through a government-sponsored randomized control trial (RCT) experiment, which is not preregistered in a clinical trial registry.

Method

Sample and Recruitment

Participants in this study were 169 low-income, low-skilled jobseekers, who enrolled in the SSP at regional centers in the Kyunggi Province of South Korea. A total of 169 participants in the study were randomly assigned into two groups at each regional center: the TIP intervention group (n = 104) and the control group (n = 65) from October 2016 to March 2017. Inclusion criteria were adults enrolled in the SSP in Kyunggi regional centers. The mean age for the TIP intervention group was 48.9 (SD = 12.35). Average age for the control group was 51.39 (SD = 10.41). Slightly over half of the sample was female in both groups (TIP intervention group = 52.9%; control group = 55.4%). The sample had a disproportionately low percentage of participants living together with their spouses (TIP intervention group = 11.2%; control group = 16.1%), implying that SSP participants mostly lived alone. About 40% of the sample did not complete high school (TIP intervention group = 41.5%; control group = 40.6%). The majority of the participants received welfare benefits, equivalently in both groups (TIP intervention group = 81.6%; control = 76.1%). Approximately 40% of the sample responded that they were healthy (TIP intervention group = 41.6%; control = 39.7%).

The TIP intervention group received 10 group sessions (2 hr per session) of the TIP program along with the SSP programmatic support such as job training and job referrals. Participants in the control group only received the treatment-as-usual SSP services. A χ² test of homogeneity confirmed sociodemographic homogeneity between two groups with no statistically significant differences in key variables, as presented in Table 1.

Table 1.
Demographic Characteristics and χ²-Test Results of Homogeneity of the Transforming Impossible Into Possible (TIP) Intervention and Control Groups.

Characteristics TIP Intervention (N = 104) Control (N = 65) χ² Test of Homogeneity

n % n % x² p Effect Size^a

Gender

Male 48 47.1 29 44.6 0.009 .757 .023

Female 54 52.9 36 55.4

Marriage

Not married 31 31.6 14 22.6 1.914 .384 .110

Married and living together 11 11.2 10 16.1

Separated, divorced, or bereaved 56 57.1 38 61.3

Education level

Lower than high school 42 41.5 26 40.6 0.015 .903 .009

High school and above 59 58.4 38 59.4

Health

Healthy 42 41.6 25 39.7 1.043 .594 .080

Having chronic disease 34 33.7 18 28.6

Not healthy or disabled 25 24.7 20 31.7

Receipt of welfare benefits

Yes 80 81.6 48 76.1 0.403 .526 .066

No 18 18.3 15 23.8

Age (year) Mean (min–max) SD Mean (min–max) SD F p Cohen’s d

48.9 (20–66) 12.35 51.39 (22–64) 10.41 1.803 .178 0.218

^a Phi and Cramers’ V were used to measure the effect size of χ² tests. While Phi was used to measure the strength of the association between two nominal variables, Cramer’s V was used in tables bigger than 2 × 2 tabulation. The values of >.25, >.15, >.10, >.05, >0 mean very strong, strong, moderate, weak, and no or very weak, respectively, in Phi and Cramer’s V (Akoglu, 2018).

Design and Procedure

Recruitment was conducted with the approval of the institutional review board of Loyola University Chicago. The original TIP program manual was translated into Korean by the Center for Research on Self-Sufficiency staff at Loyola University Chicago and was used for the experiential training held on May 10–11, 2016 in South Korea. The follow-up facilitator training took place on July 26, August 18, and August 23, 2016, with key personnel who were to be the lead trainers for all sites in Kyunggi Province of South Korea. After the original 15 session TIP program was condensed to 10 sessions to accommodate the structure of SSP and service delivery (see Figure 1), the pilot implementation of the TIP program began on August 26, 2016 at the Buchon SSP site—one of the lead program sites in Kyunggi Province.

Of 249 SSP regional centers in South Korea, the 18 SSP regional centers in the Kyunggi Province were invited and among them, 10 SSP regional centers agreed to participate in the study. Thirty-eight staff members from these centers were recruited and agreed to be trained as TIP facilitators. In a two-day experiential TIP training conducted by the lead author at the Central SSP Training Center in South Korea, the staff learned about evidence building and developmental history, mission, and core principles and fully experienced the TIP program activities. The authors collaborated with the regional centers to collect the pretest surveys at the beginning of the first TIP session and the posttest surveys after completing the last TIP session between October 2016 and March 2017. Pretest surveys for the control group were administered between October and December 2016, and posttest surveys were administered between January and March 2017.

Both the pretest and posttest surveys took approximately 30–40 min to complete, Of 187 recruited SSP participants, 18 participants (14 in the intervention group and 4 in the control group) discontinued participating in the study due to leaving the SSP. Participation in the study was voluntary; no monetary incentives were given to the participants. To minimize selection bias, all study participants were encouraged to stay fully engaged during SSP participation.

Intervention

The TIP curriculum consists of thematic reflection questions, small-/large-group sharing and activities which are integrated carefully throughout the nine themes (P. Y. P. Hong, 2016a, 2016b). Applied to South Korea’s SSP, the 15 session TIP was delivered in a condensed format with two sessions a day over the course of five to seven consecutive days for the 10-session total. Each TIP intervention group comprised five to ten participants.

TIP is designed to help low-income jobseekers to strengthen PSS by uncovering the barriers such as sickness, lack of education or career, low self-esteem, lack of self-efficacy, or hopelessness which can be obstacles to attaining and retaining employment (P. Y. P. Hong, R. Hong, Lewis, & Williams, 2019). With each session lasting roughly 2 hr, TIP participants learned to create internal and external opportunities by discovering their strengths, skills, and resources and to use them for navigating the pathway toward their goals (P. Y. P. Hong et al., 2019). A rapid-cycle evaluation was utilized to provide Korean policymakers with scientifically rigorous, timely, and actionable evidence of how the TIP program could improve psychological capital and impact policy outcomes for SSP in the South Korean sociopolitical context (Mathematica Policy Research, 2017).

Outcome Measures

Employment hope

Employment hope was measured using the Korean-translated version (K-EHS) of the original EHS (P. Y. P. Hong, Choi, & Polanin, 2014; P. Y. P. Hong, Polanin, & Pigott, 2012). Employment hope captures an aspect of the multidimensional PSS as a client-centered and process-oriented assessment (P. Y. P. Hong, 2013). Validated by Choi, P. Y. P. Hong, and Kim (2017) using a sample of South Korean low-income jobseekers, K-EHS is a 17-item four-factor measure: (1) self-perceived worth and capability (6 items), (2) motivation for future outlook (4 items), (3) goal orientation (4 items), and (4) utilization of skills and resources (3 items). These four factors were found to be consistent with those of the original EHS. While the original EHS used a 11-point Likert-type scale ranging from 0 to 10, K-EHS used a 5-point Likert-type scale with 1 indicating “strongly disagree” and 5 indicating “strongly agree.” The K-EHS is reported to have an internal consistency coefficient of .943 (Choi, P. Y. P. Hong, & Kim, 2017).

Perceived employment barriers

The PEBS (P. Y. P. Hong et al., 2014) is a 20-item scale to comprehensively assess the client-centered employment barriers. The PEBS has a five-factor 2-item structure: (1) physical and mental health barriers (4 items), (2) labor market exclusion barriers (4 items), (3) childcare barriers (4 items), (4) human capital barriers (4 items), and (5) soft skills barriers (4 items). The PEBS is a Likert-type scale ranging from 1 to 5, 1 indicating “not a barrier” and 5 indicating “strong barrier.” Each question reflects respondents’ perception of the employment barriers—that is, lack of adequate job skills. While P. Y. P. Hong et al. (2014) do not report the overall internal consistency reliability of PEBS, each factor α scores were .925, .765, .743, .755, and .904, respectively. The internal consistency coefficients in the intervention and control group of this study are .917 and .836, respectively.

Economic self-sufficiency (ESS)

ESS was measured by the Women's Employment Network ESS Scale to measure the multidimensionality of ESS (Gowdy & Pearlmutter, 1993). This continuous measure includes 15 questions that fall under four factors: (1) autonomy and self-determination, (2) financial security and responsibility, (3) family well-being and self-well-being, and (4) basic assets for community living. Each question reflects respondents’ assessment of how their financial situation for the past three months allowed them to do certain things that represent ESS—that is, pay my own way without borrowing from family or friends. Respondents rated each statement on a Likert-type scale ranging from 1 to 5, 1 indicating “not at all” and 5 indicating “all the time.” Internal consistency was reported at .89 (Gowdy & Pearlmutter, 1993).

Self-efficacy

Self-efficacy was measured using the New General Self-Efficacy Scale (NGSE; Chen, Gully, & Eden, 2001). NGSE is positively correlated with optimism, work satisfaction, and hope, while negatively correlated with depression, stress, and anxiety. NGSE is a Likert-type scale ranging from 1 to 5, 1 indicating “not at all true” and 5 indicating “exactly true.” Each question reflects respondents’ perception of self-efficacy—that is, I will be able to achieve most of the goals that I have set for myself; In general, I think that I can obtain outcomes that are important to me. Test–retest reliability is between .86 and .90 (Chen et al., 2001).

Self-esteem

Self-esteem was measured using the 10-Item Rosenberg Self-Esteem Scale (Rosenberg, 1965) to capture the self-acceptance aspect of self-esteem. Each item scores 1–5, and the total scores range from 10 to 50. Higher scores indicate greater self-esteem. Internal consistency for the Rosenberg Self-Esteem Scale was reported at .72 (Silber & Tippett, 1965).

Statistical Analysis

Survey data were collected using standardized questionnaires distributed at two time points—pre and post intervention—and analyzed using the IBM SPSS Statistics 23 (IBM, 2015). First, univariate summary statistics—mean values and standard deviations—for each outcome variables were observed and score differences between pretest to posttest were calculated for intervention and control groups, respectively, using a paired sample t-test. Second, an analysis of covariance (ANCOVA), with the pretreatment assessment scores as covariates and posttest scores as dependent variables, was utilized to test for differences between the intervention and control groups. The demographic variables that may affect the posttest scores such as education level, age, gender, and welfare receipt status were controlled.

Results

Differences at Pretest Assessments

The normality can be assumed if skewness is less than |2| and kurtosis is less than |7| (West, Finch, & Curran, 1995). The normality assumption in this study was met with the skewness values less than |1.2| and kurtosis values less than |4.1|. The skewness and kurtosis values for the five subscales are reported in Table 2.

Table 2.
Descriptive Statistics and the Results of Paired Samples t-test for the Intervention (n = 104) and Control Group (n = 65).

Pretest Posttest Result of Paired Samples t-Test

Groups Mean SD Skewness Kurtosis Mean SD Skewness Kurtosis t p Cohen’s d

Employment hope intervention 3.49 .79 −.439 .485 3.67 .71 0.078 −0.660 2.89 .005 .287

control 3.49 .70 −.017 −.967 3.38 .70 0.494 −0.207 −1.33 .189 .165

Employment barrier intervention 2.41 .74 .371 −.616 2.40 .86 0.630 0.109 0.192 .848 .019

control 2.56 .79 .367 .228 2.43 .76 0.822 1.207 −1.26 .212 .156

Economic self-sufficiency intervention 2.06 .56 .440 −.426 2.24 .68 1.109 1.738 3.55* .001 .357

control 1.91 .37 .466 .557 1.99 .53 1.094 2.448 1.39 .171 .172

Self-esteem intervention 3.10 .68 −.281 .574 3.27 .67 0.187 0.239 3.39 .001 .341

control 3.22 .53 −.208 .672 3.07 .40 −1.216 4.093 −1.84 .070 .228

Self-efficacy intervention 3.34 .82 −.142 .651 3.54 .85 −0.204 0.269 3.02 .003 .304

control 3.51 .75 .188 −.985 3.46 .77 0.300 −0.126 −0.652 .516 .008

p < .01. p < .001.

When using an independent sample t-test, a statistical difference was found between the intervention group and the control group on ESS at pretest with the intervention group reporting higher levels of ESS than the control group (t = 2.098, p = .037). However, there was no significant difference in other variables: employment hope (t = −.173, p = .863), perceived employment barrier (t = −1.288, p = .200), self-esteem (t = −1.116, p = .266), and self-efficacy (t = −1.463, p = .147).

Efficacy of TIP Intervention: Results of a Paired Sample t-Test

This study conducted a paired sample t-test to examine the overall improvement for both the intervention and control groups in their prepost scores (see Table 2). While significant differences were found between pretest and posttest scores in employment hope (t = 2.95, p = .004), ESS (t = 3.44, p = .001), self-esteem (t = 3.09, p = .003), and self-efficacy (t = 3.02, p = .003) for the TIP intervention group, no significant differences were reported for the control group. Besides, the study estimated the values of Cohen’s d to verify the experimental effect more precisely. As shown in Table 2, the results of Cohen’s d for employment hope, ESS, self-esteem, and self-efficacy indicated that the intervention group generally have medium-sized effects between .2 and .5 while the control group have small-sized effects less than .2 (Cohen, 1988). No significant change or very weak effect sizes were found in the perceived employment barrier scores in both groups.

These results from the paired sample t-test, showing the significant differences in the intervention group and no significant differences in the control group, indicate that participation in the TIP program may contribute to the psychological empowerment of participants in terms of self-esteem, self-efficacy, employment hope, and ESS.

Efficacy of TIP Intervention: Results of ANCOVA

To ensure the efficacy of the TIP intervention, we performed an ANCOVA (see Table 3). The results of ANCOVA indicated significant differences in the prepost score changes of employment hope (F = 6.46, p = .012), ESS (F = 4.64, p = .033), self-esteem (F = 7.56, p = .007), and self-efficacy (F = 2.95, p = .009) between the TIP intervention group and the control group. The partial η² score to measure the effect size of ANCOVA also showed that the values of the intervention group are significantly larger than those of the control group. According to Gray and Kinnear (2012), the effect sizes of the intervention group are large (η² ≥ .014), while those of the control group are none or small (.01 ≤ η²< .06) for all the variables (see Table 3). These results imply that the prepost increase in psychological capital and ESS measures in the TIP intervention group is significantly greater than those in the control group, suggesting preliminary evidence of TIP program efficacy.

Table 3.
The Results of Analysis of Covariance.

Group Sum of Squares df MS F p Partial η²

Employment hope Covariate (baseline) 25.195 1 25.195 79.606 .000 .351

Group difference 2.044 1 2.044 6.457 .012 .042

Error 46.526 147 0.317

Employment barrier Covariate (baseline) 22.005 1 22.005 40.018 .000 .217

Group difference 0.376 1 0.376 0.683 .410 .005

Error 79.182 144 0.550

Economic self-sufficiency Covariate (baseline) 17.739 1 17.739 74.796 .000 .339

Group difference 1.100 1 1.100 4.638 .033 .031

Error 34.626 146 0.237

Self-esteem Covariate (baseline) 14.627 1 14.627 60.703 .000 .295

Group difference 1.822 1 1.822 7.561 .007 .050

Error 34.939 145 0.241

Self-efficacy Covariate (baseline) 45.229 1 45.229 121.466 .000 .456

Group difference 1.097 1 1.097 2.947 .088 .020

Error 53.992 145 0.372

Although not reported, gender (male = 0, female = 1), age, education level (below high school = 0, high school and above = 1), and welfare receipt (yes = 0, no = 1) as control variables, and no significant differences were found for the control variables.

*p < .05, **p < .01.

Discussion and Applications to Practice

This RCT study presents the effectiveness of the TIP intervention for 169 low-income and low-skilled jobseekers from 10 SSP regional centers in the Kyunggi Province of South Korea. Psychological capital and ESS outcomes are assessed and compared between the SSP participants who received the TIP intervention and those who did not. RCT results provided preliminary evidence of the efficacy of TIP intervention in South Korea’s sociopolitical context using a rapid-cycle evaluation approach (Mathematica Policy Research, 2017).

Results from this study seem to support the implementation science framework to promote and to integrate evidence-based practice into policies. Moreover, since the PSS theory and TIP practice development follow the Bryk, Gomez, Grunow, and LeMahieu’s (2015) improvement science framework to deploy rapid tests of organizational change to help develop, test, and refine new tools, processes, work roles, and relationships (Bryk, Gomez, & Grunow, 2011; Langley et al., 2009), it is appropriate to use the RCT design with a rapid-cycle evaluation approach to quickly test the intervention and make improvements by iterative processes of learning to convert ideas into action.

The TIP intervention is found to enhance psychological capital such as employment hope, self-esteem, self-efficacy, and the ESS outcomes compared to the control group. The results of the paired sample t-test shows a significant difference in prepost score change of employment hope, ESS, self-esteem, and self-efficacy in the TIP intervention group, but not in the control group. ANCOVA results also suggest that the prepost score change is significantly greater for the TIP intervention group compared to the control group.

It is interesting to note that perceived employment barrier is not found to be significant in either the paired sample t-test or ANCOVA contrary to previous research that employment hope was found to be a full mediator between perceived barriers and ESS (P. Y. P. Hong, Choi, & Key, 2018) and that perceived barriers tend to decrease when individuals begin the process of moving forward with increased hope as part of the goal-directed PSS process (P. Y. P. Hong, 2013). This result is perhaps due to a brief duration and condensed version of TIP implementation using a rapid-cycle evaluation. Although employment hope can generate motivation to transform barriers into possibilities on one’s pathway to ESS ( P. Y. P. Hong, Kim, Hong, Park, & Lewis, 2020; P. Y. P. Hong, Polanin, Key, & Choi, 2014), it seems to require time to process and reflect on situations and experience.

Another possible reason for no statistical significance found in perceived employment barriers may be due in part to the issue of facilitator capability and fidelity. Currently, the TIP institute in the United States requires a 2-day extended TIP experiential training and a 2-day TIP facilitator training for TIP facilitators to be certified to lead and manage the delivery of the TIP program. Although the lead author offered a 2-day extended TIP experiential training, the South Korean TIP facilitators may have been left ill-prepared to adapt the PSS practice theory to the full extent in their manualized delivery of the program.

Lastly, the SSP Regional Centers were not clear about the fidelity of the TIP program in their concern to make the U.S.-based social work practice model culturally sensitive to the South Korean sociopolitical context. The condensed version of the TIP curriculum and the boot camp style of delivery in five to seven consecutive days used in the rapid-cycle evaluation approach created a local reaction bias against the structural barriers that plague the individual barrier conditions for SSP participants. Against the suggested fidelity of the TIP program, the directors of the SSP regional offices and their frontline staff who were being trained to be TIP facilitators felt that addressing barriers at a deeper level would make participants more depressed and would limit the positive empowerment that TIP would bring to their participants. Not having a local TIP fidelity expert during the posttraining implementation stage, the South Korean researchers and practitioners departed from one core component of the TIP program by easing over the barrier section and by employing local solution-focused and strengths-based practitioners to train with a heightened focus on employment hope.

Placing key focus on only the positive aspects of the PSS theory created an imbalance in their application and perhaps disrupted the integrative, recursive TIPPING process—a bottom-up lay language that emerged from TIP participants to represent the PSS process of transforming barriers to hope actions (R. Hong et al., 2019). Therefore, future RCT studies ought to be followed with a stronger commitment to implementation fidelity and contribution to learning for improvement. However, this study is meaningful in that it expands the cultural adaptation of the TIP program to the South Korean SSP population. Most importantly, using an RCT design with an improvement science approach makes it a methodologically rigorous study.

Nevertheless, our findings are consistent with previous research on PSS, showing that low-income jobseekers benefit from a process of increasing employment hope leading to ESS. The TIP program—being the social work practice model representing the PSS theory—is conceptualized to activate PSS, which enables individuals to alter their intrapsychic structure from negative and hopeless narratives to a growth mindset (R. Hong et al., 2019). It is plausible to attribute the TIP program for offering individualized opportunities for clients, including but not limited to jobseekers, to understand their values and capabilities, to self-motivate in the development, planning, and achievement of specific goals.

Some limitations need to be noted, however. First, by using a three month pretest–posttest time interval, the study falls short of testing the long term effect of the TIP intervention. Administering follow-up measures—to capture both self-report and administrative data—6 months after the TIP program will provide evidence for TIP’s long-term effect. Second, as noted previously, the condensed TIP curriculum with limited application of the barrier component may have fallen short of implementation fidelity in this study.

Despite these limitations, the efficacy of the TIP program in the South Korean sociopolitical context may contribute to a rethinking of the traditional outcome-focused SSP. The positive results of increased psychological capital among South Korea’s SSP participants provide evidence for a holistic, client-centered approach to promoting the policy outcome of ESS. As opposed to top-down, outcome-driven workforce programs, this research supports TIP’s merit as a bottom-up process-oriented approach that helps individuals engage in praxis (reflection + action) for enhancing PSS.

As such, it warrants a clinical application of the TIP intervention to social work practice. It also suggests further discussions on implementing TIP as a multilevel social work practice model designed to enhance PSS among clients at the individual level that will create a ripple effect on the mezzo and macro social policy levels (P. Y. P. Hong, 2016a). System change is theorized to occur from starting the PSS trajectory at the individual level which gradually leads to create the market demand for system-level responses to address individual needs by “glocalizing structural poverty” in framing the problem definition coupled with local participant-centered solutions (P. Y. P. Hong, 2008, 2016a). Findings add great value by highlighting a social work practice model to have such significant impact on national policy performance. Further research ought to replicate the current RCT study to determine its applicability to other populations and contexts.

To conclude, this is the first known RCT of the TIP program for low-income jobseekers. This rapid-cycle evaluation using an RCT design can help with policy learning in the tradition of improvement science. Focus groups and in-depth interviews with program participants and TIP facilitators could provide triangulation of data that could help further improve the application and implementation of TIP intervention in the South Korean sociopolitical context. Further research is needed for cross-cultural, cross-national learning of TIP efficacy not as a standalone one size fit all type model but one that possess the “glocal” respect and humility to adapt to various local needs in the community for an authentic self-sufficiency process to be owned by the people—not coerced by the neoliberal shaping of global and national policy agenda.

Characteristics	TIP Intervention (N = 104)	Control (N = 65)	χ² Test of Homogeneity
Gender
Male	48	47.1	29	44.6	0.009	.757	.023
Female	54	52.9	36	55.4
Marriage
Not married	31	31.6	14	22.6	1.914	.384	.110
Married and living together	11	11.2	10	16.1
Separated, divorced, or bereaved	56	57.1	38	61.3
Education level
Lower than high school	42	41.5	26	40.6	0.015	.903	.009
High school and above	59	58.4	38	59.4
Health
Healthy	42	41.6	25	39.7	1.043	.594	.080
Having chronic disease	34	33.7	18	28.6
Not healthy or disabled	25	24.7	20	31.7
Receipt of welfare benefits
Yes	80	81.6	48	76.1	0.403	.526	.066
No	18	18.3	15	23.8
Age (year)	Mean (min–max)	SD	Mean (min–max)	SD	F	p	Cohen’s d
	48.9 (20–66)	12.35	51.39 (22–64)	10.41	1.803	.178	0.218

		Pretest	Posttest	Result of Paired Samples t-Test
Employment hope	intervention	3.49	.79	−.439	.485	3.67	.71	0.078	−0.660	2.89**	.005	.287
control	3.49	.70	−.017	−.967	3.38	.70	0.494	−0.207	−1.33	.189	.165
Employment barrier	intervention	2.41	.74	.371	−.616	2.40	.86	0.630	0.109	0.192	.848	.019
control	2.56	.79	.367	.228	2.43	.76	0.822	1.207	−1.26	.212	.156
Economic self-sufficiency	intervention	2.06	.56	.440	−.426	2.24	.68	1.109	1.738	3.55***	.001	.357
control	1.91	.37	.466	.557	1.99	.53	1.094	2.448	1.39	.171	.172
Self-esteem	intervention	3.10	.68	−.281	.574	3.27	.67	0.187	0.239	3.39**	.001	.341
control	3.22	.53	−.208	.672	3.07	.40	−1.216	4.093	−1.84	.070	.228
Self-efficacy	intervention	3.34	.82	−.142	.651	3.54	.85	−0.204	0.269	3.02**	.003	.304
control	3.51	.75	.188	−.985	3.46	.77	0.300	−0.126	−0.652	.516	.008

	Group	Sum of Squares	df	MS	F	p	Partial η²
Employment hope	Covariate (baseline)	25.195	1	25.195	79.606	.000	.351
Group difference	2.044	1	2.044	6.457	.012	.042
Error	46.526	147	0.317
Employment barrier	Covariate (baseline)	22.005	1	22.005	40.018	.000	.217
Group difference	0.376	1	0.376	0.683	.410	.005
Error	79.182	144	0.550
Economic self-sufficiency	Covariate (baseline)	17.739	1	17.739	74.796	.000	.339
Group difference	1.100	1	1.100	4.638*	.033	.031
Error	34.626	146	0.237
Self-esteem	Covariate (baseline)	14.627	1	14.627	60.703	.000	.295
Group difference	1.822	1	1.822	7.561**	.007	.050
Error	34.939	145	0.241
Self-efficacy	Covariate (baseline)	45.229	1	45.229	121.466	.000	.456
Group difference	1.097	1	1.097	2.947**	.088	.020
Error	53.992	145	0.372

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The authors received financial support for the research by Health Profession Opportunity Grants University Partnership (HPOGUP) Research Grants [90PH0018; 90HG1003], from the Office of Planning, Research and Evaluation (OPRE), Administration for Children and Families (ACF), U.S. Department of Health and Human Services, the Ministry of Eduation of the Republic of Korea, the National Research Foundation of Korea [NRF-2016S1A5A8017747], and the Korea Foundation [2019-RF-020; 2016-RF-021].

ORCID iD

Philip Young P. Hong

References

Akoglu

(2018). User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18, 91–93.

Anderson

M. A.

Brown

Cavadel

E. W.

Derr

Kauff

J. F.

(2018). Using psychology-informed strategies to promote self-sufficiency: A review of innovative programs (OPRE Report #2018-41). Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved from https://www.acf.hhs.gov/opre/resource/using-psychology-informed-strategies-to-promote-self-sufficiency-a-review-of-innovative-programs?utm_source=facebook&utm_medium=social&utm_campaign=general

Bernstein

Brocht

Spade-Aguilar

(2000). How much is enough? Basic family budgets for working families. Economic Policy Institute.

Bryk

A. S.

Gomez

L. M.

Grunow

(2011). Getting ideas into action: Building networked improvement communities in education. In Hallinan

M. T.

(Ed.), Frontiers in sociology of education (pp. 127–162). Springer.

Bryk

A. S.

Gomez

L. M.

Grunow

LeMahieu

P. G.

(2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press.

Central Self-Sufficiency Center. (n.d.). Retrieved from www.cssf.or.kr/new_home/Main.asp

Chen

Gully

S. M.

Eden

(2001). Validation of a new general self-efficacy scale. Organizational research methods, 4, 62–83.

Choi

Hong

P. Y. P.

Kim

(2017). Evaluation of effectiveness on Korean psychological self-sufficiency program. Journal of Korean Social Welfare Administration, 57, 179–199. [In Korean]

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

10.

Daugherty

R. H.

Barber

G. M.

(2001). Self-sufficiency, ecology of work, and welfare reform. Social Service Review, 75, 662–675.

11.

Freire

(2000). Pedagogy of the oppressed. Bloomsbury.

12.

Freire

(2014). Pedagogy of hope: Reliving pedagogy of the oppressed. Bloomsbury.

13.

Gowdy

Pearlmutter

(1993). Economic self-sufficiency: It’s not just money. Affilia, 8, 368–387.

14.

Gray

C. D.

Kinnear

P. R

.(2012). IBM SPSS statistics 19 made simple. Psychology Press.

15.

Harvey

Hong

P. Y. P.

Kweli

(2010). Shared reflections on transformative practice: From challenges to client empowerment in workforce development. Work and Economic Justice, 16, 70–78.

16.

Hong

P. Y. P.

(2008). Glocalizing structural poverty: Reclaiming hope for children and families. Illinois Child Welfare, 4, 23–38.

17.

Hong

P. Y. P.

(2013). Toward a client-centered benchmark for self-sufficiency: Evaluating the ‘process’ of becoming job ready. Journal of Community Practice, 21, 356–378.

18.

Hong

P. Y. P.

(2016a). Transforming Impossible into Possible (TIP): A bottom-up practice in workforce development for low-income job seekers. Environment and Social Psychology, 1, 93–104.

19.

Hong

P. Y. P.

(2016b). Learning to TIP: Introduction. Center for Research on Self-Sufficiency (CROSS) Loyola University Chicago.

20.

Hong

P. Y. P.

Choi

(2013). The employment hope scale: Measuring an empowerment pathway to employment success. International Journal of Psychology Research, 8, 173–189.

21.

Hong

P. Y. P.

Choi

Key

(2018). Psychological self-sufficiency: A bottom-up theory of change in workforce development. Social Work Research, 42, 22–32.

22.

Hong

P. Y. P.

Choi

Polanin

J. R.

(2014). A multi-sample confirmatory factor analysis of the Short Employment Hope Scale (EHS-14). Journal of Social Service Research, 40, 339–352.

23.

Hong

P. Y. P.

Fittin

(2017, December8). Transforming impossible into possible: Pathway for increasing successful employment outcomes. Paper presented at the 2017 Northern Region Adult Education Conference, Itasca, IL.

24.

Hong

P. Y. P.

Hong

Lewis

Williams

(2019). Pathway of employment from uncovering barriers to discovering hope: Non-traditional, student-centered, relationship-based approach. Families in Society, 100, 1–14.

25.

Hong

P. Y. P.

Kim

Hong

Park

J. H.

Lewis

(2020). Examining psychological self-sufficiency among African-American low-income jobseekers in a health profession career pathway program. Social Work in Health Care, 59(3), 139–160.

26.

Hong

P. Y. P.

O’Brien

Park

J. H.

Pigott

Hong

Holland

(2019). Psychological self-sufficiency (PSS): An empowerment-based theory for workforce training and adult education. In King

Hong

P. Y. P.

(Eds.), Pathways to careers in health care (pp. 303–349). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

27.

Hong

P. Y. P.

Polanin

J. R.

Key

Choi

(2014). Development of the Perceived Employment Barrier Scale (PEBS): Measuring psychological self-sufficiency. Journal of Community Psychology, 42, 689–706.

28.

Hong

P. Y. P.

Polanin

J. R.

Pigott

T. D.

(2012). Validation of the employment hope scale: Measuring psychological self-sufficiency among low-income job seekers. Research on Social Work Practice, 22, 323–332.

29.

Hong

P. Y. P.

Sheriff

Naeger

(2009). A bottom-up definition of self-sufficiency: Voices from low-income jobseekers. Qualitative Social Work, 8, 357–376.

30.

Hong

P. Y. P.

Song

I. H.

Choi

Park

J. H.

(2016). A cross-national validation of the short employment hope scale (EHS-14) in the United States and South Korea. Social Work Research, 40, 41–51.

31.

Hong

P. Y. P.

Song

I. H.

Choi

Park

J. H.

(2018). Comparison of perceived employment barriers among low-income job seekers in the United States and South Korea. International Social Work, 61, 23–39.

32.

Hong

P. Y. P.

(2019). Neurobiological core content in the research-supported Transforming Impossible into Possible (TIP) program model. Journal of Evidence-Based Social Work, 16, 497–510.

33.

Hong

Northcut

T. B.

Spira

Hong

P. Y. P.

(2019). Facilitating transformation in workforce training: Utilizing clinical theory to understand psychological self-sufficiency. Smith College Studies in Social Work, 89, 66–82.

34.

IBM Corp. (2015). IBM SPSS statistics for windows (Version 23.0). Author.

35.

Kim

T. K.

Zurlo

K. A.

(2007). Factors that influence workfare program participants: Focusing on South Korea’s self-sufficiency program. International Social Work, 50, 551–564.

36.

Langley

G. J.

Moen

R. D.

Nolan

K. M.

Nolan

T. W.

Norman

C. L.

Provost

L. P.

(2009). The improvement guide: A practical approach to enhancing organizational performance. John Wiley.

37.

Mathematica Policy Research. (2017). Rapid-cycle evaluation. In Focus. Mathematica.

38.

Ministry of Health and Welfare. (2016). The 2016 information on the self-sufficiency program.

39.

Rosenberg

(1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

40.

Silber

Tippett

J. S.

(1965). Self-esteem: Clinical assessment and measurement validation. Psychological reports, 16, 1017–1071.

41.

Song

J. H.

Kwon

Kim

Lee

Park

J. H.

(2013). Domestic application of Economic Self-Sufficiency (ESS) and Work Hope Scale (WHS): A study on constructing a measurement for a comprehensive understanding of self-sufficiency. Journal of Korean Social Welfare Administration, 40, 31–64. [In Korean]

42.

T. Y.

(2010). A study on the influence of emotional self-sufficiency on the barriers to economic self-sufficiency among the program participants. Social Science Research Review, 26, 229–248. [In Korean]

43.

West

S. G.

Finch

J. F.

Curran

P. J.

(1995). Structural equation models with nonnormal variables: Problems and remedies. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). Sage.

44.

Yoo

T. K.

Lee

S. H.

(2011). Does mental health affect continued participation in the workfare program? The South Korean Case. International Social Work, 54, 551–564.

Characteristics	TIP Intervention (N = 104)		Control (N = 65)		χ² Test of Homogeneity
Characteristics	n	%	n	%	x²	p	Effect Size^a
Gender
Male	48	47.1	29	44.6	0.009	.757	.023
Female	54	52.9	36	55.4
Marriage
Not married	31	31.6	14	22.6	1.914	.384	.110
Married and living together	11	11.2	10	16.1
Separated, divorced, or bereaved	56	57.1	38	61.3
Education level
Lower than high school	42	41.5	26	40.6	0.015	.903	.009
High school and above	59	58.4	38	59.4
Health
Healthy	42	41.6	25	39.7	1.043	.594	.080
Having chronic disease	34	33.7	18	28.6
Not healthy or disabled	25	24.7	20	31.7
Receipt of welfare benefits
Yes	80	81.6	48	76.1	0.403	.526	.066
No	18	18.3	15	23.8
Age (year)	Mean (min–max)	SD	Mean (min–max)	SD	F	p	Cohen’s d
	48.9 (20–66)	12.35	51.39 (22–64)	10.41	1.803	.178	0.218

		Pretest				Posttest				Result of Paired Samples t-Test
	Groups	Mean	SD	Skewness	Kurtosis	Mean	SD	Skewness	Kurtosis	t	p	Cohen’s d
Employment hope	intervention	3.49	.79	−.439	.485	3.67	.71	0.078	−0.660	2.89**	.005	.287
Employment hope	control	3.49	.70	−.017	−.967	3.38	.70	0.494	−0.207	−1.33	.189	.165
Employment barrier	intervention	2.41	.74	.371	−.616	2.40	.86	0.630	0.109	0.192	.848	.019
Employment barrier	control	2.56	.79	.367	.228	2.43	.76	0.822	1.207	−1.26	.212	.156
Economic self-sufficiency	intervention	2.06	.56	.440	−.426	2.24	.68	1.109	1.738	3.55***	.001	.357
Economic self-sufficiency	control	1.91	.37	.466	.557	1.99	.53	1.094	2.448	1.39	.171	.172
Self-esteem	intervention	3.10	.68	−.281	.574	3.27	.67	0.187	0.239	3.39**	.001	.341
Self-esteem	control	3.22	.53	−.208	.672	3.07	.40	−1.216	4.093	−1.84	.070	.228
Self-efficacy	intervention	3.34	.82	−.142	.651	3.54	.85	−0.204	0.269	3.02**	.003	.304
Self-efficacy	control	3.51	.75	.188	−.985	3.46	.77	0.300	−0.126	−0.652	.516	.008