Social Workers’ Training Evaluated by a Cluster-Randomized Study

Abstract

Objective:

Using a cluster-randomized design, this study evaluates whether a skill training program for social workers increases reemployment among welfare recipients. The program focuses on systematizing follow-up work within the areas of encounters with users, system-oriented efforts, and administrative work.

Methods:

Data consist of baseline and 18-month follow-up questionnaires for welfare recipients (N = 617) in the 18 participating labor and welfare offices randomized into experimental and control groups. Full-time employment, part-time employment, and participation in active labor market programs were assessed.

Results:

After adjusting for the clustered structure of data, the findings demonstrate that social workers’ skill training increased reemployment among welfare recipients. Although neutral effects on full-time employment, there was a highly significant effect on part-time employment (p = .000). In contrast, recipients from the control group were significantly more often in activation programs at the follow-up (p = .004).

Conclusions:

Close and comprehensive follow-ups to support reemployment among welfare recipients should be emphasized.

Keywords

cluster-randomized activation policies social work evaluation evidence-based

Introduction

Full employment is essential for the functionality and legitimacy of contemporary welfare states (Esping Andersen, 1990). In Norway, as in other countries, unemployed workers’ job search process is closely monitored by means of activation plans and programs, and follow-up interviews. The activation paradigm has become an important part of social work, emphasizing tailor-made service delivery and flexibility (van Berkel & van der Aa, 2012). Nevertheless, there is a lack of knowledge regarding which elements make activation programs efficient and what the social worker’s role is in the process. A part of the problem is that there is a lack of robust evaluation methods, such as randomized controlled trials, to estimate the effects of these policies and practices (Solomon, Cavanaugh, & Draine, 2009).

In order to increase social workers’ skills related to conducting close and comprehensive follow-ups of welfare recipients, the Norwegian Directorate of Labor and Welfare developed and implemented an evidence-based skill training program called the Comprehensive, Methodological and Principle-Based Approach (CMPA). Additionally, they commissioned an independent evaluation of this program. The CMPA skill training program seeks to improve social workers’ professional competence by systematizing and improving follow-up work related to the Qualification Program, a welfare-to-work program implemented in Norway in 2007 as a means to decrease poverty and long-term unemployment. Using a cluster-randomized research design with an 18-month follow-up, this study aims to evaluate the effects of the CMPA skill training program on welfare recipients’ reemployment outcomes. Given the program’s aim of increasing social workers’ competence and skills in undertaking close and comprehensive follow-ups, making their encounters more goal-focused, and structuring the follow-up process within the three central elements of the program, the hypothesis of this study is that the program increases reemployment among welfare recipients.

The Activation Paradigm

Since the early 1990s, there has been a shift away from “passive” benefits toward the activation of unemployed workers. This shift can be understood through various ideological discourses (Lødemel & Trickey, 2001; Sabatinelli, 2010; Serrano Pascual, 2007). The first discourse sees activation as a means to control costs related to the passive receipt of benefits. Seen from this viewpoint, the unemployed who receive benefits should repay the society by participating in various activation programs. Activation is also seen as a way to prevent the development of a “dependency culture” among welfare recipients. This reflects a view that passivity leads to a deterioration of the work ethic (Dean & Taylor-Gooby, 1992). The other activation discourse emphasizes the importance of work and activation in supporting the social inclusion of welfare recipients. This perspective sees activation more as a right of the individual than as an obligation. From this viewpoint, activation becomes a means of income maintenance and serves as a way to support individuals’ reintegration into the labor market.

Accordingly, two models of activation policies have been identified (Lødemel & Trickey, 2001). The first model, the labor market attachment model, promotes a fast return to employment by emphasizing job search, work incentives, mandatory programs, sanctions, and so-called work first active labor market programs. The idea is that engaging in work-related activities of any kind is always better than being unemployed. Previous research seems to support the argument that incentives, job search training, mandatory programs, and sanctions have minor positive effects on reemployment; however, this applies mainly to the more resourceful groups of unemployed (Kluve, 2010; Malmberg-Heimonen & Vuori, 2005a, 2005b).

The other model for activation policies, the human resource development model, emphasizes that individuals should receive further education and training before returning to employment and that the societal investment in skill development for welfare recipients should take a long-term perspective. The human resource development model has been criticized, especially for its “locking in” effect (van Ours, 2004). This means that the unemployed, while participating in the programs, do not apply for work. They may also become stuck in various activation programs, postponing or hindering their ability to seek employment in the ordinary labor market.

Although the labor market attachment model is generally dominant in main parts of Europe and the United States, the Norwegian Qualification Program can be considered to be based on the human resource development model of activation (although it should be kept in mind that pure models of either type do not really exist). Eligible participants, long-term welfare recipients, are enrolled in the Qualification Program for up to 2 years while they participate in various activities with the aim of increasing their labor market qualifications. Participants’ progress is followed closely during the process; participation in the full-day program is mandatory. In Norway, the Qualification Program has become the main context for social work and it is mainly social workers who administer, coordinate, and follow up with the participants in the program.

Social Work in the Activation Context

Regardless of which ideological discourses or models welfare states adhere to when it comes to activation policies, it is the frontline workers who play a key role when implementing policies at the practical level (Lipsky, 1980). They act as mediators between organizational policies and user needs and have to resolve any conflicts between the various actors (Brodkin, 2011; van Berkel & Borghi, 2008). Although the activation paradigm has become a central part of social work, scholars have questioned whether it can be compatible with the main principles and ethics of social work (Hasenfeld, 1999; van Berkel & van der Aa, 2012). This especially concerns features related to the labor market attachment model of activation, with its emphasis on mandatory practices, work tests, and the sanctioning of noncompliance. Evaluations of activation programs have generally shown that close and frequent follow-ups of welfare recipients increase employment, although this has often been explained by the “scarce” effect, rather than effects related to the quality of the follow-up (Kluve, 2010). Accordingly, it has been questioned, whether activation policies are more about program administration than social work practices (van Berkel & van der Aa, 2012).

Although administration of passive benefits was standardized and regulated, the activation regime is characterized by flexible and tailor-made service delivery, including a shift toward measures aimed at transforming people’s personal attributes and behaviors (Meyers, Glaser, & Macdonald, 1998; van Berkel & Borghi, 2008). These services require new competences in frontline workers. In addition to program administration, counseling and motivational work are important and relational and communicative skills are therefore needed. These services also require a higher degree of professional discretion (van Berkel & van der Aa, 2012). Professional discretion may concern the choice of a “suitable” program and follow-up model for the welfare recipient, or the question of whether or not sanctions should be imposed in a given situation (Behncke, Frölich, & Lechner, 2010; Fletcher, 2011; Jessen & Tufte, 2014).

The Norwegian Qualification Program and CMPA Skill Training

To prevent poverty and social exclusion among welfare recipients, the Norwegian Labor and Welfare Administration implemented the Qualification Program in 2007. This program draws on a human resource development approach, focusing on a longer term development of participants’ qualifications, resources, and opportunities. It is a full-time program lasting up to 2 years with employment as the final goal. In Norway, the Qualification Program has become the renewed context for social work. Social workers are expected to conduct close and comprehensive follow-ups of individual participants.

To meet social workers’ growing need for more knowledge and skills related to the follow-up work within the Qualification Program, the Norwegian Directorate of Labor and Welfare developed and implemented CMPA. The CMPA aims to develop evidence-based methods for the follow-up of participants in the Qualification Program, improve social workers’ competence and practical skills, and through the independent evaluation, increase knowledge about what works within social work practices. The program also attempts to create a common language for professional practices, improving social workers’ abilities to implement “more accurate” and “tailor-made” encounters with welfare recipients (Arbeids- og velferdsdirektoratet, 2011a; Malmberg-Heimonen, Natland et al., 2014).

The CMPA program builds on research demonstrating the importance of professionals’ relational competence for the quality of social work practices (Howe, 1998; Trevithick, 2014). As effects of specific counseling approaches have been documented, the CMPA methodological tools are based on motivational interviewing (MI) and appreciative inquiry (AI) (Dematteo & Reeves, 2011; Lundahl & Burke, 2009). MI offers an important way of working with resistance, as it involves the use of active listening skills (Markland, Ryan, Tobin, & Rollnick, 2005; Miller & Rollnick, 2004). Due to its anti-oppressive nature, MI is important in the context of social work (Watson, 2011). AI encourages individuals to adopt a positive, constructive approach in order to support organizational changes. Its relevance to social work is that it challenges the problem-focused approach often dominant in the field (Dematteo & Reeves, 2011).

The CMPA program is described as innovative because it introduces a new structure and systematic approach for social workers’ follow-up work. According to the CMPA training program, the follow-up work should cover three essential elements (Figure 1). The first element, encounters with users, focuses on the relationship between the social worker and the welfare recipient. The second element, system-oriented efforts, focuses on work with collaborating partners (e.g., the participant’s social network, collaborators in welfare services, and the labor market); and the third element, administrative work, includes charting, planning, and coordinating services for welfare recipients.

Figure 1.

The main elements of the CMPA skill training program.

The training consisted of a 9-day program of four seminars held over a 5-month period. The first seminar, which gave an overview of the program, was held in May 2011. The second, third, and fourth seminars were held in June, August, and September 2011, respectively, with each seminar presenting one of the main elements of the program (encounters with users, system-oriented efforts, and administrative work). The CMPA resource group from the Labor and Welfare Administration held the seminars and they were also in charge of supervising local implementation processes. The resource group consisted of seven persons, most of them social workers and all with long-term practical experience in the field. Before the implementation took place, the CMPA resource group developed and piloted the program over a 2-year period (2009–2010). The CMPA is based on a training manual that includes all the educational material, various group-based and individual exercises, and homework tasks related to each of the essential elements of the program (Arbeids- og velferdsdirektoratet, 2011b). Other important documents produced by the CMPA resource group are the method booklet describing CMPA tools and techniques (Arbeids- og velferdsdirektoratet, 2011c), the implementation manual (Arbeids- og velferdsdirektoratet, 2011a), and the manual for the implementation of the CMPA supervision structure (Arbeids- og velferdsdirektoratet, 2011d).

It has been demonstrated that on-site training through supervision and coaching is important when learned knowledge and skills are put into practice (Fixsen, Blasé, Naoom, & Wallace, 2009; Wood, Ager, & Wood, 2011). To ensure high-quality implementation of the CMPA skill training program, a three-level supervision structure was implemented (Arbeids- og velferdsdirektoratet, 2011d). The first level of supervision consisted of the CMPA team leaders at the local Labor and Welfare offices who supervised social workers in the Qualification Program. The second level was the county CMPA representatives who supervised the CMPA team leaders. The third level was the resource group at the Directorate of Labor and Welfare, who supervised the county-level CMPA representatives and, if needed, representatives at other levels. Social workers received supervision every 14 days from the CMPA team leaders. Local CMPA implementation was further supported by two booster seminars in which social workers discussed their experiences and by a 2-day seminar for leaders from offices randomized to the experimental group.

It is also important to note that the effects of CMPA for social workers’ professional competence have been demonstrated based on this cluster-randomized data (Malmberg-Heimonen, Natland et al., 2014). After adjusting for the clustered data structure, the study showed that the program had significant positive effects on social workers’ assessments of their professional competence and quality of work supervision received 1 year after participation in the training program (18 months after baseline). The results were explained by the acquisition and mastering of combinations of specific tools and techniques, a comprehensive supervision structure, and the opportunity to adapt the learned skills to local conditions.

Data and Methods

Recruitment and Study Participants

In September 2010, 50 of the largest Labor and Welfare offices nationwide were invited by the Labor and Welfare Administration to participate in the CMPA skill training project. The Administration informed the leaders of these offices of the requirements for participation—that is, organizational readiness and participation in research—while the researchers informed them about the cluster-randomized design. Of the 50 offices invited, 18 decided to participate. A common reason for not participating was that the offices had recently undergone major organizational changes. Clusters (offices) instead of individual social workers were randomized because the skill training program was implemented at the office level and randomizing social workers would have increased the risk of contamination and problems of program implementation (Bloom, 2005; Campbell & Walters, 2014; Ivers et al., 2011).

Before randomizing the participating offices, all 103 social workers who were working on the Qualification Program, either full-time or part-time in addition to other tasks, received the T1 (baseline) questionnaire and the consent form (February 2011). Of these, 99 responded, for a response rate of 96% of the initial population of social workers. After social workers had completed the first questionnaire, the 18 Labor and Welfare offices were randomized into experimental and control groups. Nine offices were randomized to the experimental group and nine to the control group. Social workers from the experimental group offices began their skill training, while social workers from the control group offices continued with business as usual. The flow chart of the study is presented in Figure 2.

Figure 2.

The flow chart of the cluster-randomized study. Note. *recruited after randomization.

After randomization, social workers recruited participants in the Qualification Program to take part in the study (from April 2011). Altogether, 617 Qualification Program participants filled out the baseline (T1) questionnaire and consent form. The questionnaire assessed participants’ activities in the Qualification Program, unemployment and work history, their relationship with their social worker, and their views on the Qualification Program. Additionally, questions related to health, self-efficacy beliefs, and mental health were assessed. As the study was presented as an evaluation of the Qualification Program, participants were blinded for the experimental condition, while social workers, naturally, were not blinded.

It would have been preferable to enroll Qualification Program participants before the randomization of the offices took place and social workers began the CMPA training; however, this was not practicable due to planning and scheduling of the CMPA training program. Social workers did start their recruitment of Qualification Program participants in April 2011, and 36.6% of participants were recruited before the first introductory CMPA seminar (May 2011), 85.4% were recruited before the second seminar (June 2011), and the rest were recruited before the CMPA supervision structure was implemented in the local offices (January 2012). There were significant differences based on experimental conditions in the recruitment process, as Qualification Program participants from experimental group offices had been recruited earlier in the process than participants from control group offices, t(615) = –2.533, p = .012. It is also important to note, however, that within experimental group offices, women had been recruited earlier in the process than men had, t(358) = 3.36, p = .001. There was no such gender difference in recruitment of the Qualification Program participants from control group offices, t(251) = –.049, p = .961.

Of the Qualification Program participants enrolled in the study, 50.9% were women and 49.1% were men. The average age was 35.54 years (SD = 10.57). Almost half, 49.8%, had an elementary school education or lower, 36% had secondary school, and 14.2% had tertiary school or higher. Of the respondents, 57.9% had children and 41.7% were either married or cohabiting, 48.6% were immigrants (born in a country other than Norway), and 64.2% had been employed, while the rest had not been employed. The average amount of time spent in employment was 44.50 months (SD = 77.64) and the average time unemployed was 43.39 months (SD = 21.41).

When comparing with Qualification Program participants nationwide, it was determined that although the proportion of young people (under 25 years) participating in the Qualification Program is 25% nationwide, it is 23% in this study. The proportion of women is 43% nationwide, while it is 50.9% in this study. The proportion of immigrants is 40% nationally and 48.6% in this study, while the proportion of low educated people is 60% on a national basis and 49.8% in this study. Although this evaluation is not based on a nationally representative sample of Labor and Welfare offices, this comparison still shows that there are similarities between the Qualification Program participants on a national level and those taking part in this study (Ohrem Naper, 2010).

Comparisons of Experimental and Control Group Conditions

The first part of Table 1 compares office-level administrative information about experimental and control group conditions in 2010, which was the year before the offices were randomized. These comparisons demonstrate some nonsignificant trends that are important to mention. The average number of Qualification Program participants enrolled in the program in 2010 (year before randomization) was somewhat higher in experimental group offices (M = 86.4, SD = 48.8) than in control group offices (M = 67.4, SD = 28.8), t(12.96) = –1.006, p = .333. Also, the average number of Qualification Program participants that had completed the program the year before randomization was somewhat higher in experimental group offices (M = 29.9, SD = 17.2) compared to control group offices (M = 21.7, SD = 21.7), t(15.21) = –.890, p = .388. However, the success rate, measured by the average number of Qualification Program participants attaining employment, was similar for experimental group offices (M = 9.7, SD = 7.9) and control group offices (M = 8.9, SD = 10.4), t(14.91) = –.179, p = .860.

Table 1.

The Success of Randomization Based on Information From Offices (n = 18) and Participants in the Qualification Program (N = 617).

Variables	Experimental Group Offices M (SD)	Control Group Offices M (SD)	p Value
Office-level administrative data, n	9	9
Mean of QP participants in 2010	86.44 (48.78)	67.44 (28.78)	.333
Mean of completed QP programs in 2010	29.89 (17.23)	21.67 (21.73)	.388
Mean of QP participants gained employment 2010	9.67 (7.86)	8.89 (10.37)	.860
QP participants, n	360	257
Gender (% women)	54.0 (49.90)	47.0 (50.0)	.110
Age	35.96 (10.13)	34.96 (11.15)	.245
Education level, 1–5	2.59 (1.06)	2.64 (0.93)	.567
Parent (%)	61,67 (48.69)	52.53 (50.03)	.023
Immigrant (%)	57,78 (49.46)	35.80 (48.03)	.000
Previous employment (% yes)	60.63 (48.93)	69.02 (46.33)	.034
Employment duration, months	40.77 (72.88)	49.67 (83.66)	.175
Unemployment duration, months	43.45 (21.34)	43.30 (21.55)	.939
T1 self-efficacy beliefs	28.92 (6.95)	27.20 (6.82)	.003

Note. QP = Qualification Program.

The comparison of experimental and control group conditions shows that although there are several similarities between the groups of participants enrolled in the study, there are also some significant differences. Qualification Program participants from experimental group offices significantly more often have children, t(615) = –2.27, p = .023, they are highly significantly more often immigrants, t(615) = –5.51, p = .000, they have been significantly less often in employment, t(601) = 2.13, p = .034, and they have a significantly lower level of baseline self-efficacy beliefs, t(538.12) = –3.01, p = .003, than their counterparts from control group offices. However, there were no significant differences between Qualification Program participants from experimental and control group offices in relation to gender, age, education, employment duration (months), and unemployment duration. Consequently, variables that showed significant differences between experimental and control group conditions at baseline (T1) will be controlled for in the final analyses.

A similar comparison of data at the social worker level demonstrated have that there were no observed differences based on experimental condition. Social workers from experimental and control group offices were similar with respect to all measures assessed, namely, gender, age, education, workload, months in the Qualification Program, and previous experience of similar follow-up work (Malmberg-Heimonen, Natland, et al., 2014).

Follow-Up and Attrition

Of 617 initial Qualification Program participants, 430 responded to the T2 follow-up questionnaire, giving a response rate of 69.7%. For each Qualification Program participant, the follow-up was conducted 18 months after the T1 baseline questionnaire (between October 2012 and July 2013). Of T2 respondents, 53.7% were women, while 44.7% of T2 nonrespondents were women. This means that a significantly higher proportion of women than men responded to the follow-up questionnaire, t(354.6) = –2.14, p = .033. Respondents were also older than nonrespondents. The average age of respondents was 36.71 years, while it was 32.86 years for nonrespondents, which is a highly significant difference, t(354.25) = –4.22, p = .000. Furthermore, respondents significantly more often had children, t(615) = –3.07, p = .002, a higher education, t(606) = –3.08, p = .002, and previous employment than nonrespondents, t(573) = –2.20, p = .028.

However, in a randomized design, what is most important is not the extent of general attrition between T1 (baseline) and T2 (follow-up), but whether the attrition is biased based on the experimental condition. The response rate for participants from experimental group offices was 70.3%, while it was 68.9% among participants from control group offices, t(547.01) = –.37, p = .709. Although, the T2 response rate was similar between the groups, attrition analyses based on experimental condition demonstrated a highly significant difference between respondents and nonrespondents within the experimental group related to education level. Within the experimental group, lower educated participants responded to a lesser degree on T2, t(352) = –3.58, p = .000, while such a difference did not exist within the control group, t(252) = –.40, p = .688. However, none of the other study variables demonstrated a skew attrition based on experimental condition. Consequently, the level of education will be controlled for in the final analyses.

Measures

Outcomes on employment and activation

Outcomes on employment and activation were assessed as dichotomous variables based on the respondents’ main activity at the 18-month follow-up (T2). The question assessed at follow-up was “What was your main activity during the past week?” A total of 19 response options described various types of activation measures (e.g., Qualification Program, supported employment, and courses) and types of education and employment. In addition, unemployment, parental and sick leaves, and other activities were assessed. The following outcomes on employment and activation were measured by dichotomous variables: employment, full-time employment, part-time employment, and activation measures. Each outcome assessed was coded as 1, while the rest were coded as 0.

Experimental condition

The variable was coded as 0 for control group offices or 1 for experimental group offices.

Level of education

Level of education was measured by 1 = less than primary school, 2 = primary school, 3 = secondary school, 4 = bachelor’s degree, and 5 = master’s degree or higher.

Immigrant status

Immigrant status was measured by whether the respondent was born in Norway (0) or not (1).

Parental status

Parental status was measured based on whether the respondent had children under 18 years old (2) or not (1).

Previous employment

Previous employment prior to T1 baseline was coded as a dichotomous variable measuring lack of previous employment experience as 0 and previous employment experience as 1.

Duration of previous employment

Duration of employment was assessed as total months in employment prior to T1 baseline measurement.

Duration of unemployment

Duration of unemployment was measured by total months of unemployment during the last 5 years prior to T1 baseline measurement.

Self-efficacy beliefs

Self-efficacy was assessed by a 10-Item version of the General Self-Efficacy Scale (Schwarzer & Jerusalem, 1995). The scale focuses on the individual’s willingness to initiate behavior, to make efforts to complete a task, and to carry on when things get difficult. It includes items such as “I can always manage to solve difficult problems if I try hard enough,” “I am confident that I could deal efficiently with unexpected events,” “If someone opposes me, I can find the means and ways to get what I want,” and “Thanks to my resourcefulness, I know how to handle unforeseen situations.” Respondents rated items on the following 4-point scale: 1 = not at all true, 2 = hardly true, 3 = moderately true, and 4 = exactly true. The scale varied between 10 and 40. The reliability of the scale was 0.91 at T1 (Cronbach’s α).

Analysis plan

Bivariate correlations were identified for background and T1 (baseline) variables (Table 2). For the outcome variables, unadjusted descriptive information is shown in Table 3. The real effects of the skill training program for participants in the Qualification Program were determined using logistic regression (Table 4). In order to evaluate the impact of a clustered data structure, a complex samples logistic regression was also applied (Campbell & Walters, 2014) and cluster-adjusted p values are demonstrated for the experimental condition (Table 4). Additionally, cluster-specific summary statistics are shown in Table 5. The term “Larger offices” refers to offices with more than 20 Qualification Program participants in the study.

Table 2.

Summary of Intercorrelations, Means, and Standard Deviations Between T1 Study Variables.

Variable	1	2	3	4	5	6	M	SD
1. Experimental condition	—						0.58	0.49
2. Immigrant status	.22***	—					1.49	0.50
3. Education	−.02	−.02	—				2.61	1.01
4. Parental status	.09*	.39***	.03	—			1.58	0.49
5. Previous employment	−.09*	−.34***	.16***	−.07	—		0.64	0.48
6. T1 self-efficacy beliefs	.12**	.26***	.03	.18***	−.04	—	28.21	6.94

Note. *p < .05, **p < .01, ***p < .001.

Table 3.

Experimental and Control Condition Comparison of T2 Outcomes in Percentages. Unadjusted (n = 430).

	Experimental		Control
Variable	% (SD)	95% CI	% (SD)	95% CI	p Value	Cohens d
Employment	30.04 (45.93)	[24.39, 35.69]	19.21 (39.51)	[13.41, 25.01]	.011	0.253
Full-time employment	15.42 (36.18)	[10.97, 19.87]	15.25 (36.06)	[9.95, 20.55]	.964	0.005
Part-time employment	14.62 (35.40)	[10.27, 18.97]	3.95 (19.54)	[1.08, 6.82]	.000	0.373
Activation	25.69 (43.78)	[20.31, 31.07]	32.20 (46.85)	[25.32, 39.08]	.141	0.144
n	253		177

Note. CI = confidence interval.

Table 4.

Logistic Regression Analyses Estimating the Effects on Primary Outcomes: Employment (Full-Time, Part-Time) and Activation (n = 405).

	Employment		Full-Time Employment		Part-Time Employment		Active Labor Market Measures
Predictor	B (Exp. B)	SE	B (Exp. B)	SE	B (Exp. B)	SE	B (Exp. B)	SE
Constant	−2.90	0.73	−2.18	0.83	−5.64	1.14	−0.30	0.69
Experimental condition	0.61 (1.84)*	0.25	0.03 (1.03)	0.28	1.39 (4.03)**	0.43	−0.48 (0.62)*	0.23
Immigrant	0.59 (1.80)*	0.28	0.46 (1.58)	0.33	0.51 (1.66)	0.38	0.06 (1.07)	0.28
Education level	0.12 (1.13)	0.11	0.10 (1.10)	0.13	0.10 (1.10)	0.15	−0.04 (0.96)	0.11
Parent	−0.58 (0.56)*	0.26	−0.58 (0.56)	0.30	−0.32 (0.73)	0.37	0.38 (1.46)	0.26
Previous employment	0.54 (1.72)*	0.27	0.22 (1.24)	0.32	0.80 (2.22)*	0.40	−0.45 (0.64)	0.26
T1 self-efficacy beliefs	0.03 (1.03)	0.02	0.01 (1.01)	0.02	0.05 (1.05)	0.03	−0.03 (0.98)	0.02
–2 log likelihood	444.73		347.87		249.28		457.05
Nagelkerke R ²	.08		.02		.12		.04
Cluster-adjusted p values for experimental condition^a	.062	.30	.194	.33	.000	.47	.004	.28

Note. ^aAdjusted for clustered data structure (complex samples statistics); SE = standard error.

*p < .05, **p < .01, ***p < .001.

Table 5.

Descriptive Statistics of Qualification Program Participants’ Primary Outcomes per Cluster in Percentages.

Cluster Number	n ^a	Employment	Employment, Full-Time	Employment, Part-Time	Activation Measures
Experimental
2	15	6.7	0.0	6.7	6.7
4	13	23.1	15.4	7.7	30.8
5	8	50.0	25.0	25.0	12.5
9	52	46.2	23.1	23.1	17.3
10	32	43.8	28.1	15.6	21.9
13	71	21.1	7.0	14.1	32.4
16	10	20.0	20.0	0.0	20.0
17	32	31.3	18.8	12.5	31.3
18	20	15.0	5.0	10.0	40.0
Control
1	13	7.7	7.7	0	61.5
3	9	22.2	11.1	11.1	22.2
6	15	20.0	13.3	6.7	33.3
7	9	33.3	22.2	11.1	22.2
8	21	9.5	4.8	4.8	19
11	15	13.3	13.3	0.0	26.7
12	10	20.0	20.0	0.0	30.0
14	55	21.8	16.4	5.5	36.4
15	30	23.3	23.3	0.0	30.0

Note. ^aT1 and T2 respondents (n = 430).

All analyses were conducted using SPSS version 21.0. In the planning phase of the project, a power analysis was conducted based on the Guittet, Giradeau, and Ravaud (2005) model accounting for intraclass correlation coefficient (ICC). In the power analysis, it was estimated that, with 10 clusters in each arm, 286 participants would be preferable per arm with an ICC of .005. This study has nine offices in each arm, with a total of 99 social workers and 617 participants in the Qualification Program (both arms). The Norwegian Data Inspectorate and Norwegian Social Science Data Services approved the study design (Case 25275). Throughout the research process, the Norwegian Personal Data Act and Norwegian Ethical Guidelines for Social Science Research have been followed (Forskningsetiske komiteer, 2006). All data materials are based on informed signed consent from study participants (social workers and Qualification Program participants); participants have been informed about the study and the voluntariness of their participation and they were told that all information can be erased on their request at any point.

Results

Table 2 shows means, standard deviations, and correlations between baseline T1 study variables. The associations between the T1 variables and experimental condition correspond well with the findings shown in Table 1.

Table 3 shows the percentage of Qualification Program participants who are in employment at follow-up by experimental condition. Within the experimental group, 30.04% of participants are in employment, while the corresponding percentage for control group participants is 19.21%. There is a significant difference, t(428) = –2.55, p = .011, between experimental group condition and control group condition regarding employment. There is, however, no difference between the two groups regarding full-time employment. At T2 follow-up, 15.42% of the experimental group participants and 15.25% of the control group participants are in full-time employment, t(428) = –.45, p = .964. Nevertheless, although 14.62% of experimental group participants have gained part-time employment, only 3.95% of their counterparts from the control condition offices have done so. Regarding part-time employment, the difference between the groups is highly significant, t(428) = –3.63, p = .000. Although, at the time of the follow-up, a higher percentage of participants in the experimental group have gained part-time employment, a higher percentage of participants in the control group are in various activation measures. Of experimental group respondents, 25.69% are in activation, while 32.20% of control group participants are in activation. Although this is a clear trend, this comparison is not statistically significant, t(428) = 1.47, p = .141. Effect size measures displayed in Table 3 support the conclusion, showing a small effect size (Cohen’s d) for employment and activation outcomes and a medium effect size for outcomes on part-time employment, an interpretation that is also supported by the reported confidence intervals.

Table 4 shows the real effects of the intervention on reemployment and activation outcomes. The effects are estimated by logistic regression, where immigrant status, children, education level, previous employment, and T1 self-efficacy beliefs are also controlled for. The findings correspond well with the descriptive results shown in Table 3. Although there is a main effect on employment (p = .015), it is part-time employment, not full-time employment, where the most significant results are found. For part-time employment and after controlling for T1 variables, the odds ratio for employment (Exp(B)) is 4.03 for participants in experimental group offices, compared to the odds ratio for participants from control group offices, which is 1.00 as a reference category (Exp(B)). This finding is highly significant (p = .001). Furthermore, when controlling for the background and baseline variables, the results from the logistic regression demonstrate that a higher percentage of participants from control group offices are in activation measures. The effect of experimental condition on activation is significant (p = .041) with an odds ratio (Exp(B)) value of 0.62, compared to 1.00 for the control group (reference category). The cluster randomization was also acknowledged in Table 4, which shows the cluster-adjusted p values and standard errors (SEs) for experimental condition. When acknowledging the cluster randomization in the analyses, the significance of the effects on part-time employment and activation become stronger, nevertheless SEs increase (cf. Campbell & Walters, 2014).

Finally, Table 5 presents the summary statistics for the outcomes of the various clusters in the study. Cluster-specific summary statistics increase the transparency of the findings. These trends reflect the main findings from the study (compare Table 4). Summary statistics also indicate that, while outcomes on employment are driven by the larger offices (over 20 Qualification Program participants) within the experimental group (9, 10, 13, and 17), the larger offices within the control group (8, 11, 14, and 15) seem not to have had the same success.

Discussion and Applications to Practice

Using a cluster-randomized research design with an 18-month follow-up, the aim of this study was to evaluate the effects of the CMPA skill training program on welfare recipients’ reemployment and participation in activation measures. Recent active labor market policies emphasize individualized and tailor-made services, which require social workers to have well developed relational and communicative skills. Hence, a goal of the CMPA skill training program was to increase these skills and to structure the follow-up work within the activation program into the following three areas: encounters with the user, system-oriented efforts, and administrative work.

The main finding of the study is that the CMPA skill training program had a positive effect on welfare recipients’ reemployment outcomes. Welfare recipients from offices where the program had been implemented were more often in part-time work at the follow-up than their counterparts from control-group offices. In contrast, welfare recipients from control group offices were more likely to be involved in various activation programs at the 18-month follow-up. The findings show that the CMPA skill training helped social workers and welfare recipients to reach the goal of employment. The CMPA model gave social workers tools to focus on their encounters with the users, but also to acknowledge the system around the service users and the importance of documentation and administrative work. It is also important to note that the counterfactual element in this study was offices and social workers who normally worked within the Qualification Program, which means that they were already conducting close and comprehensive follow-ups with welfare recipients.

Among others, van Berkel and van der Aa (2012) have concluded that the key to successful activation programs is the provision of individualized and personalized services, but that the professionals require new knowledge and skills in order to provide these services. The results from this study support these conclusions in showing that the quality of the follow-up work done within activation programs makes a difference. As previous policies and research have largely focused on welfare recipients’ lack of qualifications as a hindrance to employment processes, these findings emphasize the importance of social work professionals’ skills and knowledge as a key for successful activation policies and programs. To be successful, these policies and programs require professionals to do much more than merely administrate them.

The Qualification Program, which was the context of this study, builds on a human resource development perspective, which emphasizes qualification as a longer term investment. Advocates of the labor attachment model of activation have been especially critical of the human resource development perspective for its locking-in effect, which means that a long-term qualification program hinders people from applying for work in the ordinary labor market. Nevertheless, these findings demonstrate that a human resource development perspective on activation, combined with skilled professionals and a high-quality follow-up, improves reemployment outcomes. It is also important to note that the Qualification Program is for welfare recipients who are far away from the labor market: The increase in employment, although only part-time employment among this group, is an important finding. Research has shown that the vulnerable groups of unemployed do not necessarily have the qualifications needed to respond to harsh activation policies, such as sanctions, incentives, and work tests (Malmberg-Heimonen & Vuori, 2005a, 2005b). For these groups of unemployed, measures with the goal of long-term inclusion seem to be more suitable.

Nevertheless, the limitations of this research should be acknowledged when interpreting the findings. Only 18 offices were included in the study. A higher number of offices would have increased the statistical power of the analysis. However, adjusting the statistical models for the clustered data structure increases the reliability of the findings, and the summary statistics for each cluster support the overall conclusions. Although this study demonstrated positive effects of CMPA, it would similarly be important to estimate the cost-effectiveness of the program, as there were significant costs related to its development and implementation.

The selection of welfare recipients to participate in the study is also an important issue to acknowledge. Although some of the offices recruited higher numbers of participants to the study than other offices, the reliability of these findings is increased by the fact that both experimental and control group offices have a similar variation in recruitment (Table 5) and the data are fairly representative compared to national data (Ohrem Naper, 2010). Also the fact that we have previously demonstrated effects for social workers’ professional competence helps to support the conclusions of this specific study (Malmberg-Heimonen, Natland et al., 2014).

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project is funded by the Norwegian Directorate of Labor and Welfare.

References

Arbeids- og velferdsdirektoratet. (2011a). Implementeringsmanual for HPMT. (CMPA implementation manual). Oslo, Norway: Arbeids- og velferdsdirektoratet.

Arbeids- og velferdsdirektoratet. (2011b). Oppfølging av brukerne i Kvalifiseringsprogrammet. Helhetlig, prinsippstyrt, metodisk tilnærming (HPMT). Opplæringsmateriell (CMPA training manual). Oslo, Norway: Arbeids- og velferdsdirektoratet.

Arbeids- og velferdsdirektoratet. (2011c). Oppfølging av brukerne i Kvalifiseringsprogrammet. Helhetlig, prinsippstyrt, metodisk tilnærming (HPMT). Metodehefte 3. utgave (CMPA method booklet). Oslo, Norway: Arbeids- og velferdsdirektoratet.

Arbeids- og velferdsdirektoratet. (2011d). Veiledningsmanual for HPMT. (CMPA supervision manual). Oslo, Norway: Arbeids- og velferdsdirektoratet.

Behncke

Frölich

Lechner

(2010). Unemployed and their caseworkers: Should they be friends or foes? IZA Discussion Paper, No. 3149. Bonn: IZA.

Bloom

H. S.

(2005). Randomizing groups to evaluate place-based programs. In Bloom

H. S.

(Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 115–173). New York, NY: Russell Sage Foundation.

Brodkin

(2011). Policy work: Street-level organizations under new managerialism. Journal of Public Administration Research and Theory, 21, 253–277.

Campbell

M. J.

Walters

S. J.

(2014). How to design, analyse and report cluster randomized trials in medicine and health related research. West Sussex, England: John Wiley.

Dean

Taylor-Gooby

(1992). Dependency culture: The explosion of a myth. New York, NY: Harvester Wheatsheaf.

10.

Dematteo

Reeves

(2011). A critical examination of the role of appreciative inquiry within an interprofessional education initiative. Journal of Interprofessional Care, 25, 203–208.

11.

Esping Andersen

(1990). The three worlds of welfare capitalism. Cambridge, England: Polity Press.

12.

Fixsen

Blasé

K. A.

Naoom

S. F.

Wallace

(2009). Core implementation components. Research on Social Work Practice, 19, 531–540.

13.

Fletcher

D. R.

(2011). Welfare reform, jobcentre plus and the street-level bureaucracy: Towards inconsistent and discriminatory welfare for severely disadvantaged groups? Social Policy and Society, 10, 445–458.

14.

Forskningsetiske komiteer. (2006). Forskningsetiske retningslinjer for samfunnsvitenskap, humanoira, juss og teologi (Research ethical guidelines for social sciences, humanities, law and theology). Zoom Grafisk AS, Norway: De Nasjonale Forskningsetiske komiteer.

15.

Guittet

Giradeau

Ravaud

(2005). A priori postulated and real power in randomized trials: Mind the gap. BMC Medical Research Methodology, 5, 5–25.

16.

Hasenfeld

(1999). Social services and welfare-to-work: Prospects for the social work profession. Administration in Social Work, 23, 185–199.

17.

Howe

(1998). Relationship-based thinking and practice in social work. Journal of Social Work Practice, 12, 45–56.

18.

Ivers

N. M.

Taljaard

Dixon

Bennett

McRae

Taleban

Donner

(2011). Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: Review of random sample of 300 trials. British Medical Journal, 343, d5886.

19.

Jessen

J. T.

Tufte

P. A.

(2014). Discretionary decision-making in a changing context of activation policies and welfare reforms. Journal of Social Policy, 43, 269–288.

20.

Kluve

(2010). The effectiveness of European labour market programs. Labour Economics, 17, 904–918.

21.

Lipsky

(1980). Street-level bureaucracy: Dilemmas of the individual in public services. New York, NY: Russell Sage Foundation.

22.

Lødemel

Trickey

(Ed.). (2001). An offer you can’t refuse: Workfare in international perspective. Bristol, England: Policy Press.

23.

Lundahl

B. W.

Burke

B. L.

(2009). The effectiveness and applicability of motivational interviewing: A practice-friendly review of four meta-analyses. Journal of Clinical Psychology, 65, 1232–1245.

24.

Malmberg-Heimonen

Natland

Tøge

A.-G.

Hansen

H. C.

(2014). The effects of skill training on social workers in Norway: Results of a cluster-randomized study. Manuscript submitted for publication.

25.

Malmberg-Heimonen

Vuori

(2005a). Activation or discouragement: Enforced participation modifying the success of job-search training. European Journal of Social Work, 8, 451–469.

26.

Malmberg-Heimonen

Vuori

(2005b). Financial incentives and job search training: Methods to increase labour market integration in contemporary welfare states? Social Policy and Administration, 39, 247–259.

27.

Markland

Ryan

R. M.

Tobin

V. J.

Rollnick

(2005). Motivational interviewing and self-determination theory. Journal of Social and Clinical Psychology, 24, 811–831.

28.

Meyers

M. K.

Glaser

Macdonald

(1998). On the front lines of welfare delivery: Are workers implementing policy reforms? Journal of Policy Analysis and Management, 17, 1–22.

29.

Miller

W. R.

Rollnick

(2004). Talking oneself into change: Motivational interviewing, stages of change, and therapeutic process. Journal of Cognitive Psychotherapy: An International Quarterly, 18, 299–308.

30.

Ohrem Naper

(2010). Kvalifiseringsprogrammets deltakere: Hvor lang er avstanden til arbeidsmarkedet? (Participants in the qualification program: How long is the distance to the labour market?) Arbeid og Velferd, 3, 17–25.

31.

Sabatinelli

(2010). Activation and rescaling: Interrelated questions in social policy. In Kazepov

(Ed.), Rescaling social policies: Towards multilevel governance in Europe (pp. 75–103). Vienna, Austria: Ashgate.

32.

Schwarzer

Jerusalem

(1995). Generalized self-efficacy scale. In Weinman

Wright

Johnston

(Eds.), Measures in health psychology: A user’s portfolio. Causal and control beliefs (pp. 35–37). Windsor, Canada: NFER-NELSON.

33.

Serrano Pascual

(2007). Reshaping welfare states: Activation regimes in Europe. In Serrano Pascual

Magnusson

(Eds.), Reshaping welfare states and activation regimes in Europe (pp. 11–35). Brussels, Belgium: Peter Lang Publishing.

34.

Solomon

Cavanaugh

M. M.

Draine

(2009). Randomized controlled trials: Design and implementation for community-based psychosocial interventions. Oxford, England: Oxford University Press.

35.

Trevithick

(2014). Humanising managerialism: Reclaiming emotional reasoning, intuition, the relationship, and knowledge and skills in social work. Journal of Social Work Practice, 28, 287–311.

36.

Van Berkel

Borghi

(2008). Review article: The governance of activation. Social Policy and Society, 7, 393–402.

37.

Van Berkel

Van der Aa

(2012). Activation work: Policy program administration or professional service production. Journal of Social Policy, 41, 493–510.

38.

Van Ours

J. C.

(2004). The locking-in effect of subsidized work. Journal of Comparative Economics, 32, 37–55.

39.

Watson

(2011). Resistance is futile? Exploring the potential of motivational interviewing. Journal of Social Work Practice, 25, 465–469.

40.

Wood

A. R.

Ager

R. D.

Wood

R. J.

(2011). Motivational interviewing: A qualitative examination of factors impacting adoption and implementation in a community-wide setting. Journal of Social Work Practice in the Addictions, 11, 336–351.