Do Start-Up Subsidies for the Unemployed Affect Participants’ Well-Being? A Rigorous Look at (Un-)Intended Consequences of Labor Market Policies

Abstract

Background:

The literature on start-up subsidies (SUS) for the unemployed finds positive effects on objective outcome measures such as employment or income. However, little is known about effects on subjective well-being of participants. Knowledge about this is especially important because subsidizing the transition into self-employment may have unintended adverse effects on participants’ well-being due to its risky nature and lower social security protection, especially in the long run.

Objective:

We study the long-term effects of SUS on subjective outcome indicators of well-being, as measured by the participants’ satisfaction in different domains. This extends previous analyses of the current German SUS program (“Gründungszuschuss”) that focused on objective outcomes—such as employment and income—and allows us to make a more complete judgment about the overall effects of SUS at the individual level.

Research design:

Having access to linked administrative-survey data providing us with rich information on pretreatment characteristics, we base our analysis on the conditional independence assumption and use propensity score matching to estimate causal effects within the potential outcomes framework. We perform several sensitivity analyses to inspect the robustness of our findings.

Results:

We find long-term positive effects on job satisfaction but negative effects on individuals’ satisfaction with their social security situation. Supplementary findings suggest that the negative effect on satisfaction with social security may be driven by negative effects on unemployment and retirement insurance coverage. Our heterogeneity analysis reveals substantial variation in effects across gender, age groups, and skill levels. Estimates are highly robust.

Keywords

start-up subsidies propensity score matching counterfactual analysis well-being unintended effects

Unemployment is known to have substantial and detrimental impact on individuals’ well-being that may last beyond the actual spell of unemployment through scarring effects (see, e.g., Clark et al., 2001; McKee-Ryan et al., 2005; Winkelmann & Winkelmann, 1998). Active labor market policies (ALMPs) can be seen as one way to shelter individuals from these negative effects on well-being by increasing the likelihood of reemployment. To date, the ALMP literature has mostly focused on objective outcome indicators of success such as earnings and employment for evaluating the success of programs (see, e.g., Card et al., 2018, for a meta-analysis). However, the literature also recently started to investigate the effects of ALMPs on subjective outcomes such as life satisfaction or self-assessed health. Examples include studies on the effects of job creation programs in Germany (Crost, 2016; Wulfgramm, 2011), a UK training program (Andersen, 2008), or job search programs in the United States (Vinokur et al., 2000; Vuori & Silvonen, 2005).¹ Expanding the set of outcomes to include the measures of subjective well-being may provide useful information for analyzing effect heterogeneity and improving policy design. For example, it may be the case that groups of individuals display similar impacts in terms of objective outcomes, but the effects on subjective outcomes may be very different, which allows further improving targeting of participants. In extreme cases, it may also be the case that effects on objective and subjective outcomes diverge such that inference based on objective measures only would provide a skewed picture of program effects on participants’ overall welfare.

While the latter case seems relatively unlikely in the case of traditional ALMPs such as training, start-up subsidies (SUS)—which are a particular kind of ALMP that aims to help jobseekers to escape unemployment by granting them temporary transfers to take up self-employment and set up a business—are at higher risk of misleading inference based on objective outcomes alone. This is because the overall effect of unemployed individuals transitioning into self-employment on subjective well-being is ambiguous. On the one hand, there is a relatively large body of literature showing the positive effects of self-employment in the general population on job satisfaction (e.g., see Benz & Frey, 2008; Hurst & Pugsley, 2011, 2016). However, self-employment is often also associated with higher earnings risk and lower social security coverage compared to regular employment (see European Commission, 2015, for European evidence), transferring more risk to the individual. This may be especially relevant among unemployed individuals who start their business from a position of relative scarcity (see Caliendo, Künn, & Weissenberger, 2019, for a general discussion), facing multiple disadvantages such as capital constraints (Meager, 1996; Perry, 2006). Together with mixed evidence of health effects of self-employment (e.g., see Blanchflower, 2004; Nikolova, 2019), this may result in unintended negative effects of SUS on subjective well-being.

Despite these theoretical concerns, little is known about the effects of SUS on subjective well-being.² In this article, we narrow this research gap by extending previous analyses on objective outcomes of the current German SUS program (“Gründungszuschuss,” dubbed New SUS, Bellmann et al., 2018; Caliendo & Tübbicke, 2019) and estimate the long-run effects of participation on subjective well-being along several dimensions. Analyzing the German program provides a useful benchmark as many SUS programs in Europe have a relatively similar institutional setup (see Caliendo et al., 2016; O’Leary, 1999 for details). We base our analysis on combined administrative and survey data, giving us access to a rich set of control variables to perform the estimation under the selection-on-observables assumption using propensity score matching (PSM) techniques (Rosenbaum & Rubin, 1983). We find positive long-run effects on satisfaction with the overall job situation but negative long-run effects on satisfaction with social security. A supplementary analysis suggests that this effect may be driven by negative effects on social security coverage. Moreover, our analysis of effect heterogeneity displays substantial differences in program impacts across gender, age categories, and skill levels. Sensitivity analyses suggest that these findings are highly robust.

The remainder of this article is structured as follows. The second section provides details of the New SUS program in Germany, offers some theoretical considerations on the effects of SUS on participants, describes the data used, and displays some descriptive statistics. The third section elaborates on the identification as well as estimation approach and details the results of our empirical analysis. The fourth section concludes.

The Program, Theory, and Data

The New SUS and Selection Into the Program

The New SUS has been in place in Germany since the end of 2011. In 2013, about 30,000 of roughly 2.5 million unemployed individuals joined the program annually, according to official statistics from the Federal Employment Agency (FEA). In order to be eligible for the program, unemployed individuals have to be the recipients of unemployment benefits (UB) I with at least 150 remaining days of entitlement when applying.³ To apply for the program, individuals need to take several steps. First, they need to write up their business plan and have its sustainability approved by some external experts, for example, from the local chamber of commerce. Second, using these documents, the individual can apply for the subsidy at the local employment agency before caseworkers make the final decision on whether the subsidy is awarded to the applicant.⁴ In their decision, the caseworker is supposed to take the reemployment probability of the applicant into account, that is, access to the program should only be granted if a reintegration of the individual into the labor market is unlikely without support.⁵ Indeed, qualitative interviews by Bernhard and Grüttner (2015) suggest that caseworkers most often reject applicants if they find a sufficient number of job vacancies for them in the local labor market. In addition, the law requires that program choice is supposed to reflect the individuals’ abilities, and hence, some entrepreneurial affinity is required. However, Bernhard and Grüttner (2015) also show that rejections due to a lack of quality in the presented business plan are relatively rare, and hence, the individual’s reemployment probability appears to be the most important confounding factor.

Successful applicants receive a monthly transfer equivalent to their UB plus a fixed transfer of €300 for 6 months after entering the program.⁶ A second and optional benefit phase—which only pays the fixed transfer for an additional 9 months—can be granted if the business is still running and further support is needed. On average, participants received a total subsidy of around €10,350 in 2012.⁷ The fixed transfer is explicitly paid to cover social insurance contributions (health care and unemployment insurance). While having health insurance is mandatory for everyone in Germany, public unemployment and, to some degree, also retirement insurance coverage is not obligatory for the self-employed.⁸ However, individuals may sign up for both types of insurance. While access to the public retirement insurance system is unrestricted, joining the public unemployment insurance system voluntarily is only possible within 3 months after the start-up and only if individuals had been in regular employment for at least 12 months during the last 2 years. Self-employed individuals who do not opt for unemployment or retirement insurance retain their entitlement to both types of insurance based on their previous contributions. On the other hand, individuals in regular employment are automatically enrolled in all types of social insurance systems. For unemployed persons, the FEA covers the cost of health and retirement insurance while no contributions to the unemployment insurance are made.

Theoretical Considerations on Program Effects and Channels

SUS programs aim to reintegrate unemployed individuals into the labor market via the route of self-employment. Theoretical justification for this type of program is given by the existence of multiple entry barriers into self-employment for unemployed individuals. First, SUS reduce potential capital constraints that are possibly due to lower personal financial means or discrimination by the capital market (Meager, 1996; Perry, 2006). Second, unemployed individuals face a disadvantage regarding (start-up-specific) human and social capital as well as labor market experience (Pfeiffer & Reize, 2000). By providing start-ups out of unemployment with a secure minimum income for a limited duration, unemployed individuals are expected to (partially) make up for these disadvantages (Caliendo, Hogenacker, et al., 2015). In addition, SUS may remedy a lack of awareness regarding self-employment as a viable employment alternative among the unemployed (Storey, 2003). By ameliorating these constraints faced by the unemployed, their labor market prospects are expected to improve through SUS participation.

Theoretical considerations regarding the effects on subjective measures such as overall life satisfaction are less clear-cut. This is because—in contrast to other more traditional types of ALMPs—participation in SUS is likely to induce higher rates of self-employment relative to nonparticipation, even if there were no differences in overall employment rates between the two states. On the one hand, this may have positive effects on subjective well-being, as self-employment in the general population is associated with nonpecuniary benefits such as greater job-related freedom (Benz & Frey, 2008; Hurst & Pugsley, 2011, 2016). For example, this is supported by Lange (2012), who finds the positive effects of self-employment on job satisfaction. On the other hand, self-employment is inherently more risky than regular employment in terms of future earnings and thus may take a toll on individuals’ well-being. This may be especially relevant among unemployed individuals who start their business from a position of relative scarcity, facing multiple disadvantages as described above. Moreover, there is mixed evidence on the association between health and self-employment. While Nikolova (2019) finds positive health effects of self-employment, Blanchflower (2004) reports increased rates of stress, exhaustion from work, a loss of sleep due to worry, and feelings of pressure among the self-employed. Overall, this may potentially lead to unintended negative effects of SUS on subjective well-being, even if the program raises the employment prospects of participants.

Data

In order to evaluate the SUS program, we use a random sample of previously unemployed participants who joined the program between February and June 2012, representing about 17% of all entries into the program during this time period. Our comparison group comprises individuals who were unemployed for at least 1 day, were eligible for the program (i.e., had at least 150 remaining days of UB I entitlement) but did not apply for it in this period. Both samples were drawn from the Integrated Labor Market Biographies (IEB) of the FEA. The IEB—containing all individuals who have ever been employed subject to social security contributions or registered as unemployed—covers the employment history of individuals and provides information on sociodemographics, previous earnings, human capital, ALMP history, and regional information. The extensive register data are enriched with informative survey data collected via two computer-assisted telephone interviews around 20 and 40 months after entering the program. In order to reduce survey costs, nonparticipants to be interviewed were selected via a prematching strategy to avoid interviewing individuals with very dissimilar observed characteristics compared to actual participants. Nonparticipants are assigned a hypothetical entry month into the program based on the month in which they were observed in unemployment and thus drawn as a comparison individual from the IEB (see Bellmann et al., 2018; Caliendo & Tübbicke, 2019, and Online Appendix B for more details on the data). The survey includes information on the intergenerational transmission of education, labor force attachment, and self-employment as well as personality traits such as the “Big 5,” risk preferences, and locus of control, which have proven important in the context of entrepreneurial decision making (see Caliendo, Hogenacker, et al., 2015, Caliendo et al., 2016, for a detailed discussion).⁹ Outcome information is also gathered through the survey and allows tracking individuals for 40 months in the panel sample. An analysis of panel attrition reveals nonselective attrition patterns with respect to our main outcomes of interest (see Online Appendix B for details). The final data set contains 1,248 participants and 1,204 nonparticipants.

As in Caliendo and Tübbicke (2019), we use information on individuals’ labor market status and net monthly earnings as objective outcome measures. Our main outcomes of interest are, however, measures of subjective well-being. In particular, we use individuals’ self-reported satisfaction with their life, health, income, job, and social security situation. These items are measured on a 7-point Likert-type scale from 1 (completely dissatisfied) to 7 (very satisfied).¹⁰ To shed some light on potential mechanisms, we also use the individuals’ unemployment and retirement insurance contributions as well as the respondent’s subjective assessment of the sufficiency of their retirement plans as additional outcomes.

Descriptives

Table 1 presents the descriptives on some of the pretreatment characteristics of our sample of participants and nonparticipants. A full overview is provided in Table A.1 in the Online Appendix A. Table 2 provides outcome descriptives for all of our measures of success of the SUS program.

Table 1.

Selected Descriptives.

	Participants	Nonparticipants	p Value
Number of observations	1,248	1,204
Sociodemographics
Age in years	42.77	43.94	.002
Share female	0.425	0.509	.000
Share with university degree	0.411	0.267	.000
Labor market history
Fraction of last 10 years in unemployment	0.101	0.170	.000
Daily last income (Euro)	81.39	61.60	.000
Share in regular employment before unemployment	0.674	0.516	.000
Share in self-employment before unemployment	0.054	0.012	.000
Intergenerational transmission
Share with father/mother having been self-employed	0.349	0.252	.000
Big five personality traits
Conscientiousness	4.383	4.106	.000
Extraversion	3.692	3.297	.000
Agreeableness	3.401	3.368	.300
Neuroticism	2.365	2.677	.000
Openness	3.723	3.255	.000
Readiness to take risks	5.777	5.316	.000
Locus of control	4.260	3.785	.000

Note. Reported are sample means, unless indicated otherwise. The p values are based on t tests of equal means. The “Big 5” and locus of control are measured on a 5-point Likert-type scale from 1 (does not apply at all) to 5 (applies completely). Readiness to take risks is measured on a 11-point Likert-type scale from 0 (not at all willing to take risks) to 10 (very willing to take risks). A complete overview of covariates used in the analysis is provided by Table A.1 in the Online Appendix.

Table 1 reveals statistically significant differences in terms of several characteristics between our treatment group and the comparison group individuals despite the prematching. For example, it can be seen that on average participants are slightly younger, less likely to be female, and generally better educated than our nonparticipants. In addition, we find that our sample of treated individuals have a more favorable long- and short-term employment history. While participants had spent on average about 10% of the last 10 years in unemployment, our comparison group was unemployed for 17% of that time.¹¹ In addition, average daily earnings prior to unemployment were higher among participants. About 5% of participants were also already self-employed before the start of their unemployment spell, whereas this was only true for 1.2% of the pool of comparison individuals. With respect to intergenerational transmission, participants are more likely to have parents who have been self-employed, which is named as one of the key drivers for becoming self-employed in the entrepreneurship literature (Lindquist et al., 2016). Moreover, participants and comparison individuals significantly differ with respect to a variety of personality traits and preferences. For instance, participants are on average more conscientious, more extraverted, less neurotic, and more open to new experiences. They also possess a higher willingness to take risks and are more strongly convinced that much of their life outcomes depend on their own actions, that is, they have a more internal locus of control (Rotter, 1966).

While the characteristics of the businesses created through the SUS scheme are not the focus of this article, Table A.2 in the Online Appendix A provides some auxiliary information in this regard. Put simply, subsidized businesses are mainly started with prior industry-specific experience from regular employment, they invest on average around €19,000—often financed entirely from own equity—and they are most commonly active in the service industry, followed by retail or wholesale and construction. In terms of business outcomes, subsidized businesses show high survival rates and substantial earnings. Slightly more than one third of businesses create jobs, although innovation activity is limited.¹²

Outcomes of interest

Table 2 presents the raw descriptive statistics for our outcomes of interest. They show that participants have more favorable objective labor market outcomes 40 months after entering the program. Participants are not only more likely to be self-employed but they are also much more likely to be employed in general (i.e., in either self- or regular employment). In addition, they have larger net monthly earned incomes. Regarding our main outcome measures of subjective well-being, we can state that they score higher on the Likert-type scale for most variables, that is, they are more satisfied with their life, health, income, and job situation in general. However, participants show lower levels of average satisfaction with their social security protection. It can also be seen that participants are less likely to contribute to the public unemployment insurance (UI) system, and they are also less likely to make contributions to a retirement plan. These tendencies hold for outcomes both measured after 20 and 40 months. However, the differences in outcome means of participants and nonparticipants shrink over time.

As comparing means of ordinal variables can be misleading, we also provide descriptive statistics on the entire distribution of subjective well-being variables in Figure 1. The graphical analysis reveals that the distribution of participants’ outcomes has more probability mass at the upper end of the Likert-type scale for life satisfaction, satisfaction with health and income, and job satisfaction. However, the opposite is true regarding satisfaction with social security, that is, nonparticipants are more likely to score high on the scale compared to participants. In our causal analysis, we will also estimate effects on the probability of scoring above the midpoint of the Likert-type scale. For example, 85% of participants are satisfied with life in general (score above the midpoint of the Likert-type scale) after 40 months while the same is only true for about 77% of nonparticipants. At the same time, 55% of nonparticipants but only 48% of participants are satisfied with their social security situation (for more details, see Table 2).

Figure 1.

Subjective well-being distributions. (A) Satisfaction with life in general. (B) Satisfaction with health status. (C) Satisfaction with income. (D) Satisfaction with job situation. (E) Satisfaction with social security. Note. This graph shows the distribution of subjective well-being of participants and nonparticipants after 40 months as measured by the individuals’ self-reported satisfaction with life, their health, income, job situation in general, and their social security situation. These items are measured on a 7-point Likert-type scale from 1 (completely dissatisfied) to 7 (very satisfied).

Table 2.

Outcome Descriptives.

	Outcomes After 20 Months			Outcomes After 40 Months
	Participants	Nonparticipants	p Value	Participants	Nonparticipants	p Value
Number of observations	1,248	1,204		1,248	1,204
Labor market outcomes
Share in self- or regular employed	0.958	0.615	.000	0.938	0.676	.000
Net monthly earned income (Euro)	1,901	886.6	.000	2,264	1,046	.000
Subjective well-being
Satisfaction with life in general
Likert points	5.677	5.021	.000	5.620	5.208	.000
Share above midpoint	0.863	0.713	.000	0.845	0.769	.000
Satisfaction with health status
Likert points	5.626	5.058	.000	5.497	5.113	.000
Share above midpoint	0.812	0.701	.000	0.807	0.727	.000
Satisfaction with income
Likert points	4.580	3.872	.000	4.591	4.208	.000
Share above midpoint	0.585	0.427	.000	0.586	0.497	.000
Satisfaction with job situation
Likert points	5.529	4.378	.000	5.366	4.740	.000
Share above midpoint	0.823	0.551	.000	0.785	0.631	.000
Satisfaction with social security
Likert points	4.192	4.249	.382	4.229	4.445	.001
Share above midpoint	0.466	0.493	.185	0.477	0.548	.001
Social security
Share not contributing to UI	0.534	0.371	.000	0.603	0.336	.000
Share not contributing to a retirement plan	0.083	0.016	.000	0.059	0.021	.000
Share deeming retirement plan insufficient	0.615	0.628	.523	0.583	0.594	.529

Note. Reported are sample means, unless indicated otherwise. The p values are based on t tests of equal means. The number of observations for the earnings and satisfaction variables is slightly lower due to item nonresponse. Satisfaction outcomes are measured on a Likert-type scale from 1 (completely dissatisfied) to 7 (very satisfied). In addition to the mean of satisfaction variables, the table displays figures on the share of individuals scoring above the midpoint of the Likert-type scale. UI = unemployment insurance.

Estimation Strategy and Empirical Analysis

This section first elaborates on our identification approach and describes the implementation of the PSM strategy. Second, our main empirical analysis regarding the long-term effects of SUS and their heterogeneity is presented. Finally, the sensitivity of our results is assessed.

Parameter of Interest and Identification Strategy

In order to estimate the causal effects of the New SUS program on labor market outcomes and subjective well-being for actual participants, we base our analysis on the potential outcomes framework usually attributed to Roy (1951) and Rubin (1974). The parameter that we want to estimate is the average treatment effect on the treated (ATT), defined as

τ_{ATT} = E [Y^{1} | D = 1] - E [Y^{0} | D = 1],

where D is the treatment indicator, taking on the value of 1 if the person received SUS and 0 otherwise. Y¹ corresponds to the outcome in the treated state and Y⁰ is the potential outcome in the untreated state. Unfortunately, the counterfactual outcome Y⁰ is not observed for participants. This fundamental evaluation problem implies that it is necessary to estimate the second expectation in Equation 1 from data on nonparticipants. However, simple mean comparisons between participants and nonparticipants are inconsistent due to selection bias. This bias may result from differences in observed characteristics X and/or unobserved characteristics U.

PSM methods as pioneered by Rosenbaum and Rubin (1983) eliminate bias due to observed characteristics by reweighting comparison individuals such that characteristics are balanced across samples. For PSM methods to give a consistent estimate of the ATT, the vector of observed characteristics X needs to be sufficiently rich to satisfy the unconfoundedness assumption, also often called the conditional independence assumption (CIA):

Y^{0} ⊥ ⊥ D | P (X),

where $P (X) = P r (D = 1 | X)$ is the propensity score (see Lechner, 2001, for details). As Lechner and Wunsch (2013) noted, this implies that the propensity score specification must include all characteristics that simultaneously influence the outcome of interest and the treatment probability. In our application, we should therefore include all covariates related to take-up of the SUS program, subjective well-being, labor market outcomes, and social insurance uptake. As previously noted, one of the main determinants of receiving SUS is the reemployment probability of participants, which is likely to be strongly correlated with their sociodemographics, educational attainment, parental background, and their labor market history. Arguably, these factors are also highly relevant determinants of our outcome measures. In addition, some measure of entrepreneurial affinity should be included in the estimation procedure. As entrepreneurial affinity is unobserved, we include several variables with a strong link to self-employment into the propensity score specification. These include parental self-employment, regional start-up activity, and a very informative set of personality traits and preferences.¹³ This is likely to substantially attenuate potential bias because personality traits and preferences have been shown to have a sizable and statistically significant impact not only on the likelihood of becoming self-employed but also on subjective well-being, insurance take-up, and labor market outcomes.¹⁴ Despite the rich data that we have, failure of the CIA poses a potential threat to the validity of our estimates, and thus, we will assess their sensitivity with respect to unobserved confounders.

In addition, the overlap assumption $P (X) < 1 \forall X$ is necessary to find suitable counterparts in the comparison group for every treated observation. If this is not the case, only the subsample ATT is point-identified. However, Lechner (2008) develops worst-case bounds in the spirit of Manski (1990), which help to assess whether or not the true ATT may be substantially different from the estimates for the subsample. Finally, it is necessary to rule out spillover effects (Stable Unit Treatment Value Assumption). Since the current German SUS program is relatively small—with only around 30,000 annual entries compared to roughly 350,000 entries into predecessor programs in 2004—this seems to be a reasonable assumption.

Implementing the Matching Procedure

The implementation of PSM requires estimating the propensity score and the imposition of common support before matching (see Caliendo & Kopeinig, 2008, for an overview). Subsequently, based on the chosen matching algorithm, the matching has to be performed, and finally, it is necessary to assess the resulting balancing quality. If the matching quality is not satisfactory, the propensity score specification needs to be reexamined, common support adjusted, or the matching algorithm changed until the matched sample can be regarded as balanced.

Estimation of the propensity score and the imposition of common support

As the first step of the matching procedure, we estimate the propensity score using a logit regression including information on the individuals’ labor market history, their sociodemographics, human capital acquisition, intergenerational transmission of education, labor force attachment, and self-employment, as well as usually unobserved personality traits and regional controls for macroeconomic conditions and self-employment activity. Overall, we use 91 control variables including some interaction terms that have been added in an iterative manner to improve resulting balancing quality.¹⁵ However, the quality of the control variables is more important than the sheer quantity of the variables included in the propensity score estimation. We have already argued above which variables are likely to influence selection into the subsidy and outcome variables simultaneously and argued that especially the availability of personality traits is important to our application. The rich nature of our data is paramount for the CIA to provide a reasonable assumption. Details on the exact specification employed can be found in Table A.3 in Online Appendix A. Generally, we aim to make the specification relatively flexible by using indicators for the categories of continuous variables rather than the continuous variables themselves in order to better balance higher moments of the confounders. Subsequently, the estimated coefficients are used to obtain predicted values of the propensity score. The distribution of these predicted values can be found in Figure 2.

Figure 2.

Propensity score distribution. Note. This graph shows the distribution of estimated propensity scores for the treated and comparison groups. The propensity score was estimated based on a logit regression with 91 variables in total, including information on sociodemographics, human capital, labor market history, intergenerational transmission, local macroeconomic conditions, personality traits, and some interaction terms chosen to maximize postmatching balance. For the exact specification and estimated coefficients, see Table A.3.

As expected due to the covariate imbalance described in Parameter of Interest and Identification Strategy section, scores are skewed toward 1 for the treated and toward 0 for comparison individuals. As stressed by Heckman et al. (1998), comparing individuals off common covariate support is a major source of selection bias. To avoid this, we impose common support by dropping treated individuals from the analysis if they lie outside the range of propensity scores among nonparticipants as described by Dehejia and Wahba (2002). This procedure was found to most strongly improve the mean squared error of matching estimators in a recent simulation study by Lechner and Strittmatter (2019).

Matching on the propensity score

Treated individuals are then matched with comparison individuals using Epanechnikov kernel matching to create a sample balanced in observed characteristics.¹⁶ Since PSM does not match on characteristics directly, it is necessary to judge the appropriateness of the propensity score specification against the resulting matching quality (Smith & Todd, 2005). Once a sufficiently balanced synthetic sample is created, mean differences in outcomes between participants and reweighed comparison individuals serve as the estimate of the ATT.

{\hat{τ}}_{ATT} = \frac{1}{N_{1}} \sum_{i = 1}^{N_{1}} Y_{i} - \frac{1}{N_{0}} \sum_{j = 1}^{N_{0}} {\hat{w}}_{j} Y_{j},

where ${\hat{w}}_{j}$ are estimated balancing weights and N₁ and N₀ are the number of treated and untreated observations in the sample, respectively. Statistical inference is then performed using resampling methods. Specifically, we obtain p values through bootstrapping the t statistic with 999 replications based on asymptotic variance approximations (Bodory et al., 2020; MacKinnon, 2006).¹⁷

Matching quality

Table 3 compares several commonly used balance indicators before and after matching. In general, one can see that matching quality dramatically increases. The number of statistically different means at the 10% level decreases from 61 to just 3 variables.¹⁸ Similarly, mean standardized bias decreases from 15.3% to 2.4%, which is below the 3%–5% threshold suggested by Caliendo and Kopeinig (2008). Reestimating the propensity score on the matched sample gives a pseudo R² of 1.8% and a corresponding p value of essentially 1. Following this approach, covariates overall no longer possess any predictive power with respect to treatment after performing the matching procedure. Balancing indicators based on Rubin (2001) measure standardized mean difference in the linear index of the propensity score (the so-called Rubin’s B) and the variance ratio of the propensity score (Rubin’s R). Ideally, these should be close to zero or one, respectively. We can see that Rubin’s B substantially decreases through the matching procedure, while Rubin’s R remains relatively close to one. Overall, the balancing quality is strongly improved by the kernel matching approach, which allows us to proceed with our causal analysis.¹⁹

Table 3.

Balancing Quality.

	Before Matching	After Matching
Number of variables with significant differences in means^a
At 1% level	46	0
At 5% level	57	1
At 10% level	61	3
Number of variables with absolute standardized bias^b
0% to less than 1%	11	28
1% to less than 3%	5	32
3% to less than 5%	11	21
5% to less than 10%	18	10
More than 10%	46	0
Mean absolute standardized bias in %	15.29	2.43
(Re)Estimation of the propensity score^c
Pseudo R²	.309	.018
p value of joint-significance test	.000	.996
Other measures
Rubin’s B^d	142.7	31.9
Rubin’s R^e	0.704	1.389
Number of variables	91	91
Number of participants off support		73

Note. Different indicators are shown for covariate balancing before and after Epanechnikov-kernel matching using a bandwidth of .15. UI = unemployment insurance.

^aNumber of variables with statistically different means is based on a t test of equality of means.

^bThe standardized absolute bias of a variable is the difference in means between treatment and comparison group as a percentage of the square root of the mean of prematched variances of both groups.

^cFollowing Sianesi (2004), pseudo R² and p value of joint significance from a logit estimation on the unmatched and the matched sample are also calculated.

^dRubin’s B is the standardized mean difference of the linear index of the propensity score of the treatment and comparison group.

^eRubin’s R is the variance ratio of the propensity score index of the treatment and comparison samples.

Main Results and Discussion

Table 4 gives our main estimates of the ATT of the New SUS program for unemployed individuals. The PSM estimates show statistically significant effects on individuals’ probability of being in either self- or regular employment as well as net monthly earned income. The program led to a 20.8 percentage point increase in the likelihood of being self- or regular employed and an increase of €910 a month in terms of net earned income 40 months after entering the SUS program.²⁰

In terms of subjective well-being, our baseline estimates suggest that the program has a short-run impact on general life satisfaction which fades over time, that is, 40 months after start of the program, the difference in life satisfaction between treated and matched controls is close to zero and not statistically significant. In order to gain a deeper understanding of the effects on overall well-being, we turn to analyzing effects with respect to certain subdomains of subjective well-being. Estimates are given in terms of both Likert points and the percentage of a standard deviation. First, we can state that there are positive short- and long-term effects of SUS participation on individuals’ satisfaction with their job situation. The treatment increases job satisfaction by about 0.28 points or 17.3% after 40 months. Point estimates for the effect on satisfaction with income are positive but insignificant at any common significance level. The point estimates on satisfaction with one’s health are negative and growing over time in magnitude but are also statistically insignificant. Finally, we can state that participation in the SUS program has a large negative effect on individuals’ satisfaction with their social security situation of about 0.51 (0.57) Likert points in the short (long) run, equal to a decrease of 32% (35.6%) of a standard deviation.

As previously noted, mean comparisons of ordinal variables may be misleading. Hence, we also estimate effects on the probability of scoring above the midpoint of the Likert-type scale (i.e., five or above) for the subjective well-being outcomes after 40 months. The results of this analysis can be found in Figure 3. The results support the zero average effects obtained by mean comparison for life satisfaction and satisfaction with health and income. Regarding job satisfaction, SUS increases the probability of scoring high on the Likert-type scale by about 6.5 percentage points. Moreover, SUS participation clearly reduces satisfaction with social security as it reduces the likelihood of scoring above the midpoint of the scale by about 17 percentage points. Thus, the evidence seems to suggest that participants—who mostly remain self-employed even after 40 months—show increased longer run worries about their social security situation. As this additional analysis supports the mean comparisons, we refrain from presenting results in such detail for the remainder of this article, rather focusing only on mean effects.

Figure 3.

Effect estimates on binary measures of subjective well-being. Note. This graph shows the effect estimates on the probability of scoring above the midpoint of the Likert-type scale for all measures of subjective well-being after 40 months; 95% confidence bands based on bootstrapping are shown.

To provide some evidence on which channels drive this last result, we also estimate the effect of the SUS program on some auxiliary objective (social) insurance outcomes. More specifically, we estimate the effects on the likelihood of contributing to the unemployment insurance system and retirement insurance plans. Our findings in Table 4 suggest that SUS participation reduces the likelihood of contributing to the public unemployment insurance scheme by over 20 percentage points.²¹ This may reflect that participants willingly choose investments in their business over joining the public unemployment insurance scheme. Alternatively, the effect may be driven by a lack of information regarding the 3-month time restriction to enter the insurance scheme for the self-employed. In any case, this implies that participants are more vulnerable to economic downturns in the future. Regarding retirement insurance, the program is estimated to reduce the probability of making contributions to some sort of retirement plan by about 4.8 percentage points after 40 months. In addition, participants are almost six percentage points more likely to view their retirement investment as insufficient at the same point in time. Taken together, this suggests that the program leads to reduced investments in retirement plans among participants, which could potentially increase the risk of old-age poverty. Unfortunately, our data do not allow us to quantify this risk. This is because we neither observe the size of contributions to retirement plans nor the accumulated benefits. Hence, we need to interpret the findings with caution.

Table 4.

Main Estimates.

	Outcomes After 20 Months		Outcomes After 40 Months
	ATT	p Value	ATT	p Value
Labor market outcomes
Self- or regular employed	0.266	.000	0.208	.000
Net monthly earned income (Euro)	740	.000	910	.000
Subjective well-being
Satisfaction with life in general
Likert points	0.152	.022	0.010	.865
in %	11.0%		0.8%
Satisfaction with health status
Likert points	−0.003	.960	−0.083	.169
in %	−0.2%		−5.8%
Satisfaction with income
Likert points	0.162	.116	0.037	.713
in %	9.5%		2.3%
Satisfaction with job situation
Likert points	0.662	.000	0.278	.009
in %	37.6%		17.3%
Satisfaction with social security
Likert points	−0.512	.000	−0.567	.000
in %	−32.2%		−35.6%
Social security
Not contributing to UI	0.217	.000	0.288	.000
Not contributing to a retirement plan	0.065	.000	0.048	.000
Retirement plan deemed insufficient	0.073	.010	0.057	.084

Note. The ATT estimates are based on kernel matching using an Epanechnikov kernel, a bandwidth of $h = 0.15$ (chosen by grid search to maximize postmatching balance in terms of pseudo R² in the reestimated propensity score regression) and common support imposition via min/max criterion using a logit regression. The estimated effects on satisfaction variables are given both in terms of Likert points and in percentage of a standard deviation. The p values are estimated by bootstrapping the t-statistic using 999 replications (Huber et al., 2014; MacKinnon, 2006). ATT = average treatment effect on the treated. UI = unemployment insurance.

Overall, our new results imply a slightly more mixed assessment of program effects compared to previous studies. On the one hand, SUS improve the employment prospects, income, and job satisfaction of participants. On the other hand, our results show negative effects on participants’ satisfaction with their social security situation as well as their UB (and retirement) insurance contributions. Hence, our analysis provides some evidence on unintended negative effects of SUS participation, suggesting that the program may potentially be improved by altering its institutional design. First, participants may need to be provided with more information on the legal constraints regarding their uptake of unemployment insurance to avoid being locked out of the system. Second, this may be combined with incentivizing individuals to increase their investments into unemployment and retirement insurance, for example, through targeted (or higher) support in the second benefit period. Taken together, these measures could reduce participants’ concerns regarding their social security situation.

Effect Heterogeneity

Knowledge about effect heterogeneity plays a crucial role in improving policy design and targeting individuals to be selected into treatment. Previous analyses of SUS programs have shown that effects on objective outcomes indeed appear to be highly heterogeneous (see Bellmann et al., 2018, Caliendo & Künn, 2011, for example). Hence, effect heterogeneity is also likely in terms of subjective outcomes. In order to analyze this, we split our sample by gender, age (above or below 45 years), and skills (being low- or high-skilled, where the latter are individuals with a craftsmanship or a university degree).

The previously described estimation steps are then repeated on these subsamples. Moreover, we perform statistical tests on the equality of ATTs across subsamples. Table 5 shows the results from this analysis after 40 months, including the number of treated and untreated individuals in each subsample as well as aggregate measures of covariate balance. While balancing quality tends to be worse than in the full sample, the mean standardized bias (MSB) remains below the 5% threshold suggested by Caliendo and Kopeinig (2008) and the covariates remain statistically insignificant in the reestimation of the propensity score in the matched sample. Comparisons with statistically significant differences in ATTs at the 10% level are italicized in the table. Note that due to limited sample sizes, even substantive differences in subsample effects may not be statistically significant. Therefore, we do not put too much emphasis on this distinction.

Table 5.

Effect Heterogeneity.

	Male		Female		Age $<$ 45 Years		Age $\geq$ 45 Years		High Skilled		Low Skilled
	ATT	p Value	ATT	p Value	ATT	p Value	ATT	p Value	ATT	p Value	ATT	p Value
Labor market outcomes
Self- or regular employed	0.221	.000	0.223	.000	0.153	.000	0.300	.000	0.178	.000	0.247	.000
Net monthly income (Euro)	1,193	.000	640	.000	766	.000	1,210	.000	864	.000	897	.000
Subjective well-being
Satisfaction with life in general	0.097	.240	0.030	.740	0.110	.155	0.050	.694	−0.034	.757	0.139	.223
Satisfaction with health status	−0.044	.634	−0.126	.258	−0.119	.202	−0.014	.902	−0.182	.099	0.104	.409
Satisfaction with income	0.270	.041	−0.067	.623	0.030	.781	0.040	.846	0.087	.570	0.146	.409
Satisfaction with job situation	0.363	.006	0.249	.117	0.270	.021	0.348	.062	0.215	.127	0.439	.032
Satisfaction with social security	−0.390	.008	−0.697	.000	−0.532	.000	−0.603	.000	−0.489	.000	−0.533	.001
Social security
Not contributing to UI	0.270	.000	0.305	.000	0.410	.000	0.143	.004	0.281	.000	0.290	.000
Not contributing to a retirement plan	0.048	.000	0.044	.027	0.052	.000	0.050	.000	0.027	.055	0.064	.000
Retirement plan deemed insufficient	0.085	.040	0.037	.398	0.019	.679	0.132	.003	0.031	.487	0.052	.255
Number of observations
Participants	717		531		691		557		704		544
Nonparticipants	591		613		626		578		432		772
Balancing quality after matching
MSB	3.536		3.920		3.510		4.956		3.818		4.326
Pseudo R²	.042		.035		.032		.067		.049		.049
p value	.727		.999		.986		.115		.207		.903

Note. The heterogeneity analysis is based on binary splits of the estimation sample. ATT estimates after 40 months are obtained via kernel matching using an Epanechnikov kernel, a bandwidth of $h = 0.15$ (chosen by grid search to maximize postmatching balance) and common support imposition via min/max criterion using a logit regression. The p values are estimated by bootstrapping the t-statistic using 999 replications (Huber et al., 2014; MacKinnon, 2006). While low skilled refers to having a vocational training degree or less, high skilled refers to having a craftsmanship degree or university degree. Estimates are italicized if ATTs are statistically significantly different in subsamples at least at the 10% level. ATT = average treatment effect on the treated. UI = unemployment insurance.

Starting with effects by gender, we find minimal differences in terms of effects on employment outcomes. However, men tend to gain much more from participating in the SUS scheme than women in terms of income. In line with this, men also display statistically significant positive effects on satisfaction with their income, while the effects for women display statistically insignificant and negative effects. For both of these outcomes—income and satisfaction with income—the effects are statistically significant different between men and women. Similarly, men display substantively stronger effects on job satisfaction than women. This may be due to the fact that men are more strongly affected by unemployment and therefore may also gain more from reemployment compared to women (Meer, 2014). Regarding the effects on satisfaction with social security, women show more detrimental effects. Note that gender differences in effects on job satisfaction and satisfaction with social security are not statistically significant, which may be due to a lack of power given the relatively small subsamples. Estimating effects by age groups shows that older individuals profit much more from SUS participation than younger individuals in terms of objective outcomes. In line with this, older individuals also display stronger effects on job satisfaction than younger participants.²² Another interesting finding in this comparison is that younger participants tend to reduce their investment in unemployment insurance (UI) substantially more than older individuals. However, older participants show more adverse effects on their assessment of the sufficiency of their retirement plans. Splitting the sample by skills, we find larger employment effects for low-skilled individuals. Moreover, our results suggest marginally significant negative effects on participants’ health satisfaction among high-skilled individuals. Finally, low-skilled participants display larger negative effects on retirement insurance contributions.

Sensitivity Analysis

For a rigorous impact evaluation, it is necessary to critically assess the applicability of identifying assumptions as the causal interpretation of our estimates crucially depends on them. In addition, the results of PSM or weighting should be analyzed for their robustness, given that these estimators require several steps of implementation with a moderate to relatively large number of discretionary choices, depending on the algorithm applied. Thus, in this section, we analyze the sensitivity of the results with respect to the matching or weighting approach chosen as well as deviations from the underlying identifying assumptions necessary for the matching approach to deliver consistent estimates.

Choice of estimator

As a first assessment of the robustness of our estimates, we test whether the kernel matching approach that we used in the previous section plays a crucial role for our results. Table 6 gives estimates for alternative estimation methods for selected outcomes.²³ First, we perform an alternative bandwidth selection for the kernel matching based on leave-one-out cross-validation (Frölich, 2005; Galdo et al., 2008). The resulting estimates are practically identical to our baseline results. Moreover, we augment our main approach by a postmatching regression to control for residual imbalance in the matched sample and find that point estimates and p values are largely unchanged compared to our baseline approach. Second, we employ inverse probability weighting (IPW) with weights scaled to unity as well as radius matching with bias adjustment as suggested by Lechner et al. (2011). For IPW, we truncate weights in the comparison group to a maximum of 4% to reduce the influence of a single observation. We chose these alternative estimators as Huber et al. (2013) and Busso et al. (2014) show that they tend to perform well in finite samples. Both of these approaches tend to deliver estimates that are even somewhat larger in magnitude than our baseline estimates. Overall, our results are relatively stable with respect to tuning parameters as well as the choice of matching or weighting estimator.

Table 6.

Sensitivity With Respect to Matching Approach for Selected Outcomes.

	Self- or regular employed		Satisfaction With				Currently Not
	Self- or regular employed		Income		Job Situation		Contributing to UI
	ATT	p Value	ATT	p Value	ATT	p Value	ATT	p Value
Baseline estimates	.208	.000	.037	.713	.278	.009	.288	.000
Kernel matching with optimal bandwidth	.210	.000	.042	.636	.290	.004	.286	.000
Kernel matching with bias adjustment	.206	.000	.031	.839	.262	.075	.294	.000
Inverse probability weighting	.214	.005	.197	.176	.299	.036	.291	.000
Radius matching with bias adjustment	.211	.000	.079	.393	.301	.027	.307	.000

Note. The table reports ATT estimates after 40 months and corresponding p values for different matching or weighting approaches. We compare our baseline estimates (from Table 4) to kernel matching estimates with an optimal bandwidth as chosen by leave-one-out cross-validation (Frölich, 2005; Galdo et al., 2008). The table also shows results based on kernel matching with postmatching regression, inverse probability weighting with weights truncated to 4% and scaled to unity as well as bias adjusted radius matching with the radius being equal to 300% of the largest distance in terms of the propensity score when using nearest neighbor matching as suggested by Huber et al. (2014). For information on the other outcomes, please see Table A.4 in the Online Appendix. UI = unemployment insurance.

Unconfoundedness

In order to assess the sensitivity of our estimates with respect to the unconfoundedness assumption, we follow Rosenbaum (2002) and use a bounding approach (see DiPrete & Gangl, 2004; Ichino et al., 2008, for other similar applications). Assume that the treatment probability is given by $P r (D = 1 | X) = Λ (x'_{i} β + γ u_{i})$ , where $Λ (\cdot)$ is the logistic cumulative distribution function. Further assuming that the unobserved confounder u_i is a dummy variable, the relative odds of two observationally identical individuals i and j receiving treatment can be written as:

\frac{1}{e^{γ}} \leq \frac{P r (D = 1 | X_{i}) / P r (D = 0 | X_{i})}{P r (D = 1 | X_{j}) / P r (D = 0 | X_{j})} \leq e^{γ},

where $Γ : = e^{γ}$ is a measure of departure from the unconfoundedness assumption. The Rosenbaum bounds approach is a worst-case scenario as it assumes that the unobserved covariate u_i is close to being a perfect predictor of the outcome. Based on this assumption, the bounding approach allows us to establish whether our estimated treatment effects would still be significantly different from zero for a hypothetical value of $Γ$ . While this strategy does not tell us whether our results are actually inconsistent due to the failure of the CIA, it provides us with valuable information on how sensitive the results are to unobserved confounders that are very predictive of the outcome. In this bounding analysis, we will assume that we overestimated the true effect for significantly positive effects and that we underestimated the true effect for negative effects. Hence, we assume that the true effects are closer to zero. Table 7 gives the critical values of $Γ$ , that is, the value for which our estimated effects would turn insignificant. In general, we can state that the long-term labor market effects are very robust to “hidden bias” as an unobserved confounder must increase the odds of receiving treatment by a factor of at least 2.58 to render our inference with respect to positive effects on earnings insignificant. The critical $Γ$ for the effect on the probability of being in self- or regular employment is even larger. Turning to the robustness of our results on subjective well-being, we can state that these are more sensitive to deviations from the unconfoundedness assumption. Effects on job satisfaction and satisfaction with social security after 40 months turn insignificant at a value of $Γ$ around 1.2 and 1.8, respectively. With respect to social security outcomes, we can see that the negative effects of SUS participation on the probability of not having unemployment or retirement insurance are similarly robust as our inference regarding positive labor market effects. The estimated effect regarding individuals deeming their retirement insurance insufficient becomes insignificant at a value of $Γ = 1.25$ . Thus, we can state that for the least robust conclusions to be overturned, an unobserved confounder would need to be 1.22 times as predictive of treatment as all covariates in our propensity score specification combined. Effects regarding labor market outcomes, satisfaction with social security, uptake of unemployment insurance, and retirement insurance would require an even stronger role of unobserved confounders. This seems to be relatively unlikely given the detailed information that we have at our disposal through the administrative and survey data. Hence, the main conclusions drawn in our analysis are highly robust to deviations from the unconfoundedness assumption.

Table 7.

Sensitivity With Respect to Identifying Assumptions.

	Baseline Estimates		Rosenbaum Bounds	Lechner Bounds
	ATT	p Value	$Γ^{*}$	ATT^low	p Value	ATT^high	p Value
Labor market outcomes
Self- or regular employed	0.208	.000	5.030	0.190	.000	0.249	.000
Net monthly income (Euro)	910	.000	2.580	767	.000	1,039	.000
Subjective well-being
Satisfaction with life in general	0.010	.865	—	−0.070	.300	0.282	.120
Satisfaction with health status	−0.083	.169	—	−0.177	.002	0.174	.309
Satisfaction with income	0.037	.713	—	−0.120	.331	0.265	.053
Satisfaction with job situation	0.278	.009	1.220	0.167	.122	0.553	.001
Satisfaction with social security	−0.567	.000	1.780	−0.718	.000	−0.284	.080
Social security
Not contributing to UI	0.288	.000	3.220	0.254	.000	0.313	.000
Not contributing to a retirement plan	0.048	.000	4.690	−0.011	.811	0.047	.000
Retirement plan deemed insufficient	0.057	.084	1.250	0.034	.278	0.092	.003

Note. Reported are the results for our assessment of the identifying assumptions for estimates after 40 months. For comparison, columns 1 and 2 contain our baseline ATT estimates and p values from Table 4. Next, the table shows the critical values for the Rosenbaum bounds on deviations from the unconfoundedness assumption. The remainder of the table gives results from the Lechner bounding approach regarding deviations from the overlap assumption. UI = unemployment insurance.

Common support

In Implementing the Matching Procedure section, we described how the analysis is restricted to treated individuals who are in the region of common support. We implemented this by discarding treated observations that have estimated propensity score values outside the range of scores of comparison individuals since no comparable untreated individual can be found for those treated individuals. Table 3 shows that this restriction leads to the exclusion of 73 participants from our main estimation procedure, which corresponds to roughly 5.8% of treated individuals in our sample. If effects are heterogeneous as indicated by our analysis, the true ATT may well be different from our estimates even if the CIA holds. In order to assess the sensitivity of the estimates with respect to this issue, Lechner (2008) developed worst-case bounds as given by

τ_{ATT}^{Low} = E [Y^{1} - Y^{0} | D = 1, S = 1] \cdot π_{1} + {E [Y^{1} | D = 1, S = 0] - {\bar{Y}}^{0}} \cdot (1 - π_{1}),

τ_{ATT}^{High} = E [Y^{1} - Y^{0} | D = 1, S = 1] \cdot π_{1} + {E [Y^{1} | D = 1, S = 0] - {\overline{Y}}^{0}} \cdot (1 - π_{1}),

where S is an indicator for being on common support and $π_{1}$ is the probability of a treated person being on support. The values of ${\bar{Y}}^{0}$ and ${\overline{Y}}^{0}$ are given by the minimum and maximum of the support of Y⁰, and hence, the difference in curly braces gives the minimum and maximum possible average effect for the subgroup of treated units off support. Replacing the terms in Equations 5 and 6 with their empirical counterparts from our sample, we obtain the bounds and p values given by the last four columns of Table 7. It is important to keep in mind that these are worst-case bounds, that is, they give the range of estimates that are consistent with the data, but they do not make any statement on the likelihood of those bounds being reached in reality. Hence, the results have to be interpreted with the necessary caution if they suggest that inference may be overturned.

Our findings show that the estimates of labor market effects are completely robust to the support problem and the lower bounds still indicate large and highly statistically significant effects on employment and income. Similarly, the estimates on the probability of not contributing to the unemployment insurance still yield a statistically significant lower bound. The results on the other measures are slightly more nuanced. Upper bounds on the effects regarding satisfaction with life in general as well as social security are marginally insignificant at the 10% level. The bounding analysis also reveals that negative effects on satisfaction with respect to health and positive effects on satisfaction with income cannot be ruled out under extreme assumptions about the missing counterfactual estimates. The data would also be consistent with null effects on the uptake of retirement insurance contributions and its sufficiency under these assumptions. Thus, our main conclusions are predominantly supported by this bounding analysis.

Conclusion

Using nonexperimental counterfactual evaluation techniques, this article estimates the long-term effects of participation in the German New SUS program on individuals’ subjective well-being. Combining this with results on objective labor market outcomes, this allows us to analyze the effects of the program using a more thorough welfare definition than previously existing evaluations of SUS programs. Using a broader welfare measure is especially important in this context as subsidizing unemployed individuals into self-employment exposes them to more risk compared with regular employment. This problem is exacerbated by the fact that social security protection is lower for self-employed individuals, and hence, the program may have unintended negative effects on participants’ well-being in the long run. Our results based on PSM suggest that the program has relatively large positive and statistically significant effects on participants’ employment prospects, income, and satisfaction regarding the individuals’ job situation. On the other hand, we find sizable, robust, and statistically significant negative effects on individuals’ satisfaction with their social insurance situation. Supplementary analyses suggest that these effects may be driven by reduced investment in unemployment insurance—making them more vulnerable to economic downturns in the future—and retirement insurance—potentially increasing the risk of old-age poverty (even though our data do not allow us to make concrete statements here). Thus, the program’s overall assessment is slightly less optimistic when taking these unintended negative effects into account. In our heterogeneity analysis, we find substantial variation in effects across gender, age groups, and skill levels. Our sensitivity analyses suggest that our main results are highly robust to deviations from the identifying assumptions and alternative choices regarding the implementation of the estimation strategy.

These findings underscore the relevance of using more subjective indicators of success to complement the analysis with respect to objective economic outcomes to improve policy design. Regarding policy conclusions, one lesson from our results is that it may be advisable to make more information on legal constraints regarding unemployment insurance for the self-employed available to SUS participants to avoid them being locked out of the system. Second, this may be combined with incentivizing individuals to increase their investments into unemployment and retirement insurance, for example, through targeted (or higher) support in the second benefit period.

Looking forward, it would be desirable to validate the nonexperimental results obtained so far with experimental evidence. This would provide researchers with the opportunity to vary the design of the program and thus ascertain whether the proposed solutions to participants’ dissatisfaction with their social security situation can be ameliorated in this way. Furthermore, SUS programs are in need of macroeconometric evaluations since they may provide further macroeconomic benefits such as additional job creation or losses due to deadweight effects that cannot be assessed using the microeconometric approach chosen here.

Supplemental Material

Supplemental Material, sj-pdf-1-erx-10.1177_0193841X20927237 - Do Start-Up Subsidies for the Unemployed Affect Participants’ Well-Being? A Rigorous Look at (Un-)Intended Consequences of Labor Market Policies

Supplemental Material, sj-pdf-1-erx-10.1177_0193841X20927237 for Do Start-Up Subsidies for the Unemployed Affect Participants’ Well-Being? A Rigorous Look at (Un-)Intended Consequences of Labor Market Policies by Marco Caliendo and Stefan Tübbicke in Evaluation Review

Footnotes

Authors’ Note

This article was prepared for the conference “Rigorous Impact Evaluation in Europe” in honor of Prof. Alberto Martini. Marco Caliendo is also affiliated with IZA Bonn, DIW Berlin and IAB Nuremberg, Germany.

Acknowledgments

The authors thank Lutz Bellmann, the editors, and five anonymous reviewers for helpful comments and the Institute for Employment Research for cooperation and institutional support within the research project number 1755. They also thank Prof. Martini for his lifelong dedication to evaluation questions and especially his drive to communicate the need for evaluations (and the associated problems) to policy makers.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors thank the German Research Foundation (Deutsche Forschungsgemeinschaft) for financial support under the project number 405629508.

ORCID iD

Marco Caliendo

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Andersen

S. H.

(2008). The short- and long-term effects of government training on subjective well-being. European Sociological Review, 24(4), 451–462.

Behrenz

Delander

Månsson

(2016). Is starting a business a sustainable way out of unemployment? Treatment effects of the Swedish start-up subsidy. Journal of Labor Research, 37(4), 389–411.

Bellmann

Caliendo

Tübbicke

(2018). The post-reform effectiveness of the new German start-up subsidy for the unemployed. Labour, 32(3), 293–319.

Benz

Frey

B. S.

(2008). Being independent is a great thing: Subjective evaluations of self-employment and hierarchy. Economica, 75(298), 362–383.

Bernhard

Grüttner

. (2015). Der Gründungszuschuss nach der Reform: Eine qualitative Implementationsstudie zur Umsetzung der Reform in den Agenturen (Forschungsbericht 4/2015). IAB Nürnberg.

Blanchflower

D. G

. (2004). Self-employment: More may not be better (Working Paper 10286). National Bureau of Economic Research.

Bodory

Camponovo

Huber

Lechner

(2020). The finite sample performance of inference methods for propensity score matching and weighting estimators. Journal of Business & Economic Statistics, 38(1), 183–200.

Bucciol

Zarri

(2017). Do personality traits influence investors’ portfolios? Journal of Behavioral and Experimental Economics, 68, 1–12.

Busso

DiNardo

McCrary

(2014). New evidence on the finite sample properties of propensity score reweighting and matching estimators. The Review of Economics and Statistics, 96(5), 885–897.

10.

Caliendo

Cobb-Clark

Hennecke

Uhlendorff

(2019). Locus of control and internal migration. Regional Science and Urban Economics, 79, 1–19.

11.

Caliendo

Cobb-Clark

Obst

Seitz

Uhlendorff

. (in press). Locus of control and investment in training. Journal of Human Resources.

12.

Caliendo

Cobb-Clark

Uhlendorff

. (2015). Locus of control and job search strategies. The Review of Economics and Statistics, 97(1), 88–103.

13.

Caliendo

Fossen

Kritikos

(2014). Personality characteristics and the decisions to become and stay self-employed. Small Business Economics, 42(4), 787–814.

14.

Caliendo

Hogenacker

Künn

Wießner

. (2015). Subsidized start-ups out of unemployment: A comparison to regular business start-ups. Small Business Economics, 45(1), 165–190.

15.

Caliendo

Kopeinig

(2008). Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys, 22(1), 31–72.

16.

Caliendo

Künn

(2011). Start-up subsidies for the unemployed: Long-term evidence and effect heterogeneity. Journal of Public Economics, 95(3-4), 311–331.

17.

Caliendo

Künn

Weissenberger

. (2019). Catching up or lagging behind? The long-term business and innovation potential of subsidized start-ups out of unemployment (Discussion Paper 12690). Institute for the Study of Labor (IZA), Bonn.

18.

Caliendo

Künn

Weißenberger

(2016). Personality traits and the evaluation of start-up subsidies. European Economic Review, 86, 87–108.

19.

Caliendo

Mahlstedt

Mitnik

(2017). Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies. Labour Economics, 46, 14–25.

20.

Caliendo

Tübbicke

. (2019). New evidence on long-term effects of start-up subsidies: Matching estimates and their robustness. Empirical Economics. Advance online publication. https://doi.org/10.1007/s00181-019-01701-9.

21.

Card

Kluve

Weber

(2018). What works? A meta analysis of recent active labor market program evaluations. Journal of the European Economic Association, 16(3), 894–931.

22.

Clark

Georgellis

Sanfey

(2001). Scarring: The psychological impact of past unemployment. Economica, 68, 221–241.

23.

Cobb-Clark

D. A.

Kassenboehmer

S. C.

Sinning

M. G.

(2016). Locus of control and savings. Journal of Banking & Finance, 73, 113–130.

24.

Cobb-Clark

D. A.

Schurer

(2012). The stability of big-five personality traits. Economics Letters, 115(1), 11–15.

25.

Cobb-Clark

D. A

Schurer

. (2013). Two economists’ musings on the stability of locus of control. The Economic Journal, 123 (570), 358–400.

26.

Coutts

A. P.

Stuckler

Cann

D. J.

(2014). The health and well-being effects of active labor market programs. In Cooper

C. L.

(Ed.), Wellbeing (pp. 1–18). American Cancer Society.

27.

Crost

. (2016). Can workfare programs offset the negative effect of unemployment on subjective well-being? Economics Letters, 140, 42–47.

28.

Dehejia

R. H.

Wahba

(2002). Propensity score matching methods for nonexperimental causal studies. The Review of Economics and Statistics, 84(1), 151–161.

29.

DeNeve

Cooper

(1998). The happy personality: A meta-analysis of 137 personality traits and subjective well-being. Psychological Bulletin, 124(2), 197–229.

30.

DiPrete

Gangl

(2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology, 34, 271–310.

31.

Duhautois

Redor

Desiage

(2015). Long term effect of public subsidies on start-up survival and economic performance: An empirical study with French data. Revue d’Économie Industrielle, 149(1), 11–41.

32.

European Commission. (2015). Recent changes in self-employment and entrepreneurship across the EU (Research note 6). Directorate-General for Employment, Social Affairs and Inclusion.

33.

Fairlie

R. W.

Karlan

Zinman

(2015). Behind the gate experiment: Evidence on effects of and rationales for subsidized entrepreneurship training. American Economic Journal: Economic Policy, 7(2), 125–161.

34.

Frölich

(2005). Matching estimators and optimal bandwidth choice. Statistics and Computing, 15(3), 197–215.

35.

Galdo

Smith

J. A.

Black

(2008). Bandwidth selection and the estimation of treatment effects with unbalanced data. Annales d’Économie et de Statistique, 91/92, 189–216.

36.

Heckman

J. J.

Ichimura

Smith

Todd

(1998). Characterizing selection bias using experimental data. Econometrica, 66(5), 1017–1098.

37.

Heckman

J. J.

Stixrud

Urzua

(2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24, 411–482.

38.

Huber

Lechner

Steinmayr

(2014). Radius matching on the propensity score with bias adjustment: Tuning parameters and finite sample behavior. Empirical Economics, 49(1), 1–13.

39.

Huber

Lechner

Wunsch

(2013). The performance of estimators based on the propensity score. Journal of Econometrics, 175(1), 1–21.

40.

Hurst

E. G

Pugsley

B. W.

(2011). What do small businesses do? Brookings Papers on Economic Activity, 42(2 (Fall)), 73–142.

41.

Hurst

E. G.

Pugsley

B. W.

(2016). Wealth, tastes, and entrepreneurial choice. In Haltiwanger

Hurst

Miranda

Schoar

(Eds.), Measuring entrepreneurial businesses: Current knowledge and challenges (pp. 111–151). University of Chicago Press.

42.

Ichino

Mealli

Nannicini

(2008). From temporary help jobs to permanent employment: What can we learn from matching estimators and their sensitivity? Journal of Applied Econometrics, 23(3), 305–327.

43.

Korpi

(1997). Is utility related to employment status? Employment, unemployment, labor market policies and subjective well-being among Swedish youth. Labour Economics, 4, 125–147.

44.

Krause

Franz

Fratzscher

(2017). Income groups and types of employment in Germany since 1995. DIW Economic Bulletin, 7(27), 267–278.

45.

Lange

(2012). Job satisfaction and self-employment: Autonomy or personality? Small Business Economics, 38(2), 165–177.

46.

Lechner

(2001). Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. In Lechner

Pfeiffer

(Eds.), Econometric evaluation of labour market policies (pp. 43–58). Physica-Verlag HD.

47.

Lechner

(2008). A note on the common support problem in applied evaluation studies. Annales d’Economie et de Statistique, 91/92, 217–235.

48.

Lechner

Miquel

Wunsch

(2011). Long-run effects of public sector sponsored training in West Germany. Journal of the European Economic Association, 9(4), 742–784.

49.

Lechner

Strittmatter

(2019). Practical procedures to deal with common support problems in matching estimation. Econometric Reviews, 38(2), 193–207.

50.

Lechner

Wunsch

(2013). Sensitivity of matching-based program evaluations to the availability of control variables. Labour Economics, 21, 111–121.

51.

Lindquist

Sol

van Praag

Vladasel

(2016). On the origins of entrepreneurship: Evidence from sibling correlations. (Discussion Paper 11562). CEPR.

52.

MacKinnon

J. G.

(2006). Bootstrap methods in econometrics. Economic Record, 82, 2–18.

53.

Manski

(1990). Nonparametric bounds on treatment effects. American Economic Review, Papers and Proceedings, 80, 319–323.

54.

McKee-Ryan

Song

Wanberg

Kinicki

(2005). Psychological and physical well-being during unemployment: A meta-analytic study. The Journal of applied psychology, 90, 53–76.

55.

Meager

(1996). From unemployment to self-employment: Labour market policies for business start-up. In Schmidt

O’Reilly

Schömann

(Eds.), International handbook of labour market policy and evaluation (pp. 489–519). Edward Elgar.

56.

Meer

(2014). Gender, unemployment and subjective well-being: Why being unemployed is worse for men than for women. Social Indicators Research, 115, 23–44.

57.

Nikolova

. (2019). Switching to self-employment can be good for your health. Journal of Business Venturing, 34(4), 664–691.

58.

O’Leary

C. J

. (1999). Promoting self employment among the unemployed in Hungary and Poland (Working Paper 99-55). W.E. Upjohn Institute for Employment Research.

59.

Perry

(2006). Are business start-up subsidies effective for the unemployed: Evaluation of enterprise allowance (Working Paper). Auckland University of Technology.

60.

Pfeiffer

Reize

(2000). Business start-ups by the unemployed—An econometric analysis based on firm data. Labour Economics, 7(5), 629–663.

61.

Puig-Barrachina

Giró

Artazcoz

Bartoll

Cortés-Franch

Fernández

González-Marn

Borrell

(2019). The impact of active labour market policies on health outcomes: A scoping review. European Journal of Public Health, 30(1), 36–42.

62.

Rodrguez-Planas

Jacob

(2010). Evaluating active labor market programs in Romania. Empirical Economics, 38(1), 65–84.

63.

Rose

(2019). The impact of active labour market policies on the well-being of the unemployed. Journal of European Social Policy, 29(3), 396–410.

64.

Rosenbaum

P. R.

(2002). Observational studies (2nd ed.) Springer.

65.

Rosenbaum

P. R.

Rubin

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.

66.

Rotter

(1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80(1), 1–28.

67.

Roy

A. D.

(1951). Some thoughts on the distribution of earnings. Oxford Economic Papers, 3(2), 135–146.

68.

Rubin

D. B.

(1974). Estimating causal effects of treatments in randomised and nonrandomised studies. Journal of Educational Psychology, 66(5), 688–701.

69.

Rubin

D. B.

(2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2(3), 169–188.

70.

Sage

(2015). Do active labour market policies promote the well-being, health and social capital of the unemployed? Evidence from the UK. Social Indicators Research, 124(2), 319–337.

71.

Sianesi

(2004). An evaluation of the Swedish system of active labour market programmes in the 1990s. The Review of Economics and Statistics, 86(1), 133–155.

72.

Smith

Todd

(2005). Does matching overcome Lalonde’s critique of nonexperimental estimators? Journal of Econometrics, 125(1-2), 305–353.

73.

Storey

(2003). Entrepreneurship, small and medium sized enterprises and public policies. In Acs

Audretsch

(Eds.), Handbook of entrepreneurship research, 18 (pp. 473–511). Kluwer Academic Publishers.

74.

Tokila

(2009). Start-up grants and self-employment duration (Working Paper). School of Business and Economics, University of Jyväskylä.

75.

Uysal

S. D.

Pohlmeier

(2011). Unemployment duration and personality. Journal of Economic Psychology, 32(6), 980–992.

76.

Vinokur

Schul

Vuori

Price

(2000). Two years after a job loss: Long-term impact of the JOBS program on reemployment and mental health. Journal of occupational health psychology, 5, 32–47.

77.

Vuori

Silvonen

(2005). The benefits of a preventive job search program on re-employment and mental health at 2-year follow-up. Journal of Occupational and Organizational Psychology, 78(1), 43–52.

78.

Winkelmann

(1998). Why are the unemployed so unhappy? Evidence from panel data. Economica, 65, 1–15.

79.

Wolff

Nivorozhkin

(2012). Start me up: The effectiveness of a self-employment programme for needy unemployed people in Germany. Journal of Small Business & Entrepreneurship, 25(4), 499–518.

80.

Wolff

Nivorozhkin

Bernhard

(2016). You can go your own way! The long-term effectiveness of a self-employment programme for welfare recipients in Germany. International Journal of Social Welfare, 25(2), 136–148.

81.

Wulfgramm

(2011). Can activating labour market policy offset the detrimental life satisfaction effect of unemployment? Socio-Economic Review, 9, 477–501.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.24 MB