The Impact on Job-Quality of Firm-Level Support to Innovation: Evidence from Natural Experiment Conditions and Linked Employer-Employee Data in Portugal

Abstract

Public support to firm-level investments in innovation is one of the main mechanisms through which the European Union promotes socioeconomic convergence among regions and the creation of quality jobs is considered a necessary condition for the convergence of disadvantaged regional economies. This paper exploits the availability of natural experiment conditions and linked employer-employee microdata in Portugal to offer empirical evidence on the impact on relevant job-quality outcomes of a large EU-cohesion-policy program to support SMEs’ innovation investments. The analysis is implemented by means of stratification/coarsened exact matching model, combined with a difference in difference scheme, suitable to the specific impact identification conditions. Our results indicate that the policy intervention in Portugal had a positive impact on job-quality outcomes, with each supported firm generating an average of 4.9 additional standard-working-time jobs, +2.9 skilled jobs, and +2.0 permanent-contract jobs, compared to a counterfactual scenario of no public support. These impacts were at a cost of about 16,100€, 27,100€ and 39,400€ in public subsidies per additional job, respectively. We also estimate that the program impact was responsible for a 2.20€ (+17.8%) increase of the per-hour remuneration. These findings are robust to sensitivity analysis, in terms of alternative matching procedures and comparison groups, and they highlight the fact that increasing job-quality is a policy goal that can be pursued, at a reasonable cost, also by means of cohesion-policy support to innovation aimed at enhancing the competitiveness of SMEs.

JEL classification: O1; R5; C23

Keywords

cohesion policy public support to innovation job-quality counterfactual impact evaluation

Introduction

Public support to firm-level investments in innovation, aimed at enhancing the competitiveness of Small and Medium Sized firms (SMEs), is one of the main mechanisms through which the European Union (EU) promotes socioeconomic convergence among European regions. Under the last programming period of 2014–2020, the European Commission has allocated around 20% of the European Regional Development Fund (ERDF)—57 billion €—to supporting SME investments in competitiveness, research and innovation, the low-carbon economy, and information and communication technologies.¹ The total amount of funds dedicated to these domains has been increasing over the years, as regional policy shifted away from investment in hard infrastructure towards business support and innovation (European Commission, 2014) and, in response to the COVID-19 crisis, the role of public support to SMEs’ innovation projects is further increasing as a major policy tool to stimulate economic recovery.

Within the domain of EU cohesion policies, the creation of durable and quality jobs is considered a necessary condition for the economic convergence of disadvantaged regions and it is cited (since the adoption of the Lisbon Agenda in 2000; European Commission 2008) as a main goal in many EU strategic documents². Indeed, the failure to create good quality jobs by the EU cohesion policies may jeopardize the development prospects of less-developed regions for two reasons. First, the expansion of low-quality jobs increases the risk of outsourcing production activities to areas with lower wages. Second, the limited availability of high-quality jobs fosters the outflow of human capital to the more advanced regions. Thus, improving job opportunities in disadvantaged regions, both quantitatively and qualitatively, is a main expected benefit of regional policies (Bartik, 2012) and one of the main goals of the EU Cohesion Policy. Moreover, boosting high-quality jobs is a crucial goal for business support schemes in economically disadvantaged regions also in order to generate a Y0 complement of education and training policies aimed at increasing the supply of skilled labor.

For all of the above reasons, providing EU policy makers with reliable empirical evidence on the job-quality impact of firm-level programs to support investments in innovation is of crucial importance.

Producing such empirical evidence, however, is not an easy task for a number of reasons. Measuring job-quality outcomes requires the rare availability of linked employer-employee microdata merged with complete program activity data. The existence of unobserved additional sources of public support generates potentially serious treatment measurement errors and firms often self-select into the program with no sources of exogenous treatment exclusions (making it hard to control for unobservable differences between the treatment and the comparison group). Finally, it is often difficult to determine the exact place in time in which the treatment occurs and the periods in which to measure the observable outcomes related to the supported investment.

Due to such difficulties, to the best of our knowledge, no reliable empirical evidence has yet been produced on the job-quality impact of firm-level support to innovation in the EU. This paper aims at filling the gap in the literature by providing reliable counterfactual impact estimates on the following job-quality outcomes: number of jobs with standard working hours³; number of jobs with high skill requirements⁴; number of jobs with permanent contracts; and level of hourly remuneration⁵.

Our analysis focuses on the direct firm-level support schemes offered by a large EU cohesion-policy intervention (referred to as the “Program for the Modernization of the Economy -POE/PRIME)”) implemented in Portugal over the first part of the years 2000s and. The program was aimed at enhancing the competitiveness of Portuguese SMEs by means of offering grants and soft loans to support firm-level investments in innovation in terms of acquisitions of physical-capital, IPR protection and IT software, R&D expenditures, and innovation activities for energy efficiency. We merged the program activity data from POE/PRIME with the linked employer-employee data collected yearly by the Portuguese Ministry of Employment and Social Solidarity on a compulsory basis. The resulting database offers desirable impact identification conditions to investigate the job-quality impacts of the EU firm-level support to innovation because of the following three reasons.

First, the POE/PRIME program has been a prominent EU-cohesion-policy intervention, with a total budget of nearly 8.3 billion €, financed in approximately equal parts by private and public sources (about 70% of the latter came from the ERDF). The POE/PRIME’s direct support to firms accounts for nearly ¾ of the total public funds of the program and it was implemented through several policy schemes that represented by far the most prominent source of public aid (in terms of both the amount of funds and number of beneficiaries) for the Portuguese firms in the 2000-06 programming period. These circumstances ensure a large external validity to the analysis and avoid the treatment contamination issues that would derive from the concurring presence of other unobserved policy interventions.

Second, the linked employer-employee administrative data used in the analysis are virtually free of the many attrition bias and measurement error limitations that plague the firm-level balance sheet data that are very often used in the evaluation of firm-support programs.

Third, in our analysis we exploit a unique period of the POE/PRIME program implementation that occurred within the years 2003-06, in which the operational rules for assigning the support deviated from the customary practice in Portugal. Within such period, the program was administered through a series of different calls for applications in which the selection into treatment was based on factors that were exogenous to the pre-treatment trends of the job-quality outcomes of the firms, determining favorable natural-experiment conditions.

To exploit this desirable impact identification scenario, we developed a preferred empirical strategy in terms of a stratification/coarsened exact matching model, combined with a difference-in-differences scheme, that is, estimated with a precise definition of the treatment spells related to the supported investments. This leads to the operationalization of different cohorts of the treated firms based on the specific pre- and post-intervention periods. Local impact estimates are then obtained for each of the different cohorts and subsequently aggregated into average treatment effects on the treated.

The results from this model are robust to sensitivity analysis, in terms of alternative matching estimators with a different composition of the comparison group, and they indicate that the cohesion-policy intervention in Portugal, by means of the POE/PRIME program, has positively affected the job-quality outcomes of the supported SMEs. This is in terms of each treated firm generating on average +4.9 standard-working-time jobs, +2.9 skilled jobs, and +2.0 permanent-contract jobs, compared to a counterfactual scenario of no public support. These impacts are at a cost of about 16,100€, 27,100€ and 39,400€ in public subsidies per additional job, respectively. We also estimate that the job-quality gains brought by the program were coupled by an increase in the average per-hour remuneration of the treated firms (+2.20€, equal to +17.8%, compared to the counterfactual scenario of no public support). These findings highlight the fact that increasing job-quality is a policy goal that can be indeed obtained also by means of cohesion-policy support to innovation aimed at enhancing the competitiveness of SMEs.

The paper is organized as follows. Job-Quality and Firm-Level Support to Investments and Innovation Projects discusses the rationale for the firm-level support to investments in innovation and the link with job-quality outcomes. POE/PRIME Program and Data describes in greater detail the data and the program features. Empirical Strategy illustrates the empirical strategy. Results presents the main results of the analysis, and Concluding Remarks and Policy Implications concludes, discussing the policy implications of our study.

Job-Quality and Firm-Level Support to Investments and Innovation Projects

Since the 1990s, work intensity and work stress have been increasing in several EU regions, with deleterious consequences for individual well-being and social inclusion (Clark, 2005; Green, 2006, 2008; Olsen et al., 2010). In addition, the new digital technologies are rapidly changing workplaces and substituting jobs, in a way that seems to polarize the workforce in terms of skills and work conditions, deepening income inequality within and across regions (Brynjolfsson & McAfee, 2014). These factors pose a great challenge to EU policy makers and explain the importance to foster the achievement of positive job-quality outcomes by means of public program interventions.

Indeed, job-quality (i.e. jobs with skilled, durable, fairly remunerated, work–life-balance characteristics)—has been shown to increase worker motivation, customer loyalty, productivity, while lowering absenteeism and turnover rates (Warr, 1999; Harter et al., 2003), and to improve firms’ performance and profitability (Boxall and Macky, 2009; Kaufman, 2010). At the regional level, an increase in high-quality jobs is also a main expected impact of local economic development policies (Bartik, 2012), including the EU cohesion-policy interventions in terms of firm-level support to investments in innovation. This is because the failure to create good quality jobs is expected to jeopardize the long-term development prospects of less-developed regions in two ways. First, a boost to local economies based on low-quality jobs may not be a stable positive outcome due to the risk of subsequent outsourcing of production activities to lower wage areas. Second, the lack of an increase in high-quality jobs encourages a subsequent outflow of human capital to the more advanced regions.

Public policies can influence the quality of work and employment in several ways (International Labour Organization, 2014; World Bank, 2013). Governments can affect payments, entitlements, working conditions, and job security, for instance, by establishing and enforcing labor market regulations. They can also exert influence on social dialog, enhancing the chances of achieving higher work standards through collective bargaining and agreements between trade unions and employers’ representatives. Finally, because job-quality outcomes are linked to technological advances, and firms (SMEs in particular) may lack autonomous resources to implement innovation projects at the optimal socio-economic degrees (Edler and Fagerberg 2017), there is also a strong rationale for considering firm-level support to innovation as an important policy tool to improve job-quality standards.

While job-quality outcomes are of high relevance in relation to public programs to support firm-level investments (in terms of projects with some innovation components), the existing impact evaluation literature of these programs focused exclusively on other firm-level outcomes such as employment, investment, sales, payroll costs, or labor productivity. In a number of cases, findings from this literature tend to indicate that the firm-level support was generally effective as a policy tool to spur some significant additionality compared to a counterfactual state of no-support, particularly when the program interventions targeted SMEs. For example, in the analysis of a large-scale capital grant support in Italy over the first decade of the years 2000, Cerqua and Pellegrini (2014) found a positive, and statistically significant effect on employment, investment, and turnover, while Bondonio and Martini (2019) estimated that a positive stimulus to additional employment growth and/or improvements in labor productivity was produced specifically from small subsidies given to small firms, rather than from large subsidies granted to large firms. Focusing on a variety of different firm-level subsidies to investments and innovation projects of the firms located in a North-Italian region over the first years of the 2000s, Bia and Mattei (2012) found positive employment impacts particularly for smaller firms and lower intensities of the support. Bondonio and Greenbaum (2006) investigated the employment impact of EU-funded incentives to investments and innovation projects in northern and central Italy during the late 1990s, with findings pointing to positive employment impacts in the target areas. In the UK, Criscuolo et al. (2012) studied the effects of capital grants to support firm investment projects in disadvantaged areas from the mid-1980s to the mid-2000s, with estimates of positive program treatment effects on employment for firms with fewer than 150 employees. In Finland, (Koski & Pajarinen, 2013) studied the effect of investment support and other subsidies granted to SMEs during the first part of the 2000s, with evidence of some positive impact on employment growth, except for the already fast-growing firms.

While these findings highlight that the support to SME investments and innovation projects can indeed produce some positive additionality on a number of relevant outcomes, they cannot shed any light on the crucially important issue of whether or not these types of policy interventions do also generate job-quality improvements. This paper fills this gap in the literature thanks to the opportunity to exploit both desirable natural experiment conditions (in terms of an exogenous treatment-assignment process that limits the selection bias issues due to unobserved heterogeneity between the assisted and non-assisted firms) and the availability of unique administrative micro-data that enables the analysis to focus directly on crucial job-quality indicators.

POE/PRIME Program and Data

The POE/PRIME⁶ program (the main Operational Program of the Third Community Support Framework in Portugal) is a large and comprehensive policy intervention that was co-funded by EU funds in the programming period of 2000-06. The program had a budget of nearly 8.3 billion €, financed in approximately equal parts by private and public sources (about 70% of the latter came from the ERDF).

The POE/PRIME’s direct support to SMEs included different program schemes that offered capital grants and soft loans to subsidize expenditures on productive equipment (buildings were usually excluded), R&D activities and several “dynamic competitiveness factors” (e.g., industrial property, fashion and design, ICT and digital economy, innovation consultancy, certification of management systems, and marketing). Grants were only used to support R&D activities and smaller investments. In the case of soft loans, firms benefitted from a period of grace of 2–3 years, depending on the nature of the projects, with zero interest. The amount of the support was defined as a proportion of the eligible expenditures, up to 75% of the total (depending on the type of investment and the size of the firm, among other factors). The public subsidy was calculated in Gross Grant Equivalent (i.e., the current value of the incentive converted into a non-reimbursable grant, if applicable), with an average value of about 150,000 euro, and had to comply with the limits imposed by EU State Aid rules.

While the different specific POE/PRIME’s program schemes differed somehow in their technical-administrative aspects, such as the minimum size and type of the admissible investments, they all shared similar sets of implementation rules and the common policy goal of enhancing the competitiveness of Portuguese SMEs by means of supporting investments in innovation. Such homogenous policy goal of the different schemes diminishes the relevance of estimating heterogeneous impacts based on the individual schemes and it is conducive (for the sake of improving the statistical efficiency and external validity of the analysis) to pooling together the program-activity data of all the POE/PRIME schemes.

Even though the POE/PRIME was not the only firm support program in Portugal during the period under analysis, it was by far the most relevant one, both in terms of public funds and number of beneficiaries. Less than 1% of the firms supported under these schemes received some kind of direct support from other programs. This scenario virtually eliminates the treatment contamination threats that may derive from the concurring presence of other unobserved public subsidies.

The POE/PRIME program activity data at our disposal cover all the investment projects submitted by the applicant firms, with information on: application date, total value of the investment project, the amount of the program support and the timing in which the supported investment took place (if applicable). This information allows us to sort the beneficiaries into different cohorts based on the exact dates of their application and investment project, virtually eliminating all the measurement errors that would arise when instead no data were available to determine the correct moment in which to measure the job-quality outcomes and the pre-intervention control variables.

The linked-employer-employee-data (LEED) used in the analysis stem from an administrative database maintained by the Portuguese Ministry of Employment and Social Solidarity. This database includes information related to both the characteristics of firms (total employment, industry classification, geographic location, legal status, ownership, number of plants, etc.) and their employees (sex, date of birth, educational background, professional category, type of contract, etc.). All information included in the LEED database is compulsorily submitted on a yearly basis by every employer firm operating in Portugal. For this reason, the database is virtually free of the many attrition bias and measurement error limitations that plague the employment information extracted from the balance sheet databases (e.g., the Bureau Van Dick’s Amadeus) often used in the evaluation of firm-support programs.

Using the firms’ National Tax Code as the common firm-identifier, we merged the LEED data with the POE/PRIME program activity data. Although the POE/PRIME program was active during the programming period 2000–6, the focus of our analysis is on the years 2003-6 because of the following reasons. First, for such period we can exploit the favorable natural experiment conditions that we describe in the next section. Second, for the 2003-6 years, we had the possibility to measure, both in the LEED and the program activity data, an adequate 3-year observation period before and after each of the yearly cohorts of treatments. Finally, the 2003–6 estimation period is highly representative of the entire 2000–6 programing period. This is because the 2003–6 period does include a high percentage (>60%) of the total number of treated firms recorded in the entire 2000–6 programming period, and the treated firms of the 2003–6 period share very similar pre-intervention characteristics and job-quality indicators with the total sample of firms supported in the 2000–6 programming period⁷.

As summarized in Table 1, the 2003–6 estimation period contains a total of 1513 firms that applied for the program incentives. Of these applicant firms, 654 (column 4) were excluded from the program incentives, based on the selection-into-treatment process described in the next section, while the remaining 859 firms (column 1) were treated by the program. Out of these 859 treated applicants, 389 of them (column 2) share a common support with 320 non-treated applicants (column 5), in terms of an exact matching of the following coarsened observable covariates: industrial sector, size, age, regional location and pre-intervention employment trend. The number of treated applicants that share a common support with the non-applicant firms is instead 856 (column 3). This is due to the presence of an ample universe of non-applicant firms (528,686 in total units, column 6).

Table 1.

Number of Firms by Treatment and Application Status. Full Samples and Units With Common Support.

		Treated Applicants			Non-Treated Applicants		Non-Applicants
		All (1)	Common Support with Non-Treated Applicants^(a) (2)	Common Support with Non-Applicants^(a) (3)	All (4)	Common Support with treated applicants^(a) (5)	All (6)	Common Support with Treated Applicants^(a) (7)
Total N. firms		859	389	856	654	320	528,686	140,899
N. firms with information on:
	Jobs with standard working hours	856	386	856	642	308	516,074	137,287
	Skilled jobs	856	385	856	654	320	519,686	140,899
	Jobs with permanent contracts	859	389	855	654	311	516,194	137,407
	Average hourly remuneration	857	387	856	654	290	514,272	135,485

(a) Common support based on exact matching on the following coarsened observable covariates of SMEs: industrial sector, size, age, regional location, and pre-intervention employment trend

As described in the bottom portion of Table1, the data on the four job-quality outcomes considered in the analysis are quite complete, and the number of missing observations due to incomplete data coverage is negligible: a maximum of three firms for the treated applicants, representing 0.3% of the total treated units (column 1), 12 firms at most for the non-treated applicants (1.8% of the total, column 4), and 2.7%, at most, for the universe of firms that did not apply to the program.

Empirical Strategy

Estimating the causal impact of the POE/PRIME program, in terms of policy-relevant average treatment effects on the treated (ATT), entails comparing the job-quality outcomes recorded in the treated firms with an estimation of the counterfactual outcomes that would have been recorded in the same units in the absence of the program. To estimate the latter, an ideal scenario would be to observe the job-quality outcomes recorded in non-treated firms with nearly identical (observable and unobservable) major job-quality determinants than those of the treated firm. As discussed in Job-Quality and Firm-Level Support to Investments and Innovation Projects, these job-quality determinants, to be controlled for in the analysis, are mostly related to the propensity to innovate of firms and, in terms of observable characteristics, they can be proxied as follows: industrial sector (coarsened into low- and medium-low-tech manufacturing industries; high- and medium-high-tech manufacturing industries; construction; trade; accommodation and food service activities; knowledge-intensive market services); the pre-intervention size class and age of the firms (which are related to the ability of both designing and implementing innovations in the production process and acquiring financing for the implementation of the innovation projects from the private credit market) and regional location.

In terms of unobservables, relevant factors are, for example, the managerial abilities and organizational culture and capacity that lead to the desire and ability of planning and designing innovative aspects in the production process and/or products. For these unobservables, non-treated applicant firms are a suitable comparison group because of the following three reasons. First, non-treated applicants submitted, in the same way as the treated firms, an innovation project as part of the application process, showing the same desire to design and promote innovation features. Second, as discussed in Job-Quality and Firm-Level Support to Investments and Innovation Projects, the propensity to innovate is related to job-quality outcomes, and for this reason, the general population of non-applicant firms is likely to have different levels of relevant unobservable characteristic than the applicant firms, due to the fact that the latter had planned an innovation project submitted for the support of POE/PRIME program. Third, during the 2003–6 estimation period, the applicant firms were selected into treatment based solely on two factors: a) the sum of the expected taxable profits, the additional labor costs and the interests paid to financial institutions (for the part of the investment not covered by the program funds), in percentage of the total investment value (at present values); b) an indicator of financial autonomy, in terms of the ratio between the firm’s equity and its total assets, intended to capture the expected risk of failure of the firm. The latter worked as an exclusion criterion: projects submitted by firms’ whose ratio of financial autonomy was below a certain threshold (which varied between 20% and 25% across all calls) were not considered for further analysis. These two factors are not theoretically linked to job-quality outcomes, and, during the estimation period, all the information on the investment project submitted by the applicant firms were predominantly prepared by external consultants. The application packages prepared by the consultants included figures that were determined by exploiting all the degrees of discretion allowed by the program bookkeeping rules in order to increase the likelihood of success in receiving the support. Thus, the resulting application scores were also significantly affected by such degrees of discretion used by the external consultants, further weakening any endogeneity of the treatment assignment process with regard to the pre-application trends of the job-quality outcomes of the application firms.

For these reasons, the features of the treatment assignment process are likely to represent favorable impact identification conditions that mimic a natural experiment scenario in which the treated applicants do not have systematically different characteristics, compared to the non-treated applicants, in terms of crucial unobservables that can have an impact on the changes of the job-quality outcome variables independently from the program support. Indeed, this occurrence is also empirically supported by the comparison of the pre-application job-quality changes between the treated and untreated applicants (Table 2): in the case of a treatment assignment process that was based on firm-level unobservables that are relevant determinants of job-quality outcomes, we should observe a sharp difference in the pre-intervention trends of the same job-quality outcomes between the treated and non-treated applicants. This would indicate, for example, that the treated firms were selected based on unobservable characteristics that determine more favorable changes of the job-quality outcomes independently from the program intervention, posing a serious selection bias threat⁸.

Table 2.

Pre-application Changes of the Job-quality Outcome Variables—Average Change (per Firm) Between the (t-3) and (t-1): Treated Firms versus Non-treated Applicants.

	ΔY [(t-1) -(t-3)] ^(a)
Outcome variable Y	Treated applicants (a)	Non treated applicants (b)	Δ (a)-(b) sd t-stat
Jobs with standard working hours (1 = 1 job)	1.85	2.41	−0.56 0.43 to 1.28
Skilled jobs (1 = 1 job)	1.61	1.29	0.32 0.35 0.91
Jobs with permanent contracts (1 = 1 job)	1.27	1.78	−0.51 0.42 to 1.21
Average hourly remuneration (1 = 1 €)	2.70	3.36	−0.64 0.39 to 1.68

(a) t = time of application.

The summary statistics of Table 2, instead, indicates that for all the four job-quality outcomes, the difference in the changes between treated- and non-treated- applicants is quite small and not statistically significant. Moreover, for three of the four outcomes (jobs with standard working hours, permanent-contract occupations and the average hourly remuneration) such slim difference is in favor of the non-treated applicants rather than the treated applicants

In relation to the selection into treatment process described above, Figure 1 summarizes the pools of possible control SMEs that can be used in the analysis (second column) with the accompanying number of treated and non-treated SMEs that share a common support in terms of crucial observable characteristics (third column to the left). In the last column to the right, Figure 1 highlights at glance the impact identification conditions related to the two options for drawing the possible controls: non-treated applicants versus non-applicant firms.

Figure 1.

Selection into treatment process, pools of possible controls and impact identification conditions.

Because the admission into treatment of the applicant firms occurred with some desirable natural experiment conditions that did not result in treatment-control systematic differences related to the determinants of job-quality outcomes (as also empirically supported in Table 2), following seminal precedents, such as Angrist (1998), a preferred option for the empirical analysis is that of focusing on comparison groups composed solely by non-treated applicants.

Due to the limited sample size (654 units, Table 1 and Figure 1) of the non-treated applicants, this option entails the limitation of reducing the estimation sample somewhat. This is because, when the pool of possible controls is restricted only to the non-treated applicants a common support of observable baseline characteristics can be found for about 45% of the treated firms (due to the fact that the number of the latter is significantly higher than that of the former). However, despite the smaller estimation sample, this is still a preferable empirical option. In fact, while differences in observable characteristics can be controlled for by implementing statistical matching procedures, heterogeneity in unobservable characteristics associated with different trends of the job-quality outcomes remains a threat to the validity of the analysis. The use of comparison groups composed of non-treated applicants (instead of including the non-applicants) restricts the treatment-comparison group to firms that are more likely to share similar unobservables. Moreover, the results from the smaller sample of treated firms with common support with the non-treated applicants are likely to be well representative of those from the entire sample of treated firms at large (as suggested empirically by the absence of statistically significant differences in the pre-treatment changes in the job-quality outcomes of the two samples of treated firms⁹, an occurrence that mitigates the concerns of a limited external validity of the results). In any case, to test the external validity of the results in a larger estimation sample, we performed a sensitivity analysis by estimating three additional empirical models (described in Sensitivity Analysis: Impact Estimation With Comparison Groups Drawn Also From Non-applicants) that adopt comparison groups drawn from the pool of non-applicants.

Impact Estimation With Comparison Groups Drawn From Non-treated Applicants

For impact estimations based on comparison groups restricted to the applicant firms, the favorable impact identification conditions previously described are exploited by means of a stratification matching combined with a difference in difference (DD) design that compares the pre-post application job-quality outcome change of the treated and non-treated applicants sharing the same baseline characteristics that are important determinants of the job-quality outcomes. The approach can be also characterized as a coarsened exact matching (CEM, Iacus et al. 2011, 2012) estimation in which, after an initial pre-processing of the data in terms of exact matching on the most relevant coarsened baseline characteristics, the analysis is implemented with a treatment-control DD comparison of the pre-post intervention changes of the average outcomes. In detail, the estimation procedure for this stratification/coarsened exact matching, combined with a DD model, can be summarized as follows:

(i) All firms contained in the POE/PRIME-LEED database are sorted into applicants and non-applicant groups, with a further division of the applicant group into successive cohorts (p) of applicants based on their year of application.

(ii) Within each cohort (p) of applicants, the relevant covariates (X) are coarsened into a number of discrete categories. These categories represent the following firm-level characteristics that in the pre-application period are shown to correlate with job-quality outcomes:

(a) Industrial sector, coarsened into: low- and medium-low-tech manufacturing industries; high- and medium-high-tech manufacturing industries; construction; trade; accommodation and food service activities; and knowledge-intensive market services;

(b) Pre-application firm size: 1–4 employees; 5–9; 10–19; 20–49; 50–99; and 100–250;

(d) Geographic location: Norte; Algarve; Centro; Lisboa; Alentejo; Açores; Madeira; and Inter-regional¹⁰;

(e) Pre-application growth trend of employment (in terms of employment change between (t-3) and (t-1), with t being the application year): workforce increase; constant; decrease; and firm not in existence at t-3;

(f) The firms’ pre-application level of the job-quality outcome (Y) used in the analysis is coarsened into two categories: one below and one above the average of the firms within the same sector and size class.

(iii) Separately for each cohort (p) of applicants sharing the same application year, we match the treated with the non-treated applicants with the same exact values of the coarsened controls (X). The result of this matching procedure is a number of cells (c) with identical categories of the controls (X).

(iv) We estimate local ATTs for each cell (c) of identical treated and non-treated matched applicants sharing a same application year (p). Our DD estimator is as follows

T_{pc}^{DD} = E [Y_{t + 3}^{1} - Y_{t - 1}^{0} {|T}_{c} =, X, p] - E [Y_{t + 3}^{0} - Y_{t - 1}^{0} {|T}_{c} = 0, X, p]

(1)

(v) For each cohort (p) of treated applicants sharing the same application year, we estimate the Average ATTs as weighted averages of the local τ^DD_pc, as follows

T_{P}^{D D} = \sum_{c} [{τDD}_{pc} * W p_{c}]

(2)

with weights Wp_{c =}

\frac{N p_{c}}{\sum^{​} N p_{c}}

computed as the number of treated firms in cell (c) over the total number of treated firms in cohort (p)11.

(vi) Finally, we estimate the Global ATTs, across all yearly cohorts of applicants, as the weighted average of τDDp, with weights proportional to the number of treated firms in each cohort (p).

Similarly as in Bondonio & Greenbaum (2018) and Bondonio & Martini (2019), theestimation procedure described above is implemented with a dynamic identification of the pre- and post-treatment periods that follows the specific years in which each treated firm started and ended the supported investment project. Compared to a standard (static) approach in which the issue of heterogeneous treatment spells is dealt with a simplistic solution of fixing for all treated firms a common pre- and post-treatment year of reference for the analysis, our approach entails estimating different local impact estimates for each subsequent cohort of treated firms, based on the varying specific years of their treatment spells. This feature of the stratification/coarsened exact matching model avoids reliance on control variables that are fixed at an initial point in time for all subsequent annual cohorts of treatments. Rather, the matching process separately pinpoints with high accuracy the specific pre-treatment periods for each cohort of treated firms, enabling the reconstruction of a specific and more reliable estimate of the counterfactual trend for all treated firms.

Compared to a one-dimensional matching (e.g., propensity score (PS) matching), a stratification/coarsened exact matching (combined with DD) approach is preferable because many of the control variables are categorical by nature (i.e., firms’ industrial sector and regional location), while others represent risk-factors for selection bias mainly across discrete intervals (e.g., the pre-application size class of the firms and the distinction between relatively new firms versus incumbent firms at the time of application). In these circumstances, as also discussed elsewhere in the literature (e.g., Ho et al., 2007; King et al., 2011), a PS matching can disguise some admissible degree of mismatching on a single control variable. Such mismatching can lead to bias in the impact estimates, and for this reason it is recommended (Ho et al., 2007) that absolute constraints are placed in the matching procedures to guarantee the perfect balancing of the covariates with the highest rank of importance for the future determination of the outcome variable. Because of these considerations, the relatively small number of our equally important controls available for the case of POE/PRIME program is conducive to adopting a stratification/coarsened exact matching (combined with DD) approach that avoids hidden residual degree of unbalance on some single covariates that could be conducive to selection bias in the impact estimates. One-dimensional (PS) matching, combined with DD, is instead adopted as part of the sensitivity analysis discussed in Sensitivity Analysis: Impact Estimation With Comparison Groups Drawn Also From Non-applicants, to further enrich the testing of the robustness of the results.

Sensitivity Analysis: Impact Estimation With Comparison Groups Drawn Also From Non-applicants

As previously mentioned, implementing a preferred estimation model with comparison groups that are restricted to applicant firms only, represents taking a one-side stance in the bias/precision trade-off declined in terms of the size of the estimation sample versus the degree of restrictions posed by the impact identification assumptions. This is in favor of the least restricted assumptions, leading to a smaller number of treated firms that can be considered in the analysis. This choice has a strong rationale because, as also previously described, such smaller sample of treated firms is very likely to be representative of all treated firms at large (as shown by pre-treatment changes of the job-quality outcomes that are similar between the restricted and unrestricted samples).

Nevertheless, as an alternative option to further check the external validity of the results, the estimation sample of treated firms could be noticeably enlarged (improving also somehow the statistical efficiency of the estimation) if also the non-applicant firms were included in the comparison group. This is due to the large sample of non-applicants (more than half-million firms) available for the analysis that makes it easier to find a common support in terms of the crucial observables baseline characteristics for nearly all treated firms. As discussed in Empirical Strategy, however, the viability of this option is challenged by the fact that treated firms and non-applicant firms are likely to differ in crucial unobservable characteristics. For this reason, alternative empirical strategies that consider comparison groups composed by non-applicant firms become feasible only if it can be assumed that treated and non-applicant firms, conditional of precisely defined observable characteristics (i.e., once they are matched on the basis of identical crucial observables), differ in terms of unobserved characteristics that are fixed effects (i.e., exerting a constant-over-time influence on job-quality, along the estimation period). These fixed effects can be controlled for by superimposing a difference in difference scheme onto the statistical matching procedure. Following a standard practice in the literature (e.g., Card and Krueger, 1994, 2000; Angrist and Pischke, 2008; Lechner 2011 and Aragón and Rud 2013), such fixed-effect assumption of the unobservables can be tested empirically by checking whether or not the pre-intervention trends of the job-quality outcomes of the treated and non-applicant firms are parallel. In our case, once conditioned on similar observable characteristics (i.e. when comparing matched samples of treated and non-applicant firms based on similar propensity scores or identical coarsened observable characteristics), the trends of the treated firms and matched non-applicants become indeed strictly parallel (confirming the conditional fixed-effect assumption). This is in terms of pre-intervention [(t-3)-(t-1), with t = year of treatment] changes of the four job-quality outcome variables that are not different¹² between the treated firms and the matched sample¹³ of non-treated firms with the same baseline characteristics.

For this reason, statistical matching models in conjunction with a difference-in-differences scheme (i.e., conditional difference-in-difference models) can also be used for the analysis when the comparison groups are composed by non-applicant firms. We exploit this possibility as a way to perform a sensitivity analysis aimed at testing the external validity of our results and to assess how stable are the results when a different stance is taken in the bias/precision trade-off. The following three statistical matching estimators are used for this task:

I) stratification/coarsened exact matching, based on the same baseline characteristics used for the preferred estimation model. These characteristics represent the factors that in the pre-application period are shown to correlate with job-quality outcomes: industrial sector (in terms of low- and medium-low-tech manufacturing industries; high- and medium-high-tech manufacturing industries; construction; trade; accommodation and food service activities; and knowledge-intensive market services); firm size (1–4 employees; 5–9; 10–19; 20–49; 50–99; and 100–25); firm age (1–4 years of age; 5 years or more); regional location (Norte; Algarve; Centro; Lisboa; Alentejo; Açores; Madeira; and Inter-regional); pre-treatment total employment growth trend (workforce increase; constant; decrease; and firm not in existence at t-3); and level of pre-treatment job-quality outcome (coarsened into two categories).

II) Propensity score (PS) radius¹⁴ matching based on the same categorical covariates, described above for the coarsened exact matching.

III) PS radius matching based on continuous functional forms of the following baseline characteristics: firm size (number of employees); firm age (in years); pre-intervention growth trend (employment change); pre-intervention level of the outcome variable.

For all of the above three statistical matching estimators, similarly to the preferred stratification/coarsened exact matching estimation, a DD scheme is superimposed onto the estimation procedure in terms of transforming the job-quality outcomes in pre-post-treatment [(t+3)-(t-1), t = year of treatment] differences. The pre- and post-treatment periods, also in this case, are determined by a precise identification process that follows the specific year (t) in which each treated firms received the program support.

Results

The main results of the analysis, in terms of average treatment effect on the treated, ATT of the POE/PRIME program are summarized in Table 3 for both the preferred empirical model and the sensitivity analysis.

Table 3.

Impact of the POE/Prime Program on Job-Quality Outcomes: Estimates From the Preferred Estimation Model and Sensitivity Analysis.

	Comparison Group Restricted to Non-Treated Applicant (Preferred Estimation)	Comparison Groups Extended to Non-Applicant Firms (Sensitivity Analysis)
Output variable Y	Stratification/Coarsened Exact Matching - DD	Stratification/Coarsened Exact Matching - DD	PSM-Radius Categorical Controls - DD	PSM -Radius Continuous Controls -DD
Jobs with standard working hours
ATT estimates (1 = 1 job)	4.9**	5.9 ***	5.8 ***	5.0 ***
(Std. Error)	(1.65)	(0.62)	(0.14)	(0.15)
N. treated firms	386	856	856	856
N. potential controls^(a)	308	137,287	137,287	137,287
Skilled jobs
ATT estimates (1 = 1 job)	2.9**	4.1 ***	3.9 ***	4.4 ***
(Std. Error)	(1.41)	(0.50)	(0.13)	(0.13)
N. treated firms	385	856	856	856
N. potential controls^(a)	320	140,899	140,899	140,899
Permanent jobs
ATT estimates (1 = 1 job)	2.0*	4.3 ***	4.1 ***	3.2 ***
(Std. Error)	(1.57)	(0.49)	(0.13)	(0.13)
N. treated firms	389	859	859	859
N. potential controls^(a)	311	137,407	137,407	137,407
Avg. remuneration per hour
ATT estimates (1 = 1 €)	2.2**	1.8 ***	0.0	0.0
(Std. Error)	(1.05)	(0.44)	(0.03)	(0.03)
N. treated firms	387	857	857	857
N. potential controls^(a)	290	135,485	135,485	135,485

(a) Total number of non-treated firms, with common support based on coarsened baseline characteristics, from which the matched comparison units are selected for the analysis.*** = statistically significant at 1% level.** = statistically significant at 5% level.* = statistically significant at 10% level.

The first column to the left of table contains the impacts estimates from the preferred model in terms of stratification/coarsened exact matching, combined with DD, with comparison groups restricted to non-treated applicants. In this case, the results of the analysis highlight that the POE/PRIME program generated a growth in the number of jobs with standard working hours that is higher than the counterfactual trend of no public support by an average of 4.9 additional jobs per firm, with and estimated standard error of 1.6, and a resulting statistical significance at the 5% level. Such a positive program impact on job-quality indicators is confirmed also for the number of skilled and permanent jobs (with an estimated growth impact of +2.9 skilled jobs and +2.0 permanent jobs per firm, compared to the counterfactual trend of no public support), and for the change in the average pay per hour [with an estimated growth impact of +2.20€ per hour (+17.8%)¹⁵ compared to the counterfactual trend of no public support]. The estimated standard errors for these latter estimates (1.41 and 1.57, for the skilled and permanent jobs, respectively) are slightly smaller than that for the jobs with standard working hours. These figures lead to a 5% statistical significance level for the estimated impact on skilled jobs, while, for the result on the permanent jobs, the statistical significance remains at 10% level due to slightly smaller magnitude of the point estimates. Given that the average value of the support assigned to each treated firm included in the analysis is about 78,720€, these impact estimates from the preferred empirical model yield the following cost of the subsidies per each additional job generated by the program: 16,065€ for the jobs with standard working hours; 27,145€ for the skilled jobs; and 39,360€ for the permanent jobs.

The last three columns to the right of Table 3 contain the results from the sensitivity analysis, in terms of the three different matching estimators, combined with a DD scheme, based on comparison groups that include non-applicant firms. For each of these alternative matching models, the treatment-control balance in the baseline characteristics of the matched firms is always achieved with no statistically significant differences (at 5% level) and with a remaining percentage bias <10%. The latter is computed in terms of the percent difference between the sample mean of the treated and the non-treated matched firms (computed as the ratio between the difference of the two sample means and the square root of the average of the two sample variances, such as in Rosenbaum and Rubin, 1985).

As expected, due to the larger estimation sample, the ATT estimates from the sensitivity analysis have standard errors that are substantially lower than those of the preferred stratification/coarsened exact matching model. As a consequence, the level of the statistical significance (1%) of these results is higher than that of the preferred estimation. The point estimates for nearly all the four job-quality outcomes and all the three alternative matching models, instead, are overall in line with those from the preferred stratification/coarsened exact matching procedure, with slightly larger point estimates than the preferred model¹⁶. Overall, these results, obtained from an estimation sample that is enlarged to include virtually all of the treatment firms, adequately support the external validity of those obtained from the preferred estimation model that exploits more favorable natural-experiment impact identification conditions at the expense of reducing the estimation sample of the treated firms.

Concluding Remarks and Policy Implications

This paper exploits the availability of favorable natural experiment conditions and linked employer-employee microdata in Portugal to offer empirical evidence on the impact on job-quality outcomes of a large EU cohesion-policy program to support SMEs’ investments in innovation. The linked employer-employee data (LEED) that we use in the analysis (covering the period 2000–2009), merged with the activity data from the ERDF-supported POE/PRIME program of Portugal, offer favorable impact identification conditions for the following reasons. First, the LEED data come from administrative records that are free of the many attrition bias and measurement error limitations that plague the firm-level balance sheet data commonly used in counterfactual evaluation of firm-level support programs. Second, the POE/PRIME program activity data ensure an adequate external validity for the results and limit the treatment contamination issues deriving from the concurring presence of other policy interventions. This is because the POE/PRIME program is a very prominent EU cohesion policy intervention, with a total budget of nearly 8.3 billion € (and direct support to firms that represented nearly ¾ of the program’s total public funds) and it was by far the most prominent source of support for Portuguese firms during our period of observation.

We exploit these favorable impact identification conditions by means of a preferred estimation model in the form of a stratification/coarsened exact matching procedure, combined with a difference-in-differences scheme that is adopted separately for each of the successive cohorts of applicant firms. The robustness and external validity of the estimates from this preferred model is tested by means of a wide sensitivity analysis in terms of using a different comparison group (the universe of all non-treated firms, instead of non-treated applicant firms) and using alternative statistical matching approaches (such as radius matching on a propensity score).

Our results indicate that the POE/PRIME program has generated positive impacts on a number of relevant job-quality outcomes, compared to a counterfactual scenario of no public support, with each treated firm creating on average 4.9 additional standard-working-time jobs, 2.9 skilled jobs, and 2.0 permanent-contract jobs. We also estimate that these job-quality gains were coupled by positive impacts on the average per-hour remuneration of the treated firms (+2.20€, equal to +17.8%).

In terms of cost per unit of impact, these results highlight that about 16,100€ of public subsidies are needed to generate each additional job with standard working hours, while this cost is about 27,100€ for the skilled jobs; and about 39,400€ for the permanent jobs. In the existing literature on business incentive programs in Europe, the cost-per-job figures related to general employment outcomes tend to be within a range of 6300€−77,500€¹⁷. In this regard, our results indicate that for the case of the POE/PRIME program, EU-cohesion-policy interventions, in terms of support to business innovation investments, generated favorable job-quality outcomes at a cost (per unit of impact) that remains in line with that attributable to general employment gains.

These findings highlight the fact that improving job-quality outcomes is a policy goal that can be pursued, at a reasonable cost, also by means of EU cohesion policy interventions aimed at supporting SMEs’ investments in innovation (in addition to labor market regulations, social dialog, and active labor market interventions, see for example, International Labor Organization, 2014; World Bank, 2013). For this reason, our results also suggest that the regional policies attempting to increase the endowment of human capital through individual-based approaches (such as education and job-training programs, as opposed to place-based approaches, see for example, Barca et al., 2012) can be successfully complemented with policy instruments that boost the demand for high-quality labor by means of supporting SMEs’ investments in innovation. These considerations are of particular relevance in the present times in which an increasing pressure is placed on the EU cohesion policy to effectively foster the creation of decent job opportunities in the less developed regions that have to deal with enduring high unemployment rates and slow economic growth.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Daniele Bondonio

Teresa Farinha

Notes

References

Angrist

J. D.

(1998). Estimating the labor market impact of voluntary military service using social security data on military applicants. Econometrica, 66, 249-288.

Angrist

J. D.

Pischke

J. S.

(2008), Mostly harmless econometrics: An empiricist's companion. Princeton, NJ: Princeton University Press.

Aragón

F. M.

Rud

J. P.

(2013). Natural resources and local communities: Evidence from a peruvian gold mine. American Economic Journal: Economic Policy 5, 1-25.

Barca

McCann

Rodríguez-Pose

(2012). The Case for Regional Development Intervention: Place-based versus Place-neutral Approaches*. Journal of Regional Science 52, 134-152.

Bartik

T. J.

(2012). The Future of State and Local Economic Development Policy: What Research Is Needed. Growth and Change 43, 545-562.

Bia

Mattei

(2012). Assessing the effect of the amount of financial aids to Piedmont firms using the generalized propensity score. Statistical Methods & Applications 21, 485-516.

Bondonio

Greenbaum

R. T.

(2006). Do Business Investment Incentives Promote Employment in Declining Areas? Evidence from EU Objective-2 Regions. European Urban and Regional Studies 13, 225-244.

Bondonio

Greenbaum

R. T.

(2014). Revitalizing regional economies through enterprise support policies: An impact evaluation of multiple instruments. European Urban and Regional Studies 21, 79-103.

Bondonio

Greenbaum

R. T.

(2018). Natural disasters and relief assistance: Empirical evidence on the resilience of U.S. counties using dynamic propensity score matching. Journal of Regional Science 58, 659-680.

10.

Bondonio

Martini

(2012). Counterfactual impact evaluation of cohesion policy: impact and cost-effectiveness of investment subsidies in Italy. Final Report prepared for DG-Regional Policy. Brussels, Belgium: European Commission. http://ec.europa.eu/regional_policy/archive/information/evaluations/impact_evaluation_en.cfm

11.

Bondonio

Martini

(2019). Are they worth it? A Counterfactual Impact Evaluation of a Decade of Investment Subsidies to Italian Firms. Italian Journal of Regional Science 2, 261-190, doi: 10.14650/93650.

12.

Boxall

Macky

(2009). Research and theory on high-performance work systems: progressing the high-involvement stream. Human Resource Management Journal 19, 3–23.

13.

Brynjolfsson

McAfee

(2014). The Second Machine Age:Work, Progress, and Prosperity in a Time of Brilliant Technologies. New York, NY: W. W. Norton.

14.

Card

Krueger

A. B.

(1994). Wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania. American Economic Review 84, 772–793.

15.

Card

Krueger

A. B.

(2000). Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania: reply. American Economic Review 90, 1397–1420.

16.

Cerqua

Pellegrini

(2014). Do subsidies to private capital boost firms' growth? A multiple regression discontinuity design approach. Journal of Public Economics 109, 114–126.

17.

Clark

(2005). What Makes a Good Job? Evidence from OECD Countries. In Bazen

Lucifora

Salverda

(eds), Job Quality and Employer Behaviour, 11–30, Basingstoke, UK: Palgrave Macmillan.

18.

Criscuolo

Martin

Overmann

Van Reenen

(2012). The Causal Effects of an Industrial Policy. NBER Working Paper Series http://www.nber.org/papers/w17842).

19.

Edler

Fagerberg

(2017). Innovation policy: what, why, and how. Oxford Review of Economic Policy 33, 2–23.

20.

European Commission (2008). Employment in Europe 2008. Brussels, Belgium: European Commission, Directorate-General for Employment, Social Affairs and Equal Opportunities.

21.

European Commission (2014). Sixth report on economic, social and territorial cohesion: investment for jobs and growth - Promoting development and good governance in EU regions and cities. Brussels, Belgium: European Commission, DG for Regional and Urban Policy and DG for Employment, Social Affairs and Inclusion,

22.

Green

(2006). Demanding work: The paradox of job quality in the affluent economy. Princeton, NJ: Princeton University Press.

23.

Green

(2008). Work effort and worker well-being in the age of affluence. In Burke

Cooper

C. L.

(eds), The Long Work Hours Culture: Causes, Consequences and Choices, 115–136, Bingley, UK: Emerald Group Publishing.

24.

Harter

J. K.

Schmidt

F. L.

Keyes

C. L.

(2003). Well-being in the workplace and its relationship to business outcomes: A review of the Gallup studies. In Keyes

C. L.

Haidt

(eds), Flourishing: Positive psychology and the life well-lived. 205–224. Washington, DC: American Psychological Association.

25.

D. E.

Imai

King

Stuart

E. A.

(2007). Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis 15, 199–236.

26.

Iacus

S. M.

King

Porro

(2011). Causal Inference without Balance Checking: Coarsened Exact Matching. Political Analysis 20, 1–24.

27.

Iacus

S. M.

King

Porro

(2012). Causal Inference Without Balance Checking: Coarsened Exact Matching. Political Analysis 20(1), 1--24.

28.

International Labor Organization (2014). World of Work Report 2014. Developing with jobs. Geneva, Switzerland: ILO

29.

Kaufman

B. E.

(2010). The Theoretical Foundation of Industrial Relations and its Implications for Labor Economics and Human Resource Management. ILR Review 64, 74-108.

30.

King

Nielsen

Coberley

Pope

J. E.

(2011). Comparative Effectiveness of Matching Methods for Causal Inference. Cambridge, MA: Harvard University Press, https://gking.harvard.edu/files/psparadox.pdf

31.

Koski

Pajarinen

(2013). The role of business subsidies in job creation of start-ups, gazelles and incumbents. Small Business Economics 41, 195–214.

32.

Lechner

(2011). The estimation of causal effects by difference-in-difference methods. Foundations and Trends in Econometrics 4, 165-224.

33.

Olsen

K. M.

Kalleberg

A. L.

Nesheim

(2010). Perceived job quality in the United States, Great Britain, Norway and West Germany, 1989-2005. European Journal of Industrial Relations 16, 221-240.

34.

Rosenbaum

P. R.

Rubin

D. B.

(1985). Constructing a Control Group Using Multivariate Matched Sampling Methods that Incorporate the Propensity Score. The American Statistician 39(1), 33-38.

35.

Warr

P. B.

(1999). Well-being and the workplace. In Kahneman

Diener

Schwarz

(eds), Well-being: The foundations of hedonic psychology, 392-412. New York, NY: Russell Sage Foundation.

36.

World Bank (2013). World Development Report: Jobs, Washington, DC: The World Bank.