Have City Deals delivered higher productivity in England? An empirical assessment of a broad-spectrum local growth policy

Abstract

Chinese

The issue of what constitutes effective regional growth policy has remained elusive, particularly for ‘broad-spectrum’ policy aimed at a large part of a country. We undertake one of the first quantitative studies looking at the City Deals in England, analysing effects on productivity. We employ a difference-in-differences model, an event study, and a synthetic control method to evaluate effects on productivity. The results are mixed and usually not statistically different from zero. While the difference-in-differences framework indicates some positive effects, possibly driven by places that were the most productive before the intervention, the event study and synthetic control methods point to, at best, small effects that diminish over time. Our findings, therefore, question the efficacy of such deals in terms of narrowing the UK’s longstanding regional inequalities, while opening up several avenues for further research to understand what worked and what did not within a ‘broad-spectrum’ local growth strategy.

Keywords

economic processes local government infrastructure policy governance regional policy productivity

Introduction

What policies bring about local growth? The question is as contentious as it is important. This is particularly true in the United Kingdom (UK), which is widely regarded as the most centralised OECD nation politically, while having one of the highest levels of regional inequality in the OECD (McCann, 2016). With the UK leaving the European Union (EU), the absence of an EU Cohesion Policy has created an even larger void in regional development policy (Di Cataldo, 2017).

In 2011, the United Kingdom embarked on a new bottom-up approach to drive productivity growth. The newly created City Deals were broad-spectrum policy solutions explicitly aimed at empowering cities through decentralisation and bespoke funding agreements. The aim was to foster (economic) spatial rebalancing across the country and not just in London and the South East (Ward, 2020). For all intents and purposes, the creation of City Deals in the UK was a modern experiment in city-led bottom-up economic development (KPMG, 2014) – but ultimately was more limited in funding, scope, and scale than EU Cohesion Policy.

While there are several case-studies almost ten years since their inception, little work has been done in (quantitatively) assessing the impact of England’s City Deals on growth and productivity. We fill this gap by conducting one of the first comprehensive quantitative studies assessing the impact of City Deals on productivity at a local level. Indeed, the fact that the City Deals were agreed in a wide set of regions and continue to be used as a template for local policy more globally (e.g. Australia) renders our study worthy of attention. We also examine a prominent claim in the previous literature that growth effects of transfers are contingent on the absorptive capacity of regions, such as human capital and quality of governance, as studied in Becker et al. (2013). We shed light on this claim by investigating the diverse effects of the City Deals for places that were in different parts of the productivity distribution before the deals were put in place, as well as for core and peripheral parts of regions.

We assess the link between the 26 City Deals agreed between 2013 and 2014 in England and productivity (Gross Value Added (GVA) per job, and GVA per hour) in local authority districts. First, we employe a difference-in-differences model with time and region fixed-effects, and a rich set of controls. Second, we conduct an event study to check that our difference-in-differences model appropriately captures productivity changes prior to the treatment, while also assessing the statistical significance of City Deals intervention throughout the treatment period. Third, we employ a synthetic control method (SCM) to help circumvent some of the deficiencies present in the difference-in-differences model. The SCM allows us to evaluate the impact of the City Deals treatment while accounting for some unobserved time-varying confounders that may bias the difference-in-differences estimates. The SCM hence represents an alternative way of defining control and treatment groups and its main strengths cover the main weaknesses of our other models.

The results of the analysis are mixed and vary across models. While our panel fixed effects model points to a statistically significant effect of City Deals on local productivity, with the impact disproportionately generated in those local authority districts that were the most productive before the deals came into effect, our event study and SCM are less conclusive on the impact of City Deals on local productivity. Our findings call into question what balancing effects the deals have had in terms of addressing economic inequalities across England’s economic geography, while potentially highlighting the importance of local institutions in better designing and delivering local growth strategies (Agostino et al., 2020; Rodríguez-Pose, 2013), as well as the importance of regions’ absorptive capacity in driving innovation and productivity (Caragliu and Nijkamp, 2012; Miguélez and Moreno, 2015). Our findings broadly echo the conclusions found on EU Cohesion Policy mainly in highlighting the heterogenous effects of place-based policy, but also in highlighting the potential importance of local institutions in delivering positive outcomes from local funding (see Bourdin, 2019; Di Caro and Fratesi, 2022; Fratesi and Wishlade, 2017; Percoco, 2017). Importantly, our analysis of the City Deals strongly indicates that any method employed must be clearly motivated and scrutinised.

We add to the limited literature on broad-spectrum local growth policy by using several statistical techniques to analyse the productivity impact of a large-scale industrial policy. We recognise that the identification problems are legion but construct several different reference categories while attempting to control for a wide set of observable and unobservable factors. In light of the scarce research on broad-spectrum policy and the sheer scope of this particular policy, we believe it deserves to be studied in much more depth than has previously been the case.

The paper is structured as follows. The following section discusses the implementation and contents of the deals, including previous research conducted on the City Deals. The ‘Data sources’ section presents the data and their sources, while the ‘Methods and empirical models’ section motivates the empirical methods we employ. The ‘Results’ section contains the empirical results, and the ‘Conclusions’ section concludes by discussing the wider implications of the study.

The City Deals

Geographic inequalities in England have plagued policymakers for decades (Martin et al., 2017). The UK has one of the highest levels of regional and sub-regional economic inequality among OECD countries (OECD, 2018), accentuated by the decline of the manufacturing sector in the 1970s and recently amplified by the agglomeration of knowledge intensive service industries. Regional policy has been a focal point of local economic development strategy for some time now, aiming to lift productivity more widely and narrow the so-called North–South divide. In response, City Deals were used as a conduit to devolve centralised decision making via the ‘Localism Act 2011’.

Through the City Deals, the UK government went further in its place-based approach to local development by actively focussing on agglomeration economies to boost local productivity (and growth). Each City Deal was organised, led, developed and negotiated directly between local entities and central government, establishing a new way of working between local and central governments in the UK, while allowing local authorities to set out their own priorities to boost growth and productivity (National Audit Office, 2015). Put differently, City Deals were aimed at capitalising on a region’s economic strengths, assets, and expertise to increase local economic growth, whilst simultaneously working to rebalance England’s economic geography.

Between 2012 and 2014, a total of 26 City Deals (in England) were agreed in two waves. While the deals were primarily agreed with cities, their reach extended beyond core cities and into peripheral local authority districts – in many cases covering the full local enterprise partnership (LEP). In designing the City Deals, three key objectives were defined as part of the Government’s White Paper to unlock city growth (HM Government, 2011): first, to grant cities the powers and tools they needed to drive local economic development (i.e. devolution); second, to set out initiatives to boost local and regional incomes (i.e. economic responsibility); and third, to strengthen local/regional governance in a new era of devolved financial and political decision-making (i.e. accountability).

More specifically, the City Deals focussed on six broad areas: infrastructure investment (mainly transport); green investment (boosting green technology, including funding green jobs and infrastructure in the form of onshore/offshore energy); human capital investment (aimed at addressing some of the skills deficit recognised by city regions, in order to better facilitate matching in the labour market and thus raise productivity); employment (improving engagement with the private sector); housing (focussing on increasing the local housing stock or improving planning reform to boost local supply of homes in local authorities); and financial devolution (allowing more autonomy on earned tax revenues as a result of stronger growth). Beyond the economic dimensions of City Deals, there was also a governance aspect surrounding each deal, which involved either political decentralisation or regional integration within or between LEPs. Such decentralisation or political integration aimed to create more accountable governance structures around devolved regions in order to build longer lasting institutions that would enable increased growth and productivity locally.

The literature around the City Deals has been piecemeal with very little quantitative analysis conducted to assess the effects of the programme. The National Audit Office (NAO) provided a preliminary analysis on the initial effects of the City Deals through several case studies focussing on Wave 1 deals. It concluded that City Deals were an important catalyst in building cities’ abilities to manage devolved funding and responsibilities, while skills and training schemes administered through the City Deals were found to have some positive effects on local workers. O’Brien and Pike (2015: 15) undertook an extensive qualitative survey interviewing lead actors across the City Deals. Based on in-depth interviews they found that City Deals essentially rejigged ‘the role of the UK state internally and through changes central–local and intra-local (city–regional) relations’. They found that the City Deals provided an ‘institutional conduit for a relatively open channel of communication for centre–local relations and enabled a controlled form of decentralisation for a highly centralised state’ (O’Brien and Pike, 2015: 19). They also found that the empowerment of local actors encouraged innovation amongst local agents from Greater Manchester’s ‘earn back’ initiative to Greater Cambridge’s ‘gain-share’ model. But while City Deals allowed for local capacity building, O’Brien and Pike questioned the efficacy of the new ‘deal-making’ framework propagated by Government to achieve local growth and maximise productivity, especially given central government funding pressures, which may have triggered some territorial competition across local agents to bid for (limited) money in delivering their identified objectives. Separately, the Centre for Cities in 2015 looked specifically at how city (and growth) deals supported the development of employment and skills policies via a qualitative framework (Clayton and McGough, 2015). They found that City Deals supported, to varying degrees, the ability of local regions to meet the demand need of local areas, enabling local stakeholders to take important steps towards ‘more effective demand-led employment and skills systems’ (Clayton and McGough, 2015: 2). They concluded that City Deals ‘had an overall positive effect on local partnership working’ while allowing local employers more say and incentives to engage with City Deal policies (Clayton and McGough, 2015: 2). More recently, Alonso and Andrews (2023) conducted a staggered difference-in-differences assessment looking at the effects of the City Deals programme on local economic performance. They found that City Deals were associated with improvements in the local economy, but were most effective where suitable institutional structures were in place.

Data sources

Our analysis focusses on England’s first two waves of City Deals as highlighted in Table 1.¹ Wave 1 was agreed in July 2012 and included the eight largest English cities outside of London.² Wave 2 was agreed by July 2014 and included the next 18 largest cities and their wider areas.³‘Treated’ local authorities were identified through the detailed City Deal plans accompanying each deal. We include any local authority district that either contributed to the formation of the City Deal proposal or was included explicitly as part of a City Deal agreement. In total, we identify 131 local authorities that were included in the Government’s City Deals (out of a total 309). In total, we estimate that the City Deals included nearly 44% of the English population alongside 40% of UK GVA.

Table 1.

City Deals agreed (2013–2014).

Wave 1

Greater Birmingham, Bristol City Region, Leeds City Region, Liverpool City Region, Greater Manchester, Newcastle City Region, Nottingham City Region, Sheffield City Region

Wave 2

Black Country, Bournemouth and Poole, Greater Brighton, Coventry and Warwickshire, Greater Cambridge, Greater Norwich, Hull and the Humber, Ipswich, Leicester and Leicestershire, Milton Keynes, Oxford and Oxfordshire, Plymouth, Portsmouth and Southampton, Preston/South Ribble and Lancashire, Southend, Stoke and Staffordshire, Sunderland and the North East, Swindon and Wiltshire, Tees Valley, Thames Valley Berkshire

Source: House of Commons Library, City Deals.

The local authority districts not included as part of the City Deals can be grouped into three categories: first, 33 local authority districts in London; second, 58 local authority districts not included in the 26 City Deals agreed in the first two waves;⁴ and third, 86 local authority districts part of a City Deal region, but not directly benefitting from a City Deal intervention.

While there is no perfect spatial unit to conduct our analysis, we see local authority districts as the best spatial unit to conduct our analysis (as opposed to LEPs or Nomenclature of Territorial Units for Statistics-3 (NUTS-3) regions – given the spatial heterogeneity of the City Deals). Indeed, not all local authorities within a given LEP or NUTS-3 region benefitted directly from a City Deal. Using local authority districts allows us to conduct a more granular assessment of the impact of the City Deals. In defining local productivity, we use two different measures: GVA per job, and GVA per hour. Data on both measures are available from the Office of National Statistics’ (ONS) Regional gross value added (balanced) by industry: local authorities dataset. The data covers all local authorities (2019 classification) in England. Our dataset covers the years 2002–2019.

The rest of our dataset is built using data from a variety of sources. Local labour market data are taken from NOMIS, which provides spatially disaggregated labour market data from 2000 to 2019. Data on local authority economic activity and share of skilled workers (NVQ 4+) are pulled from the UK’s Annual Population Survey. Data on jobs density is taken directly from NOMIS’Local Market Trends. And data on local authority population and geographic area are provided directly by the ONS.

Methods and empirical models

In assessing the impact of England’s City Deals on local productivity we first use a difference-in-differences fixed-effects model. We also build an event study to establish whether our first model meets the parallel trends assumption. This step sheds some light on whether City Deals improved productivity during each individual year in the post-treatment period. And third, we use a synthetic control method (SCM) to further probe the results of the difference-in-differences model, particularly accounting for some of the deficiencies in using a fixed-effects panel model – namely, while allowing us to control for some unobserved time-varying parameters that could impact local productivity (Kreif et al., 2016).

The difference-in-differences model and event study

Our difference-in-differences model is similar to those employed by Harding and Javorcik (2011), Crescenzi et al. (2021) and Athey and Imbens (2018). Our dataset is comprised of local authority districts in England from 2000 to 2019.

The main empirical challenge is the selection of places that would have been successful regardless of the City Deals programme. For instance, City Deals were offered to the largest cities outside of London, which may bias our estimates if growth during the period favoured larger conurbations. While the difference-in-differences framework is unlikely to completely account for all selection bias, we take several steps to mitigate this problem.

We first include fixed effects to account for time-invariant factors such as institutions and other local factors that affect productivity. We also include region-year fixed-effects at the level of the authority’s LEP, as well as individual year effects to capture broader trends associated with specific macroeconomic or regional events. These steps mitigate – but arguably do not solve – the rather complex selection issues above. Another caveat is our inability to account for when other ‘treatments’ occur simultaneously with a City Deal.

A second challenge is the parallel trends assumption. A graphical illustration of the ‘control’ and ‘treatment’ groups suggests that both groups roughly meet this assumption through a simple eye test (Figure 1).

Figure 1.

GVA per job/hour (real, natural log) for treatment versus non-treatment group (England).

We also include explanatory variables highlighted in the literature on regional productivity. The baseline estimated model is:

\begin{matrix} {L o c a l p r o d u c t i v i t y}_{i, t} = β_{0} + β_{1} {C i t y D e a l}_{i, t} \\ + β_{2} X_{i, t} + ϑ_{r, t} + ϑ_{t} + ε_{i, t} \end{matrix}

(1)

where ${L o c a l p r o d u c t i v i t y}_{i, t}$ is the natural log of GVA per job and GVA per hour (our proxies for productivity) in pound sterling (chained volume measure) in local authority district i and time t.⁵

${C i t y D e a l}_{i, t}$ is a dummy variable that takes a value 1 or 0 for each local authority district i at time t. The dummy variable switches on the year a City Deal is agreed (where City Deals are agreed in the second half of a year, we assume they come into effect the following year).

$X_{i, t}$ is a vector of control variables:

Jobs density which is defined as number of jobs per working age population (i.e. those aged 16–64) at the local authority district i at time t. Accounting for jobs density allows us to proxy for labour demand scaled for population.

Population density which is calculated as population per square kilometres in each local authority district (i) over the specified year (t). This helps us capture any sort of agglomeration effects coming from local authorities with large populations.

Spatial lag of GVA per capita– we define the spatial weight matrix using an exponential distance decay function – which allows us to capture any spatial spillovers from neighbouring local authority districts (i) over time (t), or in order words, any spatial autocorrelation in the fixed effects model. Proximity to large conurbations, or economic hubs, may support local productivity, as suggested by the New Economic Geography literature (see Graham, 2001; Office for National Statistics, 2019; Rice and Venables, 2006). Formally, the spatially lagged variable for each local authority district, i, in our dataset is defined as:

w_{i, j} = {\begin{matrix} \frac{e x p (- δ d_{i, j})}{\sum_{j = 1}^{n} e x p (- δ d_{i, j})} i f d_{i, j} < d, i \neq j, δ > 0, \\ 0, o t h e r w i s e \end{matrix}

(2)

where $δ$ is the distance decay parameter, which in our case we use an exponential of 2, _i is the local authority district of observation, and _j is the ‘neighbouring’ region.

Human capital, which we define as the share of population with an education level at, or above, NVQ 4. This is a standard variable used to proxy for education within a local authority.

Gross fixed capital formation per capita– we incorporate experimental regional data on gross fixed capital formation per capita, which gives us a proxy for capital within a local authority.⁶

Industry mix, where we include the share of GVA that comes from manufacturing activity, services activity, and public services at the local authority district level (i) over time t. This allows us to capture the effects of any sectoral shift in local authority districts that may be contributing to local productivity changes. The inclusion of the share of public services GVA is based on the findings from the UK Department for Environment, Food, & Rural Affairs (2011), which concluded that a region’s public services share of GDP was an important driver (negative) of productivity.

$ϑ_{r, t} + ϑ_{t}$ are region-year and year fixed effects, respectively, as described above.

$ε_{i, t}$ is the error term at local authority district, i, at time, t. Standard errors are clustered at the local authority district level.

Table 2 includes descriptive statistics for the variables we use in our econometric analyses. The descriptive statistics are broken down by ‘treatment’ group (i.e. those local authorities included in City Deals), and those outside the ‘treatment’ group (i.e. our ‘control’ group – excluding London) for the entirety of the sample. The most notable difference can be seen via all three measures of productivity, where the ‘control’ group average is nearly 10% higher. A big reason for this is the London productivity premium. When we exclude London from our ‘control’ group, the productivity (GVA per job) gap is only 3.4% across the entire sample period (2002–2019). Aside from GVA per job, the other notable differences between our ‘treatment’ and ‘control’ (ex-London) groups include population density and share of public services activity. On the former, given the basis of the City Deals (i.e. agglomeration-led policy), the ‘treatment’ group has a much higher population density. On the latter, the share of public services activity is also nearly 4% higher. All other variables are broadly comparable.

Table 2.

Descriptive statistics (2002–2019 avg).

Variable	Full sample			Local authorities w/City Deals (entire sample)			Local authorities w/o City Deals (entire sample)			Local authorities w/o City Deals (excl London) (entire sample)
Variable	Observations	Mean	Standard deviation	Observations	Mean	Standard deviation	Observations	Mean	Standard deviation	Observations	Mean	Standard deviation
GVA per job (£)	5562	37,909	10,718	2376	35,847	10,165	3186	39,447	10,862	2592	37,079	8801
GVA per job (ln)	5562	10.51	0.24	2376	10.46	0.22	3186	10.55	0.25	2592	10.50	0.22
GVA per hour (£)	4944	33.64	7.9	2112	31.48	5.69	2832	35.24	8.91	2304	33.43	7.41
GVA per hour (ln)	4944	3.49	0.21	2112	3.43	0.17	2832	3.54	0.23	2304	3.49	0.20
Jobs density (jobs per 16–64 pop)	5562	1.07	5.02	2376	0.79	0.16	3186	1.28	6.63	2592	0.79	0.15
Population density (per sq km)	5562	1693	2408	2376	1421	1355	3186	1895	2942	2592	761	889
Gross fixed capital formation per capita (£)	5562	5470	34,602	2376	3506	2063	3186	6935	45,632	2592	3311	1383
Share of population w/ NVQ 4+	5534	32%	11%	2357	30%	9%	3177	33%	12%	2585	31%	9%
Services share, GVA %	5562	55%	11%	2376	53%	9%	3186	57%	12%	2592	55%	10%
Public sector share, GVA %	5562	20%	7%	2376	22%	7%	3186	19%	6%	2592	18%	6%
Construction share, GVA %	5562	7%	3%	2376	7%	3%	3186	8%	4%	2592	8%	4%
Manufacturing share, GVA %	5562	13%	9%	2376	14%	7%	3186	13%	9%	2592	15%	9%

In a further step, we include the treatment area interacted with initial productivity level (quartile at treatment) in each local authority district. This informs our analysis of whether City Deals improved regional inequality, that is, lifting local productivity evenly across the ‘treated’ local areas. Through this practice, we provide more nuance to the previous discussion on whether growth effects of regional industrial policy are contingent on a region’s prerequisites (cf. Becker et al., 2013).

We also assess the potential for contamination in how we have constructed our ‘City Deals’ binary variable by separating the ‘City Deals’ variable into two parts: a core (which features the main city local authority district) and a periphery (local authority districts that are included within the City Deal proposals but are not the main City Deal local authority district benefitting from the policy intervention). By disaggregating the City Deal binary variable in our core model, we can assess whether any productivity benefits were mainly linked to core city regions, with limited spillovers to the periphery.

Synthetic control method

The synthetic control method (SCM) allows us to construct counterfactual local authority districts that mirror ‘treated’ areas in the pre-treatment period (see Abadie and Gardeazabal, 2003; Abadie et al., 2010; Nathan, 2022). If local authority districts included in the City Deals experienced higher productivity this is accounted for through a ‘synthetic control’ unit. The main advantage in using the synthetic control method is that it allows the effects of observed and unobserved predictors to vary over time – thereby potentially presenting a more robust assessment of the impacts of City Deals on local productivity, compared to the difference-in-differences approach.

To create a synthetic control for each of our treated units, we use a set of explanatory variables measured in the pre-intervention period to construct artificial local authorities that follow similar trends in the pre-treatment period for those local authorities participating in a City Deal. Any variance in the dependent variable, GVA per job, is forcibly minimised in the pre-treatment period, allowing the parallel trend assumption to be relaxed, presenting an additional advantage to the difference-in-differences framework. In effect, the SCM utilises some elements from matching and difference-in-differences methods to create a more dynamic and relevant set of control groups when assessing treatment effects. Our synthetic controls are constructed from a set of variables that are conventionally associated with supporting local productivity. These include job density, population density, share of population with NVQ 4+, labour force participation, industry mix (where we include each local authority’s GVA share of manufacturing, knowledge intensive business services, distribution, transport and public services), share of foreign-born population, share of job training (government supported training and employment programmes), gross capital fixed formation per capita, and share of self-employed – building on the fully specified fixed effects model used in the difference-in-differences model. A full summary of pre-treatment controls is listed in Tables A.1 to A.3 in the Appendix, including a list of summary statistics for the set of treated units and donor pool. The donor pool for our synthetic control estimation includes all English local authority districts not participating in the City Deals programme. For the purposes of this exercise, we also exclude all local authorities in London given the ‘London productivity premium’ that could bias the synthetic control (see Harris and Moffat, 2012; Office of National Statistics, 2021).

Unlike traditional SCM models, our analysis includes more than one treatment region (and therefore a number of synthetic controls). Accordingly, we rely on the empirical strategy set out by Wiltshire (2022) and Dube and Zipperer (2015) using the STATA program allsynth to deal with estimating the average treated effect across multiple treated units using a ‘stacked’ approach.

Turning to the weights structure, we rely on an OLS methodology, where the weights are constructed by minimising the differences between each treated unit and its respective donor pool local authority districts (as set by the default settings in the allsynth package). The pre-treatment trajectory therefore can be modelled by a convex combination of donor pool units. More information on how the SCM is constructed, including weights of the core local authority districts in the 26 City Deals agreed in England, are provided below and in the Appendix.

In terms of statistical inference, we rely on the methodological approach set out by Abadie et al. (2014). The ‘in space’ placebos test calculates the ranked RMPSE p-values for all treated local authority districts. To do this, we reassign the treatment from each treated local authority district, i, to each of the treated unit’s untreated donor pool. We sample 1000 placebos to assemble an adequate statistical distribution of the treatment effects.

Results

Baseline results

For each proxy of productivity, we estimate two models. The first is a ‘naive’ fixed-effects model (including region-year, local authority and year fixed effects) and the second is our fully specified fixed-effects panel model with explanatory variables. We present the estimates from our difference-in-differences model (1) in Table 2.

In our naive model, the City Deals variable is only statistically significant at the 10% level (for both GVA per job and GVA per hour). In our fully specified model, the City Deals coefficients are statistically significant across both our proxies for productivity, with the City Deals variable showing a statistically significant effect at the 5% level for both GVA per job and GVA per hour. The estimated treatment effect is broadly similar across both measures, with the productivity uplift associated with the City Deals intervention estimated to be 3.3% for GVA per job, and 3.4% for GVA per hour – broadly consistent with the findings from Alonso and Andrews (2023).

Models 5 and 7 build on our fully specified model, but interact the City Deal variable with each local authority district’s quartile of initial level of productivity. Judging by the point estimates alone, the effects of the City Deals programme appear to be concentrated in the top quartile of local authority districts, with the City Deals intervention lifting GVA per job by a more substantial 5.5% and by 4.1% for GVA per hour. The point estimates are reduced sequentially across the bottom three quartiles. The results are not statistically significant for the bottom three quartiles. While this does not imply that the coefficients are statistically different from each other, these results are consistent with the idea that places respond differently to treatment by place-based policies. Treated regions entering the City Deals at the top quartile of the productivity spectrum showed a statistically significant and meaningful shift in productivity, while the remainder of the local authority districts showed a modest but not statistically significant impact. Much of the rise in productivity may have been largely concentrated among the most productive local authorities entering the City Deals programme with better institutional and social capital to bid for funding and importantly, to deliver any such growth programmes.

The distributional divergence remains when we segment the local authority districts by core and periphery regions. Models 6 and 8 show that core city regions/local authority districts saw increases of 6% for GVA per job and 5% for GVA per hour. The corresponding numbers for peripheral regions were 1.9% for GVA per job and 2.5% for GVA per hour. The result for core city regions is statistically significant at the 1% level, while the coefficients representing the periphery are not.

The results of our robustness checks lend further support to the argument that the City Deals have benefitted the places that were already productive, similar to the findings for Objective 1 transfers in Becker et al. (2013)– see Table 3.

Table 3.

Regression results.

	GVA per job		GVA per hour		GVA per job		GVA per hour
	Fixed-effects	Fixed-effects	Fixed-effects	Fixed-effects	Fixed-effects	Fixed-effects	Fixed-effects	Fixed-effects
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
City Deal	0.0309* (0.0171)	0.0331** (0.0158)	0.0304* (0.0169)	0.0337** (0.0152)
City Deal × Top quartile					0.0552**(0.0227)		0.0413**(0.0196)
City Deal × Second quartile					0.0381*(0.0213)		0.0304(0.0195)
City Deal × Third quartile					0.0252(0.0207)		0.0324*(0.0192)
City Deal × Fourth quartile					0.0174(0.0193)		0.0287(0.0184)
City Deal core						0.0596***(0.0208)		0.0504***(0.0190)
City Deal periphery						0.0190(0.0179)		0.0254(0.0160)
Population density		−0.00762(0.0247)		−0.0104(0.0204)	−0.00777(0.0248)	−1.19e-05(2.43e-05)	−0.0103(0.0205)	−0.0130(0.0203)
Job density		−0.00626***(0.000613)		−0.00563***(0.000592)	−0.00629***(0.000624)	−0.00637***(0.000603)	−0.00563***(0.000595)	−0.00570***(0.000589)
(log) Spatial GVA (per capita)		0.0348(0.0471)		0.0359(0.0465)	0.0347(0.0470)	0.0465(0.0461)	0.0352(0.0464)	0.0428(0.0462)
(log) GFCF per capita		0.0464***(0.0110)		0.0434***(0.0105)	0.0455***(0.0110)	0.0464***(0.0109)	0.0429***(0.0104)	0.0433***(0.0104)
Human capital (share of NVQ 4+)		−0.0354(0.0240)		−0.0203(0.0249)	−0.0405*(0.0244)	−0.0366(0.0240)	−0.0217(0.0252)	−0.0209(0.0252)
Services share (GVA)		−0.496**(0.199)		−0.494**(0.192)	−0.518***(0.198)	−0.517***(0.197)	−0.493**(0.191)	−0.489**(0.191)
Manufacturing share (GVA)		−0.247(0.215)		−0.137(0.213)	−0.260(0.213)	−0.257(0.214)	−0.134(0.212)	−0.119(0.212)
Public share (GVA)		−1.092***(0.276)		−1.563***(0.246)	−1.095***(0.275)	−1.105***(0.276)	−1.550***(0.248)	−1.547***(0.243)
Constant	10.49***(0.00415)	10.34***(0.498)	3.487***(0.00427)	3.440***(0.504)	10.37***(0.497)	10.25***(0.488)	3.449***(0.502)	3.366***(0.501)
Observations	5562	5534	4944	4928	5534	5534	4928	4928
R ²	0.288	0.347	0.270	0.400	0.358	0.353	308	308
Year FE	Yes	Yes	Yes	Yes	Yes	Yes	0.404	0.402
Region/Year FE	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Number of local authorities	309	308	309	308	308	308	Yes	Yes

Note: Robust standard errors in parentheses.

p < 0.1. **p < 0.05. ***p < 0.01.

Event study

We re-estimate our baseline models (Models 2 and 4) with a full set of lead and lag dummy variables referring to each year in the pre-treatment period and each year during the treatment period. We also control for local year and region-year effects. The model allows us to capture any temporal dynamics of the treatment effect (i.e. City Deals). The model specification is as follows:

\begin{matrix} G V A p e r j o b / {h o u r}_{i, t} = \sum_{τ = 2}^{q} δ_{- τ} D_{i, t - τ} \\ + \sum_{τ = 1}^{q} δ_{+ τ} D_{i, t + τ} + β_{1} X_{i, t} + ϑ_{r, t} + ϑ_{t} + ε_{i, t} \end{matrix}

(3)

where q lags dummy variables ( $D_{r, c, s, t - 2}$ , $D_{r, c, s, t - 3}$ , …) and leads dummy variables ( $D_{r, c, s, t + 1}$ , $D_{r, c, s, t + 2}$ , …, $D_{r, c, s, t + q})$ check for parallel trends and anticipatory effects of City Deals. Performing an event study is important as it gives us added confidence in our staggered difference-in-differences model, allowing us to determine if there is a statistically significant difference in productivity at any point during the treatment period, while confirming no statistically significant difference in productivity during the non-treatment period (i.e. our regression model appropriately captures any variation in local/regional productivity trends before and during the treatment period) (Angrist and Pischke, 2008).

To perform the event study, we include a full set of dummies for the pre-treatment (up to nine years) and treatment period (up to six years), with the exclusion of the first-year lag, which is used as the reference year. We do this for both GVA per job and GVA per hour.

The results of the event studies are shown in Figure 2 (full sample), displaying the coefficients of the leads and lags dummy variables as well as their corresponding confidence intervals (95%) for the years preceding the start of the City Deals programme and the years of the intervention period. Our event studies show no statistically significant pre-treatment coefficient. This tells us that our choice of fixed effects (year effects, local authority effects and LEP-year effects) alongside our control variables (as included in our fully specified fixed-effects panel regression model) performs well in explaining the variation in local productivity through the pre-treatment period, supporting the parallel trend assumption in the staggered difference-in-differences model. Put another way, our preferred panel model specification works well in capturing observed and unobserved variables that may have had an impact on local productivity heading into the City Deals intervention period. However, when the treatment begins (i.e. the start of the City Deals), the coefficients obtained in the event studies show mixed results. Under the GVA per job specification, while the coefficients are positive throughout the intervention period, our event study shows no statistically significant lift in productivity across the six years of the treatment period, casting some doubt on the validity of the results obtained via the difference-in-differences model that the City Deals did in fact raise productivity. When using the GVA per hour model, we find a similar result: while the specified model does well in capturing the changes in productivity over the pre-treatment period, only one year is statistically significant when using a 95% confidence band.

Figure 2.

Event study: GVA per job and GVA per hour: (a) GVA per job and (b) GVA per hour.

Overall, the findings from the event study contradict the results of our difference-in-differences model. It is worth noting here, however, that the event study imposes a more conservative constraint in assessing the statistical significance of the treatment effect by looking at each individual year as opposed to the ‘pooled’ effect derived via the difference-in-differences framework.

We now turn to a synthetic control method to shed further insight on the effectiveness of the City Deals on local productivity.

Synthetic control method results

Turning to the SCM approach, our findings show that the impact of City Deals is indeed positive when looking at the full sample – though the estimated impact is small at 0.9% and the treatment gap closes by the end of the sample period. We also find that the aggregate SCM approach masks diverging trends in productivity performance among local authorities participating in City Deal programmes. The difference in productivity outcomes becomes starker when looking at the top and bottom quartiles of the productivity spectrum. The top 25% of local authorities by productivity going into the City Deal programme saw stronger gains relative to their synthetic controls at nearly 2.6%. Conversely, those treated local authorities at the bottom end of the productivity spectrum (as at the start of the city deal intervention) saw small effects as a result of the City Deals at 0.4% relative to the synthetic controls. Unlike our difference-in-differences model, when isolating for core cities (which we define as the largest economy included in a City Deal, thereby excluding peripheral local authorities within City Deals), the policy impact is slightly weaker (0.5%) – see Figure 3.

Figure 3.

Synthetic control method plots.

How significant are the SCM results? Using the ‘in-placebo’ tests proposed by Cavallo et al. (2013) and Abadie and L’Hour (2021), and using the framework provided by Wiltshire (2021), we calculate p-values across the local authority districts where City Deals were implemented. To do this, we calculate ranked RMSPE p-values assuming the treatment (i.e. City Deals) is applied across all untreated/donor units. A full detailed description of how the ‘in placebo’p-values are constructed is detailed by Wiltshire (2022). In all iterations, we find that our SCM results are not statistically significant across each year of the policy intervention, highlighting that the effects of City Deals, while positive, are not statistically different from zero – a finding broadly consistent with our event study results as opposed to our difference-in-differences results (Table 4).

Table 4.

SCM results.

	Treatment years (SCM results)
	0	1	2	3	4	5	Average
All treated	0.3% (0.61)	0.6% (0.49)	1.0% (0.28)	1.3% (0.17)	1.1% (0.17)	1.2% (0.18)	0.9% (0.32)
Core cities	0.0% (0.99)	0.9% (0.74)	1.1% (0.72)	1.3% (0.73)	−0.2% (0.81)	−0.3% (0.86)	0.5% (0.81)
Top quartile	2.0% (0.32)	2.6% (0.32)	3.2% (0.31)	3.5% (0.30)	2.1% (0.37)	1.8% (0.46)	2.6% (0.35)
2nd quartile	1.3% (0.26)	1.7% (0.27)	1.5% (0.32)	1.2% (0.38)	0.8% (0.48)	0.4% (0.55)	1.1% (0.38)
3rd quartile	−1.1% (0.01)	−1.2% (0.03)	−0.5% (0.09)	−0.1% (0.16)	0.2% (0.25)	0.7% (0.30)	−0.3% (0.14)
Bottom quartile	−1.0% (0.28)	−0.7% (0.41)	0.1% (0.54)	0.8% (0.58)	1.5% (0.54)	1.9% (0.49)	0.4% (0.47)

Note: Treatment effect represents gap between treated unit and synthetic control; p-values in parentheses.

Overall, our SCM approach casts some doubt on the findings from the difference-in-difference models that City Deals did in fact raise local productivity. While the treatment effect is positive, the ‘in-placebo’ significance tests highlight that the results are not statistically significant and hence there is no conclusive evidence that the deals were successful in boosting productivity growth.

Conclusions

Over the last decade, City Deals were implemented to help drive local productivity, with the aim of narrowing the UK’s widening North–South productivity gap. We conducted one of the first comprehensive studies analysing the effects of the City Deals programme on productivity.

Our most important message is that the picture is mixed and that different methods and specifications will lead academics and policymakers to different conclusions. Generally, the estimated effects of the policy are positive, but often, even in most cases, not statistically different from zero.

While our preferred difference-in-differences model highlighted a statistically significant increase in productivity (around 3%), with more productive local authority districts showing stronger effects (i.e. those local authority districts at the top end of the productivity spectrum saw an increase of 4–5.5% in productivity), both our event study and synthetic control method yielded inconclusive results, that are generally not statistically distinguishable from zero. This might not be a matter of waiting for more data as our event study and SCM model illustrate something of an attenuating effect.

Which model should we rely on? Our panel difference-in-differences model seems well specified, as evidenced by the pre-intervention period of the event study, but there are many caveats in terms of, for instance, potential endogeneity and, more importantly, selection of unobservables into ‘treatment’ and ‘control groups’. The synthetic control method (SCM) arguably circumvents some of these issues, but is not without its own faults, particularly when it comes to the calculation of the synthetic control donor and predictor weights (see Kuosmanen et al., 2021). Further, the event study’s post-treatment results are not significant in any particular year (looking at GVA per job). In our SCM models, the results are likewise not statistically distinguishable from zero.

What do our analyses suggest? In a word, caution. First, the use of a simple staggered difference-in-differences approach alone may not be fully suitable in assessing the impact of place-based policy measures. Given the potential for endogeneity, or indeed, the selection into City Deals, the effects in and of itself may reflect unobserved factors that could bias our results.

Second, the mixed results suggest that the effects of the City Deals programme either may have not been fully borne out in the post-intervention period, or indeed, are very much heterogenous, masking the efficacy of such deals. On the former, given the various dimensions of the City Deals treatment, the effects may have yet to filter through into productivity dynamics, be it through the decentralisation/devolution that City Deals enabled, or indeed through the various initiatives agreed through the City Deals aimed at supporting infrastructure or improving access to business finance and support – all of which could take a lot longer to manifest in terms of local productivity growth. On the latter, our results may also simply indicate that not all City Deals were alike, with some deals seeing larger pots of funding for investment and/or more decentralisation/devolution to boost local economic governance.

Ultimately, however, in terms of regional or local policy, the success of City Deals in driving local productivity and growth seems to be called into question. Questions around institutional capacity, including governance and expertise in bidding for funding, and ultimately delivering on growth programmes remain rife with such deal-making arrangements showing very few tangible benefits across a wider geography. Indeed, some city regions may have lacked the appropriate local/regional economic and governance infrastructure to deliver on such a policy programme, which required substantial amounts of effort to bid for funding, and large amounts of resources to ultimately deliver on agreed policy objectives. While our analyses may highlight mixed results with regards to the efficacy of the City Deals intervention, we believe more focus will need to be given to role of ‘path dependence’ in the broader context of institutional change (North, 2003), where ineffective (or inefficient) institutions may impede policy reform and productivity growth (Fernandez and Rodrik, 1991). Policies that may be appropriate for ‘far-from-frontier’ local regions may differ vastly from policies that be more suitable for ‘close-to-frontier’ local economies (Aghion and Howitt, 2006; Gerschenkron, 1962). This seems to be consistent with broader findings when it comes to assessing the effects of ‘place-based’ policy where broad based policies tend to have heterogenous effects, and are very likely linked to the presence of a number of regional and local contextual factors, including the level of initial productivity, and potentially the quality of regional/local institutions, including local economic governance – be it economic, political and/or social (Becker et al., 2013; Bourdin, 2019; Di Caro and Fratesi, 2022).

More work will need to be undertaken to fully assess the heterogeneity within the City Deals to assess more completely the merits of city-led economic development. This includes understanding how political devolution may have spurred productive capacity in local authorities, particularly through accountability and increased agency in a bottom-up development framework. An inquiry into prerequisites for when devolution might achieve the desired ends seems particularly warranted. We also need to have a better understanding of the role local institutions and social capital play in fostering and accelerating productivity as a means to narrow the North–South divide when thinking about future policy.

Going forward, with the UK Government setting in motion an agenda for ‘levelling-up’ the economy, including the use of ‘deals’ to foster local/regional economic development, we deem it important for future research to investigate the institutional setting and absorptive capacity of individual places, including which parts of the City Deals worked better than others in supporting local productivity. Finally, in conjunction with the findings from the NAO, the lack of ‘legally binding’ local targets and accountability of governance will need to be remedied going forward. Locally agreed targets between central and local governments will need to be defined and established upfront in order to build a successful framework for bottom-up development, including a transparent progress map that allows each ‘deal’ to be publicly scrutinised and assessed more thoroughly. This should help create a more ‘joined up’ and coordinated approach to tackling the UK’s longstanding regional economic inequality (Martin et al., 2021).

Footnotes

Appendix

Acknowledgements

The authors would like to thank Diane Coyle, Martin Andersson, Ozge Oner, Marco Di Cataldo, Geoffrey J.D. Hewings and Ben Gardiner for all their comments and feedback on the paper. Larsson gratefully acknowledges funding from the Kamprad Family Foundation for Entrepreneurship, Research & Charity (P20220048). All errors and omissions are our own.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Sanjay I Raja

Johan P Larsson

Notes

References

Abadie

(2021) Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature 59(2): 391–425.

Abadie

Diamond

Hainmueller

(2014) Comparative Politics and the Synthetic Control Method. American Journal of Political Science 59(2): 495-510

Abadie

Diamond

Hainmueller

(2010) Synthetic control methods for comparative case studies: Estimating the effect of California’s Tobacco Control Program. Journal of the American Statistical Association 105(490): 493–505.

Abadie

Gardeazabal

(2003) The economic costs of conflict: A case study of the Basque country. American Economic Review 93(1): 113–132.

Abadie

L’Hour

(2021) A penalized synthetic control estimator for disaggregated data. Journal of the American Statistical Association 116(536): 1817–1834.

Aghion

Howitt

(2006) Appropriate growth theory: A unifying framework. Journal of the European Economic Association 4(2–3): 269–314.

Agostino

Di Tommaso

Nifo

, et al. (2020) Institutional quality and firms’ productivity in European regions. Regional Studies 54(9): 1275–1288.

Alonso

Andrews

(2023) Can city deals improve economic performance? Evidence from England. Urban Affairs Review 60(3): 835–863.

Angrist

Pischke

J.-S

(2008) Mostly Harmless Econometrics: An Empiricist’s Companion. New Jersey: Princeton University Press.

10.

Athey

Imbens

(2017) The state of applied econometrics: Causality and policy evaluation. Journal of Economic Perspectives 31(2): 3–32.

11.

Athey

Imbens

(2018) Design-based analysis in difference-in-differences settings. Working Paper 24963, National Bureau of Economic Research, Cambridge, MA.

12.

Becker

Egger

von Ehrlich

(2013) Absorptive capacity and the growth and investment effects of regional transfers: A regression discontinuity design with heterogeneous treatment effects. American Economic Journal: Economic Policy 5(4): 29–77.

13.

Bourdin

(2019) Does the cohesion policy have the same influence on growth everywhere? A geographically weighted regression approach in Central and Eastern Europe. Economic Geography 95(3): 256–287.

14.

Caragliu

Nijkamp

(2012) The impact of regional absorptive capacity on spatial knowledge spillovers: The Cohen and Levinthal model revisited. Applied Economics 44(11): 1363–1374.

15.

Cavallo

Galiani

Noy

, et al. (2013) Catastrophic natural disasters and economic growth. Review of Economics and Statistics 95(5): 1549–1561.

16.

Clayton

McGough

(2015) City Deals and Skills: How Have City and Local Growth Deals Supported the Development of Employment and Skills Policies that Reflect Local Demand?London: Centre for Cities.

17.

Crescenzi

Di Cataldo

Giua

(2021) FDI inflows in Europe: Does investment promotion work? Journal of International Economics 132: 103497.

18.

Department for Environment, Food, & Rural Affairs (2011) Drivers of Rural Productivity. London: UK Government.

19.

Di Caro

Fratesi

(2022) One policy, different effects: Estimating the region-specific impacts of EU cohesion policy. Journal of Regional Science 62(1): 307–330.

20.

Di Cataldo

(2017) The impact of EU Objective 1 funds on regional development: Evidence from the U.K. and the prospect of Brexit. Journal of Regional Science 57(5): 814–839.

21.

Dube

Zipperer

(2015) Pooling Multiple Case Studies Using Synthetic Controls: An Application to Minimum Wage Policies. Bonn: IZA Institute of Labor Economics.

22.

Fernandez

Rodrik

(1991) Resistance to reform: Status-quo bias in the presence of individual-specific uncertainty. American Economic Review 81: 1146–1155.

23.

Fratesi

Wishlade

(2017) The impact of European Cohesion Policy in different contexts. Regional Studies 51(6): 817–821.

24.

Gerschenkron

(1962) Economic Backwardness in Historical Perspective. Cambridge, MA: Belknap Press.

25.

Graham

(2001) Productivity growth in British manufacturing: Spatial variation in the role of scale economies, technological growth and industrial structure. Applied Economics 33(6): 811–821.

26.

Harding

Javorcik

(2011) Roll out the red carpet and they will come: Investment promotion and FDI inflows. The Economic Journal 121(557): 1445–1476.

27.

Harris

Moffat

(2012) Is productivity higher in British Cities? Journal of Regional Science 52(5): 762–786.

28.

HM Government (2011) Unlocking Growth in Cities. Available at: https://www.gov.uk/government/publications/unlocking-growth-in-cities–5 (accessed 9 January 2022).

29.

KPMG (2014) Introducing City Deals: A Smart Approach to Supercharging Economic Growth and Productivity. Sydney, Australia: KPMG.

30.

Kreif

Grieve

Hangartner

, et al. (2016) Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Economics 25(12): 1514–1528.

31.

Kuosmanen

Zhoua

Eskelinen

, et al. (2021) Synthetic control methods: The good, the bad, and the ugly. EEA Conference Working Paper, s.l.

32.

McCann

(2016) The UK Regional–National Economic Problem: Geography, Globalisation and Governance, 1st edn. London: Taylor & Francis.

33.

Martin

Gardiner

Pike

, et al. (2021) Levelling up Left Behind Places: The Scale and Nature of the Economic and Policy Challenge. 1st ed. London: Routledge.

34.

Martin

Tyler

Sunley

, et al. (2017) Cities and the National Productivity Problem. London: ESRC.

35.

Miguélez

Moreno

(2015) Knowledge flows and the absorptive capacity of regions. Research Policy 44(4): 833–848.

36.

Nathan

(2022) Does light touch cluster policy work? Evaluating the tech city programme. Regional Policy 51(9): 104138.

37.

National Audit Office (NAO) (2015) Devolving Responsibilities to Cities in England: Wave 1 City Deals. London: NAO.

38.

North

(2003) Understanding the Process of Economic Change. Princeton, NJ: Princeton University Press.

39.

O’Brien

Pike

(2015) City deals, decentralisation and the governance of local infrastructure funding and financing in the UK. National Institute Economic Review 233(1): R14–R26.

40.

OECD (2018) Regions and Cities at a Glance 2018 – United Kingdom. Paris: OECD.

41.

Office for National Statistics (2019) Understanding Spatial Labour Productivity in the UK. London: ONS.

42.

Percoco

(2017) Impact of European Cohesion Policy on regional growth: Does local economic structure matter? Regional Studies 51(6): 833–843.

43.

Rice

Venables

(2006) Spatial determinants of productivity: Analysis for the regions of Great Britain. Regional Science and Urban Economics 36(6): 727–752.

44.

Rodríguez-Pose

(2013) Do institutions matter for regional development? Regional Studies 47(7): 1034–1047.

45.

Ward

(2020) City Deals. London: House of Commons.

46.

Wiltshire

(2021) Walmart supercenters and monopsony power: How a large, low-wage employer impacts local labor markets. Working Paper. University of California, Davis.

47.

Wiltshire

(2022) allsynth: (Stacked) synthetic control bias-correction utilities for Stata. Working Paper. Unpublished manuscript, Department of Economics, University of California, Davis.