Abstract
Although there have been many initiatives designed to regenerate relatively run-down and deprived parts of major urban areas, there have been surprisingly few attempts to value their benefits. This article presents the findings of research that has sought to value the benefits of urban regeneration policies. The focus has been on devising an approach that can build on the evidence provided from urban evaluations undertaken in many countries at the present time. It uses established techniques and statistical data sources that are fairly readily available. The evaluation of urban policy is subject to substantial conceptual and measurement problems and this should be recognised in interpreting valuation results and thus benefit–cost ratios. The article shows how the approach can be applied by drawing on recent UK evaluation evidence and data for England. It concludes by discussing where future research might be directed.
1. Introduction
In recent years, many countries have sought to regenerate the relatively depressed parts of their urban areas through a wide range of policy initiatives. Regeneration intervention has typically involved a series of discretionary funding programmes, operating in parallel to, although often seeking to influence, the activities of ‘mainstream’ public service delivery. The type and scale of the intervention have varied significantly.
The experience in England is illustrative with a rich array of initiatives particularly in the inner cities where the consequences of rapid and prolonged economic restructuring have been felt particularly severely. Table 1 shows where mainly urban regeneration expenditure was focused in England by the end of the past decade. Almost one-fifth was assigned to worklessness, skills and development activities, around 11 per cent went to improving industrial, commercial and infrastructure and the remainder to activities associated with homes, communities and the environment with the largest share assigned to housing growth and improvement.
Estimate of core regeneration expenditure by activity, estimated average annual expenditure in England (average spend over 2009/10 and 2010/11).
Sources: Authors’ analysis of expenditure data for 2009/10 and 2010/11 provided by Department for Communities and Local Government; RDA Finance and Governance data published on Department for Business, Innovation and Skills website (February 2010) and Homes and Communities Agency Corporate Plan (2009/10 to 2010/11).
There has been much debate about what urban regeneration initiatives have been able to achieve. Views have varied significantly and some have questioned the effectiveness of the measures adopted and, in the most extreme cases, it has even been argued that they do not provide a positive rate of return to society and it might be better not to intervene at all. It is perhaps to state the obvious that the regeneration initiatives deployed by governments should be cost-effective and represent good value for money and for this reason there has been widespread interest in evaluating urban policy and addressing the conceptual and measurement problems that constrain evaluation (Nolan and Wong, 2004; Baslé, 2006; Bartik, 2004). This article focuses on one such problem which appears to have received relatively little attention; namely, how the benefits of urban policy initiatives that produce quite a diverse range of outputs like jobs, training places and environmental improvements can be valued and then summed so that an overall benefit–cost ratio can be derived that enables policy-makers to assess the overall worth of public expenditure on urban policy. If these summary measures were in place, the overall social return from urban regeneration could be compared with that from other government policies. It would also be possible for urban policy-makers to compare and contrast the value of different policy options to society when devising intervention strategies. The lack of progress in this area has been noted (Potts, 2008) and is perhaps the more surprising given that there has been so much attention given to valuing the benefits of such expenditure in developing countries (Little and Mirless, 1974). Given the inherent conceptual and measurement problems that exist in evaluating urban policy, such estimates will always be subject to large margins of error and can only be considered as broad orders of magnitude.
It is an obvious question to ask why there is so little evidence available on the aggregate value of regeneration benefits in the light of the resources that many countries commit to them. A number of factors have constrained research. One is the sheer diversity of regeneration activity across the physical, economic and social, with an increasing emphasis on a broad social agenda that works to improve health, reduce crime and enhance social capital. Some of these impacts are more easily valued than others. Also, the impact of regeneration initiatives take time to build up and initiatives vary considerably in how long they last. The focus of valuation research should be on the real resource costs and benefits to society and it is important to avoid double-counting of benefits.
In some cases, the benefits of regeneration are traded in markets with an example being the additional jobs that an urban regeneration scheme might generate. Unless these markets are subject to distortion, there is thus a value available. In other cases—such as enhanced environmental amenity—no direct market valuation exists and techniques such as stated or revealed preference have to be adopted (Bateman et al., 2002; Bateman et al., 2006; Louviere et al., 2000; Hensher et al., 2005).
The research presented in this article has been concerned to do two things. First, it has developed an approach to valuing and aggregating the benefits of urban policy that can be readily incorporated into the basic conceptual approach that has been developed and applied in many countries to evaluate the achievements of their urban policy. It has sought to place a value on as many as possible of the relatively diverse range of outputs that are produced by urban policies, building on existing evaluation evidence, established techniques and market-based data that are commonly available in countries that have extensive urban policies. Secondly, it then demonstrates how the approach can be applied in England by drawing on evaluation evidence and data for the UK over the period 2000–10.
2. Evaluating and Valuing the Achievements of Urban Policy
The key starting point in many evaluations of urban policy in recent years is to begin by establishing a theory of change which identifies how policy objectives are expected to be achieved (Rhodes et al., 2005, 2009). This underpins the development of a logical framework, summarised in the form of a logic chain, which shows how the activities initiated by the urban policy will produce outputs that will affect core outcomes like worklessness and enhanced environmental amenity. The framework should also identify the contextual factors, external to the programme, which influence its success. Pawson and Tilley comment that Programmes are thus shaped by a vision of change and they succeed or fail according to the veracity of that vision. Evaluation, by these lights, has the task of testing out the underlying programme theories. When one evaluates realistically one always returns to the core theories about how a programme is supposed to work and then interrogates it—is that basic plan sound, plausible, durable, practical and, above all, valid? (Pawson and Tilley, 2004, p. 2).
Theory-based impact evaluation is also a central part of the approach being adopted by the European Commission to the evaluation of the European Cohesion Policy in the new programming round from 2014. Thus clearly theory based evaluations can provide a precious and rare commodity, insights into why things work, or don’t. The main focus is not a counterfactual (“how things would have been without”) rather a theory of change (“how things should logically work to produce the desired change”). The centrality of the theory of change justifies calling this approach theory-based impact evaluation (European Commission, 2011, pp. 4–5).
It should also be recognised that the logic chain described earlier should consider both the direct effects and indirect effects of urban policy. The pathways and extent to which these indirect effects arise are often not well understood and may be difficult to quantify. Thus, by way of example, the provision of better work opportunities and associated higher incomes may have direct effects in the labour market, but they may also improve health and reduce crime (Thomson, 2008; Fujiwara, 2010; Dolan and Peasgood, 2007).
Having conceptualised how urban polices are expected to bring about change, the next step is to identify a research methodology that can measure change and establish the additionality of the outputs and outcomes produced by the policy and, wherever possible, place a value on them.
The starting point in most evaluations of urban policy is to identify those economic, physical, social and environmental indicators that it is believed will be affected by policy and produce a baseline position that can help to establish a counterfactual position from which to measure change. Estimating what has been the additionality associated with urban policy presents considerable challenges and it is wise not to understate them. Our understanding of how the urban system works is imperfect; there are many other factors that bring about change besides the urban policy including the mainstream government policies and it is difficult to disentangle these effects from those of the policy. As one commentator has remarked recently There are daunting methodological problems in identifying robust causal links between interventions, programmes and policies and desired outcomes. … The processes linking funding allocations, policy priorities, mechanisms and effects are likely to be indirect, hard to identify and even harder to measure. Hence the problem of attribution—i.e. the difficulty in identifying the extent to which a particular intervention has created a specific outcome (Saunders, 2011, p. 89).
We can also expect that policy effects will take time to emerge and there will be discontinuities such that certain thresholds of activity may have to be reached before significant impacts may occur. It is also difficult to be clear as to what are the precise spatial boundaries of impact and the extent of interactions with other surrounding areas.
A common approach is to establish the value of variables that are expected to be affected by the policy in a policy off-period and then monitor how their behaviour changes relatively in the policy on-period. In the ideal case, nothing else would change between the two periods but the urban policy. In practice, a number of both policy- and market-related influences do change and some evaluators use formal econometric modelling to try and disentangle the effects of these. Examples relating to the assessment of EU Structural and Cohesion Funds include the Hermin model and the REMI policy insight model (Treyz and Treyz, 2004; Bradley et al., 2004; Bradley, 2007; and Leonardi, 2006).
The evaluation of urban policy based on formal model building is constrained by many conceptual and measurement problems and in some cases evaluators have used survey-based approaches to target carefully constructed questionnaires on the people and businesses that it is believed that the policy should impact, with a variety of control procedures in place to avoid selection bias (Bartik, 2004). Such approaches are underpinned by strong monitoring frameworks (Edwards et al., 2007). Household surveys have also been used to assess the compensating change in income that equates to the improvement in quality of life that a policy produces drawing on techniques like shadow pricing. A recent example is the evaluation of the New Deal for Communities programme in England (Lawless et al., 2010).
2.1 Valuing the Benefits
The problems associated with evaluating the impacts of urban policies are clearly very significant. In this study, we have sought to build on the evidence currently commonly provided by evaluations of urban policy in developing the valuation methodology and we have also ensured that findings are capable of being subject to robust and rigorous sensitivity analysis. Our research suggests that, in seeking to value the benefits of urban policy and derive aggregate summary benefit–cost ratios, it is perhaps best to focus on the additional outputs produced by regeneration activity—i.e. job, training place, house, etc. Estimates of these are provided in most evaluations of urban policy. Thus, by way of example, it is very difficult to assess how a labour market initiative has reduced worklessness (an outcome) but we may, subject to the limitations of evaluation research described earlier, be able to establish some broad order of magnitude as to the number of jobs it has created (an output). On this basis, a way forward is to establish the volume of additional outputs that a unit of public expenditure on urban regeneration initiatives produces in the urban area by main output type and then assign a value to these additional outputs, recognising the time it may take for them to build up and their durability. 1 In the next section, we show how the approach can be applied to English urban policy by drawing on evaluation evidence and data for the UK over the period 2000–09.
3. Applying the Approach to England
In order to demonstrate the approach, we obtained evaluation evidence for economic development and regeneration programmes undertaken in the UK over the period 2000 to 2009. We also obtained data on wages, GVA (gross value added) and land and property prices from a number of sources identified in this article. The Appendix describes the characteristics of the data that were assembled.
The data from evaluations in the United Kingdom in recent years enabled estimates of the public-sector cost per additional output from mainly urban regeneration policy expenditure over the period 2001–09. The evidence is categorised by theme and activity type, showing the mean as well as a range based on the 95 per cent confidence interval in Table 2. By presenting the low, average and high estimates, we can allow for a considerable possible variation in the potential reliability of the estimates.
Public-sector cost per additional output for urban regeneration in England, 2001/09 (2009/10 prices)
Note: The table represents the authors’ analysis of published and unpublished evaluations commissioned between 2000 and 2009 by England’s Regional Development Agencies, Scottish Enterprise, the Department for Communities and Local Government and the Department for Business, Innovation and Skills; open space and public realm unit costs informed by discussions with landscape architects and published data on local authority planning contributions requirements. Please see the appendix.
3.1 Applying Unit Costs to Generate Additional Outputs
Evidence on the public-sector cost of an additional output can then be applied to the amount of expenditure committed to provide an indication of the amount of regeneration outputs generated in the urban areas that have been the focus of urban policy. In principle, this procedure could be applied to any country in the world where the required urban evaluation evidence is available.
To show in the rest of this article how these output data can be valued, we took the average annual expenditure in England over the period 2009–11 shown in Table 1 and estimated the net additional outputs from one year of regeneration expenditure in England, based on the low, average and high unit costs presented in Table 2. Table 3 shows the estimated additional outputs from one year of recent expenditure by activity type in England. The rest of this article shows how it is possible to value these outputs and produce overall aggregate benefit–cost ratios for urban policy.
Estimated additional outputs from one year of mainly urban regeneration expenditure in England
4. Assigning Values
The second part of the framework required a monetary value to be assigned to each additional output that was produced by the urban regeneration expenditure. Each of the regeneration activity types was examined to ascertain how a value could be placed on the outputs associated with the expenditure under that category.
In relation to worklessness, skills and training, the approach was to consider the benefits provided by urban regeneration initiatives as they get people into work and enhance their skills. The evidence from a review of what is quite an extensive literature points to the benefits being reflected in increased likelihood of an individual gaining employment and higher earnings. Some work (Mcintosh and Vignoles, 2000; Powdthavee and Vignoles, 2006; McIntosh, 2004; Felstead et al., 2007; Dickerson and Vignoles, 2007) has been able to relate skill enhancement to the probability of employment and how progressions through skill levels enhance earnings. Research has been undertaken by the Department of Work and Pensions in England on how the labour market benefits that arise from policy might be valued (DWP, 2010; Greenberg and Knight, 2007; Adam et al., 2008) including any indirect effects that may arise from getting people into work most notably in the areas of reduced crime and enhanced health. The recent work of Fujiwara (2010; DWP, 2010) is of great value in this respect and the present research was able to draw on this research.
When it comes to enterprise and business development, there are perhaps fewer conceptual problems in valuing benefits with the focus being on helping businesses to start up or expand in terms of turnover, leading to the creation of employment and, in some cases, enhanced productivity. Many evaluations measure these principal outputs. Employment and productivity gains can be valued through gross value added (GVA), or economic output, with ratios of GVA per employee derived from published data (for example, Regional Accounts, the Annual Business Inquiry and the Business Register and Employment Survey in the UK).
While this basic conceptual approach is generally accepted (DTI, 2006; BIS, 2009b, 2009c), and ratios of GVA per employee are published at different spatial levels and for different sectors, there remain challenging measurement issues. First, ideally one would approach the valuation process through a detailed analysis of the particular sectors which benefit. Advanced manufacturing sectors will have a much higher ratio of GVA per employee than, for example, retailing. Where the sectors of employment cannot be ascertained, the use of the average ratio is understandable, but at the level of individual projects and programmes, sector data enable a more sophisticated analysis.
Secondly, even within a given sector, there is a legitimate question about whether it is appropriate to take the average ratio of GVA per employee when the jobs created or safeguarded through intervention may be in relatively low-skilled occupations within that sector. This is where more in-depth evaluation work pays dividends in providing clear evidence which can be used to refine the valuation approach.
The valuation of regeneration activity relating to industrial and commercial property and infrastructure has tended to draw upon two different approaches. The first considers the production benefits associated with the end use of industrial and commercial property linked to employment and gross value added. The second uses the increase in value associated with the regeneration activity using techniques like hedonic pricing on which there is a substantial literature. The former approach was used in our research and estimated the total employment accommodated in the industrial and commercial property that was regarded as additional as a result of the urban regeneration activity and then assigning GVA–employment ratios in the same way as described earlier for enterprise and business development.
The valuation of urban housing growth and improvement has been one of the more challenging areas of the research because of the diversity of activity and the fact that, in valuation terms, different activities have the potential to generate production benefits for the economy as well as consumption benefits. Table 4 summarises the diversity of benefit types and valuation approaches adopted for different housing interventions.
Housing growth and improvement: main types of benefit and disbenefit
The valuation approach recognised the possibility that there should be an explicit recognition of the production benefits of new housing which arise through its role in supporting wider economic growth (Munro, 1993). While a proportion of new housing provision is planned in response to changes in household composition, a proportion will enable net in-migration to an area. Not all of these households will work in the regeneration target area (a proportion will out-commute), but the residual will facilitate economic growth. It is possible to estimate the scale of these effects and then use data on household size, working-age population, employment rates and the GVA–employee ratios (Regeneris and Oxford Economics, 2010). Clearly the key variables involved are highly sensitive to the local context, including housing market and labour market characteristics, and the valuation approach can adopt a bespoke approach to estimation drawing on local data that reflect the characteristics of individual projects or programmes and the spatial areas that they are intended to benefit.
There has been a considerable amount of research into how to value community development activity (Gaskin and Dobson, 1997; Gaskin, 1999; Mayer, 2003; Handy and Srinivasan, 2004; Egerton and Killian, 2006; Mook et al., 2007; Pho, 2008; Bowman, 2009; Brown, 1999). For community development, our preferred approach to valuation has been to use shadow pricing and volunteer time has been valued using the English minimum wage as a proxy for the value of the input. This has been translated into gross value added (GVA) using established ratios for employment costs to GVA for sectors that accord with activities delivered by many social enterprises. For investment in community organisations, estimates of the ‘social GVA’ arising from investment in community organisations has been based on the level of local income generated and regarded as turnover.
Urban regeneration enhances environmental amenity through open space and enhancements to the public realm and can be valued using a revealed preference approach. In the case of urban regeneration, hedonic pricing has found common use since property prices are influenced by changes to local amenity and the public realm. Stated preference methods such as contingent valuation and choice experiments have in fact tended also to be used widely since they can provide valuations for outcomes that are not well represented by property markets or recreation demand behaviour (in the case of travel cost methods). In this research, the valuation of the benefits of open space and public realm was approached by using an experimental household stated preference survey administered in Seaham, a town in north-east England. 2
Valuing the benefits associated with neighbourhood improvement has presented particular challenges. A review of existing research showed that probably the best technique available with which to monetise the impact of neighbourhood renewal on residents is to use shadow pricing. There are only a limited number of examples where shadow pricing has been used to value such outcomes. Moore (2006) used data for 2003 to estimate the value of feeling ‘very’ or ‘fairly’ unsafe walking alone in the local area after dark to be approximately £9400 in household income. 3 Powdthavee (2008) found that an increase in the level of social involvements is worth up to an extra £85 000 per year in per capita household income. A third approach is adopted in the recent national evaluation of the New Deal for Communities (DCLG, 2010) which is the most recent and extensive example of the application of the technique at the neighbourhood level in England. To illustrate the broad approach, we applied findings from the recent national evaluation of NDC which used shadow pricing techniques to monetise selected outcomes.
Table 5 summarises the valuation approach and key data sources used in the research to value the benefits of urban regeneration in England.
Valuation approach and data sources
4.1 Allowing for Build-up of Benefit and Duration
To value the benefits associated with urban regeneration, it was also necessary to allow for the time it takes for them to build up and how long they last. The review of UK-based evaluation evidence described in the Appendix provided some valuable insight and Table 6 shows the values that emerged.
Values per net additional output per annum and build-up and duration of benefits
5. Benefit–Cost Ratios for Urban Regeneration in England
To illustrate how the valuation approaches described in the previous section could be used, we applied them to the evidence on net additional outputs associated with the mainly urban regeneration expenditure in England identified in Table 1. In practice, the valuation approaches could be applied to similar data from any country that has evaluated their urban policies and produced quantitative estimates of net outputs as we discussed in section 2. Because the expenditure on urban policy generates a stream of benefits over time we discounted to a present value (PV) using HM Treasury’s Social Time Preference Rate of 3.5 per cent (HM Treasury, 2008). The PV of benefits was then divided by the annual public expenditure that generated the benefits to calculate a benefit–cost ratio (BCR).
Table 7 brings together the BCRs for each of the activities, drawing on the methods and evidence set out in section 4. The central results are based on average unit costs—i.e. the average shown in Table 3—but there is clearly plenty of scope for a considerable amount of sensitivity analysis by changing the average unit cost ratio and we also show a cautious estimate.
Benefit–cost ratios by activity type: central and cautious valuation applied to outputs derived using average unit costs
Taking the cautious estimate, the approach suggests an overall benefit–cost ratio associated with regeneration expenditure to be 2.4 and thus a significant pay-back in real resource terms to society from urban regeneration policy in England. The returns from business development and land and property regeneration are particularly high. It is possible to show how comparisons can be made with other Government programmes in England during the study period. Thus, by way of example, the Welfare Programme Pathways to Work, for incapacity benefit claimants, reports a benefit–cost ratio of 1.55. The Family Intervention Work Programme Tomorrow’s People had a benefit–cost ratio in these terms of 1.32.
5.1 The Effect of Applying the Approach in Different Spatial Contexts
Clearly, market values such as Average earnings, GVA and land and property values which are assigned to net additional outputs will vary depending on the local evaluation context. In England, for example—excluding London, where high levels of GVA per employee are dominated by the financial services sector—the research found that GVA per employee ranged from just under £34 200 in the North East to approximately £42 500 in the South East (2007 prices). Variations in land and property values are even more extreme. House prices across the English regions (again outside London) in 2009 ranged from £140 000 in the North East to £253 900 in the South East. These ranges demonstrate the importance in cost–benefit analysis of using context-specific information wherever this is available. Thus, at the local or even neighbourhood level where regeneration interventions are taking place, the market values (and thus the resulting benefit–cost ratios) will be highly sensitive to factors such as supply, demand, scale, type and quality as well as (in the case of earnings or GVA) the sectors and occupations in which employment opportunities are being created or taken up.
6. Future Research
The overall objective of the research discussed in this article has been to progress thinking on how the benefits of urban regeneration can be valued and summed to produce benefit–cost ratios that can be compared with similar ratios for other programmes of government, as well as to allow different approaches to urban intervention to be compared and contrasted both within and between countries. The emphasis has been on devising an approach that can build on the evidence provided from urban evaluations in many countries at the present time. We have deliberately sought to use fairly well established techniques like stated preference when it comes to valuing increases in environmental amenity and use statistical data sources that are frequently readily available.
We should not lose sight of the considerable conceptual and measurement problems that evaluations of urban policy are subject to and thus the limitations of evaluation evidence that can only ever be regarded as providing broad orders of magnitude. We have shown how the valuation estimates can be subject to sensitivity analysis that enables the effects of changes in core evaluation evidence and contextual information on benefit–cost ratios to be identified in a clear and transparent manner.
Our research suggests that in many cases it is possible to value the outputs from urban regeneration using market-based data. In other cases, such as in the valuation of environmental amenity, it is necessary to use valuation techniques that are now becoming well established.
Over the past 20 years, urban policy-makers have devised an increasingly sophisticated set of interventions with a correspondingly diverse range of physical, economic and social outputs and outcomes. However, with increasing constraints on public expenditure, it would now seem more important than ever that urban policy-makers be able to fight their corner in defending the value of urban interventions compared with other things that government spends money on. In England, for example, funding on urban regeneration has now been reduced considerably (House of Commons Select Committee, 2011).
Against this background, it is important to be able to demonstrate clearly the value to society from new initiatives and it is hoped that the research presented in this article will stimulate further research in this important area.
Footnotes
Appendix
Acknowledgements
The authors are grateful for the assistance of other members of the research team that included Peter Wells, Angela Brennan, Ian Cole, Jan Gilbertson, Tony Gore, Richard Crisp, Anne Green, Mike May-Gillings and Zara Phang. They are also grateful to an Expert Advisory Panel that comprised Professor Ken Willis (Newcastle University), Professor Jennifer Roberts (University of Sheffield), Professor Roger Bowles (York University), Dr Daniel Graham (Imperial College), Bobby Duffy (NOP/MORI) and a large number of officers from across UK government departments for their help, support and expertise throughout the course of the work.
Notes
Funding Statement
The findings presented in this article have been derived as part of a programme of research designed to value the benefits of regeneration and funded by the Department for Communities and Local Government. The views expressed in the article are those of the authors alone.