Policy appraisal has spread rapidly throughout the OECD and beyond, as has the associated academic literature. In this paper we present the findings of a systematic review of this literature. We assess the extent to which developments in academic research and in everyday appraisal practices have informed one other. While there are signs that policy appraisal research is moving away from the ‘technical–rational model’ of appraisal, both research and practice remain heavily informed by it. The review reveals that research and practice are interacting in subtle ways, but these fall well short of what is sought by advocates of more reflexive approaches. We systematically examine the exact pattern of research–practice interaction depicted in the literature and explore how this may change in the future.
AchtnichtMRenningsKHertinJ, 2009, “Experiences with integrated impact assessment: Empirical evidence from a survey in three European member states”Environmental Policy and Governance19321–335
2.
OlssonJ AlkanBockstallerCStapletonL MEwertFKnapenRTherondOGeniauxGBellonSCorreiraT PintoTurpinNBezlepkinaI, 2009, “A goal oriented indicator framework to support impact assessment of new policies for agri-environmental systems”Environmental Science and Policy12562–572
3.
BinaO, 2007, “A critical review of the dominant lines of argumentation on the need for strategic environmental assessment”Environmental Impact Assessment Review27585–606
4.
BondAMorrison-SaundersA, 2011, “Re-evaluating sustainability assessment: Aligning the vision and the practice”Environmental Impact Assessment Review311–7
5.
BrouwerF Mvan IttersumM K, (Eds), 2010Environmental and Agricultural Modelling: Integrated Approaches for Policy Impact Assessment (Springer, Dordrecht)
6.
CarrollP, 2010, “Does regulatory impact assessment lead to better policy?”Policy and Science29113–122
7.
CashmoreMBondACobbD, 2007, “The contribution of environmental assessment to sustainable development: Toward a richer empirical understanding”Environmental Management40516–530
8.
CashmoreMBondACobbD, 2008, “The role and functioning of environmental assessment: Theoretical reflections upon an empirical investigation of causation”Journal of Environmental Management881233–1248
9.
CashmoreMRichardsonTHilding-RyedvikTEmmelinL, 2010, “Evaluating the effectiveness of impact assessment instruments: Theorising the nature and implications of their political constitution”Environmental Impact Assessment Review30371–379
10.
CecotCHahnR WRendaA, 2007, “A statistical analysis of the quality of impact assessment in the European Union”, WP 07–09, AEI-Brookings Joint Center for Regulatory Studies, Washington, DC
11.
ChilversJ, 2008, “Environmental risk, uncertainty, and participation: Mapping an emergent epistemic community”Environment and Planning A402990–3008
12.
COWI, ECA, Scott Wilson, 2004Evaluation of Approaches to Integrating Sustainability into Community Policies final report for the Secretariat General of the European Commission (Commission of the European Communities, Brussels)
13.
de RidderWTurnpennyJNilssonMvon RaggambyA, 2007, “A framework for tool selection and use in integrated assessment for sustainable development”Journal of Environmental Assessment Policy and Management9423–441
14.
DunlopCRadaelliC MRusselD, 2010, “What's an impact assessment to do? Ideas, instalments and the capabilities–expectations gap”, paper presented to the ECPR Joint Sessions of Workshops 2010, Munster
15.
EalesRSmithSTwigger-RossCSheateWOzdemirogluEFryCTomlinsonPFoanC, 2005, “Emerging approaches to integrated appraisal in the UK”Impact Assessment and Project Appraisal23113–123
16.
EEAC, 2006, “Impact assessment of European Commission policies: Achievements and prospects”, statement of the European Environment and Sustainable Development Advisory Councils Working Group on Governance, April, Brussels
17.
ElstD A M MFontarasGGenseRSamarasZ, 2005“Use of vehicle modelling tools for predicting CO2 emissions to support policy and decision making: A case study of the EU N1 vehicle legislation framework”, in Proceedings of the Transport and Air Pollution Symposium Graz, June, pp 11–20
European Commission, Commission of the European Communities, Brussels
20.
2002Communication from the Commission on Impact Assessment COM(2002) 276
21.
2004Impact Assessment: Next Steps SEC(2004)1377
22.
2009Impact Assessment Guidance SEC(2009)92
23.
European Court of Auditors, 2010, “Impact assessments in the EU institutions: Do they support decision-making?”, Special Report Number 3, European Court of Auditors, Luxembourg
24.
FontarasGSamarasZ, 2007, “A quantitative analysis of the European Automakers' voluntary commitment to reduce CO2 emissions from new passenger cars based on independent experimental data”Energy Policy352239–2248
25.
FranzJKirkpatrickC, 2008, “Improving the quality of integrated policy analysis: Impact assessment for sustainable development in the European Commission”Evidence and Policy4171–185
26.
GordonILewisJYoungK, 1977, “Perspectives on policy analysis”Public Administration Bulletin2526–30
27.
HahnR WBurnettJ KChanY IMaderE AMoyleP R, 2000, “Assessing the quality of regulatory impact analyses” Regulatory Analysis 00–1, AEI Brookings Centre for Regulatory Studies, Washington, DC
28.
HajkowiczSCollinsK, 2006, “A review of multiple criteria analysis for water resource planning and management”Water Resource Management211553–1566
29.
HarringtonWMorgensternR D, 2003Evaluating Regulatory Impact Analysis paper to OECD expert meeting (OECD, Paris)
30.
HarringtonWMorgensternR DNelsonP, 2000, “On the accuracy of regulatory cost estimates”Journal of Policy Analysis and Management19297–322
HertinJTurnpennyJJordanANilssonMRusselDNykvistB, 2009a, “Rationalising the policy mess? Ex ante assessment and the utilisation of knowledge in the policy process”Environment and Planning A411185–1200
33.
HertinJJacobKPeschUPacchiC, 2009b, “The production and use of knowledge in regulatory impact assessment: An empirical analysis”Forest Policy and Economics11413–421
34.
HolzFvon HirschhausenCKemfertC, 2008, “A strategic model of European gas supply (GASMOD)”Energy Economics30766–788
35.
HowlettMWellsteadA M, 2009, “Revisiting Meltsner: Policy advice systems and the multidimensional nature of professional policy analysis”, RP LKYSPP10–001, Lee Kuan Yew School of Public Policy, Singapore
36.
JacobKHertinJHjerpPRadaelliCMeuweseAWolfOPacchiCRenningsK, 2008, “Improving the practice of impact assessment”, Evaluating Integrated Impact Assessments, Project No. 028889, European Commission Sixth Framework Programme, Brussels
37.
JacobsS, 2006Current Trends in Regulatory Impact Analysis: The Challenges of Mainstrearning RIA into Policy-making (Jacobs and Associates, Washington, DC)
LeeN, 2006, “Bridging the gap between theory and practice in integrated assessment”Environmental Impact Assessment Review2657–78
40.
LeeNKirkpatrickC, 2006, “Evidence-based policy making in Europe: An evaluation of European Commission integrated impact assessments”Impact Assessment and Project Appraisal2423–33
41.
LiseWHobbsB F, 2008, “Future evolution of the liberalised European gas market: Simulation results with a dynamic model”Energy33989–1004
42.
LussisB, 2004EU Extended Impact Assessment ReviewInstitut pour un Développement Durable, Ottignies
National Audit Office, The Stationery Office, London
45.
2006Evaluation of Regulatory Impact Assessments 2005–06
46.
2007Evaluation of Regulatory Impact Assessments 2006–07
47.
2009Delivering High Quality Impact Assessments
48.
NessBUrbel-PiirsaluEAnderbergSOlssonL, 2007, “Categorising tools for sustainability assessment”Ecological Economics60498–508
49.
NielsenULercheD BKjellingbroP MJeppesenL M, 2006Getting Proportions Right: How Far Should EU Impact Assessment Go?Environmental Assessment Institute, Copenhagen
50.
NilssonM, 2006, “The role of assessments and institutions for policy learning: A study on Swedish climate and nuclear policy formation”Policy Sciences38225–249
51.
NilssonMJordanATurnpennyJHertinJNykvistBRusselD, 2008, “The use and non-use of policy appraisal in public policy making: An analysis of three European countries and the European Union”Policy Sciences41335–355
52.
NykvistBNilssonM, 2009, “Are impact assessment procedures actually promoting sustainable development? Institutional perspectives on barriers and opportunities found in the Swedish committee system”Environmental Impact Assessment Review2915–24
53.
OECD, 2008Building an Institutional Framework for Regulatory Impact Assessment: Guidance for Policy-makers (OECD, Paris)
54.
OECD, 2009Regulatory Impact Assessment: A Tool for Policy Coherence (OECD, Paris)
55.
OwensSRaynerTBinaO, 2004, “New agendas for appraisal: Reflections on theory, practice, and research”Environment and Planning A361943–1959
56.
PearceD, 1998, “Cost-benefit analysis and environmental policy”Oxford Review of Economic Policy1484–100
57.
PetticrewMRobertsH, 2005Systematic Review in the Social Sciences: A Practical Guide (Wiley - Blackwell, Oxford)
58.
PohekarS DRamachandranM, 2004, “Application of multi-criteria decision making to sustainable energy planning: A review”Renewable and Sustainable Energy Reviews8365–381
59.
RadaelliC M, 2004a, “Getting to grips with quality in the diffusion of regulatory impact assessment in Europe”Public Money and Management5271–276
60.
RadaelliC M, 2004b, “The diffusion of regulatory impact analysis: Best practice or lesson drawing?”European Journal of Political Research43723–747
61.
RadaelliC M, 2005, “Diffusion without convergence: How political context shapes the adoption of regulatory impact assessment”Journal of European Public Policy12924–943
62.
RadaelliC M, 2009, “Measuring policy learning: Regulatory impact assessment in Europe”Journal of European Public Policy161145–1164
63.
RadaelliC M, 2010, “Regulating rule-making via impact assessment”Governance: An International Journal of Policy, Administration, and Institutions2389–108
64.
RadaelliC Mde FrancescoF, 2010, “Regulatory impact assessment”, inThe Oxford Handbook of Regulation Eds BaldwinRCaveMLodgeM, (Oxford University Press, Oxford) pp 279–301
65.
RadaelliC MMeuweseA C M, 2009, “Better regulation in Europe: Between public management and regulatory reform”Public Administration87639–654
66.
RadaelliC MMeuweseA C M, 2010, “Hard questions, hard solutions: Proceduralisation through impact assessment in the EU”West European Politics33136–153
67.
RaynerS, 2003, “Democracy in the age of assessment: Reflections on the roles of expertise and democracy in the public sector decision making”Science and Public Policy30163–170
68.
RendaA, 2006Impact Assessment in the EU: The State of the Art and the Art of the StateCentre for European Policy Studies, Brussels
69.
RunhaarHRunhaarP ROegemaT, 2010, “Food for thought: Conditions for discourse reflection in the light of environmental assessment”Environmental Impact Assessment Review30339–346
70.
RusselDJordanA, 2007, “Gearing-up governance for sustainable development: Patterns of policy appraisal in UK central government”Journal of Environmental Planning and Management501–21
71.
RusselDJordanA, 2009, “Joining up or pulling apart? The use of appraisal to coordinate policy making for sustainable development”Environment and Planning A411201–1216
72.
RusselDTurnpennyJ, 2009, “The politics of sustainable development in UK government: What role for integrated policy appraisal?”Environment and Planning C: Government and Policy27340–354
TamborraM, 2002Tools for Sustainability Impact Assessment of EU Policies: The Contribution of the EU Socio-economic Research (European Commission, Brussels)
75.
ThielA, 2008, “Changing administrative culture and political debate? A review of impact assessment practices in the European Commission”Zeitschrift für Umweltpolitik and Umweltrecht315–27
76.
TorritiJ, 2007a“The standard cost model: When better regulation fights against red-tape”, inBetter Regulation Ed. WeatherillS, (Hart, Oxford) pp 83–106
77.
TorritiJ, 2007b, “Impact assessment in the EU: A toll for better regulation, less regulation or less bad regulation?”Journal of Risk Research10239–276
78.
The Evaluation Partnership, 2007Evaluation of the Commission's Impact Assessment System report written for the European Commission, Contract Number SG-02/2006 (European Commission, Brussels)
79.
TurnerR K, 2007, “Limits to CBA in UK and European environmental policy: Retrospects and future prospects”Environmental Resource Economics37253–269
80.
TurnpennyJNilssonMRusselDJordanAHertinJNykvistB, 2008, “Why is integrating policy assessment so hard?”Journal of Environmental Planning and Management51759–775
81.
TurnpennyJRadaelliCJordanAJacobK, 2009a, “The policy and politics of policy appraisal: Emerging trends and new directions”Journal of European Public Policy16640–653
82.
TurnpennyJLorenzoniIJonesM, 2009b, “Noisy and definitely not normal: Responding to wicked issues in the environment, energy and health”Environmental Science and Policy12347–358
83.
VibertF, 2004The EU's New System of Regulatory Impact Assessment: A ScorecardThe European Policy Forum, London
84.
ViebrockE, 2008, “Systematic literature reviews as a tool for evidence-based policy”, paper prepared for the ECPR Joint Sessions in Rennes, 11–16 April
85.
WilkinsonDFergussonMBowyerCBrownJLadefogedAMonkhouseCZdanowiczA, 2004Sustainable Development in the European Commission's Integrated Impact Assessments for 2003: Final ReportInstitute for European Environmental Policy, London
