Abstract
What a timely book! Here we are starting a fresh year (at the time of writing) with concerns about the role of evidence in public policy and the need for responsive bureaucracies at the forefront of public policy debate. It is perfect timing for a book that explores these issues within the context of evaluation. The push for improved use of evidence, organisational learning and evaluative thinking in public policy around the world is well captured in this book with contributions from a wide range of very experienced evaluators. As Kathryn Newcomer points out in the Foreword, ‘Convincing current and upcoming public sector leaders of the value that evaluation practice and thinking brings to government bureaucracies is the challenge that this volume can equip readers to address’. This is clearly a substantial challenge but one that the editors do not shy away from and they remain optimistic about the power of evaluation and evaluators to influence bureaucracy. The book explores the essential relationship between bureaucracy and evaluation, and recognises the need for adaptability in bureaucracy (as well as evaluation) especially given the increasing complexity of public policy and programmes and the speed with which they change.
The volume has a broad audience appeal including bureaucrats in national and international organisations, evaluation commissioners, managers and practitioners, and academics and students in public policy and evaluation. To these readers, it presents an interesting and relevant array of challenges in the relationship between evaluation and bureaucracy and a selection of theoretical and practical means to address these challenges. Given the deep experiential base of the authors, the book should also be of interest to those in these audiences who are interested in learning more about the challenges faced by evaluators and the evaluation function within bureaucracies, as well as the potential role for evaluation in supporting the change process.
The well-researched articles expose the reader to new ideas about the relationship between bureaucracies and evaluation. There are a number of calls to be more innovative in evaluation practice (e.g. rapid evaluation) and these are worth considering in our evaluation practice. These ideas are based on learning from the literature and from practice while also being innovative and creative in adapting to new and challenging scenarios. Throughout the book, there is a strong undercurrent of support for evaluative thinking as an essential component of organisational or bureaucratic culture. This identifies one of the key benefits of this book – it is focussed as equally on bureaucracy as it is on evaluation – readers interested in either of these can benefit from the insights of the authors.
The book takes the position that bureaucracy needs evaluation and vice versa. The authors recognise the need for change in both and much of the book is about how each of these processes might better adapt to the other without undermining the specific benefits of each. The authors strive to find the balance between keeping what is positive and constructive about evaluation and bureaucracy and what could be improved, including the relationship between them.
The book contains 12 chapters divided into three sections, starting with an excellent introduction by Tony Tyrrell. He sets the tone for the following chapters by confronting the strengths and weaknesses of bureaucracy including both its almost universal application in governments at all levels and its use as a pejorative term. The applicability and efficacy of both bureaucracy and evaluation in the increasing rate of change environment of the public sector are central themes of this book. As Tyrrell summarises, The perspectives and analyses presented in this book are grounded in empirical and other research as well as in the experiences of the authors who have practised and/or managed evaluation in many national as well as international level bureaucracies. All of this underpins a balanced and in-depth examination of the core considerations that the book sets out to explore in the relationship between bureaucracy, evaluation, and adaptability. (p. 11)
The remainder of the book is divided into three parts with a strong focus on how to improve the working relationship between evaluation and bureaucracy. The 17 authors are all experts in evaluation and highly experienced in working in and with bureaucracies, ranging from government agencies to the EU to large international aid agencies. They are mainly based in Europe with a wide diversity of expertise and experience. Through this diversity they bring a number of valuable perspectives to the discussion. If one were to look critically at the spectrum of contributors, it might have been useful to include similar experts from other continents and perhaps several from the other side of the evaluator-management equation. But there are always limits on the number and source of contributors, and Perrin and Tyrrell have managed to include an excellent selection in this volume.
The first of the three parts of the book presents four articles which focus on the art and science of working within bureaucracy covering issues of feedback, bureaucratic agility, accountability and innovation. For example, one article uses an empirical study of an area of the national bureaucracy in Poland to show that among other findings, most feedback is internally generated with the bureaucracy, evaluation is generally process driven not strategic, and feedback tends to make bureaucracy more rigid rather than open to change. This may sound dismayingly familiar to many working in bureaucracies.
The second part addresses the issue of how evaluation provides support for bureaucracies. Using the EU, the OECD and other international and national bureaucracies as sources, these three articles focus on specific functions evaluation can provide including facilitating adaptation, reducing uncertainty and distinguishing between accountability and learning. The authors present a number of enlightening discussions on the issue of different purposes of evaluation and types of use.
The third part of the book has three chapters on specific challenges for the role of evaluation. A particularly interesting chapter by Estelle Raimonda and Frans Leeuw explores the issue of ‘evaluation capture’, when evaluation processes become so embedded in the bureaucracy that they lose their autonomy and independence. Given the strong push to develop monitoring and evaluation frameworks in government agencies, this cautionary chapter is well worth a read. An extra benefit are the suggestions they provide for avoiding the bureaucratic capture of evaluation.
A particular strength of the book is that it dissects the roles of evaluation in bureaucracy from a number of timely and interesting perspectives, including adaptability, use, innovation and rapid change. These are perspectives we are all grappling with and the authors provide very useful theoretical and practical insights into these issues.
As Burt Perrin points out in the concluding chapter, given the essential role bureaucracy plays in democratic societies, our aim should be not to dismantle it but to find the right balance between administrative requirements and innovation. As the book so engagingly demonstrates, this is exactly where evaluation can be most valuable.
