Abstract
It is difficult to imagine that a practicing evaluator from North America exists who has never held the 1st, 2nd, 3rd, or 4th editions of Utilization-focused evaluation (U-FE) in their hands. Perhaps a controversial concept 44 years ago, the idea that an evaluator takes responsibility for use throughout the life of their evaluations—that they design and carry evaluations out in routine interaction with a well-defined group of users with well-specified uses clearly in mind—is now a commonplace notion. Past iterations of U-FE (1978, 1986, 1997, 2008), coupled with Patton's decades-long and persuasive advocacy of intended use by intended users contributed mightily to this shift in many evaluators’ belief that evaluation use ought to guide practice decisions.
The Evolution of U-FE
Utilization-focused evaluation's origin story is well described in this and prior editions. U-FE grew out of Patton's unshakable commitment to the role of evaluation in improving society and his frustration with its limited use. During the 1970s, he coupled empirical study of factors that facilitated use with keen observation of his and others’ practice to evolve a commonsense perspective focused on increasing the odds that evaluations proved useful and led to use. To every edition of U-FE, Patton—ever the consummate scholar practitioner—brings his accumulating experience in the field, voracious diet of reading far and deeply, learning through rebutting arguments leveled against his emerging ideas, and careful consideration of evolving scholarship on how evaluation can best meet the challenges of the day.
In the earliest editions of U-FE, Patton centered the importance of intentional cultivation of a relationship between the evaluator and those stakeholders who care about an evaluation. Naming this “the personal factor,” he highlighted evaluation as a social rather than technical endeavor. He urged evaluators to invite the stakeholders who are invested in and champions of the evaluation into decision-making processes to promote an evaluation's use and build their capability to think and act evaluatively. In later editions of U-FE, he situated an orientation to use within the context of the growing influence of complexity sciences on the evaluation field and calls for evaluators to better address issues of human diversity. The challenges of evaluation as a collaborative endeavor and of dealing with increasingly complex social issues became more prominent aspects of utilization-focused practice. In the latest edition of U-FE, co-authored with evaluator Charmagne E. Campbell-Patton, these ideas and all the familiar elements of past editions—Halcolm's wisdom, witty cartoons, stories from the field, insightful quotes, voluminous summary tables and exhibits—are present. But this U-FE is very much a new book written for a new moment in the evolution of the evaluation profession and in the evolution of the U-FE canon.
“Utilization-Focused Evaluation, 5th Edition”: What's new?
The 5th edition formally refashions U-FE as a principles-driven approach to practice, described through a combination of minimum indispensable specifications and operational principles. The book's 20 chapters are divided into three parts. Each of the book's chapters include reflective questions, exercises, and online resources. The book is anchored in current professional guidance documents originating in the United States such as the American Evaluation Association (AEA) 2018 Evaluator Competencies (AEA, 2018a), the AEA 2018 Guiding Principles (AEA, 2018b), the Joint Committee Standards on Program Evaluation (Yarborough, Shula, Hopson, and Caruthers, 2010), and the AEA statement on cultural competence (AEA, 2011).
Part I, composed of four chapters, opens with the core basic argument for orienting to use. As in older editions, Patton and Campbell-Patton define evaluation, distinguishing it from other forms of systematic inquiry. They characterize the essence of what it means to orient one's practice around use. They also identify eight common barriers to use-focused evaluations (such as preferring to prioritize rigor, independence, or compliance) and provide readers with a well-honed U-FE response. The minimum specifications for a utilization-focused practice are described next, namely honoring the personal factor through identifying and engaging intended users; focusing on the priority uses that the intended users express; intentionally cultivating process use; focusing on use from beginning to end; and adapting to changes in the context. Through these chapters, Patton and Campbell-Patton stress that the endgame is not delivering a report. Rather, the endgame is use. The “Minspecs” underscore the authors’ conviction that the quality of the evaluator's partnership with users is vital to achieving use. Judge evaluations by use, they say. Patton and Campbell-Patton conclude Part I with chapters on promoting evaluative thinking through U-FE and on the stakes of misuse and nonuse. Throughout Part I, contemporary challenges, most notably the coronavirus pandemic, are used to support the argument that use must be the evaluator's guiding concern.
Part II introduces readers to U-FE as a principles-driven approach. Drawing on Patton's 2018 book on principles-focused evaluation (Patton, 2018a), Patton and Campbell-Patton suggest that U-FE is better conceived of in terms of principles than a series of steps, as in the prior edition. They observe: “UFE principles offer an appropriate framework for dealing with real-world complexities precisely because the challenge in any evaluation design is to match the inquiry frameworks to the nature of the intervention being evaluated, the situation one faces, and the questions being asked.” (Patton & Campbell-Patton, 2022, pp. 86–87). Patton's GUIDE (guiding, useful, inspirational, developmental, evaluable) framework is introduced as the basis for deriving sound principles. The remaining chapters in Part II introduce the 10 U-FE principles, stated here in their simple form as: contextualize, personalize, customize, align, prioritize, engage users, strategize process use, adapt, mobilize follow-through, reflect, and learn. One chapter is devoted to each principle, most of which are foregrounded in the minimum specifications offered in Part I.
Part II well reflects U-FE's ongoing evolution. The chapters incorporate new scholarship and practice trends. For example, in the chapter on reflexivity, Patton and Campbell-Patton discuss recent scholarship on culturally responsive evaluation by Vidhya Shanker, Nicky Bowman, Art Hernandez, Jill Chouinard, and Rodney Hopson, among others. Data visualization and its importance for facilitating use is addressed. The chapters also include the authors’ self-critique and reflection on their past and present thinking. For instance, Patton and Campbell-Patton critique their use of the term “stakeholder” from the perspective of power and privilege. The chapters in this section of the book are laden with brief case examples focused on enduring and contemporary global challenges. Chapters integrate material from Patton's 2018 book on facilitating evaluation (Patton, 2018b), emphasizing the role of the evaluator as the architect and shepherd of a carefully designed and skillfully conducted engaged process. In addition, Part II identifies kindred approaches to evaluation that lend themselves to a utilization-focused orientation because they share some of U-FE's key premises (e.g., feminist evaluation, indigenous evaluation, equitable evaluation). The section concludes with a description of the “aha” breakthrough moments in Patton's thinking and learning over time that gradually led him to the present reformulation of U-FE, in partnership with Campbell-Patton.
Part III of the book is perhaps best described as an impassioned plea to galvanize the profession around the world's most urgent concerns. Here, Patton and Campbell-Patton adapt U-FE to pressing contemporary concerns in the profession and the world at large, but their message is principally this: it is a moral imperative for evaluation and every practitioner within the profession to evolve to meet global challenges and do so fast. In this last section, Patton and Campbell-Patton address the pervasive thoughtlessness driving our post-truth era, the deadly consequences of gross social inequities driven by forces such as capitalism and colonialism, and the unsustainability of our current ways of living on the planet. Patton and Campbell-Patton propose that without exception, all evaluations and evaluators must address these most destructive forces. The culminating mantra of this final section of the book concerns the need for transformative approaches to evaluation suited to the complexities of transformative social and structural change. Patton and Campbell-Patton draw on the concept of a syndemic (Singer & Snipes, 1992)—a syndrome of discrete epidemics that intersect and interact in ways that strengthen their collective damage—to illustrate the interconnected, systemic nature of social problems. The inadequacies of programs and evaluation approaches designed to address complex systemic issues form the heart of the argument for the present U-FE approach and its continued evolution. Pattons prior work on integrating critical systems concepts into developmental evaluation (Patton, 2011; Patton et al., 2016) and his recent articulation of Blue Marble evaluation (Patton, 2020) are prominent influences on Part III. The foundation for the next era of U-FE stands on the ground laid in these theories.
Reflections on the New Edition
This book attempts to integrate the many distinct strands and variants of Patton's theoretical contributions to practice over more than 4 decades. The treatment of topics such as developmental evaluation or principles-focused evaluation is insufficiently elaborated in this book to understand those theoretical strands on their own terms. Indeed, at times in both Parts I and II, I missed the elegance of older versions of U-FE. Knowing Patton's prior work enhanced my ability to read this book in ways that make it difficult for me to guess at how someone encountering U-FE for the first time might experience the book. The minimum specifications, ten principles of practice, and the supporting pillars of evaluative thinking, equity, and sustainability are logical outgrowths of Patton's past writing to be sure but might easily be read as an overwhelming set of considerations and guiding ideas to those being introduced to U-FE through this latest edition. The book includes multiple mechanisms to assist the reader digest the material, but there is a lot to unpack and critically examine.
As a teacher, I would do everything in my power to ensure my students know U-FE and are exposed to this book. However, the difficulty in its use in the classroom will come not simply from how packed it is with complex ideas. It will also come as a result of what the book is not, which is best captured in the words of Patton and Campbell-Patton themselves: “This isn’t a book about how to ask evaluation questions. It's not an evaluation design and measurement book. This book is about utilization-focused evaluation thinking guided by U-FE principles.” (Patton & Campbell-Patton, 2022, p. 164, emphasis in the original). A 432-page text that does not address the technical matter that must be addressed in the typical one-shot evaluation course is unlikely to be chosen. Chapters perhaps, but not the entire book. That, to me, is a tragedy. There are so many essential ideas and valuable pieces of advice in this book for future and current practitioners.
Three other matters merit mention. First, despite the global and culturally competent threads running throughout the book and the stress the authors appropriately place on decolonizing the field, it struck me as curious to not address professional ethics, principles, and competencies emanating from other parts of the world and consider how, if at all, these do and should inform U-FE. Second, there are more proofreading errors in the book than is desirable, at least one of which obscures the meaning of a sentence. Third, despite my excitement about an intergenerational text, I had hoped to find Campbell-Patton's presence more obvious than it is. The book is written with one voice overall and in the engaging style readers familiar with his past work will identify, rightly or wrongly, with Michael Q. Patton. Coupled with how much his theoretical work and prolific writing on evaluation informs the book, this is perhaps inevitable. Students especially would benefit from hearing from Campbell-Patton about her experiences learning and applying U-FE to her evaluation practice. They are likely to wonder, as I did, what she found helpful, exciting, and challenging as she moved from U-FE novice to expert and what advice she might offer emerging professionals and those entering the field today. I hope and imagine that her voice, ideas, and experiences will be more prominent in future editions.
U-FE remains a passionate sermon on central issues for the evaluation profession. It is clear on every page of the book that Patton and Campbell-Patton recognize this sermon is one that must be delivered to us again and again. Their sense of urgency is palpable. U-FE, 5th edition, offers its readers a necessary contemporary theory of how evaluation can spur action and why it must. Buy it and most importantly, use it.
