Book Review: Blue Marble Evaluation: Premises and Principles

Overview

The book is soundly structured and fluently written. It is in three parts. The first outlines four overarching principles. The second elaborates on their implications in the shape of eight operating principles. The third puts forward a theory of change for transformation and how it should be used. Each principle is preceded by an explicit premise. The focus on principles is rooted in Patton’s disdain of detailed rule books that tie down evaluators and lead them astray when faced by the uncertainties and emergent challenges of complex dynamic environments. Each principle is preceded by an explicit premise.

Principles-Focused Evaluation (2018) had already demonstrated the relevance of high-quality premises and principles in a wide range of settings: Clearly articulated principles inspire positive behavior, based on norms, values, beliefs, experience, and knowledge. But Blue Marble Evaluation’s 15 chapters do more than explicate premises and principles. Each chapter provides revealing evidence, forges fresh insights, weaves convincing arguments, and crafts specific proposals that hold promise for the future of evaluation practice.

The four chapters of Part I set the scene. The “Global Thinking” chapter features the notion of evaluators without borders in a world in which we are all subject to the same tides of technological change, increasingly severe weather patterns, and rampant pollution. The next chapter describes the Anthropocene principle and stresses the need for detailed contextual analysis. It leads to a “Transformative Engagement” chapter that advocates transformative change, given the unprecedented acceleration of deleterious human impacts on the planet.

The same chapter calls on evaluators to respect the ethical imperative of connecting humans with the natural world in sustainable ways and to acknowledge the magnitude, direction, and speed of ongoing and needed transformations. The final chapter, focused on the Integration Principle, argues that Blue Marble evaluation must tackle multiple interventions on many fronts by diverse actors.

Part II outlines eight operating principles, all embracing systems thinking. The “Transboundary Engagement” chapter proposes raising evaluation to a higher plane than the individual intervention or the nation-state: The unit of account has become the planet. The “GLOCAL Principle” chapter advocates engagement with stakeholders at multiple levels to allow promising initiatives to “go to scale” through local ownership and adaptation. This implies that social, economic, and environmental sustainability considerations must be integrated. Hence, the “Cross Silos” chapter makes clear that evaluation is uniquely placed to trespass disciplinary boundaries.

A sense of urgency permeates the “Time Being of the Essence” chapter, which recommends adoption of resilience and adaptability as sustainable development evaluation criteria. This is also one of the themes of the “Yin-Yang Principle” chapter—a plea for holistic action, transcendence of polarities, and harmonization of diverse perspectives to promote tolerance and achieve wholeness of understanding. As for the “Bricolage Methods” chapter, it outlines principles of eclecticism and context dependence, in the choice of evaluation methods, a familiar theme in Patton’s writing.

Reflecting the global perspective of Blue Marble evaluation, the “World Savvy” chapter recommends reconsideration of current evaluator guiding principles and competencies to embrace empathy, cultural sensitivity, inclusiveness, self-awareness, reflexivity, and so on. Next, the critically important and distinctive “Skin in the Game” chapter rejects the arm’s length stance of objective, neutral evaluation to embrace emotion, duty of care, and human values—a spirited reiteration of Patton’s client-centered evaluation doctrine that Michael Scriven had challenged in the 2009 Claremont Debates.

Finally, Part III is about theory. First, an evidence-based theory of transformation knits together networks theories and frameworks. Next, a Fidelity Principle is outlined. It validates and documents transformation and aims to help bring it about. The last chapter explicates the Transformational Alignment Principle according to which current evaluation practices for project and program evaluation are inadequate to evaluate transformation. The overarching conclusion is that evaluation must be transformed for positive transformation to take place and for the planet to survive.

A Development Evaluation Perspective

Before I go on, I should make clear that this review is shaped by my exposure to development activities at the World Bank. A United Nations specialized agency emerged out of the ashes of World War II following the Bretton Woods Conference that gathered delegates from 44 Allied Nations in Bretton Woods, New Hampshire. Together with its twin organization, the International Monetary Fund, the oldest and largest multilateral development bank has, since its creation, promoted market-oriented policy reforms around the world while seeking—with little success—to encourage quality economic growth that emphasizes social equity and environmental sustainability (Thomas & Dailami, 2000).

I headed the World Bank’s Independent Evaluation Group for two consecutive terms (1992–2002) after three decades of operational experience with World Bank programs. In this role, I was privileged to examine the impact of World Bank development operations independently and critically under the aegis of its board of directors, that is, at arm’s length from World Bank management. This twin exposure to development practice informed the comments that follow.

A World Without Borders?

Michael Quinn Patton, a 24-year-old Peace Corps volunteer, was posted in Burkina Faso when the moonwalk took place. Keenly aware of the cruel handicaps that artificial borders drawn by colonial powers had imposed on Africa (needless constraints on trade, disrupted interactions among peoples, and violent conflict), he was impressed by the absence of national boundaries in the Blue Marble photograph.

At about the same time, as chief of a World Bank division tasked with helping East Pakistan develop its land and water resources, I was struck by the huge economic and social costs of the partition of India. It led to the bifurcation of East and West Pakistan that proved to be politically unviable: Following a bloody civil war, Bangladesh came into being in 1971. The newly independent country faced starvation. It had to rein in population growth, achieve food security, and adapt to the disastrous effects of sea level rise.

The climate change emergency is now inducing similar challenges for the entire world. Tropical cyclones are becoming more frequent and intense. Flood risks are rising. Drought threatens agriculture in subtropical areas. Over half of the world’s megacities and 2 billion people living on coast lines are at risk. These are among the issues that underlie the global development mission that Blue Marble Evaluation outlines. They are closely linked to the globalization trends that the international development enterprise had deliberately promoted.

The Global Development Challenge

Twenty years after Neil Armstrong’s “One small step for a man, one giant leap for mankind,” I acquired a fresh perspective regarding the advantages and pitfalls of the new international economic order. At the helm of the World Bank’s independent evaluation function, I was able to take a critical view of the social and environmental consequences of the World Bank’s policy adjustment advice that had deliberately aimed to connect developing economies to the mighty engine of the global market.

On the one hand, these ideas yielded rapid economic growth and improvements in global socioeconomic indicators. On the other hand, unbridled population and rapid economic growth in an increasingly interconnected global economic system contributed to deadly environmental threats, huge increases in inequality, and growing worldwide corruption and an extraordinary proliferation of “problems without passports” (Annan, 2009). The Blue Marble evaluation response to the resulting global predicament is cogently stated at the outset of the book

Many people, organizations, and networks are working to ensure that the future is more sustainable and equitable. Blue Marble evaluators enter the fray by helping design such efforts, providing ongoing feedback for adaptation and enhanced impact, and examining the long-term effectiveness of such interventions and initiatives (p. 1).

While Blue Marble Evaluation does not explicitly state that evaluators should engage in advocacy and promote transformational change as activists, it does so implicitly: Patton views evaluators as committed participants to all beneficial social interventions and processes. He rejects the evaluation independence principle and encourages interdependence and close cooperation between evaluators and decision makers.

Thus, Blue Marble Evaluation builds on Patton’s prolific contributions to the evaluation literature. Indeed, the book is the culmination of an extraordinary intellectual trajectory: It brings together in a user-friendly and convincing way many of the far-reaching ideas and concepts that Patton has tirelessly disseminated for more than four decades as professor, trainer, keynote speaker, and author of 17 prior books and countless other publications (see, e.g., Patton, 2008, 2010, 2018).

The Blue Marble Response

Patton dedicated Blue Marble Evaluation to his grandchildren “with a mix of deep worry and great hope” (p. xi). With eloquent verve, dramatic quotes, compelling stories, and light-hearted cartoons, the book details the disruptive forces that runaway, disruptive globalization has unleashed, and the social and environmental upheaval that rapid and unsustainable growth has triggered. At the outset, Patton puts forward a unified context for the whole-earth challenge: the Anthropocene, a new epoch in world history characterized by cumulative human actions that are putting the sustainability and resilience of all natural and human systems at serious risk.

A sobering list of 10 highly destructive global challenges is laid out: (i) climate change, (ii) rising sea levels, (iii) growing concentration of wealth, (iv) virulent infectious diseases, (v) deadly pollution, (vi) human insecurity (terrorism, international criminal networks, illegal drug trade, human trafficking), (vii) refugees, (viii) water shortages, (ix) species extinctions, and (x) nuclear proliferation. Other global problems that require collective remedies, such as food insecurity, financial instability, and cyberwarfare, are also mentioned. In this context, transformational interventions of unprecedented scale are called for. In parallel, evaluation will have to be transformed as well.

The Policy Antecedents

Blue Marble Evaluation is forward-looking. It does not probe the root causes of the current global predicament. Nor does it highlight the power of vested interests or the collective action dilemmas that have long led politicians to avoid decisive action. It fails to question or probe the underlying free market ideas that have contributed to the problems that the book eloquently addresses. Yet, these ideas have dominated contemporary decision-making at national, regional, and global levels.

The same neoliberal ideas have also influenced evaluation: New Public Management concepts have characterized evaluation diffusion starting in the mid-80s (Vedung, 2010). Thus, evaluation has not always recognized that the root causes of environmental destruction are deeply embedded in the models and metrics that have guided economic policy design and implementation worldwide.

Hence, the admirable principles outlined in the book (even if they are fully endorsed and embraced by evaluators) may not suffice to turn the Blue Marble evaluation vision into reality. Looking ahead, Blue Marble evaluation will have to confront policy makers as well as evaluation practitioners with the urgent need to reconsider current evaluation criteria and indicators.

Evaluation Disenchantment

This said, while Blue Marble Evaluation sketches a changed evaluation paradigm embedded in a spirited, forward-looking disposition, it does not detract from a clear-eyed assessment of the current role of evaluation in society. Thus, it acts as a welcome counterweight to the depressed mood currently prevalent among eminent evaluation thinkers who are examining their fledgling profession from an excessively critical perspective.

According to Schwandt (2019), evaluation has yet to define and embrace an explicit understanding of its professional ethos and that “despite a repertoire of shared knowledge there remains significant disagreement in the field about what it means to practice evaluation” (p. 227). Forss (2019) views evaluation practice as a “systems-preserving activity…an intellectual effort which is inherently conservative and that assists in defending rather than challenging the powers that be, the established wisdoms, the current technologies and administrative practices” (pp. 189–190).

Finally, Dahler-Larsen (2012) has doubts about the value of the discipline and remarks that “some of the self-congratulatory rhetoric of the evaluation industry may be unwarranted. It is time to consider…whether the marginal utility of evaluation may be decreasing and whether there are sometimes good reasons for evaluation fatigue” (p. 231).

How can one explain this discontent? Is it related to the demands of an unforgiving marketplace largely controlled by vested interests? Shaped by the interplay between commissioners and suppliers of evaluation services, evaluation has become a business venture rather than a special calling. It has become a specialized form of action research or a managerial consultancy service—“one tool among many for the improvement of policies, learning and social change” (Furubo & Stame, 2019, p. xv).

Conceiving of evaluation in this way, Nielsen et al. (2019, pp. 243–265) conclude that since evaluation is not yet a profession, it cannot control the supply of evaluation services. As a result, the evaluation market has been invaded by other knowledge occupations, the boundaries between evaluation and these other knowledge occupations have become porous, and buyer power is dominating the evaluation market.

Blue Marble evaluation implicitly rejects the implications of these dour assessments. It reflects Patton’s unshakeable faith in the potential of evaluation to live up to its noble mission: influencing decision-making in the public interest whether the evaluand is an individual project, a country program, or, in the case of Blue Marble evaluation, a global initiative. It is a welcome antidote to the grim speculations that are far too prevalent under the big tent of the evaluation discipline.

A Bias for Hope

Blue Marble Evaluation is a compelling book. It displays determination, conviction, and confidence in evaluation. It is imbued with the “bias for hope” that Hirschman (1971) had championed in reaction to the dismal and parsimonious ideas of his economist colleagues. According to him, “any idea that is not encouraging is faulty” (Institute for Advanced Study, 2013, p. 30). By this, he did not imply that optimism is de rigueur. Rather, he urged constructive action whatever be one’s predictions for the future.

Similarly, Blue Marble Evaluation strikes a fine balance between positive and negative trends. It urges evaluators to hold on to hope while facing reality and aiming at balanced assessments of the state of the world. It recognizes the existential risks associated with prevailing policies, but it does not counsel despair.

Indeed, the book mentions countervailing and constructive trends that offer ground for measured and cautious optimism: reduced incidence of violent conflict; dramatic health, poverty reduction, and educational gains; new and viable renewable energy technologies; gradual phasing out of coal production; expanded areas of forest restoration; international commitment to sustainable development goals; and growing public calls for policy reform, among others.

A New Evaluation Wave?

Blue Marble Evaluation addresses not only evaluation commissioners, managers, and practitioners but also development specialists, program designers, government, private-sector and civil society leaders, grassroots activists, researchers, think tank experts, social impact investors, sustainability scientists, equity advocates, and others. The whole world is invited to the party: Evaluators on their own cannot implement Blue Marble evaluation. They are duty bound to join with others to realize the vision of an economically, socially, and environmentally sustainable world.

This exciting vision has far-reaching implications: Its realization hinges on drastic changes in the political order. In today’s divisive political climate, distrust of expertise is rampant, populism flourishes, and vested interests resist change. But there is no reason why Blue Marble evaluators should not partner with individuals and groups who may in time succeed in confronting these actors. Nor should evaluators be content with remaining peripheral to decision-making. Not all evaluators are surfing the current evidence-based wave that favors goal achievement verification rather than policy formation guided by ethical evaluation.

To be sure, evaluators must make a living. But they need not have skin in the wrong games, that is, they need not exert themselves to meet managerial demands that conflict with the public good or fail to improve the lot of disadvantaged groups. They should feel duty bound to protect a distinctive evaluation brand so that the boundaries between evaluation and other knowledge occupations are no longer blurred. They should embrace Patton’s innovative developmental evaluation practices. Doing so would make evaluation distinct from auditing, inspection, and other control activities in ways that would spare evaluation the public dissatisfaction associated with the heavy burdens imposed by overlapping monitoring, auditing, inspection, and theory-free Big Data algorithms.

Countervailing currents and turbulent streams underlie all evaluation tides. Once new ripples acquire significant momentum, a new wave begins to curl. Is a sea change in the offing? Pressures toward progressive change are building up in many societies worldwide. Public protests are proliferating. A host of private sector and civil society players are championing social impact evaluation. Indeed, development evaluators have begun pressing for a dynamic revolution in evaluation practice responsive to the universally endorsed Sustainable Development Goals (Feinstein, 2019). From this perspective, Blue Marble evaluation emerges as a critically important engine of evaluation renewal.

Conclusion

What then are the prospects of the Blue Marble Evaluation manifesto? Will its large and free-swinging ambition induce a major shift in public thinking and put the current social and environmental crisis at the center of political life in liberal and authoritarian societies alike? Will its call to arms turn the tide, change the odds, and lead the world back to sanity? Closer to home, will it help restore the commitment and restore the ideals of the evaluation pioneers? Will it generate a broad-based consensus about the identity, logic, and philosophy of operation of the evaluation enterprise?

Perhaps not. But Blue Marble Evaluation is a much needed call to action. It deserves a wide readership as a desirable counterweight to the disillusionment that has seized some evaluation thinkers. It premises and principles are sound. If adopted, they will help gather the scattered energies and latent ideals of the worldwide evaluation community. They will enhance the quality, relevance, and impact of the evaluation discipline at a critical time in its history.

As a result, Blue Marble Evaluation will raise the profile of our embattled occupation and contribute to its much needed renaissance.