Strengthening Evaluation for Development

The Challenge

Over the past decade, the distinctions between developed and developing ¹ countries have become increasingly blurred. As a consequence, some argue that “evaluation” and “development evaluation” are merging into one stream of theory and practice. Yet, a main difference remains: With few exceptions, the vulnerabilities of developing countries are magnified. The poor tend to be poorer, the vulnerable more vulnerable, institutions and systems more fragile, unstable or dysfunctional, the powerful and powerless more so, contexts less predictable, and those capacities seen by many as essential to executing conventional development models, lower.

Therefore, while the monitoring and evaluation of development is an exciting and vibrant endeavor, it has high stakes. If an evaluation is poorly designed or executed, it can have considerable and destructive consequences: People, communities, or countries already in a precarious position might lose their only chance at a better future, or policies and practices that are harmful might continue. Development evaluators have to respect and engage with such risk, and the profession has to be responsive to the ensuing challenges. Most crucially, those directing and influencing development evaluation theory and practice—evaluation commissioners and thought leaders, ² evaluators, as well as organizational leaders and managers—all have to bear the weight of this responsibility when executing their charge.

This notion also presupposes that evaluation is a valued and valuable activity that is regularly used to ensure development effectiveness. This is of course not necessarily so; the legacy of poorly executed evaluations as well as the highly political and technically challenging nature of both development and development evaluation, interferes. Yet, a firm belief in the relevance, utility, and essential contributions of evaluation must continue to guide the profession, especially in countries still struggling to find their most effective development path.

We live in extraordinary times. The rapid development of new technologies is taking society into uncharted waters, inequalities are accelerating in many previously prosperous nations and developed countries face increasing uncertainties. Power shifts and new relationships reveal alternative development models but also titanic struggles for power and influence. Yet, we can nevertheless celebrate for the first time in history that hundreds of millions have been lifted out of poverty, in record time and primarily through their own efforts, in countries until recent regarded as severely underdeveloped. This short article argues that in preparation for the exciting yet challenging time ahead, development evaluation requires a revitalized, purposeful, innovated agenda, nurtured by more visible, dynamic thought leadership from the “developing” countries themselves, with greater attention to critical issues at the development–evaluation interface.

The Development–Evaluation Interface

Development and evaluation are, or should be, in a tango with each other—the one sometimes leading, and sometimes the other, learning from each other and working together synergistically to create something meaningful. Taken together with research, they can be viewed in the same way as a strand of DNA, building a healthy body of knowledge for development. These metaphors emphasize the importance of the relationship between development and evaluation, and the need for a greater emphasis on the intersection between the two, preventing the one from mindlessly leading the other. This is not a trivial issue. It assumes that we are clear and explicit on the underlying assumptions, values, and frameworks that underpin and link the two, and that innovation in development evaluation is pursued with attention to the implications or consequences for development and its effectiveness. This is evaluation for development, rather than the evaluation of development.

For example, excluding or understating the role of power in evaluation negates its importance in development policies, strategies, and interventions. Using people as experiments and numbers while ignoring their voices during evaluation is disempowering and dismisses their voice in the course of their development. Focusing an evaluation on the interests of individuals at the cost of community harmony reflects an understanding of development where individual interests dominate those of the collective. Rigidly applying the ubiquitous “logframes” within a usually too short funding cycle for accountability in results-based management and impact evaluation neglects the critical reality of ever-evolving development contexts and slow, initially even negative trajectories of change. Tackling for impact evaluation, one strand of a development intervention without recognizing that the whole is more (or less) than the sum of the parts, or focusing on the achievement of (average) impacts without also focusing vigorously on crucial development needs such equity, transformation, institution building, accountability, sustainability, and resilience, can inflate measures of success—often at the expense of long-term, sustained, truly effective development. And as recent studies have highlighted, failing to evaluate for realities on the ground and key weaknesses identified in past development interventions, very significantly weakens the chance of development success.

Developing countries now also seek to decrease their aid dependency. As highlighted in statements at key forums such as the Fourth High Level meeting on Development Effectiveness in Busan, developing countries now more than ever insist on the need to direct their own development efforts, including referring to the many diverse models of successful development available worldwide. This trend has been accompanied by a growing indigenous focus on evaluation. In this context it is likely inevitable that development imperatives will shift toward perspectives such as those most recently articulated by leading Korean economist Ha-Joon Chang, who argues that a country can be called developed only if its high income is based on superior knowledge embodied in technologies and institutions. Interventions that focus on individuals and their small, fragmented enterprises may provide some building blocks but hardly facilitate development at national level, instead exacerbating the micro–macro disconnect that haunts development evaluation practice. Sustained development requires effective, efficient institutions, and productive enterprises supported by the collective accumulation and use of knowledge, and the expansion of those social and technological capabilities that are both the causes and the consequences of such transformation. Yet although current primarily aid-driven models such as the human development approach remind us that development has to be about more than poverty reduction, increasing income levels, or the provision of basic needs, these and other key global development discourses such as the Millennium Development Goals, the Doha Development Agenda, and the World Trade Organization discussions fail to address some of the most important components for national development. All of this has important implications for the evolution of the field of evaluation as it moves in synergy with development trends.

Strengthening the Development Evaluation Agenda

It is beyond the scope of this article to provide a detailed analysis of the development–evaluation interface. Therefore, the following only highlights a few important priorities for frontier work in development evaluation. First, more ground-breaking work is needed to bring to the forefront non-Western worldviews and values in evaluation theory and practice. The profession is poorer for the absence of a concerted effort in this regard. Second, an increasingly sophisticated understanding is required for urgent priorities that depend on complexity and systems thinking in a highly networked, competitive world—for example, evaluating impact, sustainability, transformation, and resilience, or individual, organizational, and institutional empowerment. We need to see interventions as complex adaptive systems that cannot be broken down into a sum of their parts. This implies, among many others, acquiring a clearer multidisciplinary understanding of change trajectories, interlinked systems or “theories of change,” and the many different types of relationships operating between entities. It will bring more robust attention to issues such as the dire need for experimenting and adjusting as lessons are learned in near real-time, the “micro-macro disconnect” in development, the “missing middle” in theories of change, and the unintended consequences of development interventions.

Third, alternative financing and funding models are poised to complement and even overtake the role of conventional aid mechanisms. Several types of investment by Brazil, Russia, India, China, and South Africa (known as the BRICS) and other emerging players that serve to spur development are gaining momentum in Africa, Latin America, and Asia. At the same time the private sector in developed countries appears increasingly interested in investing in financing mechanisms with seductive names such as “impact investing” and “social impact bonds.” These mechanisms may put vulnerable societies at risk unless the evaluation profession is from the beginning equipped to help stakeholders plan and assess the benefits and risks, and in particular any negative consequences following from new funding modalities—or, for that matter from any development model or strategy.

Fourth, the pendulum needs to move back from enthrallment with simplistic notions of “measuring impact” and determining “value for money” toward enabling—in parallel with these latter efforts—a smart engagement with managing for impact that ensures the capacities to achieve impacts in a sustained manner, and is based on the many lessons that have emerged from results-based management and other similar efforts. Fifth, there is a dire need to engage vigorously, in theory and in practice, with a better understanding and use of standards for evaluation quality, “rigor” and “credible evidence” that transcend incorrectly or too narrowly defined ideas of the “scientific method” and so-called magic bullets for measuring impact. Finally, credible, useful syntheses of evaluation results and lessons should be available and communicated in a manner that can truly support different worldviews of development in theory and practice.

Evaluation Thought Leadership for Development

It is time that thought leadership in evaluation theory and practice emerges more visibly from the global South and East. Champions are needed who have a propensity toward conventional as well as new indigenous evaluation paradigms. Individual sparks in developing countries need to be stoked, so that ideas can spread and ignite meaningful innovation and new directions in evaluation. There has been significant progress in building indigenous evaluation capacities and recent global efforts such as EvalPartners provide scope for much more. However, capacity strengthening efforts tend to focus on technical aspects of evaluation within established approaches and frameworks. Although welcome and essential contributions, they may not encourage or stimulate deeper questioning of these and alternative approaches and frameworks. In most developing countries, the profession is barely a decade old. These countries have rich cultures with knowledge and wisdom spanning thousands of years—often as relevant today as ever—that have yet to be applied to the field of evaluation.

This is not about “cultural sensitivity,” but rather about the fundamental questioning of worldviews, frameworks, and definitions on which evaluation theory and practice—and resultant development—have been built. The potential for new theories and practices that might revolutionize development evaluation is not yet quite clear, but fledgling efforts need to be harnessed and nurtured. The knowledge and wisdom of the rest of the world needs to complement the 50 years of advancements in the West that have established and evolved the rich body of knowledge and expertise on which we draw today.

The explosion in the development evaluation profession has attracted many poorly prepared practitioners from both developed and developing countries. There is still significant scope to engage some of the finest minds from diverse disciplines and sectors in the profession in developing countries. This will require governments and other influential entities to recognize and demand high-quality evaluation, and perceive it as a strategic, intellectually challenging endeavor that has to be informed and advanced by high-quality academic research on evaluation. More importantly, thought leadership from the global South and East needs to have the power to change practice. This means reaching and influencing evaluation commissioners, those who work with aid programs and those who work in-country with development strategies and funding modalities. Such power is still limited by capacities; innovation often takes place when what exists has been mastered. Insufficient confidence and incentives are still hampering progress. Papers and presentations are too few, with too little traction to cultivate sustained influence. Thought leadership from developing countries needs to bring about high-quality and useful analyses and innovations benefiting their own countries’ priorities and contexts; establish repositories with useful syntheses; build influential coalitions and think tanks; ensure dynamic participation at important development forums, and have a robust focus on visibility. From small beginnings, thought leaders in developing countries have to nourish and accelerate the positive trajectory of the evaluation profession worldwide.