Evaluating the environment as a global public good

The environment as global commons

Global environmental resources are public goods that are non-excludable and non-rivalrous across borders. For example, no individual or group can be prevented (excluded) from consuming or using the atmosphere. Furthermore, clean air does not benefit one nation at the expense of others, so countries are not rivals when it comes to consuming these goods. The global environment therefore falls squarely into this category.

Global environment problems are those that many countries have contributed to and no individual country can effectively address by acting alone. Examples of this include the loss of biological diversity, the spread of persistent organic pollutants in the environment and global climate change. While at the local level environmental conditions have generally been improved by economic development, the same has contributed to the worsening of global environmental problems: as we clean our surroundings, we add CO₂ and other pollutants to the atmosphere, cut down forests to make way for plantations and cattle ranches, and pave over natural habitats for urban and transportation infrastructure. Furthermore, the impacts of global environmental problems are not equally distributed and it is often poor countries – which have contributed little to global environmental change – which bear the brunt of its impacts. Climate change, for example, is likely to raise sea levels, threatening island economies, like the Maldives and the Pacific small islands, and low-lying countries such as Bangladesh. Climate change also jeopardizes agricultural production in developing countries. For example, the overall impact of a doubling of CO₂ in the atmosphere would be to reduce the gross domestic product (GDP) of developing countries by an estimated 2–9 percent (compared with 1.0–1.5 percent of GDP in industrial economies). Within developing countries, the poorest are most likely to be impacted unless mitigation and resilience efforts are stepped up. And because of the concentration of biodiverse areas in developing countries, failure to preserve biodiversity would also have greater adverse impacts on poorer nations. ¹

By removing or reforming subsidies, fostering markets, and confirming property rights, countries acting alone can improve their own environments. But even if national-level environmental concerns are addressed, global market failures call for international intervention as individual countries do not have sufficient incentives to address them as they cannot capture all the rewards of doing so. As a consequence, science shows that we are nearing the planetary boundaries – and in some cases already breaching them – when it comes to the global environment. ²

The Global Environment Facility (GEF) was set up to address global environmental issues and enhance global environmental benefits. GEF investments are predicated on the delivery of global environmental benefits in biodiversity conservation and sustainable use, climate change mitigation, protection of international waters, land degradation and sustainable forest management, and management of chemicals. Increasingly, GEF is seeking to deliver multiple environmental benefits through integrated investments across the various dimensions of the global environment. The GEF leverages its resources through co-financing and cooperation with other donor groups and the private sector. The intervention logic of the GEF is to achieve impact through a catalytic effect. The GEF projects directly address knowledge and information, implementation strategies, and institutional capacity, which in turn are intended to influence broader adoption and behavioural change in the countries, eventually leading to reduction of stress and actual improvements in the environment (Figure 1) (GEF IEO, 2014).

OPEN IN VIEWER

Figure 1.

General framework for GEF theory of change.

The GEF Independent Evaluation Office (IEO) provides evidence on the performance of the GEF portfolio and insights into what works, where, under what conditions, and why. Through a wide range of evaluations – impact, thematic, corporate, and country portfolio evaluations – the IEO has developed advanced methodologies and approaches to measure the results and impacts of the GEF portfolio and provide recommendations to improve program and policy design and implementation.

Country-level evidence from evaluations shows that GEF support has made significant contributions to institutional strengthening for environmental management. GEF support has assisted countries in determining their environmental priorities and developing and implementing national environmental policies and strategies; this has to a large extent been accomplished through enabling activities. The country environmental legal framework analyses conducted in country-level evaluations largely confirmed that GEF projects have supported national frameworks for developing environmental laws and policies in areas such as biodiversity conservation and sustainable use, biosafety, climate change mitigation, and management of persistent organic pollutants. GEF support in fulfilling countries’ reporting obligations to international environmental conventions has been relevant as well. In general, GEF support provides funding to develop national priorities (e.g. through prioritization and inventory exercises funded by enabling activities), to implement an already established national priority, or for application within an existing framework.

Measuring the actual results and impact of GEF programs and projects on the global environment is more challenging and requires different and novel approaches. But the frontier keeps advancing, with innovative solutions being developed by the IEO together with other evaluation units and research institutions.

Evaluating environment as a global public good

As global environmental problems persist, it is not adequate to only evaluate that individual projects are on track in achieving their stated objectives. This is particularly important because there is a micro–macro paradox that we have observed: while thousands of projects are mostly successful in achieving their objectives, the global environmental problems keep mounting (e.g. Berg, 2011). This is to a large extent because environmentally beneficial actions are overwhelmed by other societal forces that work against their goals, such as environmentally harmful subsidies to continued fossil fuel use and unsustainable agriculture (GEF IEO, 2014). Therefore, it is essential that evaluation also takes a holistic perspective to assess whether the interventions that we support are actually making a difference to the global environment and the lives of the people that depend on it (e.g. Uitto, 2014).

The IEO is paying special attention to moving beyond verifying project outputs and outcomes, and assessing progress towards achieving environmental outcomes and impacts by using a broader contextual approach to addressing the drivers of environmental issues. In addition to measuring environmental benefits, evaluations also seek to assess the co-benefits or socio-economic benefits of projects. The Office has also been developing methodologies on crosscutting issues such as gender, resilience, and stakeholder engagement which are now being mainstreamed into all evaluations, as evidence shows that these are essential for achieving lasting environmental benefits. With the recent GEF shift towards global programs that involve integrated approaches across countries, ³ evaluations will in the future focus on assessing the extent to which individual projects contribute to the objectives of the global programs, and the mechanisms by which these are achieved, i.e. through scaling up and replication of successful pilot initiatives.

To better deal with such integrated approaches and to gauge their impact, the IEO works to advance approaches and methodologies to evaluate global commons that will benefit the entire evaluation community. Evaluation findings and methods are shared broadly with the environment and development community so that evaluation contributes to finding sustainable solutions to the wicked problems facing the global environment. Our goal is to identify what works and under what circumstances, to advance knowledge to benefit the global environment. The Climate-Eval community of practice ⁴ set up by the IEO with donor support in 2008 has been a demonstrably useful platform to share findings and improve measurement approaches.

The GEF Council has long demanded evidence of the actual impact of GEF programming on the global environment. In responding to this demand, IEO has now conducted three impact evaluations in major areas of GEF work. Most recently, the Impact Evaluation of the Effectiveness of GEF Support to Protected Areas and Protected Area Systems ⁵ piloted a mixed-methods approach utilizing a wide variety of approaches and tools to gauge the impact of more than two decades of GEF programming in the area. The evaluation utilized remote sensing and geographical information system mapping to track forest cover change in GEF-supported and other protected areas in same biomes, as well as time series data on species abundance to track changes in relation to GEF interventions. Techniques such as propensity score matching were used where data allowed. These global analyses were supplemented with more traditional approaches, such as field visits and interviews with stakeholders. The evaluation demonstrated GEF’s contributions to slowing the rate of deforestation and species decline in protected areas.

The Impact Evaluation of Climate Change Mitigation highlighted GEF’s contributions to significant direct greenhouse gas emission reduction and broader adoption of technologies in China, India, Mexico, and Russia. The Impact Evaluation of GEF International Waters Support to the South China Sea and Adjacent Areas piloted the application of the systems approach to evaluations involving a broader assessment of how GEF provides support for activities that directly or indirectly address drivers that are expected to affect environmental degradation.

How evaluation influences policy in the GEF

Evaluation utility and influence is a perennial question that many evaluators and evaluation scholars have spent much time thinking about, and there are no magic solutions. ⁶ However, institutional mechanisms can be useful in this regard. There are several structural and process-related elements that enhance the influence of evaluations conducted by the IEO. For example, the GEF institutional structure ensures independence of the IEO. This, among other things, enhances the credibility of the evaluations conducted by the Office and enables it to provide independent feedback to the GEF partnership. The IEO reports on its evaluations directly to the GEF Council, which discusses the evaluation findings, conclusions, and the way forward. There is a management action record (MAR) on how the GEF Secretariat and agencies deal with the agreed recommendations. Yet, despite its independent status, the IEO takes a consultative approach to developing its work program and throughout the evaluation process with the intent of generating support from the relevant stakeholders.

Evaluations often influence GEF policy on process issues and on how the GEF does things. For example, the Mid-term Evaluation on System for Transparent Allocation of Resources ⁷ and Evaluation of the Accreditation Process for Expansion of the GEF Partnership ⁸ informed GEF policies on the allocation of resources and the expansion of the agency network, respectively. Evaluations led to substantial changes in the GEF approach to allocation of resources to recipient countries, including changes in the allocation formula and policy related flexible use of resources by recipient countries.

Apart from the process issues, the impact evaluations mentioned earlier were received by the Council with great interest, generating significant debate on the way forward in addressing the different sets of global environmental issues. They also helped clarify the theory of change embedded in GEF programming regarding how it aims to generate global environmental benefits.

Gender is an important dimension in the GEF theory of change, as it is one of the main avenues through which to achieve behavioural change that will lead to broader adoption of sustainable solutions to global environmental problems. To pay more attention to social and gender issues is thus neither a luxury nor an add-on, but a core element of the causal pathways to reduction of environmental stress – and eventually to environmental improvements. The fourth Overall Performance Study of the GEF (OPS4) (GEF IEO, 2010) conducted by the Office assessed gender mainstreaming in the GEF and emphasized the need for a streamlined cross-cutting gender policy in GEF operations (Awori, 2009). OPS4 noted that “Social and gender issues in GEF strategies and projects are not addressed systematically, and the GEF cannot rely completely on the social and gender policies of its Agencies.” Later evaluations and reviews show that the GEF has made progress on gender mainstreaming. A policy on gender mainstreaming was developed and adopted, which clarified the commitment to and elements of gender mainstreaming at the GEF. A gender focal point was designated at the GEF Secretariat, and a regular gender review and monitoring of the portfolio has been conducted through the annual monitoring review process since 2011.

How evaluation contributes to GEF policy influence at country level

Country portfolio evaluations demonstrate how GEF support at the country level is aligned with national priorities, shows progress toward impact at the local level, and enables countries to meet their obligations to the multilateral environmental Conventions. Evidence shows that GEF support is mostly effective at the local level, but faces challenges with scaling-up. Several country portfolio evaluations reported project-level environmental and socioeconomic impact at the local scale – including those in Brazil, El Salvador, and Jamaica; Benin, Madagascar, and South Africa in biodiversity conservation; the nine countries that form the Organization of Eastern Caribbean States in all focal areas, except adaptation to climate change – but found a lack of upscaling. Country-level evidence in Brazil and Cuba, among others, also confirmed that the most common form of broader adoption is mainstreaming, generally in the form of information, lessons, or specific results of the GEF that are incorporated into broader stakeholder mandates and initiatives such as laws, policies, regulations, and programs. The findings of country portfolio evaluations are reported to the Council and also fed directly back to national authorities in the countries, thus providing two channels for feedback and learning.

Based on a recommendation included in a study of country-level evaluations (GEF IEO, 2008) the GEF Council requested the development of the concept of integrated multifocal area approaches, including for addressing transboundary issues. This decision led to an increase in multifocal area projects in GEF portfolios. Finally, the GEF M&E Policy of 2010 set minimum standards of GEF country Operational Focal Points’ involvement in evaluations, which in turn provides more direct feedback from evaluations to the policy-makers in the countries in question.

Factors that contribute to policy influence of evaluations

Like in all fields, evaluation of global environmental programs and policies provides primarily technical inputs to decisions, but policy-making obviously is more complex. Evaluation thus is one of the factors that contribute data and information to influence policy-making. There are a number of clear lessons from the GEF experience on how to make evaluations more influential. These include timeliness of evaluations, as well as interest and demand from policy-makers. When these conditions are met, it is clear that evaluations have more traction among stakeholders.

A recent knowledge needs assessment undertaken by the IEO demonstrated a high level of use (90% of respondents) of evaluations. The IEO evaluations that have been most influential have been timely and had high levels of stakeholder engagement. The fifth Overall Performance Study responded to direct demand from the GEF Council and subsequently informed the sixth GEF replenishment process, although some of the inputs could have been more timely. The annual performance reports provide ongoing insights into the performance (outcomes and impacts) of GEF projects. They are also developed in close cooperation with the evaluation offices of the GEF agencies thus feeding back directly to them.

Experience also shows that recommendations from evaluations are more palatable when they involve win–win situations and don’t challenge institutional interests. This obviously is not always possible. Especially in the field of global environmental protection, there are winners and losers, and trade-offs may be necessary between economic development and environmental objectives (see GEF IEO, 2006). Overall, however, stakeholder engagement, especially in developing recommendations from evaluations, remains a key factor in generating buy-in that will lead to stronger use and influence of the evaluations.