Evaluation

Nowadays there are a number of evaluation approaches that specifically focus on the elicitation of stakeholder values. Multicriteria decision aid enables evaluators to go one step further by systematically showing how different stakeholder values would affect evaluative outcomes and subsequent policy decisions about a programme's future. This type of technique can greatly benefit from being embedded in a programme theory evaluation methodology. Taking into account lessons from the literature on stakeholder values in evaluation, a methodological framework combining elements of programme theory evaluation and multicriteria decision aid is presented. The framework is illustrated by means of an example.

This article investigates the use of collaborative action research in both the development and evaluation of a project designed to develop inclusive practice in Early Years and childcare settings.The purpose of the evaluation was to ascertain how practitioners understand the term ‘inclusion’, how those understandings were translated into practice and how changes in that practice might be conceptualized and carried out.The article explores the use of facilitated collaborative action research as a core element in developmental evaluation. Reflecting on other approaches such as theory of change, it considers whether the use of action research supported the critical examination and development of a project, or whether, by being so similar in design, it constricted the evaluation in terms of reliability of data, accountability and providing a framework for planning and development.

This article discusses the potential role that the creation of evaluation standards or guidelines can play in developing an evaluation culture in Spain. Rather than focus on the ethical function of these standards or guidelines, this article discusses how the process of creating such codes, whenever this process is participatory and based on real evaluation practice, can help the academic and professional consolidation and institutionalization of the evaluation function. First, the different types of existing standards are discussed. Second, the general functions that they can play are elaborated, presenting the potential benefits of creating evaluation standards in Spain and highlighting key issues. In this regard, two important factors are identified: the participation of evaluation stakeholders in the standards elaboration process, and the need for meta-evaluations or systematic studies on actual evaluation practice.

Efforts to promote ‘sustainable development’ underline a need for policy integration. Applying benchmarks on ‘environmental policy integration’ (EPI) developed by Lafferty, the article illustrates the challenge with reference to Norwegian governmental initiatives to promote a national innovation policy. The case is analysed with respect to four stylized modes (standards) for promoting EPI: environmental protection, ecological modernization, ecological communalism and sustainable development. Despite a strong mandate for policy integration for sustainable development, and efforts to promote innovation within the European Union, public policy efforts in the direction of green innovation are practically non-existent in Norway. This does not necessarily mean that green innovations are not being promoted. Whatever integration effects that are being realized with respect to the four types of standards are, however, not the result of an active and goal-directed policy by the Norwegian government. The evaluation provides a differentiated platform for more specific initiatives in the area.

Sustainable development (SD) is now a broad policy objective in the European Union (EU). This article explores how the contribution to SD of the EU Structural Funds (SFs) might be evaluated. It suggests the use of a ‘four-capitals model’ of SD, and outlines how commonly accepted principles and dimensions of sustainability might be interpreted within such a model. The article then reviews various approaches to SD indicators, concluding that an indicator framework is preferable to the various methods of aggregation of indicators that have been proposed. It then relates the indicator issue to the framework for evaluation of the SFs set out by the European Commission. Out of this discussion an illustrative indicator set for such an evaluation, related to the four-capitals model of SD, is derived.

Based on experiences from two evaluation projects, the two partners in the following dialogue discuss the meaning of ‘resistance’ to evaluation, or more specifically, resistance to evaluation touching upon values that are central in some communities. Step by step, the dialogue identifies a tension between evaluation as a modernist undertaking and the traditional values of these communities. Through the dialogue about this tension, the ‘rough ground’ in evaluation is revealed, then the implications for evaluation practice are discussed.