Evaluation

Evaluation literature has paid relatively little attention to the specific needs of evaluating large, complex industrial and infrastructure projects, often called ‘megaprojects’. The abundant megaproject governance literature, in turn, has largely focused on the so-called ‘megaproject pathologies’, i.e. the chronic budget overruns, and failure of such projects to keep to timetables and deliver the expected social and economic benefits. This article draws on these two strands of literature, identifies shortcomings, and suggests potential pathways towards an improved evaluation of megaprojects. To counterbalance the current overemphasis on relatively narrowly defined accountability as the main function of megaproject evaluation, and the narrow definition of project success in megaproject evaluation, the article argues that conceptualizing megaprojects as dynamic and evolving networks would provide a useful basis for the design of an evaluation approach better able to promote learning and to address the socio-economic aspects of megaprojects. A modified version of ‘network mapping’ is suggested as a possible framework for megaproject evaluation, with the exploration of the multiple accountability relationships as a central evaluation task, designed to reconcile learning and accountability as the central evaluation functions. The article highlights the role of evaluation as an ‘emergent’ property of spontaneous megaproject ‘governing’, and explores the challenges that this poses to the role of the evaluator.

This article presents a radically empirical and descriptive account of the most often neglected viewpoint: that of the ‘doer’. Using field autoethnographic examples from remote Northern Brazil, the article portrays the clash between the universalistic programs of international agencies and doer perspective at the fieldwork level. The see-judge-act tool helped to chart the heuristic field of daily life as a reflective practice. Three moments illustrate the confrontation of the action planned: (a) a projection of abstract theoretical assumptions based on a universalistic framework; (b) a silent period of accommodation of local realities to the imposed, descending logic of the formal discourse; and (c) the production of an exportable version of the partially hidden local situation. All three moments show the struggles to make the ‘doer viewpoint’ communicable beyond the local geographic region. In the end, fieldwork dangers were censored and inconsistencies were evened-out.

This article reports on a study that explored whether evidence can be found of a shared evaluation tradition among evaluation researchers and practitioners working in institutions in the Nordic countries. The study focused on articles in peer-reviewed, international, designated evaluation journals in the period 2000–12; it found little evidence from the analysis of these sources to support this claim. Meanwhile, the study found a clear preference of Nordic evaluators for publishing in European journals, with Sweden being the dominant source country in terms of number of publications, selection of journals in which they were published, and institutions and authors publishing the most.

Cross-cultural evaluation is an explicit approach to evaluation that considers culture a key consideration in the evaluation of programs, leading to the search for methodological practices that are commensurate with the culture, context and values of the program community. Through the exploration of three ‘problematics’, this article explores what it means to understand others and ourselves within the research context, problematizing the nature of identity and self-identity, as we begin to make sense of the multiple and often conflicting identities and categories of being that we confront both within ourselves and among our research participants in the field. How we represent our findings and how we understand the voices of others and ourselves in the text, while posing fundamental questions in any qualitative social science inquiry, becomes even more critical in cross-cultural research and evaluation.

This article aims at explaining why some Flemish (Belgian) organizations evaluate policy, while others do not. The study relies on a unique combination of two configurational comparative methods: the Most Similar Different Outcome/Most Different Similar Outcome method and crisp set Qualitative Comparative Analysis. This combination of methods helps us unravel the combinations of conditions that promote or impede policy evaluation activity in a public administration that recently underwent major changes in line with New Public Management (NPM). The analysis reveals that the impact of NPM reforms on evaluation activity should not be overestimated. The results suggest that the important conditions that explain policy evaluation activity or inactivity are the anchorage of the evaluation function, the availability of skilled personnel to undertake evaluations, evaluation demand from organizational management, and the measurability of the outputs and outcomes of the organization’s activities.

The purpose of this article is to explore the impact of performance audit on public policy. The article investigates performance audit reports and the debates they trigger in the public realm. The case of Norway is analysed using mixed methods including a questionnaire, mapping and categorizing of reports, document studies and interviews. The results show that the Norwegian Supreme Audit Institution (SAI) is primarily preoccupied with managerial issues. It is nevertheless open to interpretation whether the preoccupation with managerial issues primarily implies an efficiency and effectiveness focus or an assessment of compliance to managerial standards. Most reports receive moderate attention from the media and the parliamentary control committee. Therefore direct dialogue with the ministries appears to be important to ensure that performance audits are influential. In the public debate the SAI, the ministries and the members of parliament base their argumentation on different institutional logics. These logics can lead to different interpretations of the control system, laws and regulations and hamper the SAI’s influence.