pple effects provides insight into their links with the implementation object, that is, the intervention itself. This provides insights into how implementation strategies can influence the workings of an implementation object. Third, realist evaluation allows for an exploration of unintended and unexpected outcomes, which are often overlooked in traditional evaluation frameworks. This is crucial as unforeseen effects may shape the generation of both implementation and client outcomes. These arguments are illustrated through theoretical and empirical reflections drawn from realist evaluation studies.

28:T449,

The prime argument for independent evaluations is the avoidance of bias. Independence safeguards the evaluation process from undue influence and reinforces scientific rigour. Organisations often formalise this by separating evaluation from operations. Evaluation functions are less often located in separate organisations. This strong understanding of independence – functional and organisational – was adopted for evaluation of Sweden’s official development assistance, ODA. The Expert Group for Aid Studies, EBA, was set up after some 20 years testing out different evaluation models. How does such an organisation make itself useful and relevant to stakeholders? Does it support increased use of knowledge and ‘evidence’ in an inherently political, and increasingly politicised, policy area like ODA? We discuss how EBA is both instructed and self-governed to achieve relevance while also maintaining independence. A key argument is that formal arrangements for organisational independence have to be complemented by a shared ambition to preserve that independence in practice.

29:T49b,

The study of unintended effects of policies is a key debate among evaluation scholars. Through complexity theory, we argue that unintended effects of (international) public actions are inevitable and question the reliability of evaluations in providing a correct and complete picture of public policy. We use a machine-learning-assisted text-mining case study approach, examining 254 programme evaluations of German international development cooperation as a ‘least likely case’. While German evaluations focus more on unintended effects than Dutch, Norwegian and American evaluations, their treatment is not always correct or complete. There is an overidentification of unintended effects and a bias towards positive ones, with certain types of unintended effects overlooked. We explore explanations for the observed weaknesses, including an overreliance on linear thinking and insufficient guidance for the evaluators on identifying unintended effects. We conclude with concrete suggestions to improve implementation of the Organization for Economic Co-operation and Development guidelines that are essential to make public administration more effective and trusted.

2a:T464,

What do tools do for evaluators? An affordances framework sheds light on the multiple ways that evaluators interact with tools. “Affordances” are ways that tools enable and constrain action, but the ways that tools do so depend on underlying social conditions. I apply this framework to the case of the Organization for Economic Cooperation and Development Assistance Committee criteria, which are widely used in international development evaluation. Through interviews and document analysis, I find that the DAC criteria offer technical and social affordances—they enable and constrain ways that evaluators navigate evaluation’s technical tasks, but they also enable and constrain ways that evaluators interact with others and draw symbolic boundaries around evaluation expertise. I also find these affordances are shaped by social conditions, like an evaluator’s perceptions or position in the organization. These findings suggest the importance of assessing evaluation tools’ multiple dimensions in their social contexts to understand tools’ role in evaluation work.

2b:Tdb8,