Abstract
Issue 3 of the American Journal of Evaluation's Volume 44 has two parts. The first collects a set of articles that consider Policy, Politics, Principles, and Participation: Influences on Program Planning, Implementation, and Outcomes Achieved. The second is a Special Section dedicated to the topic of evaluator competencies and the professionalization of our field. A separate Special Section Editors’ Note contextualizes that section, with a guest editor joining us for greater insights. This From the Co-Editors synthesizes the contributions of the first half of the double issue. As always, it is our hope that AJE readers will find value in the collection and use the articles to improve their practice.
To begin, program implementation commonly involves adaptions to original program planning in response to changes in the program's or evaluation's context. These adaptations may not be reported, making it challenging to determine if the adaptations are aligned with the program's theory of change. In their article, Kate Nolt and Laura Leviton recommend a model—PRECEDE-PROCEED—to help evaluators and others involved in the program take a more active role in determining and guiding needed adaptions prior to program implementation. This article's guidance supports evaluators in tracking and reporting adaptations, which, the authors argue is contextually responsive and essential to determine program effectiveness.
Similar to the first article, the second article in this issue, Outcome Trajectory Evaluation (OTE): An Approach to Tackle Research-For-Development's Long-Causal-Chain Problem by Boru Douthwaite, Claudio Proietti, Vivian Polar, and Graham Thiele acknowledges the complex relationship between a program's theory of change, context, evaluative practice, and outcomes. The authors address issues resulting from the complexity of this relationship, particularly when evaluating international research for development (R4D) projects. The authors’ empirical work on R4D projects and their policy outcomes pushes us, as evaluators, to reconsider how a R4D project theory of change, implementation, and its history can be used to generate enhanced understanding of significant outcomes and achieving them over time.
Evaluation policies are purposefully designed to give guidance on evaluation practice. Knowing this, Isabelle Bourgeois and Stéphanie Maltais investigated the implementation of evaluation policy in the context of Canadian federal government organizations. As a result of their study, they offer nuanced depictions of how federal evaluation policies were implemented. The authors’ examination confirms not only how evaluation policy influences the character of government departments that direct evaluations and various aspects of evaluation practice but also how evaluation practice can influence evaluation policy.
Similar to policy, principles are designed to give guidance as well. The next two articles discuss how principles informed their health-related evaluation work. The first of these draws on Michael Quinn Patton's Utilization-Focused evaluation and aims to identify effective principles (EPs) to guide evaluations across a wide range of settings. Authors Erin Casey, Jan Vanslyke, Blair Beadnell, Tatiana Masters, and Kirstin McFarland describe the process of generating EPs to guide the implementation of activities of programs that seek to prevent sexual violence. This article will be of particular interest to those in the social service fields who desire to connect program practices to relevant principles and engage in collaborative, consensus decision-making evaluation approaches.
In the second of these two principles-attentive articles, Lynda Berends and Horace Wansbrough draw on empowerment and realist evaluation theories to develop principles that enable youth to participate in evaluation research on programs designed to address family youth alcohol and other drug programs. The authors persuasively argue that the principles have potential to guide collaborative evaluation approaches that advances positive, transformative change for communities. Like other articles in this issue, this article highlights the importance of evaluation planning, participation at multiple levels (personal, organizational), and effective communication mechanisms. Distinct from the other articles, this article importantly raises the unique considerations when involving youth in evaluation research.
Next, collective impact (CI) initiatives apply a systems-level approach to address social and economic problems like sexual violence and recessions. CI initiatives are often inclusive of community-based interventions at multiple sites. Given their highly dynamic nature, assessing the implementation and outcomes of CI initiatives is a problem that plagues the field of evaluation. In response, a team of authors, Sonya Panjwani, Taylor Graves-Boswell, Whitney Garney, Daenuka Muraleetharan, Mandy Spadine, and Sara Flores, conducted a scoping review to glean insights on evaluating CI initiatives. Based on evidence from their scoping review, their practical recommendations aim to bring the field closer to more holistic, relevant, and robust CI evaluations that support community-specific outcomes.
Although a common enough phenomenon, issues of power dynamics in the conduct of evaluation—especially a collaborative evaluation approach—have garnered relatively less scholarly attention. For this reason, we are pleased to publish the work of Maria Alejandra Torres-Cuello, and Luis Arturo Pinzón-Salcedo who posit that more knowledge about how power functions in evaluation practice can enhance participatory evaluation efforts. The authors draw on Michel Foucault's Conception of Power-Knowledge to examine the power issues in collaborative program evaluations. The authors make clear that power-knowledge is critical to comprehending complex social relations, which provides a foundation for managing different community interests and decision-making through each phase of the evaluation.
In closing, it is our objective to publish a diverse set of articles in each issue AJE so that the breath of our readers find something engaging to them. We believe we have met this objective in the first half of this issue and hope readers will continue on to the second half of the issue where we shine a light on the issues of the professionalization of the evaluation field.
