Abstract
Current discussion of the professionalization of the practice of evaluation is focused on matters of competencies and credentialing and largely overlooks several significant issues. These include conflating professionalizing the practice with promoting the value of evaluation as a social good, assuming there is widespread agreement on what the terms profession and professional mean, and not considering how processes of professionalization are affected by the incorporation of evaluative activities in systems of managerial control. Most glaring is the absence of vigorous discussion of what professionalism means in evaluation and what the profession itself aims to add to society, or the social good it seeks to serve. In a word, there is little discussion of the professional ethos of evaluation.
The conversation about professionalizing the occupation of evaluation is moving energetically across the globe in evaluation associations and societies. Professionalization refers to the process by which an occupation becomes a profession, and achieving that aim, at present, appears to be largely tied to efforts to identify and describe evaluation competencies, develop training opportunities based on those competencies, and create and implement means of certification and credentialing that can be used to attest to the possession of such competencies. Evidence of the global concern with professionalization of evaluation is most apparent in the EvalPartners/International Organization for Cooperation in Evaluation (IOCE) initiative as described in the Global Evaluation Agenda 2016–2020. IOCE established a working group on the topic in 2016 that defined professionalization as “a gradual, long term, context dependent process” resting on six pillars, including (1) improved access to quality education and training; (2) dissemination of evaluation knowledge and good practices; (3) harmonization of ethical guidelines and guiding principles for evaluators; (4) agreement on evaluator capability or competency frameworks; (5) recognition of the fundamental knowledge, skills, and dispositions needed to carry out work to an adequate standard of quality; and (6) the development of an enabling environment for evaluation including appropriate institutional structures that support an evaluation function (EvalPartners, 2016, p. 83).
The United Nations Evaluation Group (UNEG) has also been investigating issues related to professionalization of evaluation specifically as those matters impact the work of evaluation offices in the United Nations. UNEG (2016a, 2016b) has recently updated the technical competency framework for evaluators working in the UN as well as its Norms and Standards document (which now includes professionalism as a norm). Individual evaluation societies and associations—most recently, the European Evaluation Society, the Aotearoa New Zealand Association, and the International Development Evaluation Association—as well as other significant voices in the field including Carolyn Heider, Director General of the Independent Evaluation Group of the World Bank—have weighed in on the conversation.
In this brief essay, I comment on this professionalization movement, not, however, by arguing for or against the idea, but rather by pointing out several underexamined or overlooked issues. I point first to an obvious conflation of efforts to professionalize the practice with attempts to enhance the legitimacy and value of evaluation as a social good. Then I offer for consideration the observation that other significant topics related to defining a profession and who is a professional simply do not appear in the current conversation about professionalization of evaluation. All of that is a warm up for the main focus of the article; namely, that what is most disquieting in the conversation is the lack of vigorous discussion of a professional ethos for evaluation.
One of the first things one notices about the conversation, especially as it is framed by IOCE and EvalPartners, is that the notion of professionalizing evaluation extends beyond efforts to establish the jurisdiction and control of evaluation by its practitioners; that is, beyond what are essentially supply-side concerns. In IOCE’s perspective, professionalization also requires attending to the demand side by building national evaluation capacity and improving commissioners’ and other users’ (e.g., citizens) understanding of evaluation and encouraging them to place greater value on evaluation knowledge as a public good. Thus, for the IOCE (and presumably its member organizations), professionalization of evaluation practice is part of a larger project that “entails not only an occupational group appropriating a field or discipline as its exclusive area of jurisdiction and expertise but also the making of this field as a legitimate and valid area of knowledge and intervention on the world” (Fournier, 1997 cited in Mclaughlin & Webster, 1998, p. 784).
Yet, it is a mistake to conflate the broad aim of cultivating within society a strong sense of evaluation as a social good with the professionalization of evaluation as an occupation. What is required to achieve the latter is not necessarily what is required to achieve the former. Enhancing the preparation and training of evaluators, controlling access to the profession through means of credentialing or certification, and so forth, will not ensure that society values evaluation as a public good. Training evaluators for the purpose of improving their practice is not the same as the educating citizens in the value of critical reason. Moreover, building national evaluation capacity by encouraging the development of national evaluation policies, the creation of monitoring and evaluation systems, and the formation of more voluntary organizations of professional evaluators (VOPEs) ought to be the outgrowth of, not the impetus for, increasing social awareness of evaluation as a public good. If evaluative reasoning in any given society is neither well understood nor valued and if citizens have little capacity, interest, or opportunity to engage in critical reflection on social aims, then the building of VOPEs and the training and credentialing of evaluators is a bit like putting the cart before the horse.
Three different kinds of problems are also easily noticeable in the professionalization discussion (publicly available documents, blogs, listserv comments, etc.). First, it appears there is at least a tacit assumption of a shared understanding of the term professional and a shared focus on what professional jurisdiction and control of work means across cultures and societies. IOCE’s interest in harmonizing ethical principles, standards, and competencies across the globe would necessarily have to assume such commonality. However, the theory and history of the professions and the meaning of the term professional varies between Anglo-American and Continental perspectives (Collins, 1990; Sciulli, 2005). Furthermore, compared to the Anglo-American interest in the freedom of self-employed professionals to control the terms of their work, in Scandinavian countries, only a small minority of professionals are self-employed, and the professions are largely closely linked to the state (Jensen, Lahn, & Nerland, 2012).
Second, for those who study the professions, it is increasingly clear that the term profession is not a fixed concept but reflects a “fluid set of institutional arrangements” that include the specific jurisdictional relationships within professions and between professions and the state as well as with employers, clients, and the country within which professions are situated (Dent, Bourgeault, Denis, & Kuhlmann, 2016, p. 6). This suggests the need for far more empirical work on what the notion of an evaluation profession and an evaluation professional actually means in varying sociocultural contexts.
Third, also apparently absent in the current conversation is awareness that the grounds for understanding how professionalization is to be realized in the practice of evaluation are shifting. The traditional picture is one of evaluators as autonomous expert agents working in a collegial manner with fellow professionals, responsible for organizing and regulating their own work, exercising discretionary decision-making, and so forth. This ideal is being challenged by the ways in which evaluation practice has been and continues to be incorporated into managerialist philosophies and embedded in bureaucratic and administrative practices and standardized procedures for monitoring and performance assessment (Dahler-Larsen, 2012; Furubo & Vestman, 2011). As Taylor (2005, p. 604) has argued, evaluation has become part of a “wider logic of decentralized control” in public agencies that actually serves to deprofessionalize an expert practice.
There is an even more troubling omission in the current global conversation. The focus on matters of professionalization in evaluation is almost exclusively on competencies and establishing an agreed-upon knowledge base. What is missing is discussion of what professionalism means in evaluation and what the profession itself aims to add to society or the social good it seeks to serve. Certainly—as is quite evident for example, in the UNEG Norms and Standards—reference is often made to professionalism as manifest in the practitioner’s adherence to ethical guidelines and practice standards. And, without a doubt, there are many general references to the claim that evaluation contributes to “social betterment” (Mark, Henry, & Julnes, 2000) or the “public good” (Greene, 2005). Yet, such efforts are a weak defense of what the role of expert evaluators as civic actors in society should be; and these efforts do not speak directly to the relationship between professional evaluation expertise and citizen involvement in evaluation.
Professionalism is not simply about the attitudes and actions of individuals but also about the profession as a whole and in that way we can speak of a professional ethos. This notion captures the sum of a professional group’s moral principles, core values, epistemic and aesthetic dispositions, and aspirations that each member of the group takes into consideration in interacting with others in a professional context. Ruiz (2015, p. 201) argues that an ethos is an “internalized set of values and behaviors…a moral as well as a political phenomenon encompassing discursive and deliberative interactions, manifest goals and objectives, along with latent meanings, nonverbalized habits, aspirations, and even anxieties.” As Sullivan (1990, p. 191) has noted based on his study of the professions, “it is ethos that guides professional action more than any discreet principles or rules.”
Ethos refers to the group’s distinguishing moral character along with its sense of the civic purpose it serves. For example, a professional ethos has been defined in the physician charter adopted in 2002 by the American Board of Internal Medicine (ABIM) Foundation, the American College of Physicians Foundation, and the European Federation of Internal Medicine. The charter embraces these moral principles: the primacy of patient welfare (not to be compromised by market forces, societal pressures, and administrative exigencies), respect for patient autonomy (patients’ decisions about their care must be paramount), and promotion of social justice in the health-care system (ABIM Foundation, 2002). 1
The critical question for the global community of evaluators is how that community understands its role with respect to its clients and society more generally. This is a concern of growing importance, given that in many expert knowledge occupations, including evaluation, there is a significant tension between professionalism as a calling to a service with a moral purpose and professionalism as the competent exercise of technical (craft) skills for which one is compensated (Pittman, 2012; Schwandt, 2015; Sullivan, 2004). Craft skill and moral aspirations are not, of course, mutually exclusive, and strength of craft can be considered indispensable to realizing a strong sense of social responsibility (Brint, 2015). We would clearly prefer a skilled socially responsible practitioner to an unskilled one. Yet, technical professionalism in some form seems to be on the rise in evaluation. It is apparent in the evidence-based movement, in an increasingly technocratic and tool kit approach to evaluation that diminishes the critical voice of evaluation (Schwandt, 2008) and exacerbates the depoliticalization of political decision-making, and in the seemingly unrelenting concern with competencies and technical training among evaluation associations and societies across the globe. In his discussion of the ethical consequences of managed care and the concomitant rise of managerial (vs. medical) professionalism in medicine, Buchanan (2003) argues that a commitment to excellence in technique (excellence in the performance of the distinctive activity of a person in a particular occupational role) often excludes any direct reference to the good of clients, consumers, and constituents that is to be served by the exercise of technique, that is, a commitment to service. One can reasonably ask whether the competency movement in evaluation is evidence of primarily a commitment to excellence in technique.
Allow me to anticipate an objection here. It may be argued that a commitment to service is already well established in the field of evaluation. Support for this view might be forthcoming from standards and codes of ethics adopted by evaluation organizations. Here are a few examples:
UKES (United Kingdom Evaluation Society) Guidelines for Good Practice in Evaluation: Evaluators should “be prepared to argue the case for the public right to know in evaluation in specified context.”
Code of Ethics of the Australasian Evaluation Society: “Members should consider the interests of the full range of stakeholders in their evaluation work, including the broader public interest, and in particular, the potential impacts of differences and inequalities in society.”
African Evaluation Guidelines—Standards and Norms of the African Evaluation Association: “The evaluation should be designed to efficiently meet the needs of all target stakeholders. The evaluation should be designed and conducted in compliance with rights and moral and physical integrity of stakeholders and their community…. Evaluators should respect the dignity and the human value in their interaction with people involved in the evaluation in such a way these people do not feel threatened or harmed physically or culturally or in their religious beliefs.”
Guiding Principles of the American Evaluation Association: The principle of “Responsibilities for General and Public Welfare”—Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation), include relevant perspectives and interests of the full range of stakeholders when planning and reporting evaluations, and acknowledge “obligations that encompass the public interest and good.”
Aotearoa/New Zealand Evaluation Standards: One of its four guiding principles, the Ethic of Care: “Kindness, respect, humanity and reciprocity are extended to all involved. Care is taken to ensure that the dignity of everyone is enhanced. Indigenous and other cultural worldviews, concepts and protocols are valued. Cultural sensitivity, safety and inclusion occur. This also applies to ethnic, religious, gender, disability, age, sexual orientation and any other ways of being. Individual and collective rights and protections, including privacy and confidentiality are negotiated.”
These, and similar statements, point in various ways to the noteworthy ethical aim of respectfully attending to stakeholder perspectives and interests in the conduct of evaluation. Yet, these statements have little to say about the public good that is to be served by these commitments to accounting for and portraying stakeholder understandings. Nor do they offer a clear sense of how the field of evaluation views the agency of ordinary people in situations where they (as stakeholders with various roles and stakes) are affected by evaluation; where agency is understood as the capacity of communities and groups to act cooperatively and collectively on common problems across differences in viewpoints, perspectives, and interests.
Admittedly, decades of discussion in political philosophy testify to the fact that defining what the public good is or what is in the public interest are not simple matters. But the field of evaluation does not need to enter that extensive debate. It can address the issue of what public good it serves by answering the question, “What role is to be played by civic actors in evaluation practice?” Jacob (2011, p. 69) points to an important distinction in this regard between citizens regarded as consumers of public goods or as partners invested in the production and management of public goods, with this divergence partially explaining the gap between the approaches of “evaluation for the people and evaluation with the people.” At issue is whether civic actors (all relevant stakeholders in various roles and viewpoints) are, essentially, required actors in evaluation processes directed at the public good or whether they are consumers of the knowledge produced by evaluation experts (even if those experts take into account, in more or less participatory ways, the perspectives, values, and interests of those in various stakeholding roles).
The distinction between citizens as required actors in or consumers of evaluation has considerable merit as a focal point for debating the professional ethos of evaluation. It resonates with a distinction that Datta (2011) drew between the public interest and democratic populist political stances in evaluation. The former focused primarily on the provision of unbiased, fair information to consumers; the latter more intimately tied to the concept of citizens actively engaged in deliberative practices. However, to reiterate, the debate in question is about professional ethos not methodology or political stances; it is a matter of what moral principles, core values, epistemic and aesthetic dispositions, and aspirations shape the ways in which evaluators interact with (and see their obligations to) others in a professional context.
My personal view is that the professional ethos of evaluation ought to be grounded in a rationale for citizens as required actors in evaluation. Support for that way of thinking can be found in the literature on democratic (or civic) professionalism (e.g., Dzur, 2008; Sullivan, 2004) as well as in the literature on civic agency and public work (e.g., Boyte, 2007, 2014; Ostrom, 1999, 2010). Martin Reynolds and I are in the process of explicating a justification for evaluation as public work that combines insights from these literatures with ideas from critical system heuristics and the literature on knowledge utilization. For present purposes, I simply sketch a few key notions here.
In the perspective of democratic professionalism, the people who the professional serves are not regarded as clients or consumers but as citizens with a stake in a professional decision. While the professional exercises the responsibility that comes with having expertise, he or she also shares authority and knowledge with citizens. As Kreber (2016) explains, the function of professionals in society in this perspective is: To provide opportunities or spaces for the public to debate issues of concern, explore solutions, and make decisions together…professional expertise is there to inform and help the public understand the issues more deeply…and appreciate why making decisions about these issues is often complex, and perhaps also why certain decisions that were made in the past did not have desired consequences. (p. 113)
An ethos of democratic professionalism for evaluation evokes parallels with forms of participatory evaluation that are genuinely collaborative, share authority and responsibility for decisions, and promote an ethical (vs. pragmatic) justification for such work. Yet, democratic professionalism is not grounded in romanticized ideas of empowerment. Rather, it issues from deliberative democratic theory and a conception of democracy as a way of life (not just a set of formal political arrangements). In this sense, it shares something with the approach to deliberative democratic evaluation. Yet, even the deliberative democratic approach in evaluation still strongly emphasizes the central role of the evaluator and that the evaluator is responsible for representing the voices of the least heard or the least well off in society. Democratic professionalism as manifest in evaluation does not assume an ideological, emancipatory commitment on the part of the evaluator or an a priori advocacy stance in favor of the particular ways of thinking and acting of some specific group of citizens or stakeholders. What it represents is a way of working with evaluative knowledge in everyday situations of debate and decision-making. Such a way of working is emancipatory only in the sense that, and to the degree which, it enables evaluators and citizens to engage in learning, solution questioning, and successfully arguing their concerns and addressing their problems in a variety of local, regional, and national arenas.
The intent of this brief essay is to raise the prospect that the contemporary discussion of professionalization in evaluation is too narrowly focused on matters of competencies, evaluator training, and evaluation capacity building. What is missing is examination of whether the field of evaluation as a specialized knowledge occupation fully understands its own professional ethos and, consequently, whether that ought to be debated more publicly than is apparent at present. This kind of examination matters because the field has not yet articulated and justified in a coherent manner the common good to be served by the exercise of evaluation expertise and how citizen participation in decisions affecting their well-being and expert evaluation knowledge are related. To be clear, absent discussion of more weighty questions of ethos and the public role for evaluation, it is not the case that the field will somehow become dysfunctional. Rather, the field will simply operate with a rather ethically thin and narrow conception of itself as a technical occupation (as is currently mirrored in the equally “thin” discussion of professionalization of evaluation confined to issues of competencies and credentials). This examination also matters because a vision of evaluation as largely the exercise of technical expertise hardly measures up to EvalPartners’s promotion of the field as contributing worldwide to “a transformed global community characterized by transparency, accountability, and progress towards the common good” (EvalPartners, 2016, p. 3).
It may well be that current concerns with competencies, credentialing, and certification are all that matters to the majority of practitioners in the field. Perhaps the field assumes that the shared fundamental culture of the practice of evaluation has long since been established and resides in something like a scientific ethos where the contribution of professional evaluation to society is framed in terms of principles, values, dispositions, and aspirations associated with the expert, scientific critical appraisal of whether and to what extent a policy, program, or project works (or does not work) for whom, and (possibly) why and how. If so, then there is no more to be said. The field should then accelerate efforts to achieve technical professionalism and a commitment to excellence in technique. However, if the field is perplexed by what actually comprises its shared understanding of moral principles, values, aspirations, and ways of behaving, then we need a much more public and energetic discussion of the professional ethos of evaluation.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.