Abstract
Audit and evaluation play crucial roles in the oversight function of organizations. Both professions share commonalities that have led some to call for “enhanced oversight” through stronger collaboration of the two functions. Our experience suggests that joint assessments can help promote a holistic three-dimensional view of performance, which looks both inside and outside organizations to see whether inputs matched with outputs and outcomes. However, tangible differences in paradigm and approach need discussion before further convergence of the functions is countenanced. The separation of the two functions ensures their independence, which is a guarantee of the credibility of the auditors and evaluators’ work.
Introduction
In an increasing number of countries and organizations, there is growing pressure for adequate scrutiny and demand for accountability, driven by social media, which highlights glaring performance deficits.
The causes of such deficits are diverse. In some cases, failures have arisen from non-compliance with rules and regulations that are central to good governance. In others, failures occurred because of slavish or unthinking application of rules or procedures which are not delivering results, or worse, are doing harm. And in other cases, failures arise from faulty policies, built on flawed assumptions that are never subject to the rigorous scrutiny that would expose their inability to produce results.
As key producers of evidence and promoters of accountability, evaluation and audit functions have important roles to play in addressing and preventing such failures. This has led some to ask about whether audit and evaluation work can or should work more collaboratively to improve accountability for results. There are several examples globally at all levels of contradictory results emerging from audit and evaluation, many of which have led to reputational damage.
This article draws on evidence from the evaluation and audit experiences within the United Nations Development Programme (UNDP), where audit is conducted by the Office of Audit and Investigations (OAI), and Evaluation is carried out by the Independent Evaluation Office (IEO), the largest evaluation office in the United Nations system. The experience from joint and independent assessments conducted by IEO and OAI showed that, although the two oversight functions preserve their identity, they could bring complementary skills and tools to the table to ensure a holistic performance assessment of UNDP’s contribution to development in terms of compliance, value for money, and impact on people.
Moreover, this article explores the potential risks related to the convergence of the two functions. Joint assessments also face significant challenges given differences in the standards, values, and perspectives of the two professional traditions. Despite the similarities, the differences between financial audit, performance audit, and evaluation need to be respected.
Audit and evaluation: Understanding the two traditions
Active discussion has been taken place in the literature on audit and evaluation functions and practices.
Researchers and institutions provide various definitions of audit. According to the International Organization for Standardization (ISO), audit is a “systematic, independent and documented process for obtaining objective evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled” (ISO 19011:2018, 2018).
In the public management literature, Mayne (2006) provides a general definition of audit as the action of checking, determining if result, activity, or statement meets predetermined criteria. As highlighted in both definitions, audit assessment relies on criteria widely established by the profession and tends to be characterized by compliance, respect of norms, and standards which represent an important tool for the auditors. Different types of audits have been identified in the literature, among which performance audit has become the most common. Indeed, Barzelay (1997) distinguished performance audit from traditional audit, which includes financial and compliance audits. The financial audit refers to the control of the internal financial system while compliance audit involves the examination of financial and budgetary operations with respect to the established rules and regulations (Mayne, 2006).
As for performance audit, more attention is given to the performance of the organization with respect to its objective rather than its capacity to follow the rules and regulations. Shand and Anand (1996) distinguished two types of performance auditing: substantive performance audits which pursue the objective of assessing efficiency and effectiveness of programs or organizations and the systems and procedure performance audits. According to the International Organization of Supreme Audit Institutions (INTOSAI), performance auditing is “an independent examination of the efficiency and effectiveness of government undertakings, programs or organizations, with due regard to economy, and the aim of leading to improvements.” Subsequently, performance auditing requires a high degree of judgment and auditors enjoy more flexibility in their choice of subjects, audit objects, methods, and opinions, compared with financial auditing which applies relatively fixed standards (INTOSAI, 2004).
Within organizations, performance and traditional audits are carried out both internally and externally. Internal audit is undertaken by auditors who are part of the institution while external audit is performed by auditors who are independent of the organization. The UNDP Charter defines Internal Audit as an independent, objective assurance and consulting activity designed to add value and improve the operations of UNDP. It helps UNDP accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of governance, risk management, and control processes.
Although this definition might refer to performance auditing, its focus has been more oriented to management performance than evaluation of program results and impact.
Regarding the evaluation function, several definitions can be found in the literature, related to its focus, purpose, and methods used. Vedung (2017) defined evaluation as a “careful retrospective assessment of the merit, worth, and value of administration, output, and outcome of government interventions, which is intended to play a role in future, practical actions situations.” This definition emphasizes the purpose and focus of the evaluation function driven by results measurement against the objective of the jurisdiction. At the institutional level, the United Nations Evaluation Group (2016) states that evaluation is an assessment, conducted as systematically and impartially as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area or institutional performance. It analyses the level of achievement of both expected and unexpected results by examining the results chain, processes, contextual factors and causality using appropriate criteria such as relevance, effectiveness, efficiency, impact and sustainability. An evaluation should provide credible, useful evidence-based information that enables the timely incorporation of its findings, recommendations and lessons into the decision-making processes of organizations and stakeholders.
Evaluation, thus, often examines processes on top of performance and results.
Critical to the definition is that evaluation makes a value judgment about past performance, intending to stimulate debate about what the future should be. It is at the level of making the judgment that there is often contestation—more than regarding the evaluation findings and recommendations—as this is a judgment rendered on the evaluand to its oversight or board, and, while these are judgments about a program, policy, or strategy, there is inevitably vested interest in such judgment.
Audit and evaluation: Commonalities and disparities
The audit and evaluation functions have been the subject of a number of papers and debates in the literature, focusing on the similarities and disparities between the two functions and the question of whether or not they should be integrated for costs efficiency and effectiveness of the organizations. Indeed, audit and evaluation play important roles in the oversight and accountability frameworks of organizations. They are both retrospective and follow systematic processes to describe and assess programs and practices. As performance auditing has become more common in the practices, this article focuses more on performance audit and less on traditional audit.
So, what can be considered similarities and what distinguishes each practice?
Commonalities
Defining the boundaries and identifying commonalities or disparities between performance audit and evaluation can be a rather challenging task considering the existing overlaps between both, but it is important to determine whether and how they can complement each other.
Evaluation and performance audit are considered established tools for promoting accountability and learning (Barrett, 2001; UNDP, 2016; UNDP-IEO, 2019). Both are forms of a systematic inquiry used to assess institutional performance, implementing consistent procedures or methods for collecting and analyzing information, which includes review of existent documents and statistical data, stakeholders’ surveys, and focus groups. They concentrate on gathering evidence to support judgments about the merit or worth of entities, programs, activities, and functions in terms of economy, efficiency, and effectiveness. Both practices are guided by a set of norms and standards (Mayne, 2006)—with shared values, but also specific features for each of them—and exercised by professionals with specific skills and experience (Chelimsky, 1996).
Similarities also exist in the methodological approaches used in the two disciplines. Pollitt and Summa (1997) noted that auditors and evaluators are getting closer to each other in the methods used, with a marginal difference in the tools available to them. The authors linked the closeness of the two disciplines with the broad adoption of performance auditing.
Disparities
Notwithstanding the similarities between audit and evaluation functions, there are significant differences that need to be recognized. As a discipline, audit has a much longer history with origins that, according to some authors, can be traced back to ancient civilizations, when it was used to keep records and verify public accounts. But the modern form of audit, began only with the Industrial Revolution, becoming progressively an extension of management. From its original scope, it has evolved and segmented in several types, depending on the interrelationship among participants—internal and external—and the purpose and focus—compliance, financial, operational, information system, and investigative (Barzelay, 1997; Chelimsky, 1996; Mayne, 2006).
While audit originated in the field of accounting, focusing on the accuracy of financial statements of a particular entity, evaluation has its roots in social sciences and only developed as a formal professional practice after the 1950s (Chelimsky, 1985). The evaluation tradition stands out from the audit tradition for its focus on causal inference in examining the impacts of policies and programs, including unintended consequences. The tradition of audit is narrower in focus and is based on checking compliance with criteria while evaluation is designed with a more comprehensive approach that seeks to answer the question “what should be done next,” indicating that the evaluation function looks beyond the organizational architecture, and looks at interventions in the broader landscape beyond their internal logic.
The differences between the two functions lie mainly in their role and mandate, the methodological approach, and the scope of the questions. Audit and evaluation put emphasis in different variables. Performance audit usually focuses on how the entity operates and the legal and compliance aspects, verifying and assessing the adequacy of controls and processes, as well as the institutional efficiency in managing risk. Evaluation gives prominence to the assessment of outcomes and impacts, implementing scientific methods to determine causation and attribution, to learn and contribute to decision-making for further improvement.
In his or her approach, the evaluator is expected to be sensitive to contextual factors, belief, manners, and customs of the social and cultural environments in which he or she works (UNDP, 2016) while the auditor is guided by the professional standards of the practice. Therefore, it is more likely for the evaluator to capture the social and behavioral mechanisms operating in public sector and decision-making process than the auditor as emphasized by Leeuw (1996). Those factors play a significant role in the effectiveness of organizations, particularly in the international development context marked by the complexity of development issues.
Overall, because of their distinct background, influences, and orientation, auditors and evaluators tend to approach their work in quite different manner (Chelimsky, 1996; Divorski, 1996). There are key differences in the purpose and the type of questions that interest each of these disciplines. Both fields address normative questions that audit uses mainly for accountability purposes, whereas evaluation does it to connect activities with effects (Chelimsky, 1985). Audit might also ask descriptive questions, but descriptive and contextual questions are more frequent in evaluation. The mandate of auditors does not usually allow them to question or comment on the merit or worth of overarching policies, which are typically taken as given. The major difference is that only evaluation asks cause-and-effect questions to establish whether policies or programs have produced the desired changes, which may lead them to question and recommend fundamental changes to the design of those policies or programs.
Independence, the key to audit and evaluation credibility and social change
There is a broad consensus that the independence of audit and evaluation functions from management system guarantees the credibility of their assessments (Ahlenius, 2000; Davis, 1990; Reichborn-Kjennerud and Johnsen, 2011). Audit has a longer tradition of protecting the independence of the function than evaluation. In many countries, the audit function has been established and independently institutionalized by the constitution to enhance democracy through accountability (Ahlenius, 2000). The evaluation independence, on the other hand, originated from the debates on ethical challenges faced by evaluators during evaluation processes. These include mainly pressure from client or commissioning party, issues regarding the evaluators’ motives and work quality as well as conflicts related to the reporting of evaluation results (Morris, 2007; Morris and Clark, 2013) which compromise the credibility of the evaluation function and its ability to influence decision-making. For these reasons, some institutions such as UNDP have adopted sophisticated policies to ensure that the evaluation function and practice are truly independent, but this is not the case in many other multilateral organizations and public entities.
In UNDP, centralized evaluations are conducted by the IEO while decentralized evaluations are carried out by country offices following the guidelines provided by the evaluation policy to guarantee the credibility of the function. Unlike to the IEO which enjoys structural independence from the UNDP management, the OAI is a core unit of the management system. However, the OAI exercises operational independent in performing its duties as provided by the UNDP Charter. It conducts independent, objective assurance and advisory activities in conformity with the International Standards for the Professional Practice of Internal Auditing (UNDP-OAI, 2017b).
In the literature, there are different strands in the theoretical, philosophical, and political orientation of evaluation. One of the dominant strands positions evaluation as a form of management support or an extension of management. Barrett (2001) sees evaluation as a systematic assessment aiming at providing better information to managers to improve program performance. In such a context, independence is less of an issue, although it is important for the evaluation to play its role internally and its utility.
For others, evaluation is viewed as a support for democracy and transformation because of its ability, largely in post-colonial and new democratic countries, to promote democratic values of transparency and accountability. In this context, independence becomes critical because evaluation provides a professional judgment on past performance, which is necessary as part of a diagnosis, in order to ignite the conversation that will lead to future improvements in peoples’ lives.
Based on IEO’s experience, the evaluators’ independence from management, both organizational and behavioral, is crucial for the evaluation function to be credible and serve accountability. Yet, “independence” can be understood very differently by evaluators and the institutions for which they work as evidenced by a recent debate among reputed specialists in the fields of audit, evaluation and development assistance (Wilton Park, 2018).
Organizational independence is related to the degree of autonomy that the evaluation unit has from the operational management and decision-making bodies within an institution. In the case of UNDP, both audit and evaluation functions enjoy organizational independence. Behavioral or substantive independence refers to the capacity of the evaluator to be free of bias when conducting and reporting the assessment. It can be argued that, even when organizational independence is guaranteed, there is a risk of bias, embedded, for instance, in the choice of focus, approach, or methodology, and in the particular values and backgrounds of the evaluators. The challenge, hence, is for the evaluator to provide an “objective assessment warranted by good reasons and evidence and being transparent with respect to any particular standpoint that he or she adopts in the evaluation” (Wilton Park, 2018).
In our view, evaluation has the potential to be a tool for social learning and citizens’ empowerment, as evaluation is a means to enhance learning within organizations and among stakeholders to support better decision-making (UNDP-IEO, 2019). However, for this to take place, it is critical that institutional frameworks are developed to support evaluations that are independent, not subject to the direct control of internal or external influences. Evaluations are more likely to be credible and transparent and more useful and more used if they are produced by evaluators that are able to report without fear or favor. Although evaluation is not directly meant to make a society more democratic, credible evaluations can deepen democracy by creating the conditions for holding institutions accountable for their performance, which in turn can enhance the trust that the citizens place in their political institutions (Naidoo, 2016).
Collaboration between the two functions: Perspectives and cautions
Over the last decade, with decreasing financial resources, tighter budgets, and a more active citizenry, more governments are under pressure to account for their actions. At the same time, funding constraints and the search for more coherence and coordination in development assistance have also driven increasing calls within the United Nations system and from donor countries for oversight units to work in a more collaborative manner. Advocates of this alternative claim, not without reason, that it promotes synergies, avoids duplications, and reduce costs.
Combining efforts makes even more sense considering the complexity, inter-relatedness, and broadness of the sustainable development goals (SDGs). The question is whether and how auditors and evaluators can work together to foster transparency and accountability in a way that improves programs’ effectiveness and promotes good governance.
Although evaluation can inform evidence-based policy making, other disciplines also contribute. The wide scope of the 2030 Agenda for Sustainable Development combined with the cross-sectoral nature of the goals, and the greater number of actors involved in its implementation have entailed the engagement of professionals from various fields in different types of assessment. Given the global nature of the SDGs and their objective to guide countries in reaching a development that is economically, socially, and environmentally sustainable, the evaluation and audit functions appear as critical oversight functions and institutional tools to analyze performance at policy, program, and organizational levels at different scales to advance in the achievement of the SDGs. A greater partnership between these oversight entities also becomes a necessity considering the strong emphasis of country-owned reviews of the implementation of the 2030 Agenda and the shortcomings in national evaluation capacities. Not all countries have strong evaluation institutions, but most do have oversight institutions (ombudsman offices, human right commission, parliamentary committees, etc.) which play an accountability role.
As part of the oversight function, performance audits and evaluations can help to ensure that development resources are wisely spent and expected results are achieved in a sustainable manner. They are both powerful tools for policy development and implementation. The difference between the two disciplines is less in purpose than in the identity and the framework of the institutional relationship in which their activities are carried out. Together they can better achieve their oversight, accountability, and learning purpose provided that the following conditions are satisfied:
First, the measurement issues must be considered (Chelimsky, 1996; Crawford et al., 2004; Van Thiel and Leeuw, 2002). Measuring compliance is not the same as measuring performance and outcomes. It must be recognized that financial audit, performance audit, and evaluation are all part of an oversight architecture, and play different roles and provide different information.
Second, it has to be acknowledged that none of the disciplines is inherently superior to the other. Each has its own strengths and challenges. The bigger strength of audit is its higher level of credibility, attributed to a greater reputation, the support of strong professional societies, well established and largely followed standards, and the perception of addressing issues of concern to the public. Evaluation, however, can explain why things are working or not, can be flexible in design and practice, has a longer history of addressing the attribution question, and acknowledges complexity and uncertainty (Mayne, 2006).
Third, evaluation and audit can work and reinforce each other, benefiting from their respective experience and building on their knowledge without compromising their own identity. Many organizations, including UNDP, have both an internal audit function and an evaluation function. Historically, evaluation and audit units have work in isolation, implementing their own strategy and working plans, with interactions with other sectors limited mostly to a few joint assessments. What is needed is a clear understanding of each function to properly serve their purpose and their clients.
Fourth, while audit and evaluation can and sometimes are covered within the mandate of the same unit or institution, the audit and evaluation functions are distinct and cannot be merged without damaging the strengths of either. As recognition of the similarities between audit and evaluation and concern about budgetary constraints increased, the temptation to combine both functions has arisen. Although there are clear advantages in collaboration, this discussion needs to advance cautiously.
Fifth, assessing the worth of programs cannot be done without the freedom to assess and comment on the impact of policies on people, and it is here where evaluation offers an irreplaceable perspective.
Overall, given the decline in the differences between the two functions (Brooks, 1996) and the disadvantage of separation—increased costs, duplication of work, and risk of misleading when the two functions found different results on the same subject (Chelimsky, 1996), a collaboration between audit and evaluation professions is expected. However, their separation ensures independence and credibility, freedom for the evaluation function to develop critical skills, respond to policy questions with strong studies, and easy feasibility (Chelimsky, 1996). For those reasons, Chelimsky called for caution when it comes to integrating the two functions. Moreover, Chelimsky (1996) and Divorski (1996) emphasized the difference in background between evaluators and auditors who tend to have different mindsets on the same subject. Furthermore, auditors and evaluators face challenges on performance measurement, which are related to measurement issues (Crawford et al., 2004; Van Thiel and Leeuw, 2002). For Chelimsky (1996) these issues are driven by the difference in training between auditors and evaluators.
Prospects for collaboration will be highest between audit and evaluation functions that are internal to an organization. In such cases, there will likely be overlapping concerns and interests and a need for regular interaction to manage the tensions that may arise from this. At the simplest level, there is cause for regular communication and information sharing to avoid duplication, minimize oversight fatigue, and to share insights. In some cases, there may be a more significant convergence of interests, where more substantive collaboration, in the form of joint analytical work may be merited. The promised benefits of this may be that auditors are able to learn from and understand better the dominant concern of evaluators to understand the impacts of organizational activities. From this, auditors may be able to reflect on the justification for organizational rules and policies, compliance, which is often the dominant focus of their work. Conversely, evaluators can potentially come to a better understanding of the impact that organizational rules and procedures can have in the production of the results or impacts produced by an organization, and the internal levers that can be used to improve accountability for performance.
Lessons learned from audit and evaluation joint and independent assessments within UNDP
Within UNDP, the centralized evaluation and audit functions are administered by the IEO and the OAI, respectively. The policies that establish the mandate of these units are quite alike. They both provide protections on independence and share core concerns around the UNDP’s effectiveness and accountability. There are also similarities between some of the performance audit work done by OAI and the consideration of operational and administrative issues of some IEO evaluations. However, there are some important differences in terms of mandate, outputs, and provisions regarding their position and independence within the organization.
The UNDP Charter of the OAI states that OAI will audit risk exposures in relation to UNDP’s governance, risk management, and controls, and will support UNDP in ensuring (a) achievement of the organization’s strategic objectives, (b) reliability and integrity of financial and operational information, (c) effectiveness and efficiency of operations, (d) safeguarding of assets, and (e) compliance with agreements, legislative mandates, regulations and rules, and policies and procedures (UNDP-OAI, 2017b). However, the UNDP (2016) evaluation policy provides that IEO conducts independent evaluation which pursues the objective of lessons learning, accountability, and improvement of national evaluation capacity to strengthen progress toward the SDGs. A key difference is that the IEO Director is accountable to the UNDP Executive Board while the OAI Director reports to the Administrator.
Independent assessments by IEO and OAI
In 2017, both OAI and IEO conducted independent assessments of the Global Fund grants geared toward HIV/AIDS in Angola. OAI performed an audit assessment of two program outputs, namely, reducing the burden of HIV/AIDS and strengthening the national response to HIV/AIDS in Angola covering the period from 1 January 2016 to 31 March 2017. The same program was evaluated by the IEO within the framework of the Independent Country Program Evaluation (ICPE) in 2017.
The audit report showed that the assessment was in line with the OAI’s mandate and has focused on the adequacy and effectiveness of the governance, risk management and control processes related to (a) governance and strategic management, (a) program management, (c) sub-recipient management, (d) procurement and supply management, and (e) financial management (UNDP-OAI, 2017a). The focus of the audit assessment is consistent with the performance auditing, as mentioned in the literature, indicating that audit assessment within UNDP is more management-oriented than program results and impact.
In contrast, the ICPE conducted in Angola in 2017, assessed the progress achieved with the Global Fund grants directed to HIV/AIDS. By looking at results and factors contributing and hindering them, the ICPE team found that UNDP was able to support national institutions’ capacity building, systems, law, and policies for the equitable delivery of HIV and related services (UNDP-IEO, 2017). Moreover, the evaluation revealed that the grants helped to prevent HIV infection, particularly among youth, reducing mother-to-child transmission, increasing the access to antiretroviral treatment and viral load measurement, and so on.
Drawing on this example, audit and evaluation, as mentioned in the literature, may differ by their purpose, scope, and methodological approach used. OAI’s assessment appeared more oriented to management and operations issues in relation to results rather than the results itself, while evaluation is results and impacts–oriented. Therefore, a collaboration between the two functions can contribute, on one hand, to provide the evaluators with evidence on management and operations factors that may affect results, and on the other hand, provide the auditors with evidence of the impact of management and operations issues on the achievement of development results within the organization.
Joint assessment by IEO and OAI
Strengthening the collaboration between the two functions has been a subject of interest for both units for long. In 2015, the IEO and the OAI decided to conduct a joint assessment on the institutional effectiveness of the organization to assess whether the programmatic and operational measures introduced by UNDP were likely to impact the quality of the organization’s work in the field. The joint assessment compelled the offices to work across traditional boundaries, share methodologies, and settle a common credible approach. Overall, it represented a unique learning opportunity for UNDP in which evaluators were bound to look more closely at quantitative analysis and normative questions, while auditors had to study performance from the perspective of cause and effect on results and people.
The initiative was successful because of the commitment of the IEO and OAI’s leadership. Since it was completed, both offices exchange more information and conduct missions together when possible. Nevertheless, the whole process was riddled with many challenges for both Offices and the organization that generated important lessons that could be taken into consideration when doing similar exercises in the future.
Challenges ranged from the way to name the study, the communication channels, and protocols, to the critical issue of the purpose, scope, and methodology of the assessment, among other basic components that included the key questions that the assessment should answer. Evaluators and auditors had to agree on data collection methods, strategies for triangulation, and what constitute credible evidence for both.
These are some of the conclusions and lessons:
Joint exercises may need a team leader with appropriate authority to make timely decisions and meet deadlines. Sharing decision-making was quite challenging in this experience, reflecting that it does not always work effectively in joint assessments. In addition, the decision-making process needs to be agreed before the assessment work starts and rigorously followed during the implementation.
Agreeing on operational concepts, such as “institutional effectiveness” and “results-based management,” was critical to adopt a commonly informed approach and methodology that were acceptable for both auditors and evaluators. It contributes to building common ground between auditors and evaluators in joint assessments to facilitate negotiations of means and tools to be used.
Although the interest and approaches to the methodology are different, performance auditing and evaluation can complement each other, strengthening the analysis and contributing to a more sound assessment. This experience shows that joint assessments allow the use of various methodological tools to capture a holistic view of the performance of program, policy, or organization.
Writing the report in a way that could ensure its independence, credibility, and utility proved to be quite challenging given the differences in the reporting line and approaches between both units. The quality review process, which was conducted separately by experts in each field, was also difficult and long as the experts were assessing findings and the underlying mechanisms from different angles. For future exercises of this kind, it would be best for the report to have a common section and add separate sections, specific to each function. However, the review for quality assurance should be done in one single process by professionals with experience in both.
Despite the concerns and reluctance that the assessment generated within the organization (for fear to losing resources, point out failures, etc.), the joint report was rich in evidence and made a strong statement for transparency, accountability, and learning in the United Nations.
The report forged a common language through the negotiations between evaluators and auditors that was critical since concepts were understood differently. A compelling reason for collaboration was the discussion on “evidence.” Defining more explicitly what constitutes evidence in the evidence hierarchy, how quantitative and qualitative data are processed and valued, and what triangulation and mixed methods mean, advanced both offices’ capacity, knowledge, and visibility. These challenges are consistent with the difference in background and the measurement issues raised in the literature.
The evaluation was well received by the Executive Board and had positive benefits from both the strategic and operational perspective: it demonstrated that it is possible to take a coherent and comprehensive approach to examine core performance concerns of compliance, value for money, and impact on people by bringing together the different skills and perspectives of evaluators and auditors.
Conclusion
Audit and evaluation have much in common. They have standards that share core principles of independence, objectivity, transparency, and professional judgment. They also share core concerns around the performance of public institutions, and the efficient, effective, and ethical use of taxpayers’ money. They are important actors and agents for promoting accountability and evidence-informed policy.
However, audit and evaluation are different functions, with their ethical guidelines and bodies of professional practice as well as traditions and particularities that preclude the merging of the two disciplines.
Recognizing this, it is important to understand how the unique identity and strengths of the performance audit and evaluation functions can be mutually reinforced in promoting accountability and providing a deeper understanding of the causal factors affecting development results.
In a context where questions are asked both about the financial compliance and the eventual results, it makes sense to seek greater collaboration between audit and evaluation. Our experience is that such collaboration can provide a more holistic or three-dimensional view of performance, which looks both inside and outside organizations to see whether inputs matched with outputs and outcomes. Joint assessments (when possible) and greater cross-referencing by the audit and evaluation offices can provide a coherent body of evidence to oversight protectors, parliaments, boards, and funders, to name a few. They can also help to rationalize resources, provide opportunities for robust debates on methodologies and the philosophical assumptions underpinning methodological choices. If this can be done without mandate creep or the dilution of the identities, and the strengths of these unique functions and disciplines, it can enrich and strengthen each of them.
For this to happen in a more thoroughgoing manner, oversight units from the two disciplines need to reflect on how they can work together and reinforce each other. From an evaluation perspective, whether this provides pay-off will depend on the continued maturation and development of the profession which will enable it to play a more assertive role in informing policy debates. This will require the establishment of models of practice that preserve and protect the independence of evaluation, as a pre-requisite for the quality and credibility of its interventions.
While the call for collaboration is likely to continue, it must be dealt with carefully recognizing the differences in approach and philosophies. This must be spelt out clearly upfront. For example, in the collaboration, more time should have been spent on upfront clarification, as tensions emerged in the process when different views were held of what constitutes evidence, and how this is evaluated. While it was eventually resolved, a clearer process and protocol should have been developed.
Footnotes
Acknowledgements
This paper was informed by inputs from David Slattery, Ana Rosa Soares, and Gedeon Djissa.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
