Indigenous Made in Africa Evaluation Frameworks: Addressing Epistemic Violence and Contributing to Social Transformation

Abstract

Transformative change is needed if the world is to achieve the sustainable development goals. Such change requires attention to culture, ethics, and values. We discuss the need to be responsive to the voices of Indigenous scholars in terms of the assumptions that guide methodological choices in the evaluation of international initiatives. We describe an Indigenous paradigmatic framework and then narrow the focus to a Made in Africa approach to evaluation that is designed to redress the epistemic violence perpetrated by the use of a Western cultural lens to determine evaluation approaches.

Keywords

Indigenous framework Made in Africa evaluation Transformative evaluation Sustainable Development Goals

Calls for transformative change at the societal level in order to achieve the sustainable development goals (SDGs) inherently raise issues about culture, ethics, values, and paradigmatic stance in development work. In this article, we discuss the lack of international agencies’ responsiveness in their framing of evaluation to issues of culture, ethics, and values from an Indigenous paradigm perspective and argue that this can harm communities and result in lack of achievement of the SDGs. We bring into the discussion voices of scholars who critique the dominance of Western culture and evaluation approaches in development evaluation. We argue that the benefits of listening to the voices of evaluation experts who use Indigenous approaches for evaluation establish the importance of addressing the epistemic violence that occurs when there is only one dominant voice. The use of Indigenous approaches to evaluation challenges the culture, ethics, and values that have directed the evaluation terms of reference that will potentially leave a legacy of falling short of achieving the goals of Indigenous communities. We narrow the focus of the Indigenous lens to the Made in Africa approach to evaluation (Chilisa, 2020) to illustrate specific implications for considering culture, ethics, and values in addressing context, needs, and relevance of intervention in development evaluation.

Evaluation and the Reliance on Dominant Paradigms

It is evident that despite the important role that evaluation plays in development, it has become a colonial prejudice that reinforces uneven and biased power relations (Bowman & Dodge-Francis, 2018; Cavino, 2013; Held, 2019). The African Evaluation Association and the Bill and Melinda Gates Foundation convened a group of evaluation experts in Uganda who were from across the global South to explore the dissatisfaction with the implementation of evaluation theory and practice based on a philosophical paradigm from the global North. The conveners produced a concept paper that recognized that reliance on the dominant paradigms from the global North for commissioned evaluations, monitoring and evaluation systems, and evaluation education does not attend to the intricate contextual issues shaped by societal cultures, traditions, and priority needs of people in the global South (South to South Cooperation, 2018). While the United Nations (UN) evaluation language invariably acknowledges and encourages partnerships of the North and South and the South to South cooperation in addressing development challenges, it does not support or has no strategy for a just epistemic approach in terms of the lens used for project design and evaluation.

The call for a more just approach to evaluation is not limited to the experts who attended the South to South convening in Uganda. Jeng (2012) notes that blind reliance on Eurocentric models, strategies, techniques, and research methods often lead to inadequate assessments, wrong prescriptions, and flat evaluation models. The reliance on external evaluators in most international development contexts reinforces poorly designed interventions, reproduced through inadequate evaluation methodologies and evaluation models (Chilisa, 2020). Often, the programs that are implemented are either not a priority to the communities or are not sustainable given the contextual environment in which they take place. Communities have no voice in the initiation of the programs; their goals, purposes, and the evaluation designs; implementation; methodology; or analysis and reporting. Evaluation reports and recommendations are submitted to the funders in formats that they require and, in most cases, are not submitted to the communities. Mainstream evaluators fail to address the broader struggles of low- and middle-income countries (LMIC) and Indigenous peoples, which include sovereignty, self-determination, and decolonization (Cavino, 2013).

Culture, Values, and Ethics in International Organizations: Epistemic Violence

The values of the international community are evident in the statement of transformative vision that the UN (2015) provided in their resolution adopting the SDGs. The resolution includes the following description of a dream for the future:
In these Goals and targets, we are setting out a supremely ambitious and transformational vision. We envisage a world free of poverty, hunger, disease and want, where all life can thrive. We envisage a world free of fear and violence. A world with universal literacy. A world with equitable and universal access to quality education at all levels, to health care and social protection, where physical, mental and social well-being are assured. (p. 3)
We clearly see the values that are expressed in this vision. We support those values that drive a desire to lessen poverty, hunger, disease, and violence; to increase access to education and health care; and to improve the environment. However, what we do not see in the commitment to transformational change is a commitment to the inclusion of Indigenous approaches to evaluation. Thus, people in LMIC have no or limited roles in planning the methodological strategies needed to support an accurate understanding of problems, crafting appropriate interventions, or designing an evaluation that is responsive to their culture, context, and needs. This tendency to silence the voices of local Indigenous voices has been characterized as harsh epistemic violence.

An example of the ethical implication of this lack of involvement of Indigenous voices in the determination of appropriate interventions is found in the strategies used to prevent malaria in Africa. Malaria no doubt poses a health risk threat to some countries in Africa and threatens the SDG goals on good health and well-being. Donors have adopted a one-size-fits-all solution and are quick to distribute mosquito nets. This is despite growing reports that some recipients of mosquito nets use them for fishing, while others prefer not to sleep under the nets because of the discomfort they experience (Gettleman, 2015). Eating the fish that are caught in the nets leads to health problems because the fish absorb the insecticide that is on the nets. Sleeping without the nets has the obvious consequence of increasing the risk of contracting malaria. But what if some communities have other ways of preventing malaria? Where are the stories of Indigenous malaria prevention methods in the evaluation reports? Where are the alternatives to mosquito nets from local communities? There is an unfortunate assumption that when beneficiaries use mosquito nets for fishing, they have no alternative in place but sit to die from exposure to mosquito bites and malaria. It is these weird assumptions and silencing of local communities that make evaluation another colonizing discipline.

The field of evaluation has for a long time neglected the axiological, ontological, and epistemological assumptions that inform evaluation (Billman, 2019, Shadish et al., 1991). This neglect especially in evaluation texts, for example, Bamberger’s (2012) text on evaluation in non-Western settings suggests that there is an assumption that all evaluators adhere to Western axiological, ontological, and epistemological assumptions (Billman, 2019). An evaluation grounded on Western axiological, ontological, and epistemological ways of knowing force out other ways of knowing leading to “systemic, institutionalised fragmented knowledge” (Billman, 2019, p. 69).

The culture of international organizations constitutes blatant epistemic violence that is likely to limit the achievement of the SDGs in Indigenous communities. The epistemic violence of the international organizations and donor agencies is particularly evident in the approaches to evaluation that are discussed and mandated with regard to the SDGs. The International Development Evaluation Association (IDEAS) published Evaluation for Transformational Change in which they provide guidance as to how evaluation can contribute to transformational change. The cultural values of these organizations are manifest in the call for evaluators to become fluent in systems thinking in order to understand how projects, programs, and policies can support lasting systems changes (Magro & van den Berg, 2019). They suggest that evaluators can contribute to transformation through learning to use big data, artificial intelligence algorithms, risk analysis, Bayesian statistics, and new technologies.

The language throughout the IDEAS publication reflects the dominant Western perspectives with very little reflection of the values and culture of those whose knowledge systems have been excluded from the evaluation discourse. This is reminiscent of Fals Borda’s (1980) commentary on the dominant paradigm’s lack of attention to discuss ethical or human implications of methodological work. He asserted that there was a need to lift the mask of objectivity and look under the wig of neutrality in order to develop methodological approaches that were rooted in the values, culture, and ethics of communities in low-income countries. In contrast, the international organization’s assumptions seem to imply that clarity exists with regard to the nature of the problem and solutions, thus the role of the evaluator is to provide data about the effectiveness of policies, systems, and programs in a politically neutral environment. This tendency to divorce methods–based paradigmatic assumptions from theory and assume political neutrality has become the trading license of epistemic violence.

While the choice does not have to be either/or (systems theory vs. Indigenous), the absence of a central place for the Indigenous voices from Africa and other parts of the world in this methodological space is indicative of an ongoing colonial mindset. How do donors react, for example, to unintended outcomes of mosquito nets? The focus of UN evaluation reports, for example, is on justifying continued involvement with the country programs by showing relevance to both country and UN mandate. It would appear unintended outcomes of development portfolios get very little attention. This lack of representation or authority to define problems, solutions, and evaluation strategies by the formerly colonized and Indigenous peoples from across the world rests on the power differences between members of these communities, their governments, and international organizations and donor agencies. These power differences occur in a fertile ground of epistemic violence. The use of a lens that incorporates the views of those whose voices have been excluded from the evaluation discourse has the potential to challenge this epistemic violence and creates a basis for the design of evaluations and interventions that incorporate Indigenous values, culture, and ethics.

Paradigms and Theoretical Branches in Evaluation: Adding the Indigenous Paradigm

Paradigms inform the methodologies and methods, research processes, priorities, choices, actions, and dissemination of evaluation findings (Mertens, 2009, 2018, 2020). Consequently, evaluators need to engage with the ontological, epistemological, and axiological assumptions that inform their evaluation methodologies. Mertens and Wilson (2019) present four paradigms and associated evaluation branches in detail in their textbook while recognizing that an Indigenous paradigm is emerging from scholars from that community. Indigenous scholars such as Wilson (2008) and Kovach (2009) have also argued for an Indigenous paradigm. The argument is that Indigenous paradigms need their own space (Chilisa, 2020; Held, 2019; Kovach, 2009; Romm, 2018; Wilson, 2008), so that their intersection with Western approaches can be critically examined. Chilisa (2012, 2020), Held (2019), and Romm (2018) have started to include it as a fifth paradigm in the typology of paradigms. Mertens and Wilson (2019) described four branches of evaluation that align with the “big four” philosophical paradigms: The methods branch with its emphasis on precision of quantitative methods aligns with the postpositivist paradigm. The values branch prioritizes judgment and is aligned with the constructivist paradigm, while the use branch prioritizes the use of evaluation findings and is aligned with the pragmatic paradigm. The social justice branch prioritizes equity and social justice and is aligned with the transformative paradigm.

We argue that Indigenous pathways to evaluation should emanate from Indigenous world views and philosophies, and Indigenous knowledge not available to nonindigenous evaluators. An evaluation methodology separated from its overarching paradigm is not sufficient for addressing epistemic violence and decolonization of Western thought (Chilisa, 2020; Held, 2019; Kovach, 2009; Wilson, 2008). In support of this view, Chouinard and Cram (2020) discuss epistemological, ecological, and methodological dimensions of cultural practice in evaluation. Culture infuses all contexts, while context grounds all aspects of the evaluation (LaFrance & Nichols, 2010). The epistemological, ecological, and methodological practice requires us to situate the evaluation of SDGs in LMIC in its paradigmatic space.

Carden and Alkin (2012) argue that the evaluation in LMIC deserves a distinct evaluation branch that addresses context. Supporting this view, Chilisa (2020) has added a fifth branch, the needs and context branch, aligned with an Indigenous paradigm (see Figure 1).

Figure 1.
Five branches of evaluation theory. Source: Chilisa (2020).

In the needs and context branch, the evaluator’s role is to establish the extent to which a program or policy addresses the priorities and needs of the beneficiaries and is culturally and contextually relevant for the local populations. The needs and context branch invites evaluators to take seriously a multiple-constructed reality grounded in material, social, and spiritual context reflective of an Indigenous paradigm. The needs and context of the communities in the drive to achieve SDG goals in LMIC can be addressed through relevant paradigmatic-situated evaluation strategies that question epistemic violence and address paradigmatic and multicultural validity.

Table 1 provides a brief introduction to the ontological, epistemological, axiological, and methodological assumptions that characterize a generic Indigenous paradigm. We locate the Made in Africa evaluation (MAE) approach in the Indigenous paradigm’s needs and context branch and illustrate the contextualized principles aligned to the generic Indigenous paradigm that can guide evaluation under the context and needs branch in an African setting (Chilisa, 2015, 2020).

Table 1.
Ontological, Epistemological, and Axiological Assumptions for Indigenous Paradigm and a Transformative Paradigm.

Philosophical Assumption Indigenous Paradigm Transformative Paradigm

Reality Multiple constructed realities grounded in material, social, and spiritual context and marked by the interconnectedness of the living and the nonliving and relational existence Rejects cultural relativism; recognizes various versions of reality that emanate from different social positionalities; conscious recognition of the consequences of privileging one version of reality over another

Knowledge Knowledge is subjective, objective, relational and include spirituality and vision Knowledge is socially and historically located; respectful interactions are necessary to create trusting relationships and address issues of power

Ethics and values (axiology) Values reflections on paradigmatic validity, and social as well as epistemic justice. Evaluation guided by principles of relationality, respect reverence, responsibility, reciprocity, reflexivity, responsiveness, and decolonization Respect for cultural norms; promotion of human rights and support for increased social, economic, and environmental justice; reciprocity

Methodology Transformative participatory lens for mixing indigenous qualitative and quantitative methods with Western quantitative and qualitative methods Transformative lens for cyclical mixed methods designs; building respectful relationships; consideration of cultural and contextual factors

Source: Adapted from Chilisa (2020), Heidi (2019), Romm (2018).

In the following sections, we describe the assumptions of the MAE approach and how it reflects assumptions of an Indigenous paradigm within a needs and context branch. We explore the implications of the MAE and its openness to inclusion of other paradigmatic perspectives as part of its nature.

MAE

The MAE is an approach that embraces African resistance to blind borrowing of Western values and standards to evaluate programs in Africa. Guided by Ubuntu, “I am because we are,” MAE offers ways to resist normalized positions of Western dominance: damaged, focused, and deficit discourses on evaluation in Africa (Chilisa & Malunga, 2012; Cloete, 2009; Ofir & Shiva, 2012; Sibanda, 2019). It is a deliberate attempt to adapt evaluation tools, instruments, strategies, and theory models, as well as to develop evaluation practice, theory, and methodologies emanating from local cultures, Indigenous knowledge systems, African philosophies, and African world views while at the same time embracing other knowledge systems (Chilisa, 2020; Chilisa, et al., 2017; Gaotlhobogwe et al., 2018). Built on the principle of motho ke motho ka batho (a person is because of others, I am because we are), MAE has no boundaries between Africa and the rest of the world. This ethical principle incorporates the ideas of wholeness and relationality leading to the position that the evaluator has an obligation to promote the transformation of all humans and the physical world (Ramose, 2020). Thus, the Indigenous paradigm manifest in the Made in Africa approach is suitable to integration of other paradigmatic perspectives, a stance that Johnson and Stefurak (2013) label as dialectical pluralism.

For example, the African ethical principle of motho ke motho ka batho holds that evaluators have an ethical responsibility to design their work to support positive transformation in the human and physical world because we are all related. This aligns with the pursuit of social, economic, and environmental justice as an ethical remit in evaluation, an assumption found in the transformative paradigm (Mertens & Wilson, 2019). In the MAE ethical view, there are no boundaries between knowledge systems; thus, it can be integrative, bringing together Western and Indigenous perspectives. It promotes global partnerships of knowledge systems and of evaluation actors and stakeholders. It seeks to stamp out decontextualized evaluation and the silencing of non-Western ontological, epistemological, and axiological assumptions in evaluation. What follows is a discussion of how ontological, epistemological, and axiological assumptions emanating from an Indigenous paradigm are applied in the MAE approach.

Ontological Assumption of an MAE

The ontological assumption associated with an MAE approach holds that Africans are to play a greater role in solving their own problems; thus, questions on who prioritizes, initiates, and designs community programs and projects are essential. All areas of culture, living experiences, and Indigenous knowledge systems must be utilized to come up with a methodology through which the realities can be known. Reality is contextual and cultural bound. The evaluator should examine the history of the program, the location and its people, all connections and interconnections and interrogate how spirituality, relational power, political discursive, and historical temporal power (Cavino, 2013) shape the evaluand. Under this assumption, the main question addressed by the evaluation is: Whose priorities and aspirations are addressed by the evaluation? In Africa where there is scarcity of even the most basic needs, most interventions will be relevant to the community. However, the question should not be whether interventions are relevant but rather whether they are a priority to the communities. The UN, for example, addresses relevance in a generic way by aligning development portfolios to national development plan priorities, SDGs, and the UN mandate. But what if government priorities are politically motivated, serving the interest of the elites? What if communities perceive their problems and solutions to the problems differently?

There is, nevertheless, a growing realization of the tensions between what is relevant for government and what is a priority for the community. The UN Independent Evaluation Office evaluation in Africa is beginning to address this tension by deliberately interrogating prioritization of activities and projects and the need for analysis of context, community needs and reflecting on the value that projects add to communities. The Organization for Economic Cooperation and Development (OECD) in their review of the Development Assistance Committee (DAC) evaluation criteria have highlighted the importance of prioritizing needs and aspirations of beneficiaries and problematizing the relevance and the contextually and culturally relevant dimensions of evaluation practice (OECD/DAC, 2019).

Relational Epistemological Assumptions

The Made in Africa approach aligns with the relational epistemological assumption, meaning that no one person holds knowledge. We know through a network of relations that include the living and the nonliving. Truth is therefore a dialogue on what is known and what can be known. Take, for example, the community knowledge that a clinic built on sacred ground will impact negatively on the well-being of the health seekers. This is the knowledge that is consistent with a material, social, spiritual, and contextual construction of reality and should have a bearing on how evaluators go about their work. It points to the necessity of a dialogue on the production of knowledge and the evaluation methodology with a diversity of stakeholders that include the funders, the project implementers, the community and the beneficiaries, among others. The approach invites a mixed-methods approach where mixing is conceptualized along the idea of integrating nonindigenous and Indigenous approaches or methods (Chilisa, 2020). Macdonald et al. (2009) propose dialogic methods that can be used for understanding particular aspects of the problem addressed by the intervention, integrating visions, world views, interests, and values.

Made in Africa Axiological Assumptions: Ethics and Values

We discuss axiological assumptions under nine common principles across Indigenous research and evaluation:
Relationality: In the context of MAE, core values are based on an I/we relationship. The emphasis is on belongingness, togetherness, interdependence, relationships, collectiveness, love, and harmony. There is an emphasis on valuing community strength and building community relationships to inform evaluation intent, motive, and methodology. Communities are recognized as knowers who can pass judgment on the relevance of SDG intervention and can prioritize their needs. Relationality also points to the need to recognize the evolving international context within which evaluation takes place and the value of safe networks and relationships between funders and beneficiaries. It points to the need for evaluators and funding agents to establish long-lasting relationships with communities.

Responsibility: It is about the role of an evaluator in pursuing social, economic, and environmental justice, resisting dominant ideologies that silence local communities, and contributing to the worth, health, unity, and harmony within the community and with all stakeholders. In Africa, the African renaissance calls for Africans to consciously decide on whose side they are, the community side or that of the funder or both. Evaluators are called upon to ensure restoration, protection, and revitalization of community knowledge that can add value to community needs and priorities and can lead to sustainability of interventions. Why have we not had alternatives to mosquito nets based on knowledge production between local communities in Africa and evaluators? This question is pertinent more so when we consider evaluation to be a learning endeavor. The evaluator needs to reflect on lessons learnt and how they further the evaluation of the SDGs in its attempt to address development in LMIC and make responsible recommendations. The MAE is built on the African logic of its not either or but both/and. In the spirit of Ubuntu, “I am because we are,” MAE calls for the building of responsible relationships and partnership of knowledge systems.

Reverence: Indigenous research recognizes the critical nature of spirituality and values as an important contribution to ways of knowing. Many Indigenous people place value on sacred sites and spiritual practices. The I/we relation recognizes a material and spiritual reality. Ignoring spirituality can thus cause tensions between communities, stakeholders, funders, and evaluators with a Western methodological orientation. Imagine, for example, carrying out an evaluation of a health facility that is ignored by the community. The evaluator with a deficit approach about the “other” will label poor utilization of the health facility as backwardness, while a respectful and responsible evaluator will consider as knowledge that the facility was built on sacred space and the need to relocate it if the community is to use it.

Reciprocity: Evaluators need to ask the fundamental question of the value added to the community by the intervention and what the community brings to the intervention. Muwanga-Zake (2009) has referred to this as value validity and addresses the extent to which the interventions contribute to personal and social transformation and the people hold themselves equally responsible for the outcome. Take, for example, a situation where funders address the SDGs on education by building a school in a remote area in Botswana. The community should show care and responsibility by reflecting on how they give back to the funders by ensuring the success and sustainability of the intervention.

Respectful representation: Respect requires that the process from the initiation of the intervention to the evaluation questions asked, the methodology, the data collection procedures, and reporting and dissemination of the report is guided by the community and that the community has ownership and access to the data collected. Relevant programs reported by evaluators as prioritized and immediate needs by communities are indicators of respectful relations and respectful evaluation.

Reflexivity: The principle of responsibility is tied to reflexivity. Researchers continuously reflect on their position within existing powers. From an Ubuntu (“I am because we are”) position, reflexivity entails a journey of the evaluator and the intervention recipients into each other’s life. The Ubuntu disrupts and decolonizes dominant deficit thinking by promoting compassion, care, togetherness, and empathy (Chilisa, 2020). Thus, the evaluator with the stakeholders reflect on the theory of change, evaluation theory and models, and together imagine other ways of conducting evaluation that emanate from Indigenous knowledge systems and how to integrate multiple knowledge systems to conduct culturally and contextually relevant evaluation that is responsive to the priority and needs of communities.

Responsivity: The SDGs are complex challenges that involve communities changing their priorities quickly. Responsiveness is the ability of evaluators to learn from the process, recognize the evolving changes, and adapt their approaches and methodologies to become change, context, and culturally sensitive and appropriate.

Rights and regulations: This calls for ethical protocols that accord communities the rights and opportunities to prioritize their needs and participate in the design and evaluation of the interventions and have ownership of data produced and the reports. Communities should have the right to require evaluators to use the African evaluation guidelines (AfrEA, 2019) to guide the evaluation process.

Decolonization: The ninth principle associated with the MAE approach to evaluation’s axiological assumption calls for African resistance to resist blindly borrowing Western values and standards to evaluate programs in Africa and capacity building of African policy analysts, researchers, and evaluators to enable them to promote adaptation of evaluation tools, instruments, strategies, and theory and model adjustment to ensure relevancy to African settings; and the development of novel evaluation practice, theory, and methodologies emanating from local cultures, Indigenous knowledge systems, and African philosophies such as Ubuntu. If evaluators and evaluation reports cannot influence the way interventions are designed, programmed and monitored, and evaluated to address priorities and needs of communities, then their work adds little value to communities and are not likely to influence the achievement of the SDGs.

Methodology of an MAE

The methodology of MAE supports the use of a transformative mixed-methods approach that values the use of dialogue methods (Macdonald, 2008; Mertens, 2018) and community conversations to integrate knowledge systems. There is a holistic construction of evaluation knowledge to produce evidence, and community knowledge is used as a basis for further improvement and sustainability of projects. Evaluators listen to metaphors on the environment that have a relationship to the project and use dialogue methods and community conversations to integrate community set standards, stakeholders’ standards, and donors’ standards to evaluate worth and merit.

The MAE principles can be summarized as follows:
MAE values paradigmatic validity. Every evaluator who works in the African context should recognize and be guided by a paradigm that is contextually relevant and relational; emanates from the realities, ways of knowing and values of the Africans; and articulates their needs and priorities. A methodology separated from its overarching paradigm is not valid (Kovach, 2009; Wilson, 2009).

Ubuntu philosophy transcends all processes of evaluation. Ubuntu principle makes provision for “the rationale, modulation and interconnectedness of the categories of race, class, gender, ethnicity and their respective isms…all those things which Europeans and westerners view as either/or opposites, binaries or dichotomous thinking” (cited in Goduka, 2000, p. 29).

African self-determination and rebirth inform the evaluation process where the evaluation agenda, who designs and evaluates the intervention, what is evaluated, and how it is evaluated, and findings disseminated are defined by African realities within an Ubuntu context that takes into account the realities of others (Chilisa, et al., 2017). The African evaluation ethics guidelines are recognized and utilized by the evaluators.

In the context of the Ubuntu “I am because we are,” local Indigenous knowledge and practices that include spirituality and religiosity are brought together with Western knowledge to design interventions and contextualize the evaluation process in ways that are respectful, valuing community strength, and serving the priorities and needs of the communities. It values a transformative mixed-methods approach where mixing is defined in terms of the bringing together of Western knowledge and Indigenous knowledge (Chilisa, 2020; Cloete 2009, Cloete & Auriaacombe, 2019; Mertens, 2018). The international community of researchers is inviting us to challenge the conventional thinking of seeing mixed methodologies as mixing the dichotomy of methods that are either quantitative or qualitative and to focus more on integration of knowledge systems. The main argument is that Indigenous and Western knowledge should be integrated to acknowledge and enhance participation of Indigenous peoples as knowers and creators of their own destinies, increase the relevance of research to their needs, enhance rigor in the evaluation process and the dissemination of findings in ways appropriate for both academic and community settings.

The nine principles of relationality, responsibility, reverence, reciprocity, respectful representation, reflexivity, responsivity, rights and regulations, and decolonization are the hallmarks of an ethical evaluation process.

Indigenous groups around the world have developed approaches to evaluation that emanate from their own cultural roots as can be seen in the work from Aotearoa (New Zealand; Cram & Mertens, 2015), Native American Indians (LaFrance & Nichols, 2010), and Africans (Chilisa, 2020). In line with the focus on culture and context, Indigenous evaluators discuss culturally responsive Indigenous evaluation (Bowman & Dodge-Francis, 2018; Chouinard & Cram, 2020) and evaluations informed by Indigenous paradigms and world views. The work sheds light on how to incorporate the voices of evaluation beneficiaries in ways that further the transformative agenda.

Paradigmatic Solidarity

While the Indigenous paradigm and its associated contexts and needs branch represent a unique perspective in terms of evaluation assumptions, it also shares some characteristics with other paradigms, for example, the transformative paradigm in that both are concerned with engaging with the full range of stakeholders in culturally responsive ways with the goals of increasing social, economic, and environmental justice (Cram & Mertens, 2015; Mertens & Wilson, 2019). This points to the porous nature of paradigms and the valuable opportunity dialectical pluralism in that Indigenous and non-Indigenous evaluators can work together, thus avoiding the current trend where evaluation is dominated by external evaluators who use predominantly decontextualized methodologies and are often ignorant of the context and culture of the people (Chilisa, 2020). The social justice dimension of the transformative paradigm also brings an essential language that evaluators can use to address power differentials among the diverse stakeholders ranging from donors with conflicting interests to the competing interests of stakeholders, governments, and beneficiaries.

Another illustration of dialectical pluralism in this context is the integration of aspects of the Indigenous paradigm with the postpositivist paradigm that can be found in the evaluation of a youth-oriented evaluation of an HIV/AIDS prevention project in Botswana (Chilisa & Tsheko, 2014). They framed the evaluation using the Indigenous framework; however, part of the evaluation design included a randomized controlled trial (RCT). Before the implementation of the RCT, the evaluators used Indigenous methods to collect cultural knowledge and nurture respectful relationships that informed the entire evaluation process. This dialectical stance allowed them to integrate the knowledge of the Indigenous knowledge systems, which are typically marginalized, into the dominant paradigm’s methodologies. The result was use of the evaluation findings in the early part of the study to inform the development of a culturally responsive intervention that reflected the cultural, values, and ethics of the Botswanan culture. It also led to the collection of data throughout the study that were based on Indigenous ways of knowing such as yarning, talking circles, and storying.

Well-performed evaluation should inform planning, development of interventions, and outcomes (Bellagio Leaders Forum, 2012). The evaluation should focus on the contribution of development to the world of individuals, their relatives, others, and the environment upon which they depend. Thus, it should generally contribute to societal progress by providing invaluable information to policy and decision makers and advancing understanding of how development can best be approached locally and globally. The UN dream of a better future should embrace efforts to stamp out practices of epistemic violence in the design and evaluation of SDG interventions by creating a space for the inclusion of Indigenous frameworks. If the development projects are situated in Africa, for example, a Made in Africa approach can be used with other tried and tested approaches to promote context and culturally responsive evaluation that address the needs and priorities of African people (Chilisa, 2020).

Conclusions

International organizations tasked with the evaluation of the SDGs propose the use of Western/Northern frames for the evaluations (Magro & van den Berg, 2019). We argue that this approach to evaluation will result in the design of evaluations and interventions that are not responsive to the culture and complexity of LMIC and those who have been excluded from the evaluation discourse. Evaluators are challenged to influence the design of intervention and evaluation and practice in general by probing the way interventions are designed and the value added by interventions when communities are allowed to be innovative. Communities’ voices need to come out not only as recipients of packaged interventions but as active participants who can adapt interventions to respond to the dynamic and complex rapidly changing contexts, so that intervention responds to their needs, priorities, and aspirations. Decades of silencing of African innovativeness of African people in addressing malaria serves as a clear example of the colonizing tendency of evaluation theory and practice. The use of an Indigenous lens, exemplified in a MAE approach, challenges the wisdom of ignoring the pragmatic lens that inform evaluation that, according to Billman (2019), results in fragmented knowledge. The integration of the Indigenous and the assumptions and approaches of other paradigms through a dialectical process provides insights into how evaluations can be designed to be more inclusive and responsive to the culture, values, and ethics in LMIC contexts. We do not want to repeat the mistakes of the past when the Millennium Development Goals missed the mark on communities in LMIC, Indigenous populations, women, and people with disabilities. Evaluators can advocate for paradigmatic situated evaluation that claim the voices of those who have been excluded from the evaluation discourse as a way to be responsive to marginalized populations and potentially to increase the attainment of the SDGs. This is not an easy process, but the voices of Indigenous scholars provide guidance for traversing this difficult terrain.

Philosophical Assumption	Indigenous Paradigm	Transformative Paradigm
Reality	Multiple constructed realities grounded in material, social, and spiritual context and marked by the interconnectedness of the living and the nonliving and relational existence	Rejects cultural relativism; recognizes various versions of reality that emanate from different social positionalities; conscious recognition of the consequences of privileging one version of reality over another
Knowledge	Knowledge is subjective, objective, relational and include spirituality and vision	Knowledge is socially and historically located; respectful interactions are necessary to create trusting relationships and address issues of power
Ethics and values (axiology)	Values reflections on paradigmatic validity, and social as well as epistemic justice. Evaluation guided by principles of relationality, respect reverence, responsibility, reciprocity, reflexivity, responsiveness, and decolonization	Respect for cultural norms; promotion of human rights and support for increased social, economic, and environmental justice; reciprocity
Methodology	Transformative participatory lens for mixing indigenous qualitative and quantitative methods with Western quantitative and qualitative methods	Transformative lens for cyclical mixed methods designs; building respectful relationships; consideration of cultural and contextual factors

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Donna M. Mertens

References

African Evaluation Association. (2019). The African evaluation guidelines: Final draft. https://afrea.org/the-african-evaluation-guidelines/

Bamberger

. (2012). Introduction to mixed methods in impact evaluation. InterAction: Rockefeller Foundation.

Bellagio Leaders Forum. (2012). Africa thought leaders’ forum: An evaluation and development expanding thought leadership in Africa. Proceedings of the Bellagio Conference. Bellagio, Italy

Billman

J. A. H.

(2019). Tackling the wicked problems in the field of evaluation [Unpublished PhD dissertation]. Indiana University of Pennsylvania.

Bowman

(Mohican/Lunaape) Dodge-Francis

(2018). Culturally responsive Indigenous evaluation and tribal governments: Understanding the relationship. New Directions for Evaluation, 159, 17–31.

Carden

Alkin

M. A

. (2012). Evaluation roots: An international perspective. Journal of Multi-disciplinary Evaluation, 8(17), 102–118.

Cavino

H. M.

(2013). Across the colonial divide: Conversations about evaluation in indigenous contexts. American Journal of Evaluation, 34(3), 339–355.

Chilisa

(2015). A synthesis paper on the Made in Africa evaluation concept. African Evaluation Association. Retrieved February 4, 2020, from https://afrea.org/wp-content/uploads/2018/06/MAE2-Final-31st-august.pdf

Chilisa

(2020). Indigenous research methodologies (2nd ed.). Sage.

10.

Chilisa

Major

T. E.

Gaotlhobogwe

Mokgolodi

(2017). Decolonizing and indigenizing evaluation practice in Africa: Toward African relational evaluation approaches. The Canadian Journal of Program Evaluation, 30(3), 312–328.

11.

Chilisa

Malunga

(2012, November 14–17). Made in Africa evaluation: Uncovering African roots in evaluation theory and practice [Paper presentation]. African Thought Leaders Forum on Evaluation for Development Expanding Thought Leadership in Africa . The Bellagio Centre.

12.

Chilisa

Tsheko

G. N.

(2014). Mixed methods in indigenous research: Building relationships for sustainable intervention outcomes. Journal of Mixed Methods Research, 8(3), 222–233.

13.

Chouinard

J. A.

Cram

(2020). Culturally responsive approaches to evaluation. Sage.

14.

Cloete

(2009). Evidence-based policy analysis in South Africa: Critical assessment of the emerging government-wide monitoring and evaluation system. Journal of Public Administration, 44(2), 291–311.

15.

Cloete

Auriacombe

(2019). Revisiting decoloniality for more effective research and evaluation. African Evaluation Journal, 7(1). https://doi.org/10.4102/aej.v7:1.433

16.

Cram

Mertens

D. M.

(2015). Transformative and indigenous frameworks for multimethod and mixed methods research. In Hesse Biber

Johnson

R. B.

(Eds.), The Oxford handbook of multimethod and mixed methods research inquiry. (pp. 91–109). Oxford University Press.

17.

Fals Borda

(1980). Science and the common people. In Dubell

(Ed.), Research for the people—research by the people. Selected papers for the international forum on participatory research in Ljubljana, Yugoslavia, (pp. 13–40). Linkoping University.

18.

Gaotlhobogwe

Major

Koloi-Keaikitse

Chilisa

(2018). Conceptualizing evaluation in African context. New Directions for Evaluation, 159, 47–62.

19.

Gettleman

(2015, January 24). Meant to keep malaria out, mosquito nets are used to haul fish in. New York Times. https://www.nytimes.com/2015/01/25/world/africa/mosquito-nets-for-malaria-spawn-new-epidemic-overfishing.html?auth=link-dismiss-google1tap

20.

Goduka

I. N.

(2000). African or indigenous philosophies; legitimizing spiritually centered wisdoms within the academy. In Higgs

Vakalisa

N. C. G.

Mda

T. V.

Assie-Lumumba

N. T.

(Eds.), African voices in education, 63–83. Lansdowne: Juta.

21.

Held

(2019). Decolonising research paradigms in the context of settler colonialism: An unsettling mutual collaborative effort. International Journal of Qualitative Methods, 18, 1–16.

22.

Jeng

(2012). Rebirth, restoration and reclamation: the potential for Africa-centred evaluation and development models. African Thought Leaders Forum on Evaluation and Development, Bellagio, Italy.

23.

Johnson

R. B.

Stefurak

(2013). Considering the evidence-and-credibility discussion in evaluation through the lens of dialectical pluralism. New Directions for Evaluation, 138, 37–48.

24.

Kovach

(2009). Indigenous methodologies: Characteristics conversations and contexts. University of Toronto Press.

25.

LaFrance

Nichols

(2010). Reframing evaluation: Defining an indigenous evaluation framework. The Canadian Journal of Program Evaluation, 23(2), 13–31.

26.

MacDonald

Bammer

Deane

(2009). Research integration using dialogue methods, Canberra, Australia, ANU E Press.

27.

Magro

van den Berg

R. D.

(2019). Systems evaluation for transformative change: Challenges and opportunities. In van den Berg

R. D.

Magro

Mulder

S. S.

(Eds.), Evaluation for transformational change (pp. 131–155). IDEAS.

28.

Mertens

D. M.

(2009). Transformative research and evaluation. Guilford Press.

29.

Mertens

D. M.

(2018). Mixed methods design in evaluation. Sage.

30.

Mertens

D. M.

(2020). Research and evaluation in education and psychology (5th ed.). Sage.

31.

Mertens

D. M.

Wilson

A. T.

(2019). Program evaluation theory and practice (2nd ed.). Guilford.

32.

Muwanga-Zake

J. W. F.

(2009). Building bridges across knowledge systems: Ubuntu and participative research paradigms in Bantu communities. Discourse: Studies in the Cultural Politics of Education, 30(4), 413–426. https://doi.org/10.1080/01596300903237198

33.

Ofir

Shiva

K. S.

(2012). Evaluation in developing countries: What makes it different? In Donaldson

S. I.

Azzam

Conner

R. F.

(Eds.), Emerging practices in international development evaluation (pp. 11–24). Information Age.

34.

Organization for Economic Cooperation and Development/Development Assistance Committee Evaluation Network. (2019). OECD/DAC network on development evaluation: Better criteria for better evaluation. https://www.oecd.org/dac/evaluation/revised-evaluation-criteria-dec-2019.pdf

35.

Ramose

M. B.

(2020). Motho ke motho ka batho, as African perspective on popular sovereignty and democracy. In Jenco

L. K.

Idris

Thomas

M. C.

, (Eds.), The Oxford handbook of comparative political theory. Oxford Press.

36.

Romm

(2018). Responsible research practice: Revisiting transformative paradigm in social research. Springer.

37.

Shadish

W. R.

Cook

T. D.

Leviton

L. C.

(1991). Foundations of program evaluation: Theories of practice. Sage.

38.

South to South Cooperation. (2018). Addressing asymmetries in global evaluation: Elevating the knowledge and influence of the global South. Final proposal. Phase 1: 2018–2021. https://afrea.org

39.

United Nations. (2015, September 25). Resolution adopted by the general assembly: Transforming our world: The 2030 agenda for sustainable development.

40.

Wilson

(2008). Research is ceremony: Indigenous research methods. Fernwood.