Abstract
Indigenous evaluation literature is a powerful approach to upholding the aspirations of Indigenous Peoples. As a site of knowledge production, literature shapes the evaluation processes and practices that serve self-determined priorities: This article offers a synthesis of international Indigenous evaluation theory, to move beyond culturally competent evaluation as a framework for prioritising Indigenous values and aspirations. We acknowledge that ‘culture’ and ‘context’ remain necessary and strategic principles in evaluation design, particularly when evaluation of Indigenous programs continues to be strongly bound by funder and settler government priorities. However, we argue that such principles are insufficient, drawing instead on the broader field of Indigenous evaluation theory that centre Indigenous sovereignties, for programs that enact Indigenous aspirations. Insights from our theoretical review indicate Indigenous evaluation frameworks can play an active role in moving beyond neoliberal discourses of evaluation often adopted by governments and other funders, towards relational responsibilities that inform Indigenous approaches to validity. We take up the frameworks of relationships, relevance, and responsibility (3 Rs) as providing an important framing to foreground relational accountabilities and place-specific priorities in evaluative practice for Indigenous-led social change.
What We Already Know
• Evaluation theorists and practitioners are already familiar with calls to include culture and context in evaluation practices, particularly when it comes to Indigenous program evaluation. • In line with the development of culturally competent and relevant evaluation practices, the literature in Australia has also begun to move towards the role of values in sovereign evaluation methodologies.
The Original Contribution the Article Makes to Theory and/or Practice
• The authors offer an original synthesis of international Indigenous evaluation values and frameworks. • Our analysis of international Indigenous evaluation frameworks in the literature presents a possibility, in which evaluation upholds learning and growth for wisdom, in the name of self-determined futures.
Introduction
This article presents a synthesis of Indigenous evaluation literature, to centre Indigenous aspirations in program design and delivery that align with Indigenous values. In Australia, where the authors write, an annual stock take of the effectiveness of government-led initiatives for Indigenous peoples routinely reports on the failure of such initiatives to improve outcomes (Schultz, 2020). Simultaneously, a growing research base explains how Indigenous educational, social and economic outcomes improve when Indigenous communities have decision-making authority (Chandler & Lalonde, 1998; Eva et al., 2024; Murphy, 2014). We argue Indigenous evaluation frameworks play an important role in the shift required to support the quality of programs that Indigenous communities either provide for themselves or can expect from government (Morley, 2015). This study seeks to provide insight into the themes emerging from Indigenous scholarship that can be taken up in a wide range of evaluative contexts, but particularly those seeking to move beyond the discourses of competition and standardisation embedded in government-funded programs and associated evaluative practices (Bryant, 2024; McCausland, 2019).
Indigenous communities and organisations seek approaches to evaluation that are not simply about assessing effectiveness, but embed steps to actively further their rights, needs and aspirations (Finlay et al., 2021). In this article we discuss the different purposes that evaluation might serve when self-determined approaches are foregrounded. For example, governments can implement evaluation practices in ways that have little regard for cultural validity. Conversely, Indigenous organisations, particularly in service delivery and program design, require culturally valid information that supports Indigenous governance (Hemming & Rigney, 2008). In making the case for centring Indigenous theories, peoples and governments in evaluation processes, Mohican/Lunaape scholar Waapalaneexkweew and Dodge-Francis write that ‘evaluation should be a tool of transformation, improvement, and empowerment to solve chronic issues in society’ (2018, p. 27). In other words, Indigenous approaches to evaluation are significant for upholding self-determination through knowledge-producing practices and their implications. Such approaches, when community-led, enable an understanding of what and how initiatives enable meaningful change. This article offers a move towards evaluative practices that attend to Indigenous peoples’ multiplanar and interspecies relationships and intergenerational responsibilities, with significant attention to the relevance of evaluative practices and findings for local priorities (Hopson & Cram, 2018).
On the Need for Indigenous Evaluation
Many Indigenous organisations and programs encounter the discipline and practice of evaluation through funding regimes, as government or philanthropic entities seek to identify the efficacy of financial investments in social domains. Historically, the purpose of western evaluation practice has been expressed as a process of determining and assessing merit (Scriven, 1991; Stake, 2004; Stufflebeam, 2001). Programs are measured against ‘established standards’ (Scriven, 2001, p. 302) and values are assessed against enactment (Patton et al., 2015); a top-down approach to accountability that functions as ‘enabling and normalising comparison, competition and control’ (de St Croix, 2022, p. 700). Program evaluation often then centres on clarifying a ‘theory of change’; describing the context in which a program is created and assumptions about how individual, behavioural, programmatic, institutional, or structural change occurs. In addition to a theory of change, evaluation processes also often include an associated logic model, which usually represents a linear relationship between aims, inputs, outputs, and outcomes. In this way, ‘program evaluation is a systematic process of data collection and analysis designed to address issues of program improvement, measure program effectiveness and the attainment of outcomes and serve decision-making and accountability purposes’ (Chouinard & Cram, 2019, p. 1).
Evaluation is often conceptualised as a systematic and valid approach to meet funding accountability requirements, to wit, justifying the introduction of ‘market rationality in non-economic domains’ by applying neoliberal values of competition and economic efficiency to domains of personal and public life (Giannone, 2016, p. 497). Against a backdrop of global standardisation of performance benchmarks, ratings and indicators (Giannone, 2016, p. 503), there have been some moves to identify how developmental, transformative or empowerment evaluation practices might serve other aims (Cram & Mertens, 2016; Patton et al., 2015). In particular, Indigenous contexts and aspirations are often governed by philosophies far removed from market forces and their incumbent logics of accountability. Ancestral or land-based approaches, Indigenous efflorescence (Roche et al., 2018) or resurgence (Simpson, 2017), for example, offer other values that might be centred in evaluation practice. In contexts where the state is not the primary funder, assessments of merit can be underpinned by different values that may well be in dialogue with but not generated by market logics. Accountability to ancestors, land and water, intergenerational transmission of language and culture, or various projects of self-determination offer different priorities which can be supported by and enabled by evaluative practices. It is to this context that we bring questions around what can be learnt from international Indigenous evaluation scholarship. 1
Putting the Review in Context: Evaluation for an Indigenous-led Social Change Program
As authors, we each bring our own cultural, social and linguistic contexts within which we live and develop our understandings. All three of us live and work in Narrm (Melbourne), Australia. Elizabeth identifies as Māori, with research experience in secondary and tertiary education. She moved to Australia 11 years ago. Nikki is a queer Gomeroi woman, from Country located across the Queensland/New South Wales border in Australia, and who brings a professional interest in sociology and policy. Fi is a white invader/settler and queer woman with Irish, British and Scottish ancestry, who grew up on Kheeray Whurrong Country and holds a commitment to centring Indigenous sovereignties in evaluation and education work. As authors, we are connected through our relationship to the Atlantic Fellows for Social Change (AFSE) Program and committed to working critically at the nexus of Indigenous education, social transformation and evaluation practices.
This theoretical review has been undertaken as part of an Indigenous-led social change program, the Atlantic Fellows for Social Equity (AFSE), which is based at the University of Melbourne on the unceded 2 Country of the Wurundjeri – Woiwurrung Peoples. Mid-career participants in the program are largely Indigenous, with a few non-Indigenous Fellows working with Indigenous communities, that come from across Australia, Aotearoa New Zealand, and the Pacific Islands. We use the term Indigenous and First Nations interchangeably, when referencing Peoples who have ancestral ties to the first inhabitants of a place, while simultaneously recognising a wide range of definitions and constraints. 3 A master’s degree is a key component of Fellows’ first year of AFSE, which involves participants engaging with research literature around social change to strengthen their Indigenous community-oriented projects. Participants are then supported to implement their social change projects. AFSE holds an associated responsibility to approach evaluative praxis in ways that centre Indigenous ways of knowing, doing, and being.
Review Outline
There are two key ideas that we take up from a theoretical review of the literature. First, we outline the context in which evaluation plays a role in governance through policy, and the associated imperative to increase non-Indigenous evaluators’ capacity to attend to culture and context (Chouinard & Cram, 2019). Second, we analyse the ways Indigenous epistemologies have been articulated, engaged with and argued for in Indigenous evaluation work. Cultural competence forms a significant part of earlier Indigenous evaluation literature and is the current emphasis found in Australian scholarship (Gollan & Stacey, 2021). The work, alongside culturally relevant evaluation, tends to be centred on how culture and context shape the validity of evaluation practice. Theories of both cultural competency and culturally relevant evaluation make the case for evaluators to move into higher standards of capacity to understand and respond to the particularities of the cultural context in which they work, in a move to improve the validity of evaluation, particularly for multicultural communities (Kirkhart, 2010). While significant and necessary as a response to barriers and harm, an approach oriented by the concerns of cultural competence does not necessarily centre Indigenous sovereignties nor aspirations.
We undertake this review to consider the key values and theoretical frameworks the literature offers the AFSE program for evaluation praxis to support Indigenous projects of resurgence, across communities and Lands. In light of the limitations of cultural competence approaches for a program premised on Indigenous sovereignties, the second part of the review involves an engagement with the role of Indigenous epistemologies in evaluation work. We take up the recent values proposed in Indigenous evaluation literature (Hopson & Cram, 2018; Smith, 2018a) of the 3Rs – relevance, relationships, and responsibility – to argue that they both provide a meaningful alternative to impact discourse and offer a way for conceptualising evaluation as a relational enactment of responsibilities to enact community priorities. The final section of the article takes up the 3 Rs approach to explore how relational interdependence is central to knowledge creation and how we might approach evaluation practices that are designed in accordance with Indigenous peoples' theories, methodologies, and aspirations.
Methodology
This theoretical review of recent literature on Indigenous evaluation frameworks sits alongside a more established corpus of Indigenous scholarship focused on the relationships, processes and practices associated with colonial states’ dispossession of Indigenous peoples. This article is underpinned by the work of Indigenous scholars like Linda Tuhiwai Smith (Ngāti Awa and Ngāti Porou) (2013, p. 1), who identifies the damage of colonial forms of knowledge production and, equally, the possibilities present when academic processes reflect Indigenous priorities. In the context of evaluation research, Smith (2018b) foregrounds that ‘the big political agenda of Indigenous research is for the paradigm shifts and transformations that overturn colonialism and create new relations of power with the nation-state so that Indigenous wellbeing and ability to be self-determining is achieved’ (Smith, 2018b, p. 33). With these possibilities in mind, the review is informed by Graham Hingangaroa Smith’s (Te Aitanga a Hauiti, Ngāti Porou, Ngāti Apa, and Ngāti Kahungunu) Smith (2005) emphasis on the responsibility held in circular praxis to transform Indigenous lifeworlds. In other words, we take up the responsibility for evaluation to be enacted in the ongoing cycle of reflection and action in colonial contexts, to the benefit of Indigenous communities within a framework of Indigenous futurities (Moodie & Maddison, 2023). 4 It is important to note while there is an abundance of empirical evaluation published about Aboriginal and Torres Strait People in an Australian context (Altman & Russell, 2012; Nakata, 2007); there is room for further methodological work that extends from Indigenous community priorities and is translated into evaluation scholarship. 5 In undertaking this review, we searched literature across key evaluation journals published in comparable settler colonial contexts to honour and learn from the work already practiced and published in Indigenous evaluation. As with the concept of Indigeneity, while there is much to be gained from international literature on evaluation, we remain cognisant of the specificities of Country and communities.
Method: Reading the Literature for Self-Determined Evaluation Frameworks
Our assessment of the field is focused on recent publications in Indigenous evaluation across these contexts over the past two decades (2002–2022). This includes Turtle Island (North America) (LaFrance et al., 2015; Waapalaneexkweew (Nicole Bowman) & Dodge-Francis, 2018) and Aotearoa New Zealand (e.g. Kawakami et al., 2008; Wehipeihana & McKegg, 2018) where Indigenous-led evaluation practices are relatively well established. We undertook two phases in our search for literature.
Phase 1
• Search period: We initially searched for texts from 2012–2022. • Journals searched are Evaluation Matters, American Journal of Evaluation, Canadian Journal of Program Evaluation, and New Directions in Evaluation. • Key search terms included ‘First Nations’, ‘First Peoples’, ‘Aboriginal’, ‘Torres Strait Islander’, ‘Native’, ‘Tribal’, ‘Indigenous’, ‘Māori’, and ‘evaluation’. • Inclusions: Articles were included in on the basis of offering both an explicit evaluation framework and the presence of Indigenous scholars on the writing team. • Exclusions: ∘ Articles without an explicit theoretical framework (mainly empirical). ∘ Articles that were solely authored by non-Indigenous scholars.
Phase 2
• Timeframe expanded: Based on initial search results, we expanded the timeframe to include significant texts referenced from 2002, which contextualise subsequent work. • Snowball literature identification: The initial search terms, while useful starting points, do not speak to specific Land-based identities and associated terms. On this basis, we sought place-specific supplementary literature, including grey literature, based on texts referenced in phase 1 literature. • Expanded authorship: Some scholarships by non-Indigenous authors were included when cited in numerous articles by Indigenous authors as relevant to the development to their work.
Indigenous Evaluation Literature: Approaches and Relevance
In the following table, we offer an overview of all relevant literature identified. In reading through the literature, we addressed the questions outlined previously by attending to: • the broad approach of the work (e.g. Indigenising evaluation and culturally relevant evaluation); • the Indigenous Lands or territories on which the evaluation took place (where explicitly named); and • the key concepts (e.g. relationality) in relevant frameworks developed or applied during the evaluation.
Culturally competent, relevant and responsive evaluation
Discussion
The Limitations of Cultural Competence
Whether described within cultural competence or culturally relevant evaluation, the concepts culture and context constitute a significant strand of evaluation literature. Within the literature, culture is used as a term to signify a range of concepts, practices and relationships. The term is also represented as a specific set of values or as representative of a particular epistemology, within the argument that epistemic recognition improves evaluation validity (Hood et al., 2005; Kirkhart, 2010). Chouinard and Cousins (2007) assess cultural competence as the dominant conceptual framework, based on their review of early empirical literature in cross-cultural evaluation between 1997 and 2006. The authors note that ‘while there are no agreed upon terminologies, definitions, or even methodologies, what these approaches all share is the recognition that culture and context matter, and that there are no universally agreed upon rules or abstractions that can be applicable in all contexts’ (Chouinard & Cousins, 2007, p. 40). The significance of this assessment of the literature is the introduction of the dual frameworks of ‘culture’ and ‘context’ to broader evaluation practice (Hood et al., 2015), now taken up largely in culturally responsive evaluation approaches (Chouinard & Cram, 2019; Waapalaneexkweew (Nicole Bowman) & Dodge-Francis, 2018).
Cultural competence reveals the normative and often invisible functions of western values through evaluation practice. While culture is defined in many ways, cultural competence often deploys the concept of culture as a kind of shorthand to describe the expression of values and worldviews (Masters-Awatere, 2015; Masters-Awatere & Nikora, 2017; SenGupta et al., 2004). This shares a common thread with evaluation, which is equally concerned with identifying values and merits of programs and practices in ways that seek to acknowledge cultural specificity but which ultimately adopts this as a variable, rather than foundational purpose. Evaluators’ work is therefore undertaken in a context where values tend to exist mostly as invisible, normative systems of merit that routinely shape program design, policy and evaluation practice (Hood et al., 2014; Kirkhart, 2010). In this way, culturally competent evaluation responds to the risks often incurred by government policy-making when those policies, or programs informed by those policies, inadequately respond to the values, needs and aspirations of Aboriginal and Torres Strait Islander peoples. Evaluation oriented by cultural competence frameworks largely focuses on the responsibilities inherent in the role of the evaluator, as a tool for evaluation (Patton, 2014). These responsibilities are especially evident in non-Indigenous and settler’s evaluation practices across cultural contexts other than their own (Wehipeihana et al., 2010). In light of the ways evaluation is deployed in colonial governance (Altman & Russell, 2012; Smith, 2018b), the emphasis on culture in evaluation can be read as an attempt to accommodate cultural specificity rather than recentre Indigenous peoples’ sovereign priorities for transformative processes and practices.
Calls to responsibly engage with cultural context actively negotiate the power that evaluation holds in policy and decision-making, including in funding processes. For example, SenGupta et al. (2004) make the case for culturally competent evaluation by outlining the implications for programs designed without cultural specificity in mind which, they argue, are premised on uncritical responses to stereotypes and, therefore, produce inaccurate representations of culturally premised deficits. The importance of culture, values, and power in evaluation practice has been acknowledged in the field for at least a decade. The American Evaluation Association has articulated their commitment to cultural competence since 2011 for example (American Evaluation Association, 2011). In the Association’s statement, the rationale for the competencies is grounded in increased validity, ethical requirements, and epistemological congruence. That is, the Association makes the claim for cultural competence as central to valid and ethical conduct. The theory of change underpinning much of the work with ‘cultural context’ is that a recognition of specific cultures by white evaluators will hold increased validity and epistemological congruence (American Evaluation Association, 2011). Such concerns functionally obscure the specific sovereign concerns and rights of First Peoples, independent of the settler state.
Developments in Indigenous evaluation theory: Epistemological and methodological considerations
Indigenous Evaluation Frameworks
Epistemological Considerations for Orienting the Purpose and Practice of Indigenous evaluation
In the second category of Indigenous evaluation articles that emerged (see Table 2) there was substantial theoretical work undertaken to explicate the role of Indigenous epistemologies and methodologies in evaluation practices. In their emphasis on epistemology, scholars highlight evaluation as a knowledge-producing practice (Cram & Mertens, 2016), revealing and contesting the epistemological violence of whitestream evaluation through an emphasis on what constitutes valid Indigenous evaluation. Self-determined knowledge production is not constrained by the recognition of culture and context within evaluation practices, but is foundationally reoriented in its the purpose, towards Indigenous sovereignties (Bowman, 2020; Cram, Tibbetts, et al., 2018; LaFrance & Nichols, 2008): From an Indigenous perspective, for evaluation to be true and useful – that is, a good evaluation – the evaluator must have an understanding of the self-determination that fuels the goals and aspirations of Indian communities to preserve, restore, and protect their cultures and ways of doing things (LaFrance & Nichols, 2008, p. 18).
This work is generative for naming what underpins internally valid evaluation, which simultaneously orients its purpose and holds implications both for ways of doing evaluation and for the qualities evaluators require in the process.
Land and language are central in the theory surrounding Indigenous evaluation purpose and process. Hopson and Cram (2018), for example, argue for the recognition of the context in which evaluation takes place as a complex ecology, foregrounding First Nations communities’ relationships to place and Land. Further, Indigenous peoples’ relationships to their Lands are co-produced with knowledges and language of places. One example of this is demonstrated in the ways that Lakota language specifically invokes the meaning of evaluation itself as ‘inquiry for the future’ (Robertson et al., 2004). Centring the value of interrelatedness across human and non-human entities is in this way crucial to meaningful Indigenous evaluation. Relationships thus comprise the interconnected realities of Indigenous Lands and community-oriented responsibilities. Further, ‘to decolonise means that standards of good evaluation practice are defined by local values and protocols’ (Hopson et al., 2012, p. 63). This emphasis on local Indigenous sovereign protocols and priorities underpins the formative work of LaFrance and Nichols (2008, 2009), whose evaluation framework was developed through extensive community meetings, gatherings and opportunities for feedback across multiple Indigenous lands and communities.
Engaging with sovereign knowledge practices means that evaluation itself can contribute to resurgence of language, culture and decision-making authority through the recentring of Indigenous epistemologies and values. Writing from a Cree context, Kopp et al. (2021, p. 212) redefine evaluation as wisdom-seeking. That is, Indigenous ontologies raise a range of possibilities for conceptualising knowledge itself; as variously revealed, empirical and traditional (LaFrance & Nichols, 2008). Further, historically excluded forms of spiritual and ceremonial knowledge are central to the process of recentring Indigenous knowledges (Kopp et al., 2021; Wilson, 2008). This multidimensional, cohesive approach to engagement with knowledges is mirrored in the work of Kopp et al., who identify that unlike narrow western forms of health evaluation, ‘Indigenous teachings on health and wellness are grounded in spirituality, ceremony, language, collective wellbeing, reciprocity, balance, and good relations’ (2021, p. 213). That is, Indigenous evaluation includes the protocol, ceremony, and spirit (Kennedy et al., 2015) in doing evaluation work according to Indigenous ways. This second body of literature is therefore significant for orienting the purpose of good Indigenous evaluation and its epistemological grounding. We now turn to Table 3 to engage with examples of the frameworks and methodologies that emerge from these foundations.
Values Across Indigenous Evaluation Frameworks
The final body of literature in this review highlights the role of values in meeting the transformative needs and aspirations of Indigenous evaluation (L. T. Smith, 2019; Tuck, 2009, p. 49). Unangax̂ scholar Eve Tuck (2009) offers a critique of ‘theories of change’ for the ways they tend to reify an apparent binary of reform and revolution. In doing so, Tuck offers a way to navigate this through her elaboration of four ‘vantage points’ (2009, p. 49), or epistemological shifts, that articulate what change, specifically, is being sought and what Indigenous-led processes are required to achieve that change. Tuck (2009) writes about these epistemological shifts as the inner angles of a circle, in dialogue with each other and with no hierarchical sequencing. Within this framework, sovereignty, contention, balance and relationships work as key vantage points, necessary in a properly conceived Indigenous evaluation framework. That is, ‘theory of change’ is articulated in terms of the importance of and dynamics inherent in: • acknowledging and advancing Indigenous sovereignty, • seeing contention as educative and disruptive of power imbalances, • pursuing balance in the distribution of knowledge and responsibility, and • elevating the collective peoplehood of Indigenous communities, their lands, and priorities.
In other words, Indigenous evaluation centres self-determination, as a core orienting principle of evaluation work. Such an approach is oriented by cultural values and knowledge in a particular political context shaped both by colonisation and ongoing commitment by Indigenous peoples to the strengths (Morelli & Mataira, 2010) of their cultures and languages.
Identified by evaluator Fiona Cram (Ngāti Pahauwera) Cram (2018) as the first characteristic of Indigenous evaluation practices, acknowledging and upholding sovereignty raises a range of questions for evaluation praxis. As Joan LaFrance (Turtle Mountain Chippewa) and Richard Nichols’ (Santa Clara Pueblo) note, ‘…for tribes, sovereignty derives from our sense of place, our language, history, and culture. It is deeper than simply a legal or political relationship. Evaluation has a responsibility to support nation building’ (2008, p. 25). The move to recentre Indigenous sovereignty establishes both the scope and imperative in evaluation practice for attending explicitly to the social, legal, and political context in which it is enacted (Bowman-Farrell, 2019). This approach focusses on nation-to-nation engagement that considers Indigenous governance as an intelligible and necessary entity for evaluation decision-making. An explicit politico-legal conceptualisation of evaluation systems stands in contrast to the present way evaluation is employed to reify state-Indigenous relations (which frequently serve to undermine self-determination) (Waapalaneexkweew (Nicole Bowman) & Dodge-Francis, 2018) 6 .
An emphasis on evaluation as an act of relationality underpinned by culturally specific values is not new (Chouinard & Cousins, 2007, p. 461), though it continues to be deepened and theorised in increasingly nuanced and generative ways (Morelli & Mataira, 2010; Smith, 2018a; Trotman et al., 2018; Waapalaneexkweew (Nicole Bowman) & Dodge-Francis, 2018). LaFrance and Nichols (2009), for example highlight ‘recognising gifts’, as an implicitly strengths-based approach to maintaining relationships through telling the evaluative story of Indigenous programs. Relationality is not simply interpersonal. The politics of relationality are explicated in Bowman-Farrell’s (2019) work, where she outlines the range of mechanisms through which nation building takes place, and the responsibilities of evaluation practices to identify and respond to these through Tribal Critical Systems Thinking. Bowman-Farrell attends to the logics underpinning colonial contexts and as an ‘emerging framework for raising critical consciousness, activating culturally responsive practices in systems, and emancipating public government, policy, and practices’ (2019, p. 350). Relationality can be taken up according to cultural context, and informs evaluation frameworks in similarly variable ways.
The 3Rs – relationships, relevance, and responsibility – offer another way of conceptualising Indigenous evaluation (Hopson & Cram, 2018). In other words, of the 3 Rs, Cram and Hopson write that: … critical to increasing our usefulness in complex ecologies are the relationships that enhance our ability to connect with and transform communities, the responsibilities we have to ensure that our work meets the diverse and dynamic needs of stakeholders and other interested groups, and a more grounded, functional, and integrated understanding of the relevance of our work. (Hopson & Cram, 2018, p. 10, emphasis in the original).
In this context, the 3 Rs approach is an example of the ways in which values can be operationalised in community-based contexts for meaningful programmatic evaluation practices, and in ways that honour the priorities that emerge from specific Indigenous Lands. This values-oriented evaluative framework provides an opportunity to conceptualise meaningful but non-prescriptive approaches to assessing quality and utility in program evaluation. The three theoretical strands in the literature highlight both the imperative to understand current evaluation assumptions and the possibilities for otherwise, together. The emphasis on Indigenous community-specific values supporting Indigenous decision-making, program design and understanding the impact of these on Indigenous lives.
Methodological Implications and Possibilities of the 3 Rs Approach
The 3 Rs framework points to the ways in which relational interdependence is central to meaningful research in Indigenous communities and, similarly, evaluation practices that are responsive to context (Tuck & McKenzie, 2014). In this way, relationships are conceptualised in the literature as an expansive form of interrelated being – as complex ecologies (Hopson & Cram, 2018). Relational validity, then, ‘prioritizes the reality that human life is connected to and dependent on other species and the land’ (Tuck & McKenzie, 2014, p. 4). This approach to relationality points to a significance of the ‘where’ of evaluation: knowledge is held in, and produced by, land (Tuck & McKenzie, 2015). Evaluation as a relational practice shapes our emphasis on our responsibilities to Indigenous relationships, lands, languages and cultures, and the quality of evidence that can be drawn on by Indigenous decision-makers.
While recent literature in evaluation largely speaks to the importance of ‘evidence-based’ (Royse et al., 2015) or ‘best-practice’ approaches in programs and their operation, there is a simultaneous turn towards recognising the legitimacy of practice as an evidential basis for evaluative validity. Specifically, there has been a move towards the recognition of practice-based evidence as a meaningful way of engaging with the validity of community-oriented Indigenous knowledges (Abe et al., 2018). That is, practice-based evidence offers a re-imagined approach to what constitutes data and knowledge in the Indigenous evaluation literature (Naquin et al., 2008). At the same time, there are practices emphasised across the literature as constituting congruent evaluation with identified values. One example of the ways of tending to relationships and responsibility, to ensure relevance, is in establishing an Indigenous evaluation committee: Johnston-Goodstar (2012) emphasises the role that Indigenous community members play, and the centrality of a structure of community oversight for both upholding responsibilities to community and ensuring the relevance of evaluation to local priorities. Relational responsibility forms a central element of evaluation: First, we create, form, and commit to relationships – which define who we are and to which we are held accountable – and everything we do must incorporate the principles of respect, reciprocity, and responsibility, otherwise defined as good relations and relational accountability (Kopp et al., 2021, p. 216).
In this way, values underpin the validity of evaluation work, and the practices established to uphold locally held responsibilities.
While the principle of relationships highlights the knowledges every person brings to a programmatic context, it also foregrounds the ways these translate to reciprocal gifts in the collective (LaFrance & Nichols, 2008). According to Hopson and Cram (2018, p. 10), there are three primary questions that guide the ‘relationship dimension’ of Indigenous evaluation, including: What key interactions, variables, or stakeholders do we need to attend to (or not)? Whose interests and what decisions and relationships drive the context of the work? How can we attend to important interactions amid competing interests and values through innovative methodologies, procedures, and processes? (Hopson & Cram, 2018, p. 10)
The value of relationality expands an evaluation axiology, where Indigenous peoples’ priorities are elevated. Importantly, an emphasis on relationality highlights the ways that Indigenous methodology work in Australia already offers significant relevant conceptual frameworks (Martin, 2006; Moreton-Robinson, 2017; Rigney, 2006; Walter, 2018, are just some examples). In other words, there is literature that enables locally specific ways of translating relationality into evaluative praxis. The work of tending to relationships in and through evaluation creates the impetus to balance recognising the gifts offered by each individual and the wide range of contexts and communities within which individuals enact responsibilities to such relations.
Conclusion: Re-Centring Indigenous Sovereignty in Evaluative Thinking; Purpose, Values, and Validity
Our analysis of culture and context highlights that the associated theory of change is frequently oriented towards increased validity and epistemological congruence (American Evaluation Association, 2011). We have argued that while it is necessary to make the case for culture and context within mainstream evaluation settings premised on settler colonial knowledges and power relations, they are insufficient to a context premised on Indigenous sovereignties as the basis of evaluation design. Further, approaches that foreground evaluator’s recognition of culture and context can at times obscure the specific sovereign concerns, rights, and complex lifeworlds of First Peoples.
In contrast, the emphasis on place-specific protocols and values in the literature offers not just the possibility but the necessity of enacting evaluation in relationally responsible and relevant ways, for which Indigenous worldviews constitute the foundation. In considering this, we take up value-led approaches centred on relationships, relevance, and responsibility. While we have turned towards Indigenous evaluation outside of Australia, the tenet of relationality across such texts serves to reinforce the relevant and transferrable role of pre-existing Indigenous methodologies in research. Our emphasis on evaluation as relationality in service of self-determination decentres and undermines the current normative role of evaluation in an Australian context. For our purposes, this analysis is significant for enabling us to take an approach that centres key principles grounded in responsibility, but which does not preclude Fellows’ specific protocols and methodologies from guiding the practices. This approach is significant for translating key international learning into a locally grounded context. Our analysis mirrors lessons from literature in terms of reflection for growth, grounded in Indigenous-led priorities. The next step is to practice such approaches, and to return with new lessons. Our analysis of the literature presents an alternative vision, in which evaluation upholds learning and growth for wisdom, in the name of self-determined futures.
Footnotes
Acknowledgements
We would like to acknowledge Professor Linda Tuhiwai Smith for her feedback on an early draft. We are deeply grateful to the generous and insightful reviewers who shared their critical feedback – thank you for strengthening this work.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.