A Human Rights-Based Evaluation Approach for Inclusive Education

Abstract

This article reports on ways in which United Nations human rights treaties can be used as a normative framework for evaluating program outcomes. In this article, we conceptualize a human rights-based approach to program evaluation and locate this approach within the broader evaluation literature. The article describes how a rights-based framework can be used as an aspirational set of indicators for program evaluations to promote activities that align with internationally agreed-upon human rights norms. We then describe a case study of the evaluation through which this method was developed, including its sampling design, methodology, and findings. The United Nations International Children’s Fund (UNICEF) inclusive education evaluation described highlighted the need for conceptual clarity around what inclusive education is, and the importance of contextualized innovation toward meeting the educational rights of children with disabilities. Human rights perspectives and evaluation designs can help create such clarity, but should also be used with care.

Keywords

convention on the rights of persons with disabilities disability evaluation human rights inclusive education United Nations

The contemporary conceptualization of human rights in diplomatic and programmatic discourses began gaining popularity after the creation of the United Nations and the subsequent 1948 Universal Declaration of Human Rights (UDHR). The 30 universal rights spelled out in the Declaration include life, rights to liberty, security, movement, nationality, opportunity to marry, to own property, freedom of thought, assembly, work, rest, leisure, well-being, education, and cultural participation, among others. The UDHR further spells out that all humans should be free from slavery or servitude, torture or inhumane punishment, and arbitrary arrest, and have equal protection under the law (United Nations, 1948).

The universal rights and freedoms spelled out in UDHR have guided the work of the United Nations for over seven decades and have been proposed as a set of principles that outlast political shifts in any nation. de Waal (2016) observed that “human rights should make no distinction between political allies and adversaries: all should be held to the same standard” (p. 12). Universal rights have become a cornerstone to United Nations activities, but Member States and United Nations organizations have also acknowledged gaps and failures in the upholding of certain rights for specific populations.

For this reason, a series of subsequent treaties have been developed and signed by Member States. Additional treaties, such as the 1965 International Convention to End Racial Discrimination, the 1979 Convention to End Discrimination Against Women (CEDAW), the 1989 Convention on the Rights of the Child (CRC), the 2003 International Convention on the Protection of the Rights of All Migrant Workers and Members of their Families, and the 2008 Convention on the Rights of Persons with Disabilities (CRPD) have acknowledged that human rights are at once universal and plural, because “human rights are about promoting human rights for all … (and) human rights may also be about delving deeply into issues of identity, survival, and dignity of particular groups” (Mégret, 2008, p. 496, author's emphasis). Mégrét's work is particularly important for rights-based conversations because the notion of plural rights decenters normative assumptions (which in the case of persons with disabilities, could be considered ableist) and allows for an acknowledgment of pluralism within humanity while highlighting that there are different routes, resource needs, and considerations necessary to ensure all can claim the universal rights outlined in normative instruments.

The theme of rights has spread widely as a centerpiece of United Nations treaties, with the most recent example of the Framework Convention on Global Health, which frames public health as a human rights issue (Gable & Meier, 2013). A common feature among all the post-UDHR treaties described above is a commitment to “progressive realization” of rights, meaning that framers acknowledge that nation-states are at different starting points for upholding rights, but all must be progressively working toward enactment of the rights spelled out in treaties. For example, the CRPD emphasizes the importance of inclusive education for children with disabilities, yet many countries maintain segregated systems of education for this population. For a country to progressively realize human rights in education it would need to make immediate commitments toward inclusion, and progressively implement these commitments over time (i.e., gradually closing all segregated schools in favor of inclusive schools).

Concepts outlined in treaties are also increasingly informing the work of international organizations, such as non-government organizations and the United Nations organizations that engage in programmatic work for international development. In part, this shift has been the result of a growing recognition that needs-based or service-delivery approaches have failed to substantially reduce poverty (UNICEF/UNESCO, 2007, p. 9) and may be reinforcing inequalities and rights violations in countries. As part of the United Nations Program for Reform, launched in 1997, all entities of the United Nations system are now mandated to mainstream human rights into their activities and programs. This led to an inter-agency process of negotiation in the adoption of a UN Statement of Common Understanding that was acceptable to all UN agencies. The statement provides a conceptual, analytical, and methodological framework for planning and monitoring development activities based on international human rights standards that should be included in UN programming and subcontractor tasks. Coomans (2012) observed that organizations have begun to adopt to these requirements by using “rights language,” or terminology that makes “an explicit reference to universal human rights” (p. 274). This language has informed programs focused on the economic, social, cultural, civil, and political rights of program participants. Further, both non-governmental and international organizations have begun to frame themselves as duty holders for upholding rights in programming.

Uvin (2007) acknowledged the shift to rights-based language in educational organizations and labeled the willingness of organizations to focus on rights as a “revolution” in programming. According to Uvin:
as with most ethically desirable aims, organisations seeking to promote human-rights outcomes through the use of aid have a very easy place from which to start: themselves. Ensuring that their internal personnel management and decision-making procedures are non-discriminatory, non-exclusionary, transparent. (p. 604)

In sum, Uvin noted that a human rights focus provides organizations with a framework for programming and day-to-day operations that has a clear orientation and prescriptive language.

Human Rights and Evaluation

In this article, we demonstrate how a human rights framing can inform evaluation design. We define a human rights-based evaluation approach as an evaluation that uses human rights conventions to develop evaluation questions, interpret evaluation data, and make recommendations. In the paragraphs below we outline a rights-based approach that we undertook for a recent United Nations organization formative evaluation. We describe our underlying framework and provide an overview of the findings of the evaluation. We then locate this work within the broader evaluation literature, highlighting both the promise and limitation of rights-based evaluation in relation to theory-based (Coryn, Noakes, Westine, & Schröter, 2011), participatory (Cousins & Earl, 1992) and decolonizing evaluation (Cram, 2015). Through our transparent assessment of the arguments for, and limitations of, rights-based evaluations we hope to contribute to ongoing conversations about structural inequalities with the readership of this journal (Caldwell & Bledsoe, 2019) and other global evaluation organizations.

In this case, we report on an evaluation that was focused on the rights of children with disabilities as a case example for the development of rights-based evaluation. This case example was selected because disability-inclusive education is a priority for United Nations International Children’s Emergency Fund (UNICEF), which reports that among the 240 million children with disabilities worldwide, nearly half do not have access to schooling.

In the paragraphs below we outline the underlying tenets that informed how we designed this particular type of evaluation and describe a case example of how it was implemented. We argue that human rights-based evaluations can inform programs that provide opportunities for individuals or populations marginalized or minoritized in society. We describe this approach through a case example of an evaluation of disability-inclusive education programs.

Rights-Based Evaluation: An Overview

In this article, we define human rights as the entitlements that are afforded to all humans, because of their status as humans in this world. Quite often social and human services programs are evaluated because there is (1) a need to be accountable to a sponsoring or funding agency; (2) competition for scarce funds requires that only one program (the most efficient) can be funded; (3) new interventions being tried; and (4) a need for some objective assessment about the worth of the social programs (Royse et al., 2010). Evaluations may also occur for internal accountability (i.e., is this program working?) or, even more simply, to find out if what was hoped for really happened (Patton, 1986).

Evaluation designs are generally intended to determine the impact of programs, identify areas of concern or priority amid program activity (i.e., formative evaluations), or provide real-time feedback to program staff (developmental evaluation). Bamberger, Tarsilla, and Hesse-Biber (2016) suggest that evaluators generally seek statistical, theory-based, or narrative evidence that indicates whether programs have achieved their objectives. The scope of evaluations is generally limited—“even those (evaluations) that are considered the most ‘rigorous’ do not permit the identification of outcomes that were not specified in the program design” (Bamberger et al., 2016, p. 155). Limitations on evaluations allow for a level of focus on the outcomes that are important to the organization or sponsoring agency.

In the formative evaluation of inclusive education for children with disabilities that we report on in this article, both the sponsoring agency (UNICEF) and the evaluation consultants it hired (all of whom are co-authors of this paper) sought to include a normative framework for the evaluation. The desire for a normative framework was based on field-based knowledge that the education of children with disabilities varied widely across global UNICEF programming and that children with disabilities have been historically marginalized from education systems in general (UNESCO, 2020). In this evaluation, for example, we examined programming in 22 countries but guided by a single organization's commitment to a human rights model. Because of this, there were concerns on behalf of the evaluation team that evaluation data capture a range of types of engagement, even if these outcomes were not intended by the projects under evaluation. Previous UNICEF evaluations highlighted that, in some places, children with disabilities were completely ignored and out of school. In other instances, any engagement with children with disabilities (whether aligned or not aligned with children's rights) was considered a point of pride, and in still others, children with disabilities were effectively included in their communities and local schools (Johnstone et al., 2019).

To tease out these outcomes, the evaluation team decided to frame inclusion as a human right by drawing upon the language of the Convention on the Rights of Persons with Disabilities (CRPD). Two specific parts of CRPD were utilized for guidance in this evaluation. First, Article 24, which focuses on education for children and adults with disabilities. Second, General Comment Number 4, which provides additional implementation expectations beyond the Convention itself. Drawing upon these documents, we set out to examine how programming aligned with the parameters of rights-based inclusive education outlined by CRPD. Such an approach was not new to United Nations agencies. Coomans (2012) stated that:
Back in 1998 the UN Secretary-General called upon all entities within the UN system to mainstream human rights into their activities and programs. Since then, many UN agencies have adopted a human rights-based approach which is supposed to underlie the carrying out of their mandate. (p. 274)

However, the application of rights into evaluation design was an important consideration for the UNICEF evaluation office. We sought to extend the UN mandate by incorporating the CRPD into the evaluation methodology. Our rights-based approach was also deemed appropriate for the evaluation, which examined activities in 22 countries, because “UN Country Teams play an important role in translating rights into action … rights language only means something when it is backed by a translation of clear and concrete activities” (Coomans, 2012, p. 294).

The CRPD is the first-ever multilateral United Nations treaty designed to specifically identify the rights, including education, of persons with disabilities as a specific rights-bearing population. Of relevance to this evaluation was the definition of inclusive education included in CRPD's General Comment 4 (United Nations, 2016), which stated that inclusive education is:
A fundamental human right of all learners. Notably, education is the right of the individual learner and not, in the case of children, the right of a parent or caregiver. Parental responsibilities in this regard are subordinate to the rights of the child.

A principle that values the well-being of all students, respects their inherent dignity and autonomy, and acknowledges individuals’ requirements and their ability to effectively be included in and contribute to society.

A means of realizing other human rights. It is the primary means by which persons with disabilities can lift themselves out of poverty, obtain the means to participate fully in their communities, and be safeguarded from exploitation.¹ It is also the primary means of achieving inclusive societies.

The result of a process of continuing and proactive commitment to eliminating barriers impeding the right to education, together with changes to culture, policy, and practice of regular schools to accommodate and effectively include all students. (p. 3)
Also important to this evaluation was the CRPD's Article 24, which focuses on “an inclusive education system at all levels and lifelong learning.” Implementers of CRPD’s vision often call for a “twin track” approach that outlines that inclusion in local schools is a fundamental human right.² (Track 1). In order to achieve that right, education systems need to be accessible and provide, when necessary, accommodations and supplemental supports for children with disabilities (Track 2).

When aligned, the two tracks create a direction to uphold both a universal human right (education) and a plural right (supplemental accommodations and aids so that children with disabilities can experience their right to education). An example of the twin-track approach would be students who are blind accessing specialized braille reading classes through a trained special educator, while also training future cohorts of special educators to support students with vision disabilities in inclusive classrooms.

Framing the evaluation to interrogate the extent to which UNICEF Country Offices are upholding rights through twin-track approaches allowed for flexibility in how country programs could promote and support the right to inclusion but kept a focus on the specific additional rights of children with disabilities to avoid the detrimental effects of what Buchner et al. (2020) call the “invisibilization of disability” (p. 12) and an inherent elasticity in how governments describe inclusive education (Slee, 2011). For example, often in larger inclusion agendas, Buchner et al. argue that children with disabilities may get lost in reforms if they are not considered as a unique identity group. The authors, citing Article 31 of the CRPD (which focuses on statistics and data), call for disaggregated data to be collected within evaluations “to collect appropriate information, including statistical and research data, to enable them to formulate and implement policies to give effect to the present Convention” (p. 12). Thus, we specifically focused on a twin-track approach in relation to the right to inclusive education and the general guidance provided by the CRPD.

Typology and Case Study of a Rights-Based Evaluation

The rights-based approach that we developed had aspects of an empowerment-based evaluation, a professionally driven evaluation (Fetterman & Wandersman, 2005), and a theory-driven evaluation. For example, in empowerment-based evaluations, advocates within organizations dictate their own goals for evaluations, which was clear in this evaluation from the initial terms of reference for consultants—although readers should note that the evaluation design originated in an internal evaluation office and was not driven by any country or regional office. At the same time, Lackey, Moberg, and Balistrieri (1997) acknowledge that there are instances when organizations seek legitimacy through external evaluations.

In this case, we developed a hybrid approach that sought to capture UNICEF's stringent commitment to the rights of all children with the legitimacy that could be gained through using a universally agreed-upon set of norms (the CRPD) and content experts who could conduct the evaluation and interpret data in relation to agreed-upon norms (see Figure 1 for an example of how the process was developed). The human rights evaluation approach, in this case, was decided upon through constructive conversation between internal evaluation experts (who oversee, but do not conduct evaluations) and external content experts who have deep knowledge of the CRPD and its interpretation of inclusive education. Within the broader evaluation literature, we would consider this approach a “theory-driven process evaluation” (Chen & Chen, 2005) mainly because the approach had a theoretical orientation (human rights can inform inclusive practice), but there were no specific causal analyses built into this evaluation.

Figure 1.
A model for human rights-based evaluation.

Human rights-based evaluations (i.e., evaluations in which questions and analysis used human rights conventions as an evaluative lens) hold great promise and can simultaneously be informed by other evaluation models (as described above). Such evaluations can acknowledge local implementation innovations and the importance of context in relation to programming while at the same time “hold … a commitment to social justice as its fundamental core” (Naraian, 2016, p. 2). We argue that engaging in evaluations through a human rights framework offers another form of evaluation that may add novel approaches to theory and practice in the field (King & Alkin, 2019) and present a potentially transferable approach to the development of equity-focused evaluations (Catsambas & Bauer, 2015).

Throughout the remainder of this article, we seek to explain an evaluation methodology that is guided by a conceptual lens: human rights. This lens is particularly useful for evaluating programs focused on individuals who are marginalized in societies and may experience the suppressed opportunities to enjoy rights, freedoms, and opportunities. We hope this provides an additional tool for local and global evaluations that guide or complement evaluation design.

In the following section, we provide an overview of the specific methods used in the UNICEF evaluation as a case example. We present these and encourage scrutiny by fellow evaluators in support of developing the robustness of human rights-based evaluation approaches. In this way, organizations can begin to link evidence-based evaluation approaches with lofty human rights ideals. Satterthwaite (2012) argued that “a turn to evidence-based practice would emphasize specific methods to identify, document, and understand human rights violations” (p. 255). We concur with Satterwaithe and argue that formative and summative evaluations conducted through a rights-based lens can highlight rights-friendly practices that can be utilized as practical exemplars in large multinational organizations such as UNICEF.

Human Rights-Based Evaluation: A Case Study

The sections below provide an overview of the specific methodology and findings of a formative evaluation that used a human rights-based design. The methodology we employed for this evaluation was informed by normative program theory, which “provides guidance on what goals or outcomes should be pursued or examined and how the treatment should be designed and implemented” (Nkwake, 2013, p. 69). However, in this evaluation, we avoided a strict prescription of how inclusive education should be conducted. Rather, in this case, we focused on creating shared definitions (i.e., defining inclusion), broad principles (the right to inclusion), and concepts (twin-track approaches) that can guide local activity and innovation. Because this was a global evaluation, we expected and appreciated that there would be diverse programming activity across contexts.

The evaluation was formative and therefore designed to identify both promising and concerning practices. In this case, promising practices were those that aligned with the CRPD and focused on a rights-based approach to inclusion (UN ENABLE, 2007). The United Nations Department of Economic and Social Affairs (DESA) describes these as non-discrimination, recognition in the interaction between gender and disability, promotion of accessibility, meaningful partnership of people with disabilities, accountability mechanisms, awareness-raising of disability, utilization of effective partnerships, initiatives that are results-based, demonstrate measurable change and are appropriately resourced, replicable, and sustainable (United Nations Department of Economic and Social Affairs, 2011). Henderson, Mannan, and Power (2017) used a similar framework (EquiFrame) to evaluate community-based rehabilitation activities in the health sector. Examples of concerning practices were programs that (un)intentionally excluded children with disabilities from participation, spoke about persons with disabilities but neglected to consult them directly, or failed to collect any data on the impact of interventions on children with disabilities.

Three main methods were utilized: a desk review of United Nations documents related to disability and inclusive education, a desk review of annual and other reports from a sample of 22 countries, key informant interviews with stakeholders in partner organizations working on inclusive education initiatives with UNICEF, and site visits to conduct case studies of three countries. These reviews all aimed to answer the questions: (1) To what extent do UNICEF activities align with a rights-based approach to inclusive education (as outlined by the CRPD), and (2) In what ways (if any) are twin-track approaches to inclusive education implemented in country offices? However, the sponsoring agency UNICEF was also interested in gathering data on other domains, so we utilized data to answer 13 evaluation questions that covered both descriptive (what is happening) and normative (how program activities align with the CRPD) questions.

A stratified sample was used for data collection. UNICEF works in countries with a wide range of circumstances that may influence how inclusive education is implemented. Our sample, therefore, included countries from each of the seven operating regions and countries. The sample for the portfolio review and subsequent field visits was also selected to be representative of a range of inclusive education activities, from no activity to high levels of engagement. A table outlining the country offices selected for the evaluation is found in Table 1. The sample included:
Representation of countries ranging from weak to championing on inclusive education indicators of from an internal Strategic Monitoring Questions document. A minimum requirement was that all countries have an education portfolio (if not a dedicated inclusive education portfolio).

Regional representation (at least two countries per region).

To the extent possible, the inclusion of countries whose activities were impacted by humanitarian crises (as recommended in the inception meeting).

Diversity of economic circumstances.
Finally, although the evaluation was focused on UNICEF activities within countries, we confirmed that all sampled countries were signatories of the CRPD, and many of these countries also ratified the treaty. Table 1 provides an overview of the country office activities that were evaluated. The 22-country sample was arrived upon through conversations with country offices. Those that fit the criteria explained above and who could provide documentation of inclusive education activities in the past four years were included. Prior to reaching the final sample, some country offices removed themselves from the evaluation because they felt they were not focused on inclusive education as part of their portfolio of country office activities. Evaluators noted that complete lack of activity by some country offices in the final evaluation, but these countries were excluded from fieldwork and desk reviews because they had no programs or reports to evaluate.

Table 1.
Sample of Country Offices Selected for Portfolio Review and Site Visits.

Region Weak Implementation^a Initiating Implementation Established Implementation Championing Office

East Asia and Pacific Vietnam Thailand

Myanmar

Europe and Central Asia Ukraine Armenia

Armenia

Eastern and Southern Africa Mozambique Madagascar South Africa

Kenya

Latin America and Caribbean Ecuador Uruguay

Middle East and North Africa Djibouti Jordan

Morocco

South Asia Pakistan Bhutan Nepal India

Western and Central Africa Cameroon Togo Ghana

Source: Johnstone et al. (2019).

Abbreviation: UNICEF = United Nations International Children’s Emergency Fund.

a
As determined by internal UNICEF ratings on Country Office Activity Reports.

Because of the human rights-based approach to the evaluation, we felt it necessary to consider potential influences such as the economic situation of countries, the region in which activities were taking place, and potential humanitarian crises which could impact education program delivery. We believed this decision to be defensible based on Hansen, Klejnstrup, and Andersen's (2013) conclusions that evaluator knowledge and thick description of content and variables can contextualize claims, even when rigorous randomized samples are not present. We also feared that a random sample might over-identify nations from regions or with particular track records on inclusion. This evaluation was never intended to be experimental in nature, but we align with Hansen et al.'s conviction that a “well described” sample and well-understood programming are important to any evaluation, including one that has a rights-based framework. For this reason, our sample was devised as described above.

Evaluation Findings

In this section, we present a brief overview of findings in tabular form. The presentation of findings is intended to provide a contextual overview for readers to understand the 13 evaluation sub-questions and findings that emerged from the evaluation. Although the organization itself was interested in a wide range of indicators, including geographic coverage of programming, a focus on different levels of schooling (from pre-primary to post-secondary), and the role of headquarters in supporting global initiatives, a central theme of human rights permeated all of the evaluation findings and provided solid footing for lessons learned.

By using a human rights-based normative instrument (i.e., the CRPD) as part of the deductive coding process for qualitative data, we learned that UNICEF did not have a consistent or shared definition of inclusive education across headquarters and country office implementation. Through the evaluation, we learned that there were primarily three conceptualizations of “inclusive education” being used in guidance and program implementation. The first conceptualization was a universal approach to inclusive education that focused on “all” children and contemplated how a broad category of “vulnerable” children could be included. This conceptualization was aligned with the CRPD's vision of inclusive education by supporting the rights of all children to education (CRPD Track 1 alignment) but in most cases did not consider the additional programming accommodations or modifications needed for children to realize this right (CRPD Track 2 gap), despite its focus on vulnerability. In other countries, UNICEF was supporting the education of children with disabilities who were receiving highly segregated settings, often in settings that were segregated from other children in the community and frequently far from the children's hometown. Finally, we also discovered several models that focused on inclusive education for all children (CRPD Track 1 alignment) with supportive infrastructure for children with disabilities (CRPD Track 2 alignment). For the purposes of the evaluation, we called these successful models “disability-inclusive education.”

A second important finding of the evaluation was the way in which programs were implemented. The CRPD calls for the right to education to be experienced at all levels of education, supported by high-quality experiences in classrooms and disaggregated data to track progress. A human rights perspective lens helped tease out the strengths and gaps of programming. For example, UNICEF focused heavily on access to education at the early levels (pre-primary and primary) but was not focused on quality experiences for students once included or the development of opportunities for students with disabilities beyond the early years of education. Further, despite important gains in access to education for children with disabilities, there is virtually no data on either the educational experiences (quality) or outcomes of such access. A rights-based framing highlighted issues of access, quality, and progression in education that are outlined in the CRPD and allowed the evaluation team to identify programming priorities after the formative evaluation. Table 2 provides a full overview of the evaluation questions and findings. They are listed as an example of how human rights-based evaluation framing—in this case, through the lens of the CRPD, its Article 24, and General Comment Number 4—can inform an evaluation framework, questions, and relevant organizational data to act upon. It is beyond the scope of this article to discuss evaluation findings themselves in detail. A summary table of all evaluation findings, linked with the original project evaluation questions, is provided in Table 2.

Table 2.
Evaluation Findings.

Evaluation Question Evaluation Finding

What are the conceptual underpinnings, frameworks, and evidence base that inform UNICEF framing of inclusive education as it applies to children with disabilities? UNICEF does not have a consistent or shared definition of inclusive education.

Is UNICEF framing of inclusive education sufficiently aligned with the CRPD and General Comment No. 4? UNICEF framing of inclusive education concerning the CRPD varies widely. Some documents and activities explicitly align with the CRPD while others are out of alignment with the Convention.

Is there a shared understanding, among Education and Disability-focused staff of how UNICEF conceptualizes Inclusive education as applied to children with disabilities? If not, what/where are the discrepancies? Discrepancies exist between headquarters, regional offices, Country offices (Nyers, 1999), and field-level staff exemplifying various understandings of what is inclusive education and how it should be implemented. At the country office level, there is limited agreement on what disability-inclusive education is, and no clear pathway for local focal point staff to receive further clarification.

What is the range of activities, interventions, or Strategies that UNICEF supports in delivering inclusive education? Across country offices, UNICEF supports an extraordinary range of activities. Among these, teacher development (training), policy support, and early intervention appear to be the most common. UNICEF plays a clear role in thought leadership around inclusive education through the convening of national and international experts for reports, workgroups, etc.

What activities, interventions, or strategies are being implemented in relation to child outcomes? UNICEF has successfully documented new enrollment of children with in schools, but data is lacking on the quality of education children experience in schools and the outcomes of inclusive education for children with disabilities.

Are interventions and implementation strategies aligned with the guiding principles of the CRPD? Most UNICEF Cos align activities with some aspects of the CRPD, but At times have also supported segregated education (where children with disabilities lack access to mainstream education with same-aged peers without disabilities).

What is the coverage of the interventions in relation to the demand for lifelong learning communities they serve, specifically related to children with disabilities? UNICEF programming focuses heavily on access to pre-primary and primary but does not typically consider how children might transition from primary or segregated schooling into higher levels of schooling.

What is the extent of geographic coverage of programs For children with disabilities? In general, UNICEF country offices focused on supporting programming across various regions of their countries. Only in rare cases was programming narrowly focused on capital cities.

What are the data and technical resource needs and sources? UNICEF has been successful in advocating for disability-disaggregated data on internal reports and national data systems. Data captures incidence and enrollment, but not often the quality of outcome indicators.

What programming tools are available to UNICEF country offices (data, guidance, etc.)? UNICEF country offices have produced multiple tools and guidance documents in some areas but are lacking in others. UNICEF efforts have primarily focused on early childhood and primary education, but less frequently address higher levels of education or whole system inclusion.

To what extent are available tools adequate for disability-inclusive education programming? Internal capacity for developing tools for adequate disability-inclusive education programming is a growth area for UNICEF. Disability focal points at times have little or no formal training in disability-inclusive education.

What evidence is there, if any, of UNICEF contribution to a change of attitudes toward children with disabilities? Anecdotal and proxy data suggest a change in community attitudes toward children with disabilities when UNICEF initiates sensitization activities, but more data is needed in this area.

What evidence is there, if any, of UNICEF contribution to a change in educational outcomes for children with disabilities? Is progress equitable (in terms of gender, income level, locality, etc.)? Data from a variety of sources indicate that UNICEF efforts have primarily focused on access to education for children with disabilities, with no information about the quality of educational experience or outcomes for these children.

Source: Johnstone et al. (2019).

Abbreviations: CRPD = Convention on the Rights of Persons with Disabilities; UNICEF = United Nations International Children’s Emergency Fund.

Human Rights-Based Evaluations: Limitations

Niccolai (2020) observed that rights-based approaches can provide a solid grounding for equal opportunity for individuals through ensuring rights are upheld in programming. At the same time, Niccolai (2020) acknowledges that in a rights-based orientation “the conditions of freedom of human beings depend … on ‘provisionally benevolent institutions’” (p. 116). In our case, the rights of children with disabilities may be wrapped into national policy and UNICEF programming, but are at risk of being taken away if national or organizational agendas shift. Organizations themselves change priorities periodically, and there is a danger for children with disabilities if organizational priorities shift away from human rights. Rights-based evaluations, then, may be subject to the same political vagaries within organizations as other forms of evaluation.

A second limitation to this type of evaluation is that human rights take a universalist approach and the assumption that ideals outlined in human rights documents are the ideas of local communities. Human rights outline universal ideals of what ought to be, predominantly from a Western perspective (Uvin, 2007) that imposes Western moral and political theory and neoliberal economic ideals (Nyers, 1999). Basing an evaluation on universal rights risks a neocolonial imposition of Western values onto other contexts and may not account for the local needs and interests of the communities that those rights set out to protect. A natural consequence for communities focused on sovereignty and their own approaches to disability education is that a rights-based approach may introduce a neocolonial approach to evaluation (Cavino, 2013). In this way, further research that can connect human rights and decolonizing evaluation (Cram, 2015) is needed, especially in relation to sovereign territories within nation-states and for groups that seek intellectual sovereignty from imperial or settler colonization.

A third limitation is the concept of rights itself is largely approached through an individualistic frame. Further work is needed to translate how the rights of individuals can be upheld within broader collective rights in any given society. To this end, Singal and Muthukrishna (2014) contend that:
The notion of collective rights … must always be considered important to any debate on disability rights in the global South … when concepts and discourses such as human rights are disassociated from the complexities of local contexts, their relevance is directly drawn into question. (p. 298)

Fourth, this approach takes a disability-affirmative position, but further interrogation is needed regarding the production of impairment through global violence imposed, often, from the North to the South. For example, Soldatic and Grech (2014) highlight how structural adjustment programs (SAPs), free trade agreements, impositions such as chemical-dependent monocropping, and settler colonialism have been all linked to impairment and need to be part of a broader praxis related to impairment, disability rights, and broader economic, environmental, and social justice agendas. Soldatic and Grech's recommendations were not included in the case study but should be a consideration in rights-based approaches.

Fifth, because of the technical nature of rights-based evaluation, Satterthwaite (2012) warned that there might be a privileging of evaluation opportunities for “translator figures” who are proficient with rights discourses, laws, and policies. Referring to “translator figures,” Satterthwaite (2012) stated that human rights translators will be:
Increasingly empowered because they possess technical expertise in techniques of metrics and measurement and in human rights law. Such translators are frequently hired as consultants to create, evaluate, or advise on the use of indicators. This empowerment sometimes comes at the expense of grassroots, ‘‘local,’’ or embedded forms of knowledge, and the drive to quantify tends to ‘‘render technical.’’ (p. 255)

Satterwaite's point is not lost in this discussion as an increased focus on rights-based evaluations may create new exclusions within the field itself. The evaluation team all had strong backgrounds in disability rights as part of their portfolios, and all happened to be from the United States. Finally, Le Fanu (2014) argues that donor-recipient relations are often managerialist in nature and may act as a form of surveillance on the daily activities of recipient organizations. Le Fanu calls for more “inclusion” in the form of greater control by local implementing organizations (which may or may not align with centralized evaluative frameworks).

The risks and limitations identified above were all present in this evaluation. Certainly, as Niccolai asserted, the rights of children with disabilities could easily become less of a priority for UN organizations (although this will not happen soon because UNICEF's 2022–2025 strategic plan identifies disability inclusion as a priority area). The second limitation identified by Uvin and others was addressed through the plural rights framework we embraced. Indeed, universal human rights can be limiting, but a plural rights perspective offsets these limitations. Finally, this evaluation fell short of identifying the colonial forces that may have been the root of impairment for children in the sample countries. Further work is needed from this perspective for future evaluation to have a comprehensive understanding of impairment, disability, exclusion, and inclusion.

Despite its limitations, this evaluation work was a form of reciprocal evaluation capacity building (Catsambas & Bauer, 2015). Prior to the project, neither the UNICEF office nor consultants had considered a CRPD-focused human rights evaluation, but through consultation with one another developed a schema and approach that yielded informative results. One reason for this was that one of the original goals was cross-national comparison and some form of framework was required. A rights-based focus emerged from consultation between internal and external evaluators as an appropriate framework. It is unknown the extent to which this evaluator co-creation experience informs Catsambas and Bauer (2015) framing of evaluation capacity building, but it is likely more engagement with evaluators from a broad spectrum of backgrounds may have even further enhanced the final product. In this case, consultants did not build the capacity of headquarters staff, but learning was reciprocal. Certainly, further consultations with national-level stakeholders would have enhanced the design and framing of rights-based approaches. We believe that conscious effort and forethought are required to further develop rights-based evaluations that can address all the risks we outlined above.

Conclusions and Future Directions

This article proposes and provides a practice-based example of a human rights-based approach to evaluation. Our use of the CRPD in this evaluation reflected a commitment to a generally agreed-upon set of principles by nations around the world—including those in our sample. In this way, we avoided external interventionism related to educational expectations by using a set of principles familiar to policy makers in both UNICEF and its partner national governments.

Rights-based evaluations hold great potential in spelling out the ways in which plural actors can experience universal human rights in programs. In this case, our aim was to recognize the right to inclusive education in UNICEF programming (based on its own mission alignment with the Convention) to inform future institutional programming. The disability rights-affirmative positions taken in this evaluation article are central to this evaluation approach, and future rights-focused evaluations should interrogate the structural, economic, environmental, and political conditions that may predict why children are being excluded from schools.

We conclude that a rights-based evaluation approach (which includes using a human rights convention to develop evaluation questions, interpreting evaluation data through a rights lens, and make recommendations informed by rights frameworks) can be a powerful tool in ensuring that programming avoids approaches that are patronizing toward populations, focus on charity over development, or reproduce exclusions that may already exist in a society.

This article presented the opportunities and challenges associated with taking a human rights-based, normative approach to evaluation. The approach we described relies on a broad-based theory of universal human rights which posits that all humans, by nature of their humanness, are entitled to freedoms to (i.e., the right to education) and freedoms from social ills (i.e., freedom from persecution, abuse, etc.) (United Nations, 1948) and an acknowledgment of the need for plural rights perspectives (Mégret, 2008).

The findings from this study may lead to several future directions for human rights evaluation. As noted above, this evaluation was informed by both universal and plural rights conceptualizations (Mégret, 2008). To this end, the approaches and methods may be transferable to other populations, as all humans may expect to live with basic, universal human rights, and there are multitudes of other groups who face systemic exclusion and oppression, and for whom evaluating programs through a lens of human rights may be beneficial.

Another direction related to inclusive education specifically is to test the hypothesis that greater adherence to human rights in programming leads to, or predicts, greater inclusion outcomes. In this evaluation, we witnessed the importance of centering human rights in a UNICEF evaluation as a codified approach for evaluations. Instruments such as the CRPD can be easily incorporated into future evaluation designs for either other excluded populations or to examine the impact of rights-based orientations on program outcomes. In all cases, local contextualization will also need to be a key feature of these designs.

Finally, we recommend further research that can expand and operationalize human rights-based evaluation as a theory-based evaluation approach. To do so, we draw upon Coryn et al.'s (2011) principles of theory-driven evaluation to spell out the next steps for evaluators and researchers to consider. These are:
Develop a full formulation of human rights theory for evaluation. This may require a “plural” approach that asks, “human rights for whom?”

Expand questions to identify the formulation and prioritization of particular rights.

Identify human rights-based planning, design, and execution of programming.

Continue human rights-focused construct development and measurement.

Identify human rights breakdowns and side effects, determining program effectiveness or efficacy, and causal explanations (based on Coryn et al., 2011).
The formative evaluation and its human rights-based approach described in this article have provided indications and exploratory direction for a form of evaluation that seeks to uphold the safety, dignity, and freedom of all humans in development programming. In the case of this evaluation, our human rights-based approach helped define and spell out parameters of rights-based inclusive education for children with disabilities. In evaluating both conceptualization and implementation, we hoped that a rights-based discourse would reinforce an organizational ontology (Dahler-Larsen, 2018) or at least an organizational orientation in UNICEF that inclusive education is a human right and therefore must be an underlying tenet to education programs for children with disabilities. This right cannot be met when conceptualizations include segregationist approaches to education³ or education that assumes that all children learn and experience inclusion in the same way. The norms we espouse and that drove this work were not based on evaluators’ opinions or organizational trends. Rather, explanations of what exactly is meant by human rights-based education and its implementation are found in United Nations treaties, such as the CRPD. In this case, human rights-based evaluation acted simply as a mirror to human rights-based conceptualizations of, and programming in, education.

Evaluation Question	Evaluation Finding
What are the conceptual underpinnings, frameworks, and evidence base that inform UNICEF framing of inclusive education as it applies to children with disabilities?	UNICEF does not have a consistent or shared definition of inclusive education.
Is UNICEF framing of inclusive education sufficiently aligned with the CRPD and General Comment No. 4?	UNICEF framing of inclusive education concerning the CRPD varies widely. Some documents and activities explicitly align with the CRPD while others are out of alignment with the Convention.
Is there a shared understanding, among Education and Disability-focused staff of how UNICEF conceptualizes Inclusive education as applied to children with disabilities? If not, what/where are the discrepancies?	Discrepancies exist between headquarters, regional offices, Country offices (Nyers, 1999), and field-level staff exemplifying various understandings of what is inclusive education and how it should be implemented. At the country office level, there is limited agreement on what disability-inclusive education is, and no clear pathway for local focal point staff to receive further clarification.
What is the range of activities, interventions, or Strategies that UNICEF supports in delivering inclusive education?	Across country offices, UNICEF supports an extraordinary range of activities. Among these, teacher development (training), policy support, and early intervention appear to be the most common. UNICEF plays a clear role in thought leadership around inclusive education through the convening of national and international experts for reports, workgroups, etc.
What activities, interventions, or strategies are being implemented in relation to child outcomes?	UNICEF has successfully documented new enrollment of children with in schools, but data is lacking on the quality of education children experience in schools and the outcomes of inclusive education for children with disabilities.
Are interventions and implementation strategies aligned with the guiding principles of the CRPD?	Most UNICEF Cos align activities with some aspects of the CRPD, but At times have also supported segregated education (where children with disabilities lack access to mainstream education with same-aged peers without disabilities).
What is the coverage of the interventions in relation to the demand for lifelong learning communities they serve, specifically related to children with disabilities?	UNICEF programming focuses heavily on access to pre-primary and primary but does not typically consider how children might transition from primary or segregated schooling into higher levels of schooling.
What is the extent of geographic coverage of programs For children with disabilities?	In general, UNICEF country offices focused on supporting programming across various regions of their countries. Only in rare cases was programming narrowly focused on capital cities.
What are the data and technical resource needs and sources?	UNICEF has been successful in advocating for disability-disaggregated data on internal reports and national data systems. Data captures incidence and enrollment, but not often the quality of outcome indicators.
What programming tools are available to UNICEF country offices (data, guidance, etc.)?	UNICEF country offices have produced multiple tools and guidance documents in some areas but are lacking in others. UNICEF efforts have primarily focused on early childhood and primary education, but less frequently address higher levels of education or whole system inclusion.
To what extent are available tools adequate for disability-inclusive education programming?	Internal capacity for developing tools for adequate disability-inclusive education programming is a growth area for UNICEF. Disability focal points at times have little or no formal training in disability-inclusive education.
What evidence is there, if any, of UNICEF contribution to a change of attitudes toward children with disabilities?	Anecdotal and proxy data suggest a change in community attitudes toward children with disabilities when UNICEF initiates sensitization activities, but more data is needed in this area.
What evidence is there, if any, of UNICEF contribution to a change in educational outcomes for children with disabilities? Is progress equitable (in terms of gender, income level, locality, etc.)?	Data from a variety of sources indicate that UNICEF efforts have primarily focused on access to education for children with disabilities, with no information about the quality of educational experience or outcomes for these children.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the UNICEF Evaluation Office (grant number Formative Evaluation of Disability-Inclusive Educa).

ORCID iD

Christopher J. Johnstone

Notes

References

Bamberger

Tarsilla

Hesse-Biber

. (2016). Why so many “rigorous” evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute. Evaluation and Program Planning, 55, 155–162. https://doi.org/10.1016/j.evalprogplan.2016.01.001

Buchner

Shevlin

Donovan

M. A.

Gercke

Goll

Šiška

Janyšková

Smogorzewska

Szumski

Vlachou

Demo

Feyerer

Corby

(2020). Same progress for all? Inclusive education, the United Nations Convention on the rights of persons with disabilities and students with intellectual disability in European countries. Journal of Policy and Practice in Intellectual Disabilities, 18(1), 7–22. https://doi.org/10.1111/jppi.12368

Caldwell

L. D.

Bledsoe

K. L

. (2019). Can social justice live in a house of structural racism? Question for thefield of evaluation. American Journal of Evaluation, 40(1), 6–18. https://doi.org/10.1177/1098214018815772

Catsambas

T. T.

Bauer

. (2015). Creating a global movement in evaluation. The American Journal of Evaluation, 36(2), 256–269. https://doi.org/10.1177/1098214015573548

Cavino

H. M

. (2013). Across the colonial divide: Conversations about evaluation in Indigenous contexts. American Journal of Evaluation, 34(3), 339–355. https://doi.org/10.1177/1098214013489338

Chen

H. T.

Chen

H. T.

(2005). Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Sage.

Coomans

. (2012). On the right(s) track? United Nations (specialized) agencies and the use of human rights language. Verfassung Und Recht in Übersee, 45(3), 274–294. https://www-jstor-org.web.bisu.edu.cn/stable/43239646

Coryn

C. L. S.

Noakes

L. A.

Westine

C. D.

Schröter

D. C

. (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32(2), 199–226. https://doi.org/10.1177/1098214010389321

Cousins

J. B.

Earl

L. M

. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14(4), 397–418. https://doi.org/10.3102/01623737014004397

10.

Cram

. (2015). Lessons on decolonizing evaluation from Kaupapa Māori evaluation. Canadian Journal of Program Evaluation, 30(3), 296–312.

11.

Dahler-Larsen

. (2018). Theory-based evaluation meets ambiguity: The role of Janus variables. American Journal of Evaluation, 39(1), 6–23. https://doi.org/10.1177/1098214017716325

12.

de Waal

. (2016). Writing human rights and getting it wrong. Boston Review, 41(3), 1–14. https://www.bostonreview.net/articles/alex-de-waal-writing-human-rights/

13.

Fetterman

Wandersman

(2005). Empowerment evaluation principles in practice. Guilford Press.

14.

Gable

Meier

B. M

. (2013). Global health rights: Employing human rights to develop and implement the framework convention on global health. Health and Human Rights, 15(1), 17–31. https://www.hhrjournal.org/2013/10/global-health-rights-employing-human-rights-to-develop-and-implement-the-framework-convention-on-global-health/

15.

Hansen

Klejnstrup

N. R.

Andersen

O. W

. (2013). A comparison of model-based and design-based impact evaluations of interventions in developing countries. American Journal of Evaluation, 34(3), 320–338. https://doi.org/10.1177/1098214013476915

16.

Henderson

Mannan

Power

. (2017). Disability inclusive development good practices: Level of commitment to core concepts of human rights. Disability, CBR and Inclusive Development, 29(3), 32–55. https://doi.org/10.5463/dcid.v28i3.608

17.

Johnstone

C. J.

Hayes

Cohen

Niad

(2019). Inclusive education: A formative evaluation. https://www.unicef.org/evaluation/reports#/detail/16612/formative-evaluation-of-inclusive-education-for-children-with-disabilities

18.

King

J. A.

Alkin

M. C

. (2019). The centrality of use: Theories of evaluation use and influence and thoughts on the first 50 years of Use Research. American Journal of Evaluation, 40(3), 431–458. https://doi.org/10.1177/1098214018796328

19.

Lackey

J. F.

Moberg

D. P.

Balistrieri

. (1997). By whose standards? Reflections on empowerment evaluation and grassroots groups. American Journal of Evaluation, 18(2), 137–146. https://doi.org/10.1177/109821409701800205

20.

Le Fanu

. (2014). International development, disability, and education: Towards a capabilities-focused discourse and praxis. International Journal of Educational Development, 38, 69–79. https://doi.org/10.1016/j.ijedudev.2014.04.001

21.

Mégret

. (2008). The disabilities convention: Human rights of persons with disabilities or disability rights? Human Rights Quarterly, 30(2), 494–516. https://www-jstor-org-443.web.bisu.edu.cn/stable/20072851

22.

Naraian

. (2016). Inclusive education complexly defined for teacher preparation: The significance and use of error. International Journal of Inclusive Education, 20(9), 946–961. https://doi.org/10.1080/13603116.2015.1134682

23.

Niccolai

(2020). What is happening to the norm? Gender as paradigm of a deformalized neolegal positivism. In Condello

(Ed.). New rhetorics for contemporary legal discourse (pp. 101–118). University of Edinburgh Press.

24.

Nkwake

(2013). Working with assumptions in international development program evaluation. Springer.

25.

Nyers

(1999). Emergency or emerging identities? Refugees and transformations in world order. Millennium, 28(1), 1–26. https://doi.org/10.1177/03058298990280010501

26.

Patton

M. Q.

(1986). Utilization-focused evaluation (2nd ed.). Sage Publications.

27.

Royse

Thyer

Padgett

(2010). Program evaluation: An introduction (5th ed.). Cengage Learning.

28.

Satterthwaite

. (2012). Measuring human rights: Indicators, expertise, and evidence-based practice. American Society of International Law Proceedings, 106, 253–256. https://doi.org/10.5305/procannmeetasil.106.0253

29.

Singal

Muthukrishna

. (2014). Education, childhood and disability in countries of the south—Re-positioning the debates. Childhood (Copenhagen, Denmark), 21(3), 293–307. https://doi.org/10.1177/0907568214529600

30.

Slee

(2011). The irregular school: Exclusion, schooling and inclusive education. Taylor & Francis Group.

31.

Soldatic

Grech

. (2014). Transnationalising disability studies: Rights, justice and impairment. Disability Studies Quarterly, 34(2). https://dsq-sds.org/article/view/4249/3588

32.

UNICEF/UNESCO. (2007). A human rights-based approach to education for all. https://www.right-to-education.org/sites/right-to-education.org/files/resource-attachments/A%20Human%20Rights-based%20Approach%20to%20Education%20for%20All_0.pdf

33.

United Nations. (1948). Universal declaration of human rights. United Nations. https://www.un.org/en/about-us/universal-declaration-of-human-rights

34.

United Nations. (2016). General Comment No. 4 on the Right to Inclusive Education. Convention on the Rights of Persons with Disabilities. https://www.ohchr.org/Documents/HRBodies/CRPD/GC/RighttoEducation/CRPD-C-GC-4.doc

35.

United Nations Department of Economic and Social Affairs. (2011). Best practices for including persons with disabilities in all aspects of development efforts. http://www.un.org/disabilities/documents/best_practices_publication_2011.pdf

36.

United Nations Educational, Scientific, and Cultural Organization (UNESCO). (2020). Global education monitoring report 2020: Inclusion and education: All means all. https://unesdoc.unesco.org/ark:/48223/pf0000373718

37.

United Nations ENABLE. (2007). United Nations Convention on the Rights of Persons with Disabilities and Optional Protocol. United Nations. https://www.un.org/disabilities/documents/convention/convoptprot-e.pdf

38.

United Nations Office of the High Commissioner on Human Rights. (2008). Convention on the rights of persons with disabilities. https://www.ohchr.org/en/instruments-mechanisms/instruments/convention-rightspersons-disabilities

39.

Uvin

. (2007). From the right to development to the rights-based approach: How “human rights” entered development. Development in Practice, 17(4/5), 597–606. https://doi.org/10.1080/09614520701469617

Region	Weak Implementation^a	Initiating Implementation	Established Implementation	Championing Office
East Asia and Pacific		Vietnam	Thailand
East Asia and Pacific		Myanmar
Europe and Central Asia		Ukraine	Armenia
Europe and Central Asia		Armenia
Eastern and Southern Africa		Mozambique	Madagascar	South Africa
Eastern and Southern Africa			Kenya
Latin America and Caribbean		Ecuador	Uruguay
Latin America and Caribbean
Middle East and North Africa		Djibouti	Jordan
		Morocco

South Asia	Pakistan	Bhutan	Nepal	India
Western and Central Africa	Cameroon	Togo	Ghana