Abstract
Youth participatory evaluation (YPE) engages young people in the evaluation process. Over the last 20 years, the field has emerged, but more work is needed as evaluators explore the potential for youth participation within their own evaluation practices. This article describes a practice-oriented matrix for visualizing and conceptualizing the role of young people in evaluation. In this article, we describe the matrix, provide examples from practice to illustrate its potential, and raise considerations for the field, including the role of adults in YPE, and the benefits and limitations for youth and for the evaluation process. We conclude with potential future directions of YPE for the field.
Youth participatory evaluation (YPE) engages young people in the evaluation process. It can be adult led, youth led, or intergenerational, but at the core, involves young people in the practice and process of evaluation, including the conceptualization, data collection, analysis, and use of evaluation findings (Goodyear & Checkoway, 2003; Sabo Flores, 2003; Sabo Flores, 2007). Engaging youth in evaluation efforts can lead to positive changes in organizational practices and can increase positive impacts on youth and communities (Roholt, Baizerman, & Hildreth, 2013).
Over the last 20 years, there have been multiple efforts in the field to conceptualize and explore the potential of YPE practices (Ashton, Arnold, & Wells, 2010; Checkoway & Richards-Schuster, 2003; Goodyear & Checkoway, 2003; Powers & Tiffany, 2006; Sabo Flores, 2003; Sabo Flores, 2007; Teixeira, 2015). Some of the earliest scholarship emerged in the late 1990s in conjunction with the international passage of the United Nations Convention on the Rights of the Child series of international projects on youth engagement. In the early 2000s (2001, 2002), a set of convenings at the Wingspread National Conference Center brought together a group of academics, foundations, youth, and practitioners to focus on the potential for youth participation in research and evaluation. Through these meetings, the group developed a “Wingspread Declaration” of principles for practice and a special issue on the field (Goodyear & Checkoway, 2003; Checkoway, Dobbie, & Richards-Schuster, 2003). In addition, there were a series of books, special issues, articles, and tool kits that helped to shape the field (e.g., Arnold & Gifford, 2015; Camino, Zeldin, Mook, & O’Connor, 2005; London, Zimmerman, & Erbstein, 2003; Sabo Flores, 2003; Sabo Flores, 2007). By the mid-2000s, there were regular panels and papers on YPE at the American Evaluation Association conference, including the development of the Youth Focused Evaluation Topical Interest Group (Richards-Schuster, Sabo Flores, White, & Arnold, 2015). However, despite the growth in practice and research, there are still gaps in understanding YPE in practice.
This article emerged out of various consultations and workshops with community-based organizations and their evaluators who were seeking simplified tools and resources to help understand the potential for YPE practices. While there are case studies and scholarship exploring practices (Arnold & Gifford, 2015; Roholt et al., 2013; Sabo Flores, 2007), we heard from community organization staff the need for a quick tool that could help staff visualize and buy in to the idea of YPE and to quickly see the range of potential for the practice within their own evaluation efforts. As more organizations begin to explore the potential for YPE, this article aims to contribute to the development of practice by describing a simplified practice-oriented matrix and identifying important considerations needed in the process of involving youth within evaluation efforts.
Youth Participatory Evaluation (YPE)
The idea of YPE draws from elements of action research, collaborative and participatory evaluation, empowerment evaluation, and utilization-focused evaluation (Delgado, 2006; Fetterman & Wandersman, 2007). Participatory evaluation is rooted in a paradigm that understands evaluation to be contextually based, socially constructed, democratic, and political (Cooper, 2018). It stands in contrast to the idea of evaluation as a “closed or linear” process (Guba & Lincoln, 1989 in Cooper, p. 51). Participatory evaluation focuses on the process of evaluation, the developmental effects on youth from the participation, as well as the product of the evaluation and the changes that result (Goodyear & Checkoway, 2003; Powers & Tiffany, 2006; Sabo Flores, 2003; Sabo Flores, 2007).
YPE practices extend concepts of participatory evaluation primarily through the specific involvement of youth in the process and a focus on the relationships between and roles among youth and adults within the evaluation context. Like other forms of participatory evaluation, YPE contextualizes the lived experiences of youth, positions youth as experts in their lives, and draws on youth knowledge and perspectives in the practice of understanding programs, organizations, and communities. However, different from approaches that engage adults, YPE explores evaluation through a developmental lens and is aimed at developing skills and capacities (Powers & Tiffany, 2006). Young people, in relation to adults, are in a different developmental stage of life, whereby they have the ability to learn and grow from an experience in ways that can substantially impact their life direction (Lerner, Almerigi, Theokas, & Lerner, 2005). Thus, YPE approaches often focus on the specific developmental stages of youth and explore the benefits to youth participants through a positive youth developmental framework (Ozer, 2016; Powers & Tiffany, 2006; Sabo Flores, 2007).
YPE has the potential to reposition young people as valuable contributors who have knowledge not available to adults without their consultation and guidance. When adults seek the guidance of youth, the inherent power and worth of youth is valued. In particular, youth participatory approaches may be particularly powerful for youth of color, enabling them to challenge, critique, and reshape understanding of conditions and to see themselves as positive leaders rather than to internalize racial stereotypes (Ozer, 2016).
YPE also offers youth hard and soft skills that can prepare them for success in college and future work settings. Youth may expand their social capital by engaging in the research process, gathering evidence, presenting a coherent and well-supported argument, and using critical thinking to analyze a social setting or social problem, which are all skills that are necessary for postsecondary success and for well-paying careers (Anyon, Bender, Kennedy, & Dechants, 2018). Connections with adults in institutions of power can also help youth develop a “thicker” network, as well as behaviors, skills, and knowledge valued by those in positions of access and power (Stanton-Salazar, 2010). This form of evaluation builds directly on the framework of positive youth development, which repositions youth as assets to their communities, rather than as problems that need to be fixed (Lerner et al., 2005).
Adults are very important in YPE, given their access to knowledge and relationships that can either expand or contract youth’s role in the inquiry process. Adults have a critical role for identifying and welcoming youth as coresearchers, helping them gain a sense of belonging, and creating a setting where their thoughts and contributions are respected (Zeldin, Christens, & Powers, 2012). Understanding the roles of adults and the importance of navigating power with youth are important for YPE. Lacking this understanding can create power imbalances that may negatively impact young people’s sense of their worth, particularly when they are seen as less knowledgeable or prepared than their adult counterparts.
Within evaluation there are existing tools to explore the variety of participatory approaches and practices, but they tend to focus on community-based participatory or empowerment evaluation and less specifically on youth. There is a continued need for simplified approaches to specifically visualize and conceptualize YPE as a form of practice.
Youth Participation Matrix
The matrix (see Table 1) presented in this article emerged from a workshop with youth development providers in a major city in the United States. The workshop’s goal was to provide practical ideas to youth development organizations seeking to increase youth participation in their quality improvement work. The matrix was piloted at the workshop to help the organizations visualize YPE broadly speaking. After the workshop, we began to hear from other practitioners in other communities—from program managers to evaluators—interested in having for practical tools to help visualize the potential for YPE in their organizations or communities. Thus, the matrix began to have a broader value.
A Practice Matrix: Involving Young People in Evaluation. (Adapted from Chekcoway & Richards-Schuster, 2003).
The matrix evolved from an earlier effort to explore the potential of youth participation in evaluation (Checkoway & Richards-Schuster, 2003). This earlier effort laid out the types and roles of youth evaluators, including youth as subjects, youth as consultants, youth as partners, and youth as directors. This was an important first step to begin to outline the ways that young people engage in research and evaluation work. Some scholars have used this initial paper as a framework for meta-reviews of youth roles within community-based participatory research or practice (Jacquez, Vaughn, & Wagner, 2013). But, it was primarily a conceptual piece that was not developed to be a practice tool for evaluators interested in exploring the potential for youth involvement in evaluation.
While a useful first step, this initial conceptualization is often limiting in its practical use because it falls short in helping practitioners and evaluators specifically visualize the possibilities for youth participation in evaluation in ways that could be replicated or adapted. Thus, to help expand understanding about the roles of young people in evaluation, we conceptualized this matrix, which provides illustrative examples of what participation might look like across multiple levels of evaluations, for example, of programs, organizations, and communities. The goal was to develop the earlier conceptualization into a matrix for practice that could be used by both evaluators and practitioners and have implications for further evolving the field.
It is critical that we identify the limitations of the matrix as caveats for the reader. First, this matrix was conceived and conceptualized within a U.S. context, so it may need further adaptation for global use depending on the way youth participation is framed and conceptualized. Second, it is important to note that the examples are just illustrations of potential roles, and they are not meant to be comprehensive of all the potential ways that young people could engage in evaluation. Third, this matrix is meant to be a tool or framework to visualize youth participation in evaluation and its potential for practice. We do not assume it to be applicable in all contexts.
Describing the Matrix
The matrix has three dimensions: Dimension 1 describes the roles young people can play: consultants, collaborators, partners, and leaders, including prototypical activities youth would engage in as evaluators; Dimension 2 articulates youth’s role based on levels of evaluation: program evaluation, organizational evaluation, and community evaluation; and Dimension 3 offers critical considerations for successful youth involvement in evaluation. This matrix builds on but still differs from the original matrix, which only focused on roles of youth in evaluation, namely youth as subjects, youth as consultants, youth as partners, and youth as directors (Checkoway & Richards-Schuster, 2003). We believe that the value of this new matrix is less in the three dimensions independently but rather in the cells that link the dimensions together to offer illustrative examples of what youth participation in evaluation could look like based on role, level, and the related considerations for evaluation stakeholders. In total, the matrix gives practitioners a scaffolded view of how youth can be increasingly involved in different types of evaluations so they can make conscious choices about engagement and support to youth.
Dimension 1: Roles of Youth
In this matrix, there are four variations of youth participation across the top: youth as consultants, youth as collaborators, youth as partners, and youth as leaders. In these roles, young people could engage in evaluation at various stages of the work, from the planning to the data collection or the reporting. For the purpose of this article, we describe the roles and contributions of young people broadly without specific attention to stage of evaluation. However, we recognize that this added element could be of use in a future adaptation of this matrix. Because this content draws on an earlier paper (Checkway & Richards-Schuster, 2003), we focus briefly on describing these roles, and we expand on the implications of each role for evaluation processes.
Youth as consultants
Youth as consultants may be the most common form of youth participation in evaluation (Jacquez et al., 2013). In this form, youth’s expertise is used to help shape and guide the evaluation to be more effective. Youth who are consultants stand to benefit minimally from the evaluation, though may appreciate the opportunity to share their opinions with adults.
Youth as collaborators
Youth as collaborators is the idea that young people are engaged in parts of the evaluation but not all of the evaluation. This form has similar features to aspects of collaborative evaluation in which “clients” or stakeholders inform and support aspects of the evaluation process in a collaborative process, but the lead evaluator is in charge (Fetterman, Rodríquez-Campos, Wandersman, & O’Sullivan, 2014).
Youth as partners
In this role, young people work alongside adults in the development of the evaluation from the conceptualization, to the method development, to the analysis and reporting. In practice, this might involve a youth–adult evaluation team or could be an adult evaluator working alongside a youth or team of youth. This role draws on concepts from participatory evaluation practices and youth–adult partnerships, in which youth and adults work together with attempts to create equal power and shared decision-making between youth and adults (Cousins & Whitmore, 1998; Fetterman et al., 2014; Zeldin et al., 2012).
Youth as leaders
Youth as leaders is the idea that young people lead an evaluation themselves with young people’s voices and ideas at the forefront of the evaluation process. The youth team would decide what they want to know, how they want to know it, and what they learned from the process. This form is closest to empowerment evaluation, where the community or program participants are in control of the evaluation, and adults serve as coaches or critical friends, or to transformative participatory evaluation practices, in which the evaluation itself transforms the participants and the organization/community (Cousins & Whitmore, 1998; Fetterman et al., 2014).
Dimension 2: Levels of Evaluation
Dimension 2 represents an addition to the initial framework and was born out of discussions from practice and with practitioners about what these roles could look like in practice. Given the interest of this article in helping evaluators visualize and engage YPE in practice, we highlight the levels of evaluation as the lens through which to conceptualize the roles young people can play. For this reason, we developed this dimension focused on three practice levels of evaluation: program/project evaluation, organizational evaluation, and community evaluation. There are other variations that could have been used, including types of evaluation (such as summative, formative, empowerment), but for the purposes of providing a simplified tool to visualize practices, we chose to ground the second dimension in practice levels of evaluation. These levels are defined below.
Program/project evaluation
Arguably the most common type of evaluation, program and project evaluation, would involve young people so that evaluation stakeholders gain more specific understanding about the learning, outcomes, experience, and satisfaction with a specific program or project. For evaluators, especially those that have not engaged young people before, this is likely one of the easiest ways to conceptualize the involvement of young people. If young people are viewed as experts and the best source of knowledge about their experiences, then it is possible to visualize the role of young people in consulting on program evaluation-level questions, collaborating on aspects of information gathering, partnering on the whole design and development of the evaluation, or even developing and leading their own evaluation based on their own experiences.
For example, engaging youth as consultants in program evaluation could mean engaging a team of youth in providing feedback on a program evaluation survey meant to ask youth participants to share their experiences. As consultants, young people would give feedback on the specific wording of questions (e.g., Do they make sense? Will young people understand the question? Will they want to answer this?). They may also give feedback on the length of the survey or the questions themselves (e.g., What else might young people want to say about their experiences? What else might the organization/program ask?). This form might be most useful for formative evaluation or utilization evaluation practices. In formative evaluation, youth expertise may be very valuable in shaping the formation of a program and its evaluation (P. Chen, Weiss, & Nicholson, 2010; Powers & Tiffany, 2006). For utilization-focused evaluation, adults who value youth input may be more likely to be active in the evaluation and use the evaluation results when youth serve as consultants on the evaluation design (Roholt & Mueller, 2013).
Organizational evaluation
Organizational evaluation would expand beyond a program or project and explore the overall work of the organization or perhaps the workings of the organization with various constituents. Although this may not seem to be a likely role for young people, a goal of the matrix is to challenge evaluators to think differently about the potential for young people’s involvement in all aspects of the evaluation. There can be real value for youth development and youth-serving organizations to involve young people in the evaluation process—perhaps in helping to redesign how intake processes gather primary data from youth participants to lower barriers to entry, working with staff to develop questions for youth participants, or even engaging youth in regular staff evaluations or training based on evaluation outcomes.
For example, young people, as part of a broader organizational evaluation, might be responsible for conducting interviews with other young people about their experience in the organization. Similarly, young people might be asked to analyze the transcripts from the youth focus group data to assess what they think are the key themes or ideas that young people had about their experience in the organization. Some research argues that peer youth interviewers may elicit more authentic responses from youth who are interviewed (as subjects) or youth data analysts may see patterns or themes that adult researchers miss (Fine & Cammarota, 2008; Sabo Flores, 2007). Thus, engaging youth in organizational evaluation may help provide better feedback to the evaluation of the overall organization.
Another example could be the roles youth could play on an overall organization’s evaluation team. In this case, young people would work alongside adults in ongoing ways to assess the impact of the organization’s work and to seek out opportunities to evaluate the organization at multiple levels, from program to organizational structure.
Community evaluation
Community-level evaluations are often closely related to youth participatory research or community-based research practices and focus on developing knowledge about the community. This knowledge could be about critical community issues, community strengths or challenges, community factors that impact the organization, or other scans or mapping projects that help organizations better understand the environment in which they operate. Young people might engage in various aspects of evaluation or data gathering to expand understanding of community issues from their perspectives, including how youth experience a particular community or setting.
For example, engaging youth as leaders in community evaluation could mean a team of youth would develop its own assessment of issues in the community. This team might be a youth leadership board or youth organizing collective that wants to conduct its own survey to assess what young people think the major issues facing young people in the community are. In this case, the youth team would develop its own questions, create the survey, and discuss the ways to distribute the survey in the community. The team would analyze the findings and decide to make a presentation to the organization’s board, school leaders, or elected officials about the findings to inform organizational- or community-level discussions about issues, programs, policies, and future directions. They may share the results through infographics, videos, or websites they create. While possibly less common than youth as consultants, this level of youth participation maximizes youth’s expertise and the use of the evaluation for organizational or community change.
To help illuminate the role of young people within each level of evaluation, the matrix provides a set of examples to help visualize practice. Our hope is that these cells can help evaluation stakeholders to consider the range of potential for engaging youth, the types of activities that might be used, and the nuances and considerations they should make when planning for successful youth involvement in evaluation.
Dimension 3: Critical Considerations for Success
Using this matrix in practice raises a set of important critical considerations for organizations interested in using or strengthening youth participatory practices. At the most basic level, any use of this matrix must accompany a discussion about process and implementation. This discussion would include exploring the basic steps in the process of actually engaging youth in organizations. The most problematic approach to increasing participation is to jump into the process without any understanding or intentionality around the steps needed to engage youth. However, for organizations that have already engaged youth and are interested in strengthening or deepening their practices with youth around evaluation, there are additional critical considerations that we argue must be part of discussions by stakeholders thinking about involving youth. These considerations include the role of adults in supporting youth involvement, the level of knowledge about evaluation that youth need for authentic involvement, and a focus on the resources and contexts needed to support young people. We have included these considerations in the matrix, by role of young people and by level of evaluation.
Roles of adult engagement within YPE
As noted, one of the differences between YPE and other forms of community-based evaluation is the role of adults. In all forms, even with youth as leaders, adults play some role within the evaluation process. Thus, it is critical to consider the skills and supports needed by adults to support youth. The matrix lists brief considerations for the roles of adults as they relate to the roles of youth.
Adults often need to recognize the real limitations that may exist because of young people’s life and developmental stage and design the evaluation process to be responsive to their needs. These may include demands related to school, after-school activities, sports, home responsibilities, and employment. This may entail a need for adults to spend extra time doing individualized check-ins with youth to see whether they understand the evaluation process and whether their life outside the evaluation process is allowing them to fully engage in the work (Chen et al., 2010). This is particularly important because youth may feel less empowered to speak up to adults if they are struggling. Adults may also need to check in with youth about how they are experiencing interactions with adults and peers on the evaluation team and about how the adults can create activities that engage youth fully in the process, as youth may feel less empowered to speak up about the process if adults do not seek their input directly and regularly.
Level of evaluation knowledge needed by youth
In addition, regardless of roles, adults often need to identify the training and capacity skills required to engage in evaluation. In some cases, training with young people may need to start with valuing the evaluation as a process, understanding their right to ask questions, and acknowledging themselves as experts in their lives and in their communities (Checkoway & Richards-Schuster, 2003; Ozer, 2016). In other cases, training focuses on evaluation skills including ethics, methods, and analysis. The level of evaluation training required will depend on the role of the youth. Youth who consult on evaluations would likely need less training than youth who are partnering or leading evaluations. Training manuals such as those by Sabo Flores (2007), London, Zimmerman, and Erbstein (2003), or by Arnold and Gifford (2015) offer examples.
Training for youth on evaluation, however, is only one piece of training needed. When we examine the roles of adults, it raises questions about adults’ capacity and training needs to work with youth, as many evaluators are not trained in youth work. Similarly, while many youth development professionals are trained to work with youth, they are likely to have limited knowledge around evaluation. The recent closing plenary of the American Evaluation Association’s 2018 Conference featured a panel of youth evaluators who spoke eloquently to these points and to the importance of adult evaluators respecting the roles and contributions of youth in evaluation efforts. More work is needed to examine the intersections between youth development professionals and evaluators and the skills needed to support young people in this shared role. This remains an area for future study.
General organizational resources and contexts need to involve youth
Exploring the organizational context is crucial before engaging in youth evaluation work. Scholars have pointed to the importance of organizational qualities that help support young people in leadership roles (Jennings, Parra-Medina, Hilfinger-Massias, & McLoughlin, 2006; Powers & Tiffany, 2006; Roholt et al., 2013; Roholt & Mueller, 2013). These qualities include staff and organizational leadership buy-in, physical places in the organization for young people to work, opportunities for young people to engage in meaningful work, and the staffing and resources needed to sustain the work (Jennings et al., 2006). The resources needed may depend on the scope and role of the work. For example, evaluators aiming for more extensive roles for youth in the evaluation may need to consider offering financial stipends or payment for the youth’s time. Buy-in from adults and youth is necessary regardless of scope or role (Powers & Tiffany, 2006). In addition to stipends or payment for youth, other considerations include transportation for youth to get to and from meetings, food for meetings, resources for implementing their evaluation ideas, and additional education and training needed for capacity building. In addition, if youth partner with adults in the evaluation process, it will be necessary to examine logistics such as meeting times (e.g., do meetings happen during the day when youth might be in school?).
Without buy-in and support, youth’s participation can often become tokenized and marginalized rather than valued as an authentic experience that is important and meaningful. Ideally, it aligns with an organization’s values around youth participation and the necessity of youth voice for running effective programs or organizations that meet youth needs or for building communities that are inclusive of youth. As these values become stronger, there is more opportunity for youth’s roles as evaluators to deepen in those settings.
Related to the organizational context, it is critical to examine the extent to which stakeholders are prepared to “hear the ideas” that emerge from the youth evaluators (Chen et al., 2010). It is one thing to create an authentic experience, but if the ideas are not heard, it can lead to marginalization and frustration (Pritzker, LaChapelle, & Tatum, 2012). Adult evaluators often must do important work to ready the adult stakeholders to listen and trust evaluation findings generated by youth, especially the more engaged the youth are in their role in the evaluation (Richards-Schuster & Timmermans, 2017; Roholt et al., 2013).
Using the Matrix: Exploring Case Examples for Illustration
To support use of the matrix, we draw on selected examples from our own practice with YPE efforts in the Detroit region to help illustrate the roles and forms of practice.
Involving Youth as Consultants in Program-Level Evaluation: Informing Quality Standards for Assessing Youth Programs
The Youth Development Resource Center (YDRC), an intermediary organization supporting youth development organizations with evaluation in the city of Detroit, was charged with creating a set of quality standards for youth providers in Detroit. After the key adult stakeholders met and reviewed quality standards from other cities, the adults decided to engage youth in the process of developing the quality standards. Although youth were not involved in all stages, the YDRC staff consulted youth through hosting a series of youth input sessions, engaging 75 youth in defining what great youth programs looked like to them. The final quality standards are documented in the tool used by a network of 175 youth-serving organizations to increase their understanding of the core indicators of quality, as described in the youth’s language from these sessions, and to help illustrate what quality looks like in practice in the language of youth development professionals and best practice research. In particular, youth shaped language in the standards on youth experiences of inclusion or exclusion in programs, including that of Lesbian, Gay, Bisexual, Transgender, Questioning (LGBTQ+) youth. This was an important addition that many of the adults had not included in the first draft. Written in youth’s own words, the Diversity, Access & Inclusion quality standard subsequently opened multiple conversations with adults about the rights of LBGTQ+ youth to participate in after-school and summer employment opportunities in the city without experiencing bias. Without youth as consultants on the creation of the quality standards, it would have been much easier for those adults, who were themselves excluding youth from opportunities because of sexual orientation or gender identity, to disengage from conversations about the importance of young people’s experiences and the inclusion of all youth in programming without bias or discrimination. The role of adults in this process was to continually center youth’s input as critical to creating program quality standards for the community of youth-serving organizations.
Involving Youth as Collaborators in a Large-Scale Program Evaluation: Evaluation of a Summer Dialogues Program
The Youth Dialogues program is a summer program for youth in the Detroit region to dialogue around issues of race and ethnicity. In many years of the program, youth evaluators collaborated with adult program evaluators to incorporate a youth perspective as part of the overall evaluation. Many of these youth had been participants in the program in previous summers, so they were uniquely qualified to bring their own expertise to developing an evaluation. They were given small stipends for their participation and training based on the type of evaluation work they used. For example, one summer, the youth team developed a set of interview questions about student experiences. These questions focused on participants’ learning and perspectives. The interviews complemented the pre- and post- surveys that the students were already completing. The youth were trained in interviewing techniques and protocols before engaging in the interview process. Other examples of youth’s collaboration included the addition of students drawing their experience before and after the program and encouraging students to use “voting boxes” to provide ongoing feedback about their learning at various points throughout the program. The evaluators used these learnings to develop a set of word clouds to examine student learning ideas at the beginning, middle, and end of the program. Each of these evaluation efforts contributed to the overall understanding of the evaluation and added a level of creativity and engagement to the evaluation. In this evaluation, the role of adults was to engage the youth in developing ideas based on the overall evaluation. The adults collaborated with the youth to assess ideas, develop training as needed, and support the youth’s contribution within the overall evaluation effort.
Involving Youth as Partners in Organizational-Level Evaluation: Developing and Piloting an Afterschool Quality Assessment Tool
The Youth Program Quality Assessment developed by the David P. Weikert Center for Youth Program Quality is a standardized tool used to help promote quality instructional practice in after-school and youth development programs. In most cases, adults are trained to observe and rate the quality of the program offerings using the assessment tool. However, in one example in Detroit, an organization, Urban Neighborhood Initiatives, engaged youth as partners by having youth co-design a more youth-friendly program quality observation tool. This tool was used at the program and organizational levels to understand youth experiences across the organization’s various youth programs. The organization hired a team of three high school–aged youth led by a young adult intern to develop the tool with staff support. The team spent the summer designing and testing their program observation tool with the variety of youth employment and summer camp programs the organization offered. They used the results to give recommendations to the organization for future program improvements. The youth were also part of the organization’s youth advisory board and are now serving as part of the board of directors’ program committee, further deepening their leadership role of the organization. The youth spent a second summer refining the tool with the possibility of making it available to other youth-serving organizations, and the youth evaluator who led the process was hired full time by the organization to continue the work and further embed youth-led evaluation into the organization. The adult staff have scaffolded youth from the youth as consultants role to youth as leaders role over time.
Involving Youth as Leaders in Community-Level Evaluation: Young People Develop Their Own Community Social Justice Assessment
In the Detroit area, a team of youth leaders, the Metropolitan Youth Policy Fellows, led the development of a large-scale social justice assessment of the metropolitan region focused on youth’s needs in the community. While there was support from adults to help prepare and support the youth’s ideas, the young people led the development of the questions, the format and administration of the assessment, and the analysis and recommendation processes. Through meetings, the youth discussed what they wanted to assess, how to shape the questions, and then decided to conduct an online survey. They conducted their own network mapping to recruit for a broad-based group of survey participants. The youth evaluators used the findings from the survey to develop a report on young people’s ideas for their community and a set of recommendations for policy officials, including recommendations about youth transportation and equity in opportunities for after-school activities, and the desire that adults have high expectations of young people around education. The report and a video report were shared with foundation officials, community leaders, and were featured on the foundation’s website. The report led to the development of a special grant fund aimed at developing new programs that would address the recommendations in the report. In this example, adults played a supportive ally role to the work of the youth. Adults gave feedback to the ideas, provided training, supported the logistics of meetings, kept communication flowing, and facilitated the youth’s sharing their work with adult stakeholders.
Using the Matrix: Potential for Practice
The purpose of the matrix is to help practitioners visualize YPE as a tool within their work. While it is meant to be illustrative, and not comprehensive, we believe that the matrix can be helpful in a variety of ways.
First, this type of concrete tool can be helpful for those organizations that are open to the idea of YPE but need help in visualizing the practice. For example, in a workshop that we conducted with community practitioners interested in engaging YPE for the first time, the matrix offered a visual to help participants think about the potential for YPE in their organizations. For some, just the ability to think about and see concrete examples of what YPE could look like helped them to think about what YPE could be implemented in their organizations.
Second, using a matrix helps evaluators to see how youth participation efforts can complement and enhance already existing efforts rather than replace them by visualizing the various possibilities and levels of participation. In this way, a youth-driven program evaluation could support and strengthen already existing program evaluation efforts leading to deeper, richer, and more nuanced data collection and analysis without replacing already existing practices.
Third, the matrix itself can be used as a tool to challenge those who might be skeptical of the idea of youth participation in evaluation by providing illustrative examples of the potential of YPE within an organization. Sometimes an example can provide a basis for discussion that can lead to even the most novice or resistant thinker on YPE to move one step closer to seeing its potential.
Fourth, the matrix can also be adapted for assessing and considering youth engagement more broadly within an organizational context. Thus, it is possible to explore how the category “levels” and “illustrative examples” could be repopulated to explore youth engagement in a variety of different settings.
For example, colleagues in a major public school system adapted the youth evaluation matrix to help principals and school leaders visualize youth involvement in school programs, school-level leadership, and district-level leadership. Their adaption of this matrix helped school leaders begin to visualize the potential for youth engagement and to think about small- and large-scale implementation possibilities. In addition, it helped them to consider the ways that youth participation could complement existing leadership and engagement efforts, especially of other stakeholders such as parents.
Understanding the Benefits and Limitations to Youth Involvement in Evaluation
The goal of this article is to offer a matrix that can be useful as a tool to help stakeholders to consider the potential of youth participation in evaluation. However, we argue that, regardless of role or level in the matrix, it is essential to understand the benefits and limitations to youth involvement. While there are many points to consider, we raise a few that we feel are important for discussion.
On benefits, there is research to indicate that youth participation in evaluation has direct benefits to youth. Research has shown that young people gain skills, confidence, and are more likely to stay engaged, among other outcomes (Foster-Fishman, Law, Lichty, & Aoun, 2010; Ozer, Ritterman, & Wanis, 2010). There is also research to suggest that youth evaluation leads to new information, better programs, and improved ideas for organizations and the community (Chen et al., 2010; Roholt et al., 2013). However, more research is needed to truly understand the range of benefits and to assess the impact of this approach on influencing practice. Knowledge and research of this type would help move the field from being a novelty to an essential form of practice in youth-serving organizations and systems.
YPE, however, also has limitations. It is critical to consider the potential issues in involving youth in evaluation, in particular, the ethics involved and the potential for harm to young people (Fleming, 2010; Pritzker et al., 2012). Efforts to engage young people in evaluation must consider the ethics of engagement. This includes understanding the necessary protections to ensure that youth evaluation does not lead to harm, especially harm in the youth who participate and harm to the organizations. For example, organizations need to consider the potential vulnerabilities of young people evaluating the organizations or systems that they interact with and how findings—positive or negative—could impact them. It is important that organizations that engage youth in evaluation follow ethical guidelines to ensure that young people’s participation is supported and that their participation in the evaluation does not impact their participation in the organization. In addition, it requires that organizations and other stakeholders be able to acknowledge and address the findings that emerge from the youth’s evaluation work (Costley, Elliott, & Gibbs, 2010; Sabo Flores, 2007). Adults who work with youth in evaluation efforts should be cognizant of the potential risks and help young people and other stakeholders to be aware of and navigate them as needed (Roholt et al., 2013). Differing types of organizations such as schools or governmental care systems (e.g., foster care, juvenile justice) may need additional types of protections and review processes, given the vulnerabilities of young people within those settings.
In addition, it is important to raise critical questions about the agency of young people in deciding how they want to participate (S. Chen, Poland, & Skinner, 2007). In our experience, young people should be given the right to determine when and how they want to participate and to assume that their participation can change over time. In addition, organizations should consider the various social, cultural, organizational, and contextual factors that might influence or impact participation.
Understanding the benefits and naming the limitations are important as the effort to engage youth in evaluation moves forward. These are essential questions and ones that should be discussed and examined in future projects and research.
Conclusion
In closing, YPE is building as a field. In this stage, those evaluators who committed to youth participatory approaches should be both refining practices and raising questions, while still helping to support organizations looking to embrace youth participatory work for the first time. As such, this article’s contribution to the field is to share a practice matrix, a practice tool for visualizing and conceptualizing the role of young people in evaluation, and, in doing so, raise some of the considerations that evaluators interested in developing or adopting a youth participatory practice should take into account.
There are limitations in our current article, and we raise more questions about the field than we answer. For example, more research and discussion are needed to describe the roles of young people in evaluation in more complicated and critical ways, through answering questions about the process, benefits/limitations, and context. Our goal is to continue to develop the field of youth participation evaluation with a more in-depth conceptual focus on the “how” and the considerations for practice. This matrix is one example to achieve this, and we hope that it can be a starting point for use and adaptation in the field.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.