Using Social Psychology to Constructively Involve Colleagues in Internal Evaluation

Abstract

Internal evaluators can lead positive, helpful, promotive interactions among colleagues to make evaluation more appropriate, understandable, and accessible. As the motivations and abilities of non-evaluator colleagues are highly varied, interpersonal skills help facilitate cooperation and promote evaluation. Social interdependence theory from social psychology can assist with understanding teamwork and developing interpersonal competence. While the literature links social interdependence theory with evaluation, there is limited understanding of its application in practice. In this article, we share our internal evaluation experiences in three Australian nonprofit organizations to elucidate how we harnessed social connections and interpersonal dynamics to achieve cooperative goals. The purpose of this article is to share our process for using theory and literature to assist with reflecting, planning, and acting, around the way we interact with colleagues. Evaluators may find this process useful when considering how to personalize professional development around interpersonal competencies for evaluation.

Keywords

internal evaluation social interdependence evaluation teamwork nonprofit

Internal evaluators can lead positive, helpful, promotive interactions among colleagues to make evaluation more appropriate, understandable, and accessible. However, it can be a constant challenge to develop and maintain cooperative interactions around evaluation with people who are not evaluators (Mark et al., 2011). Although many evaluators work under challenging circumstances, internal evaluators must continually consider factors such as organizational culture, leadership dynamics, professional relationships, and bureaucratic processes, while working with the same colleagues over the long term. Being an internal evaluator involves more than just evaluation. According to Volkov (2011), internal evaluators have many roles including change agent, evaluation educator, evaluation capacity builder, management and decision-making supporter, consultant, researcher, technician, analyst, advocate, organizational learning supporter, facilitator, generalist, planner, collaborator, independent observer, judge, and information specialist (p. 32). External evaluators often need to quickly develop rapport and understand new contexts (King & Stevahn, 2013, p. 227). In contrast, challenges for internal evaluators include engaging with colleagues around evaluation on a daily basis, over potentially many years, and navigating the multifaceted issues that arise when attempting to meet the demands for information from stakeholders inside and outside the organization (Campbell & Lambright, 2017; Mayne, 2017; Rogers et al., 2019a, 2019b).

Evaluators can find it difficult to effectively explain the benefits of evaluation to people who are not evaluators (Mason & Hunt, 2018). In an organizational setting, employees may have negative emotions toward evaluation as it can make them feel threatened and anxious (Donaldson et al., 2002; Whitehall et al., 2012). They may be worried about exposing failures or anticipate interrogation of their personal weaknesses. They may resist participating or actively try to prevent evaluation from occurring (Edmonson, 2003; Whitehall et al., 2012).

Internal evaluators need to have strong interpersonal skills because the motivations and abilities of colleagues to gain access to, understand, and use evaluative information will be highly varied (Rogers et al., 2019a). Skillfully interacting with colleagues and demonstrating respect for individuals are key to successful evaluation practice (American Evaluation Association, 2018; Garcia & Stevahn, 2019). However, interpersonal competency is unlikely to be taught in formal evaluation courses (Davies & MacKay, 2014; Gullickson et al., 2019). There currently is a gap in the literature on how to identify, develop, and apply interpersonal skills in practice (Archibald et al., 2018; Boyce, 2017; Garcia & Stevahn, 2019; Ortega & Coulborn Faller, 2011). Although the importance of the personal factor for increasing evaluation use by engaging key stakeholders appears in the evaluation literature (Patton, 2008), less guidance is available on interpersonal dynamics for the purposes of establishing evaluation buy-in from colleagues in intraorganizational contexts (Alkin & King, 2017; Baron, 2011; King & Stevahn, 2013; Mayne, 2014).

This article is built on the assumption that active and ongoing interaction among internal evaluators and colleagues will result in evaluations with meaningful and relevant design, high validity, and increased utilization. As such, consideration of interpersonal dynamics can assist with encouraging, promoting, and facilitating the participation of colleagues in evaluative activities. These assumptions are at the foundation of evaluation approaches that include collaborative evaluation (O’Sullivan, 2012), empowerment evaluation (Fetterman & Wandersman, 2005), participatory evaluation (Cousins & Chouinard, 2012), and utilization-focused evaluation (Patton, 2008). These approaches, with the addition of the evaluation capacity building literature (Stockdill et al., 2002), have influenced our work and shaped our thinking. We also recognize that adult learning, workplace learning, and organizational change theories, among others, are useful for facilitating teamwork, addressing how individuals respond to learning and change, recognizing colleagues as individuals who have their own knowledge systems, and understanding that personal experiences can and should influence evaluation activities (Preskill & Boyle, 2008b). However, for this article, we drew most heavily upon interactive evaluation practice because of the focus on interpersonal dynamics (King & Stevahn, 2013). We appreciated and valued this approach because it intentionally and directly considers interpersonal interactions: “the intentional act of engaging people in making decisions, taking action, and reflecting while conducting an evaluation study” (King & Stevahn, 2013, p. 14). Interactive evaluation practice uses the empirically supported theoretical framework of social interdependence theory from social psychology, to guide actions and structure the evaluation to support cooperative interaction for success.

In this article, we share our internal evaluation experiences in three Australian nonprofit organizations to explain how we harnessed social connections and interpersonal dynamics to achieve cooperative goals. We found that undertaking internal evaluation in nonprofit organizations has unique challenges and is therefore a useful context in which to reflect upon interpersonal dynamics related to evaluation. The purpose of this article is to share our process for using theory and literature to assist with reflecting, planning, and acting, around the way we interact with colleagues. First, we provide background about internal evaluation in nonprofit organizations. Second, we review the literature on social interdependence theory and its connection with evaluation. Third, we describe how social interdependence theory has helped us focus our attention on interpersonal interactions. Our process involved using the basic elements of cooperative teamwork as a framework for thinking about the evaluation literature and our practical experiences in activity planning. Fourth, we present the outcome of this illustrative peer reflection process. We do not intend our outcome to be prescriptive for other evaluators, but it could be adapted by readers to suit their context. However, the main contribution of this article is the process. It may be instructive for others who are considering how to personalize professional development around interpersonal competencies for evaluation.

Context

Nonprofit organizations are separate from government and for-profit entities and often have a social and altruistic purpose (Hardwick et al., 2015). Nonprofit organizations are under increasing pressure to demonstrate their effectiveness, learn and improve, be accountable to funders in a competitive environment, and produce evidence for multiple audiences (Braverman, 2013).

Evaluation is a tool that can help nonprofit organizations meet the increasing demand for information. Harman (2019) states, “Evaluation helps social program staff improve their services, it helps them increase their impact, motivate funders and donors, and monitor the extent to which they are achieving their mission” (p. 23). However, the problems that social and human service nonprofits are seeking to solve are complex and not easily measurable (Gill, 2010; Kelly, 2019). A multitude of factors and demands are placed upon nonprofits; social, economic, and political factors affect the role of evaluation; community expectations are high and there is often a lack of coordination across service delivery, clinical, and sector boundaries (Campbell & Lambright, 2017; Gilchrist & Butcher, 2016).

We concur with the literature that suggests internal evaluators can influence decision-making and organizational learning, align evaluation with context, support improvements, build capacity; advocate for change, practice reflection, and critical thinking; and promote evaluation among their colleagues, when there is an enabling contextual and structural environment for evaluation (Leviton, 2014; Love, 1991; Mathison, 2011; Sonnichsen, 2000; Volkov, 2011). To enhance evaluation literacy or undertake any of the influencing behaviors listed above, we believe that internal evaluators need to work cooperatively with groups of people in the organization. The size of the groups, the duration of engagement, and the location of the interaction will vary depending on the organization. However, in all circumstances, cooperative interactions with people will be an essential element of the internal evaluator’s role. King and Stevahn (2013) state “The more organization development is the target of an evaluation, the more the dynamics between evaluator and stakeholders should be positive and supportive, both in evaluation decision-making and implementation” (p. 39).

Social Interdependence Theory

Social psychology can assist with understanding teamwork as interpersonal competence is essential for constructive interactions with colleagues. Social psychological theories can strengthen evaluation practice by helping evaluators understand the challenges they face regarding interpersonal interactions and trust maintenance (Mark et al., 2011). Social interdependence theory is useful for studying groups, understanding teamwork, helping to define the group, and exploring what gives a team purpose (Tindale & Anderson, 2002). A review article on social interdependence theory, consolidating over 750 research studies conducted over 110 years, demonstrates that extensive research has been conducted in diverse educational, business, and social service organizations and supports the conclusion that the theory has wide generalizability and considerable external validity (Johnson, 2003). More recent reviews highlight the significant contribution that social interdependence theory has made to influencing social policy and understanding cross-cultural dynamics in business and organizational settings (Johnson et al., 2019; Maton, 2017; Tjosvold et al., 2014).

A brief early reference to social psychology in internal evaluation appeared in the 1983 special issue of New Directions for Program Evaluation (now known as New Directions for Evaluation). Social psychologist, Morton Deutsch (1949) was referenced in recognition that the organizational workplace was a psychological group because coworkers share a common interest, have roles that interlock, pursue interdependent goals, and identify themselves as members (Deutsch, cited in Neigher & Metlay, 1983). Later, Johnson et al. (2011) drew together the intersecting fields of evaluation and social psychology. They suggested that social interdependence was a suitable theory for evaluators to learn how to facilitate cooperation among stakeholders. Since then, the link between social interdependence theory and evaluation has been made more explicit: “Social interdependence theory helps us understand how to structure evaluation goals and tasks in ways that will foster cooperative interaction among participants for success in conducting evaluation studies” (King & Stevahn, 2013, p. 49).

Lewin (1935, 1948) proposed the concept of interdependence among group members whereby any member can influence the dynamics, members are connected through common goals, and movement toward achieving the goals can be a source of motivation (Lewin, cited in Johnson & Johnson, 2009). Deutsch (1949) extended the theory to illustrate that positive interdependence occurs when individuals’ perceive they can only accomplish their goals by working together, while negative interdependence occurs when individuals view accomplishment as the failure of others (Deutsch, cited in Johnson, 2003). Deutsch validated the theory of cooperation and competition on the premise that the type of intentionally structured interdependence determines how individuals interact with each other and the outcomes of those interactions (Deutsch, 1949 cited in Deutsch, 2011). Deutsch (2011) stated,

The basic psychological orientation of cooperation implies the positive attitude that “we are for each other,” “we benefit one another”; competition, by contrast, implies the negative attitude that “we are against one another,” and in its extreme form, “you are out to harm me.” (p. 25)

This theory of cooperation and competition was extended to develop social interdependence theory (Deutsch, 2011; Johnson, 2003; Johnson & Johnson, 1989).

Social interdependence exists when “individuals share common goals and each individual’s outcomes are affected by the actions of others” (Johnson & Johnson, 2002, p. 11). Social interdependence theory includes cooperative and competitive goals and predicts different interactions likely to evolve from each. When cooperation is occurring, the actions of team members will support one another, demonstrate positive emotions toward each other, and their work will be open to each other’s influence (Johnson & Johnson, 2014). This psychological process, positive cathexis, refers to the investment of energy in activities beyond the personal. It is the opposite of a negative cathexis psychological process where no energy is expended in colleagues or work, no substituting of actions occurs among individuals, and there is no openness to influencing others or being influenced (Johnson & Johnson, 2009). According to social interdependence theory, when the elements of cooperative teamwork are considered in the design of activities, this can result in increased productivity, psychological health, and positive interpersonal relations (Johnson & Johnson, 2002). Cooperation promotes greater effort to achieve than competitive or individualistic approaches and produces greater social support and quality of relationships. Applying social interdependence in internal evaluation means team members’ perspectives are incorporated during decision-making processes and can result in team members feeling valued, having ownership of mutual outcomes, and developing a greater positive understanding of self and others (Johnson et al., 2011).

Elements of Cooperative Teamwork

The main components of social interdependence theory—interdependence, interaction, and outcomes (Deutsch, 1949, 2011)—became the foundation for further research that identified and empirically validated five elements for operationalizing effective teamwork (Johnson & Johnson, 2003, 2009). Those elements provide evaluators with information useful for successfully structuring social relationships to achieve cooperative goals. Specifically, positive interdependence defines how participants are mutually linked to achieve a shared goal that can be represented by the phrase—we all need each other for success. This mutually interdependent relationship motivates individual accountability (doing one’s part), promotive interaction (helpful behaviors among colleagues), the use of social skills (accomplishing tasks and maintaining positive relations), and group processing (reflecting on what is working well and what needs adjusting; Johnson & Johnson, 2003, 2009). Figure 1 shows the elements of cooperative teamwork in relation to social interdependence theory.

Figure 1.

The elements of cooperative teamwork in relation to the theory of social interdependence. Adapted from Johnson and Johnson (2009), p. 367.

Positive Interdependence

Positive interdependence is about establishing a shared goal that links the actions of individuals to others; the success of an individual is facilitated by the extent to which all other individuals are successful (Stevahn, 2013, p. 34). Each participant acknowledges how their efforts are connected and coordinated with others to complete a task (Johnson & Johnson, 2002). Johnson (2003) frames positive interdependence as “knowing that one’s performance affects the success of one’s group mates seems to create forces for responsibility that increase one’s efforts to achieve” (p. 938). Creating a shared identity, structuring mutual goals, contributing and dividing resources, determining complementary roles, and clarifying the joint rewards can be strategies to assist with establishing positive interdependence (Johnson et al., 2011).

Individual Accountability

Individual accountability occurs when all team members feel like they have a personal responsibility to their team to complete their share of the work. This element also means that team members facilitate the work of others and minimize hindrances (Johnson & Johnson, 2002). Stated comprehensively: “Failing oneself is bad, but failing others as well as oneself is worse. The more a person is liked and respected by group mates, furthermore, the more responsibility he or she will feel toward group mates” (Johnson & Johnson, 2009, p. 368).

Promotive Interaction

This element is about establishing and strengthening the level of encouragement provided among the team (Johnson & Johnson, 2003). Johnson and Johnson (2003) state that promotive or face-to-face interaction exists “when individuals encourage and facilitate each other’s efforts to complete tasks and achieve the group’s goals” (p. 175). This means promoting each other’s success and creating opportunities for meaningful interpersonal encounters. Team members can provide each other with encouragement, help, assistance, praise, resources, information, materials, and support in a timely and efficient way that contributes to achieving the group goal (Johnson, 2003; Stevahn, 2013). This element also incorporates providing constructive feedback, advocating to increase group effort and focus, influencing each other, and trusting and being trustworthy (Johnson & Johnson, 2003).

Social Skills

For the fourth element, social skills, Johnson (2003) suggests that success relies upon group members having, and being motivated to use, the interpersonal and small group skills needed for cooperation (p. 939). These interpersonal competences include communication skills, conflict resolution skills, cultural competency, trust and trustworthiness, flexibility, and democratic facilitation skills (Stevahn, 2013). Johnson and Johnson (2003) suggest individuals need to have and be motivated to use these interpersonal skills: “(a) getting to know and trust each other; (b) communicating accurately and unambiguously; (c) accepting and supporting each other; (d) resolving conflicts constructively” (p. 176). At the group level, they also suggest mastering the skills of ensuring team members are committed to the mutual goals, communicating accurately among the team, providing leadership and influence, incorporating flexible and democratic decision-making procedures, and constructively resolving conflicts (Johnson & Johnson, 2003).

Group Processing

The regularity of opportunities to reflect on group functionality can influence effectiveness of the team. Group processing is about reflecting on how the team is functioning and how things could improve (Johnson, 2003). Johnson et al. (2011) suggest that this occurs when the group takes time to openly discuss the degree to which the team members are achieving their goals and the health of their working relationships. It is about clarifying and improving how the team carries out processes and assesses the level of individual accountability so that everyone can be more effective in achieving the group goal. Stevahn (2013) suggests that the success of the team can be promoted by discussing team dynamics, considering the extent of progress toward the goal, and implementing changes that will enhance teamwork. Taking time to incorporate reflection can assist with clarifying how important the overall goals are, increasing the involvement of group members, and finding the resources needed to succeed (Johnson & Johnson, 2009).

Application to Evaluation

Social interdependence theory can assist with understanding how internal evaluators engage with their own team and the teams they work with, establish mutual goals, manage conflict, and promote cooperation while facilitating interactive evaluation practice (King & Stevahn, 2013; Stevahn & King, 2005, 2016). Internal evaluators can use social interdependence theory (Deutsch, 2011; Johnson & Johnson, 2005) and the basic elements of cooperative teamwork (Johnson, 2003) to guide, reflect on, and make sense of their interactions with colleagues. Social interdependence theory reminds internal evaluators of how important cooperative goals are for promoting and encouraging positive and constructive interactions among colleagues. Internal evaluators can then use the basic elements of cooperative teamwork as a framework to operationalize the theory and support the achievement of cooperative goals. In fact, Johnson et al. (2011) comment that: “The better evaluation staff understands social interdependence theory and the more carefully they structure positive goal interdependence among themselves and the other stakeholders, the more successful their evaluation will be” (p. 312).

Hence, social interdependence theory and the elements of cooperative teamwork are highly relevant for evaluation because of the potential for evaluators and program staff to have different goals that, without intentional mitigation, may result in conflict (Johnson et al., 2011). However, there is minimal understanding about how internal evaluators can apply this theory in practice. It is the intention of this article to address this gap by illustrating how we as internal evaluators used the theory and elements to understand and synthesize the literature, reflect on our own work, and plan actions for constructively involving colleagues in internal evaluative processes.

Our Process

This section shares our process for using theory and literature to reflect, plan, and act on the way we interact with colleagues. We use our internal evaluation experiences in three Australian nonprofit organizations to elucidate how we harnessed social connections and interpersonal dynamics to achieve cooperative goals. We each have over 15 years’ experience working in teams in nonprofit organizations (mainly in Australia) and almost as long conducting evaluations and working in internal evaluation roles. We have been in internal evaluation positions that have supported organizations of between 5 and 200 people over the long term to build evaluation capacity, commission evaluations, undertake internal evaluations, develop organizational evaluation cultures, and embed evaluative thinking. We work in three different human service nonprofit organizations and are all undertaking, or have recently completed, doctoral research on topics relating to internal evaluation.

We included ourselves in the inquiry to learn from each other and consolidate our practical expertise. We based our inclusion on four components: (i) our examination of the literature, (ii) our expertise in evaluation and nonprofit organizations, (iii) our experience in self-reflection and analysis of each other’s work, and (iv) our participation in and observations of ongoing conversations among colleagues working in evaluation and nonprofit organizational contexts. Our process combines the theory and literature with our professional experiences to present an analysis of what cooperative interactions with colleagues could look like and how these can be cultivated. Figure 2 provides an overview of our four-step process.

Figure 2.

Overview of process to develop the framework.

Step 1: Literature Review

The lead author conducted and published a literature review on evaluation champions in the Journal of Multidisciplinary Evaluation. The article included a focus on internal evaluation, nonprofit organizations, and social interdependence as part of her PhD research project (Rogers & Gullickson, 2018). Articles found during this process were coded against terms that included internal evaluation and social interdependence. In relation to social interdependence theory, over 45 articles were identified that explained the historical development, reviewed the progressive changes, summarized the elements, consolidated the evidence, and provided critiques (e.g., Froncek & Rohmann, 2019; Johnson, 2003; Johnson & Johnson, 2002; Tjosvold et al., 2003). Stevahn’s (2013) descriptive study, although based in an educational setting, provided a particularly informative interpretation and presents a range of practical strategies.

Specifically focused on evaluation, the literature on interactive evaluation practice provided a rich source of examples that could be tailored to the intraorganizational context (King & Stevahn, 2013; Stevahn & King, 2005, 2016). The literature on evaluation capacity building and utilization-focused evaluation offered a plethora of strategies for engaging participants in evaluation processes (King, 2007; King & Volkov, 2005; Patton, 2008; Preskill & Boyle, 2008a, 2008b; Taylor-Powell & Boyd, 2008). These publications resonated strongly because they collate research findings from real-world experiences and contain practical examples that could assist internal evaluators working with colleagues over the long term in organizational contexts.

Step 2: Literature and Theory Analysis

To analyze the combination of theory and practice-based literature, we used the five elements of cooperative teamwork to organize the information into positive interdependence, individual accountability, promotive interaction, social skills, and group processing (Johnson & Johnson, 2003). To guide the process for analyzing the literature, we organized the material into a table structured against a key question. Each question related to an element of cooperative teamwork:

Positive interdependence: How do you set common goals that link all individuals?

Individual accountability: How are individuals held accountable for their contribution?

Promotive interaction: How do you ensure there are opportunities to connect, provide encouragement, and have meaningful interpersonal interactions?

Social skills: How do you communicate, resolve conflict, value diversity, respect difference, and self-reflect?

Group processing: How do you incorporate opportunities for reflection and consider effectiveness of the team?

Step 3: Practice, Literature, and Theory Analysis

The key questions above also provided a way for us to link our professional experience with theory and literature to reflect on our own practice. We collectively discussed our different ways of working with colleagues around evaluation. We shared our challenges, gave suggestions, reflected upon strategies that did not work, and highlighted effective processes. Using the elements of cooperative teamwork meant the dialogue was structured and provided us with a framework upon which we could assess our strengths and weaknesses. We identified gaps, clarified commonalities, and came to an agreement on the final sample of practical strategies. Tables 1 –5 record relevant examples from the literature and our summarized responses to the questions to produce a consolidated list of practical strategies under each element of cooperative teamwork.

Table 1.

Positive Interdependence: How Do You Set Common Goals That Link All Individuals?

Examples From Practice	Consolidated Strategies
Develop diagrams of how the program should work with evaluation embedded Support team members to determine the priorities based on their interest, concerns, or needs Enable team members to have ownership over the collaborative process and see how their contribution has been incorporated Connect evaluation conversations with the purpose of the organization Collaboratively develop the theory of change and distribute a physical copy to all team members	Brainstorm to discover needs, expectations, values, and issues Generate lists of priorities Create diagrams of connections between program and evaluation Use metaphors in the form of words or images Discuss how individual interests connect to goal Map concepts to show connections Articulate the team goal with timelines and tasks Share tangible examples of how evaluation can benefit the programs, the team members, and the organization Set goals and offer rewards and incentives Develop team identity (seating or location, clothing, history connections, differences from other teams) Develop a strategy for evaluation that links with organizational policies and procedures Develop a memorandum of understanding or charter of team values Generate a written purposeful, detailed, long-term plan
Key references Stevahn (2013); King & Stevahn (2013); King (2007); Preskill & Boyle (2008b)

Examples From Practice

Consolidated Strategies

Develop diagrams of how the program should work with evaluation embedded
Support team members to determine the priorities based on their interest, concerns, or needs
Enable team members to have ownership over the collaborative process and see how their contribution has been incorporated
Connect evaluation conversations with the purpose of the organization
Collaboratively develop the theory of change and distribute a physical copy to all team members

Brainstorm to discover needs, expectations, values, and issues
Generate lists of priorities
Create diagrams of connections between program and evaluation
Use metaphors in the form of words or images
Discuss how individual interests connect to goal
Map concepts to show connections
Articulate the team goal with timelines and tasks
Share tangible examples of how evaluation can benefit the programs, the team members, and the organization
Set goals and offer rewards and incentives
Develop team identity (seating or location, clothing, history connections, differences from other teams)
Develop a strategy for evaluation that links with organizational policies and procedures
Develop a memorandum of understanding or charter of team values
Generate a written purposeful, detailed, long-term plan

Key references
Stevahn (2013); King & Stevahn (2013); King (2007); Preskill & Boyle (2008b)

Table 2.

Individual Accountability: How Are Individuals Held Accountable for Their Contribution?

Examples From Practice	Consolidated Strategies
Support colleagues to develop short evaluation bulletins as part of work plans Demonstrate the usefulness of a tool to a person in a leadership position who has the power to embed as a requirement across the organization Record decisions in minutes or rationale log to ensure the information is accessible by all at any time Use an outcomes framework to connect individual roles to organizational outcomes	Email, remind, hint, flag to ensure follow up Use calendars, To-Do charts for targeting benchmarks, specifying roles, responsibilities, and deadlines Create agendas, record decisions in meeting minutes and rationale logs to make process transparent and accessible Develop guides and templates for step-by-step instruction Mutually agree upon and record team rules Incorporate evaluation into orientation process, job descriptions, program development, guidelines for grants, and work plans Develop a data management system to identify evaluation needs, explain processes, store templates, clarify success indicators, and disseminate findings
Key references King & Volkov (2005); Stevahn (2013); King & Stevahn (2013); King (2007); Taylor-Powell & Boyd (2008)

Examples From Practice

Consolidated Strategies

Support colleagues to develop short evaluation bulletins as part of work plans
Demonstrate the usefulness of a tool to a person in a leadership position who has the power to embed as a requirement across the organization
Record decisions in minutes or rationale log to ensure the information is accessible by all at any time
Use an outcomes framework to connect individual roles to organizational outcomes

Email, remind, hint, flag to ensure follow up
Use calendars, To-Do charts for targeting benchmarks, specifying roles, responsibilities, and deadlines
Create agendas, record decisions in meeting minutes and rationale logs to make process transparent and accessible
Develop guides and templates for step-by-step instruction
Mutually agree upon and record team rules
Incorporate evaluation into orientation process, job descriptions, program development, guidelines for grants, and work plans
Develop a data management system to identify evaluation needs, explain processes, store templates, clarify success indicators, and disseminate findings

Key references
King & Volkov (2005); Stevahn (2013); King & Stevahn (2013); King (2007); Taylor-Powell & Boyd (2008)

Table 3.

Promotive Interaction: How Do You Ensure There Are Opportunities to Connect, Provide Encouragement, and have Meaningful Interpersonal Interactions?

Examples From Practice	Consolidated Strategies
Encourage team members to share evaluation bulletins on internal social media platforms Promote the success of one group with another to acknowledge the initial achievement and build momentum Allocate homework to be completed in small teams to make the information relevant and then share with wider group Publicly acknowledge and thank individuals for their contribution with thank you cards that link senior management and evaluation achievements	Use every occasion to promote evaluation: e-mails, social events, hallway discussions, sharing resources, and asking questions Celebrate achievements and provide recognition at any opportunity Provide ongoing, timely, personalized technical capability building assistance Facilitate routine learning, brainstorming, and big picture thinking sessions Establish communities of practice, advisory groups, peer support structures, meetings, workgroups, task forces, internships, mentoring arrangements to support others, provide feedback, challenge, and explore points of view Enlist people who advocate for evaluation, motivate others, ask critical questions, help act on findings, and assist with socializing into the organization’s daily operations Structure teams around topics or arrange workspaces to break down silos and encourage cross-fertilization Offer meaningful incentives: allowances, release time, acting in a leadership role, peer recognition, demonstrate professional skills, use of data to validate work, and improve programs, and flexibility to integrate evaluation
Key references Taylor-Powell & Boyd (2008); Preskill & Boyle (2008b); King & Stevahn (2013)

Examples From Practice

Consolidated Strategies

Encourage team members to share evaluation bulletins on internal social media platforms
Promote the success of one group with another to acknowledge the initial achievement and build momentum
Allocate homework to be completed in small teams to make the information relevant and then share with wider group
Publicly acknowledge and thank individuals for their contribution with thank you cards that link senior management and evaluation achievements

Use every occasion to promote evaluation: e-mails, social events, hallway discussions, sharing resources, and asking questions
Celebrate achievements and provide recognition at any opportunity
Provide ongoing, timely, personalized technical capability building assistance
Facilitate routine learning, brainstorming, and big picture thinking sessions
Establish communities of practice, advisory groups, peer support structures, meetings, workgroups, task forces, internships, mentoring arrangements to support others, provide feedback, challenge, and explore points of view
Enlist people who advocate for evaluation, motivate others, ask critical questions, help act on findings, and assist with socializing into the organization’s daily operations
Structure teams around topics or arrange workspaces to break down silos and encourage cross-fertilization
Offer meaningful incentives: allowances, release time, acting in a leadership role, peer recognition, demonstrate professional skills, use of data to validate work, and improve programs, and flexibility to integrate evaluation

Key references
Taylor-Powell & Boyd (2008); Preskill & Boyle (2008b); King & Stevahn (2013)

Table 4.

Social Skills: How Do You Communicate, Resolve Conflict, Value Diversity, Respect Difference, and Self-Reflect?

Examples From Practice	Consolidated Strategies
Develop visual maps of concepts, synthesis lists, and collaboratively prioritize and display flow charts and process maps Connect with colleagues who work in a way that is kind, friendly, empathetic, and patient, yet also task-oriented and organized Value sources of friction against the process as opportunities to tailor tools and incorporate diversity Connect with frontline workers at any opportunity to tailor information, foster genuine interest, and acknowledge their contribution and unique role in connecting with the community or target audience	Take time to build relationships and develop rapport using engaging participatory ways of sharing Use a strengths-based approach, consider diverse needs, and incorporate adult learning principles Display an attitude that is kind, friendly, empathetic, and patient, yet also task-oriented and organized Plan and manage communications about evaluation to cultivate demand and engagement Use wide variety of accurate and unambiguous communication strategies and multiple platforms: email, memos, agenda items at meetings, presentations, organizational social media posts, and bulletins on notice boards Develop a pre-planned procedure for dealing with conflict and openly acknowledge negative positions or damaging statements by thanking the participant for sharing their perspective and then finding an expedient constructive strategy for resolving the conflict Act with integrity, be trustworthy and respectful, listen carefully, ask questions, and be open minded Synthesize information collaboratively and express links between practice and evaluation using visual tools, mind mapping, whiteboards, flowcharts, process mapping, lists, and sticky notes
Key references Johnson & Johnson (2009); Taylor-Powell & Boyd (2008); King & Stevahn (2013); Stevahn (2013); Preskill & Boyle (2008a)

Examples From Practice

Consolidated Strategies

Develop visual maps of concepts, synthesis lists, and collaboratively prioritize and display flow charts and process maps
Connect with colleagues who work in a way that is kind, friendly, empathetic, and patient, yet also task-oriented and organized
Value sources of friction against the process as opportunities to tailor tools and incorporate diversity
Connect with frontline workers at any opportunity to tailor information, foster genuine interest, and acknowledge their contribution and unique role in connecting with the community or target audience

Take time to build relationships and develop rapport using engaging participatory ways of sharing
Use a strengths-based approach, consider diverse needs, and incorporate adult learning principles
Display an attitude that is kind, friendly, empathetic, and patient, yet also task-oriented and organized
Plan and manage communications about evaluation to cultivate demand and engagement
Use wide variety of accurate and unambiguous communication strategies and multiple platforms: email, memos, agenda items at meetings, presentations, organizational social media posts, and bulletins on notice boards
Develop a pre-planned procedure for dealing with conflict and openly acknowledge negative positions or damaging statements by thanking the participant for sharing their perspective and then finding an expedient constructive strategy for resolving the conflict
Act with integrity, be trustworthy and respectful, listen carefully, ask questions, and be open minded
Synthesize information collaboratively and express links between practice and evaluation using visual tools, mind mapping, whiteboards, flowcharts, process mapping, lists, and sticky notes

Key references
Johnson & Johnson (2009); Taylor-Powell & Boyd (2008); King & Stevahn (2013); Stevahn (2013); Preskill & Boyle (2008a)

Table 5.

Group Processing: How Do You Incorporate Opportunities for Reflection and Consider Effectiveness of the Team?

Examples From Practice	Consolidated Strategies
Facilitate discussions, undertake confidential surveys, informally check in to take the temperature, and constantly gauge level of interest to get feedback from the team Share simple visual assessments of progress toward overarching plan, such as a traffic light guide Source and incorporate ideas from the literature Bring together leaders and other key stakeholders after implementation to provide feedback on process and implementation Role modeling personal reflection, learning, and development Openly reflect on the similarities of the personal aspirations of other team members with the goals of the evaluator	Routinely ask for feedback from all colleagues to track what worked well, improvements required and suggestions; confidential surveys, a suggestion box, facilitated discussions, external confidant Document process throughout; record attendance, requests from colleagues, key quotations Share a calendar to document key demands Routinely source and share new ideas from the literature Present visual representations of progress mapped against overarching plan Incorporate a feedback mechanism in the decision-making process and communication plan Use an assets-based collaborative approach to boost interest and engagement Use evidence from data management system, or informal sources to draw out successes, challenges, failures, improvements required, and gauge interest Assess process using engaging participatory techniques such as co-operative interviews, round-robin/check-in, fist-to-five, and dot votes/bar graphs Establish an advisory group Evaluate the evaluation initiative to generate new knowledge, increase accountability, provide evidence, and assess impact
Key references King & Stevahn (2013); King & Volkov (2005); Taylor-Powell & Boyd (2008); Stevahn & King (2016); Stevahn (2013); Patton (2008); Preskill & Boyle (2008a)

Examples From Practice

Consolidated Strategies

Facilitate discussions, undertake confidential surveys, informally check in to take the temperature, and constantly gauge level of interest to get feedback from the team
Share simple visual assessments of progress toward overarching plan, such as a traffic light guide
Source and incorporate ideas from the literature
Bring together leaders and other key stakeholders after implementation to provide feedback on process and implementation
Role modeling personal reflection, learning, and development
Openly reflect on the similarities of the personal aspirations of other team members with the goals of the evaluator

Routinely ask for feedback from all colleagues to track what worked well, improvements required and suggestions; confidential surveys, a suggestion box, facilitated discussions, external confidant
Document process throughout; record attendance, requests from colleagues, key quotations
Share a calendar to document key demands
Routinely source and share new ideas from the literature
Present visual representations of progress mapped against overarching plan
Incorporate a feedback mechanism in the decision-making process and communication plan
Use an assets-based collaborative approach to boost interest and engagement
Use evidence from data management system, or informal sources to draw out successes, challenges, failures, improvements required, and gauge interest
Assess process using engaging participatory techniques such as co-operative interviews, round-robin/check-in, fist-to-five, and dot votes/bar graphs
Establish an advisory group
Evaluate the evaluation initiative to generate new knowledge, increase accountability, provide evidence, and assess impact

Key references
King & Stevahn (2013); King & Volkov (2005); Taylor-Powell & Boyd (2008); Stevahn & King (2016); Stevahn (2013); Patton (2008); Preskill & Boyle (2008a)

Step 4: Reflection and Consolidation

Table 6 summarizes Tables 1 –5 to provide a high-level overview template useful for constructively promoting evaluation and supporting colleagues in nonprofit settings. The first column lists the five elements of cooperative teamwork. The second column contains brief statements on how to operationalize each element. The third column points to key areas for focus. The fourth column provides key examples in practice.

Table 6.

Constructively Promoting Evaluation and Supporting Colleagues With Examples.

Element of Cooperative Teamwork:	How to:	Key Area:	Examples in Practice:
Positive interdependence	Link individuals to success of the group	Collaboratively develop a plan	Create diagrams of connections between program and evaluation
Individual accountability	Hold individuals accountable	Use a system to manage information	Create agendas, record decisions in meeting minutes and rationale logs to make process transparent and accessible
Promotive interaction	Provide encouragement	Create opportunities for engaging	Celebrate achievements and provide recognition at any opportunity
Social skills	Communicate clearly	Tailor communication appropriately	Use a strengths-based approach, consider diverse needs, and incorporate adult learning principles
Group processing	Reflect on how well the group is functioning	Schedule opportunities for reflection	Incorporate a feedback mechanism in the decision-making process and communication plan

To provide a short explanation of these requirements, positive interdependence can be operationalized by collaboratively enacting a plan. We have found it useful to work together to generate lists of priorities, create diagrams of how the program should work and link them with evaluation, articulate goals with timelines and tasks, and jointly generate a written purposeful, detailed, long-term plan. Individual accountability can be operationalized by systematically managing information and helping to clarify team members’ roles and responsibilities. The systems could begin with email reminders, benchmarking charts, step-by-step processes, and extend through to a database that connects to individual work plans. Promotive interaction can be operationalized by creating opportunities for engaging. Using every occasion to promote evaluation could involve establishing communities of practice, advisory groups, peer support structures, meetings, workgroups, task forces, internships, and mentoring. Social skills can be operationalized by tailoring communication appropriately. To enhance understanding and use of information, we suggest using a wide variety of accurate and unambiguous communication strategies and multiple platforms to connect to different people. Finally, group processing can be operationalized by scheduling opportunities for reflection. Using formal and informal sources of information, we suggest routinely asking for feedback from the team and incorporating feedback mechanisms at the outset of interactions.

Discussion

Social interdependence theory has strong internal and external validity in organizational settings and rigorous evidence to support the attainment of the purported outcomes (Johnson & Johnson, 2009). The value of using such a theory is knowing that it is grounded in a strong research base. As Johnson and Johnson (2002) stated, “The basic elements are a regimen that, if followed rigorously, will produce the conditions for effective cooperation” (p. 24). This gave us the confidence to extend our own experience through a formal self-reflection process. We have been able to use the elements of cooperative teamwork to frame our practical strategies and guide future interactions. We used this theoretical framework in combination with examples from the literature. This has enabled us to work constructively and with an increased degree of intent to support our colleagues with evaluation.

Internal evaluators need to work cooperatively with other people over the long term. They are required to think about how to set common goals, how to hold individuals accountable, how to structure opportunities to connect, how to communicate information, and how to incorporate reflection no matter what the purpose of their organization or context. However, we are aware that there are a multitude of variables in each organization that may act as inhibitors or enablers to prevent or support internal evaluators’ effective engagement with their colleagues around evaluation. Leadership support, the culture of evaluation, processes, and resources may all influence the level of success that an internal evaluator has in promoting evaluation. Acknowledging these factors and assessing the contextual conditions is important. However, our suggestions for how to operationalize the elements of cooperative teamwork for evaluation could be useful if the internal evaluator is intending to support a group of people working toward mutual goals. The elements of cooperative teamwork may be useful for application in a range of situations, including assisting internal evaluators in strategically incorporating interpersonal elements for their activities into practice to give their team the best chance of success.

Our process has resulted in a framework (Table 6) that can assist internal evaluators to structure the way they engage with their colleagues to promote access, understanding, and use of evaluation. This could form the basis of a checklist for internal evaluators to personalize and plan how they engage with their colleagues. The strategies included in this article offer a wide variety of examples that internal evaluators could adopt. However, these are intended as a catalyst for thinking about what may be possible and would need to be adapted and contextualized for each unique situation. In its simplest form, the five basic elements of effective cooperative teamwork could be used as a framework for interactive reflection on internal evaluator practice as it provides a way of personalizing professional development.

In our own practice, we used the basic elements to plan concrete actions to involve organizational members in internal evaluations. Applying this framework has meant considering the key areas during annual and monthly planning processes, discussing the elements of teamwork with immediate colleagues and other internal evaluators, and further brainstorming creative and engaging ways of sharing evaluation with colleagues that align with the elements. One author has inserted strategies against each key area in the framework, printed it onto a single page, and glued it above her desk to act as a constant reminder. Another regularly returns to the tables above for inspiration of what to implement next in the continual journey to build capability and evaluation literacy in her organizational setting. On an ongoing trajectory to improve and find ways of connecting our colleagues with evaluation in meaningful and appropriate ways, this framework has become another tool in our toolbox that helps us work with intent and an additional degree of confidence.

Value is in the Process

We suggest that the overarching value of this article is in the process of self-reflection using theory and literature to develop our interpersonal competencies as internal evaluators. Internal evaluators could use the process presented in this article to make sense of their practice and involve organizational members in ongoing internal evaluation, plan strategies, and reflect on effectiveness. The article shares a process that internal evaluators could use to engage in meaningful professional development and self-reflection.

This process has supported us as internal evaluators to think big, take a bird’s-eye view of the strategic landscape, connect with others across the wider organization, reflect on our journey as evaluators, seek advice about future requirements, and find examples from the literature and other organizations to support our work. The team members we are working with, as they should be, are incredibly proud of their work. As nonprofit staff, they are driven by helping other people. So too are we as internal evaluators who want our work to contribute to addressing social inequalities and challenges. As such, meaningful interaction between the more rational task of the evaluator and the strong humanistic value approach of the practitioner can be achieved through authentic engagement, respect, and mutual understanding. Reflective activities often remind us of how far we have come. We are reminded of the old ways of doing and thinking and marvel that we did not recognize how naive we once were. As we continue to learn and grow, a year down the track we will be reviewing again and marveling at how naive we are now. As internal evaluators, we recognize that the beauty of our roles is that we have the time and space to see the entire program from a different perspective than other team members. This specific reflective process has enabled us to broaden our perspective and further contribute to enhancing our effectiveness.

Further Research

The value of social interdependence theory is in relation to the strong focus on interpersonal interactions and the dynamics among individuals in the group. This theory recognizes that specific social skills are required to create psychologically safe environments in which to undertake evaluation. Evaluation can cause fear and anxiety as there are risks to an individual’s personal image as a result of participating in evaluative activities (Edmonson, 2003). When the elements of this theory, such as the establishment of mutual goals and building in opportunities for reflection at the initiation stage, are implemented in practice, the level of psychological safety is increased and the fear of being penalized for making a mistake is reduced (Edmonson, 2003). However, further research is required to explore whether social interdependence theory does offer enough guidance for evaluators to minimize the potential negative effects of power imbalances or unfair processes. A recent article by Froncek and Rohmann (2019) calls for further consideration of stakeholders’ socioemotional needs and suggest that the procedural justice literature can help evaluators understand that an individual’s relationship to the group is critical. We suggest that the next steps for the field of evaluation could be to conduct further research into how the practical framework outlined in this article could be implemented while simultaneously increasing the focus on the socioemotional interests of the group and being conscious of minimizing the potential negative consequences.

Conclusion

Preskill and Boyle (2008a) asked evaluators and their organizations how they chose their evaluation capacity building methods and were “struck by how many said that it was a very ‘organic’ process” (p. 156). We believe that, while having an attitude of going with the flow, being flexible, seizing opportunities when they arise and acknowledging the dynamics of momentum are all important qualities for internal evaluators, we should not be working in an ad hoc way. Internal evaluators should carefully consider colleague interactions in evaluation and plan their approach accordingly. Acknowledging the potential power of interpersonal skills to harness group dynamics will provide internal evaluators with a head start.

By drawing on social interdependence theory, internal evaluators can purposefully seek increased productivity, psychological health, and positive interpersonal relations that a solid body of research has demonstrated is possible when the five elements of cooperative teamwork (positive interdependence, individual accountability, promotive interaction, social skills, and group processing) are combined in practice. Learning from the literature about what has worked in other settings can provide internal evaluators with strategies that may be applicable in their setting. By combining theory, practice-based literature, and professional expertise, we offer a process to build internal evaluators’ capacity to enact, promote, and use intraorganizational evaluation with their colleagues.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Alison Rogers

Leanne M. Kelly

References

Alkin

M. C.

King

J. A.

(2017). Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation, 38(3), 434–450. https://doi.org/10.1177/1098214017717015

American Evaluation Association. (2018). American Evaluation Association: Competencies. Retrieved October 10, 2019, from https://www.eval.org/page/competencies

Archibald

Neubauer

L. C.

Brookfield

S. D.

(2018). The critically reflective evaluator: Adult education’s contributions to evaluation for social justice. New Directions for Evaluation, 2018(158), 109–123. https://doi.org/10.1002/ev.20319

Baron

(2011). Designing internal evaluation for a small organization with limited resources. New Directions for Evaluation, 132(Winter), 87–99. https://doi.org/10.1002/ev.398

Boyce

A. S.

(2017). Lessons learned using a values-engaged approach to attend to culture, diversity, and equity in a STEM program evaluation. Evaluation and Program Planning, 64, 33–43. https://doi.org/10.1016/j.evalprogplan.2017.05.018

Braverman

M. T.

(2013). Negotiating measurement: Methodological and interpersonal considerations in the choice and interpretation of instruments. American Journal of Evaluation, 34, 99–114. https://doi.org/10.1177/1098214012460565

Campbell

D. A.

Lambright

K. T.

(2017). Struggling to get it right. Nonprofit Management and Leadership, 27(3), 335–351. https://doi.org/10.1002/nml.21249

Cousins

J. B.

Chouinard

J. A.

(2012). Participatory evaluation up close: An integration of research-based knowledge. Information Age Publishing.

Davies

MacKay

(2014). Evaluator training: Content and topic valuation in university evaluation courses. American Journal of Evaluation, 35(3), 419–429. https://doi.org/10.1177/1098214013520066

10.

Deutsch

(1949). An experimental study of the effects of cooperation and competition upon group process. Human Relations, 2(3), 199–231.

11.

Deutsch

(2011). Cooperation and competition. In Coleman

P. T.

(Ed.), Conflict, interdependence, and justice: The intellectual legacy of Morton Deutsch (pp. 23–40). Springer. https://doi.org/10.1007/978-1-4419-9994-8_2

12.

Donaldson

S. I.

Gooler

L. E.

Scriven

(2002). Strategies for managing evaluation anxiety: Toward a psychology of program evaluation. American Journal of Evaluation, 23, 261–273. https://doi.org/10.1177/109821400202300303

13.

Edmonson

(2003). Managing the risk of learning: Psychological safety in work teams. In West

M. A.

Tjosvold

Smith

(Eds.), International handbook of organizational teamwork and cooperative working (pp. 255–276). John Wiley & Sons. https://doi.org/10.1002/9780470696712.ch13

14.

Fetterman

D. M.

Wandersman

(Eds.). (2005). Empowerment evaluation principles in practice. Guilford Press.

15.

Froncek

Rohmann

(2019). “You get the great feeling that you’re being heard but in the end you realize that things will be done differently and in others’ favor”: An experimental investigation of negative effects of participation in evaluation. American Journal of Evaluation, 40(1), 19–34. https://doi.org/10.1177/1098214018813447

16.

Garcia

G. L.

Stevahn

(2019). Situational awareness and interpersonal competence as evaluator competencies. American Journal of Evaluation, 41(1), 107–124. https://doi.org/10.1177/1098214018814941

17.

Gilchrist

Butcher

(2016). Introduction. In Butcher

Gilchrist

(Eds.), The three sector solution: Delivering public policy in collaboration with not-for-profits and business (pp. 3–22). ANU Press.

18.

Gill

S. J.

(2010). Developing a learning culture in nonprofit organizations. Sage Publications. https://doi.org/10.4135/9781452272030

19.

Gullickson

A. M.

King

J. A.

LaVelle

J. M.

Clinton

J. M.

(2019). The current state of evaluator education: A situation analysis and call to action. Evaluation and Program Planning, 75, 20–30. https://doi.org/10.1016/j.evalprogplan.2019.02.012

20.

Hardwick

Anderson

Cooper

(2015). How do third sector organisations use research and other knowledge? A systematic scoping review. Implementation Science, 10(1), 84. https://doi.org/10.1186/s13012-015-0265-6

21.

Harman

(2019). The great nonprofit evaluation reboot: A new approach every staff member can understand. Charity Channel.

22.

Johnson

(2003). Social interdependence: Interrelationships among theory, research, and practice. American Psychologist, 58(11), 934–945. https://doi.org/10.1037/0003-066x.58.11.934

23.

Johnson

(1989). Cooperation and competition: Theory and research. Interaction Book Company.

24.

Johnson

(2002). Cooperative learning and social interdependence. In Tindale

R. S.

Heath

Edwards

Posavac

E. J.

Bryant

F. B.

Suarez-Balcazar

Henderson-King

Myers

(Eds.), Theory and research on small groups (pp. 9–35). Plenum Press. https://doi.org/10.1007/0-306-47144-2_2

25.

Johnson

(2003). Training for cooperative group work. In West

M. A.

Tjosvold

Smith

K. G.

(Eds.), International handbook of organizational teamwork and cooperative working (pp. 167–184). John Wiley & Sons. https://doi.org/10.1002/9780470696712.ch9

26.

Johnson

(2005). New developments in social interdependence theory. Genetic, Social, and General Psychology Monographs, 131(47), 285–358. https://doi.org/10.3200/mono.131.4.285-358

27.

Johnson

(2009). An educational psychology success story: Social interdependence theory and cooperative learning. Educational Researcher, 38(5), 365–379. https://doi.org/10.3102/0013189X09339057

28.

Johnson

(2014). Aprendizaje cooperativo en el siglo XXI [Cooperative learning in 21st century]. Anales de Psicología, 30(3), 841–851. https://doi.org/https://dx-doi-org.web.bisu.edu.cn/10.6018/analesps.30.3.201241

29.

Johnson

Stevahn

(2011). Social interdependence and program evaluation. In Mark

Donaldson

Campbell

(Eds.), Social psychology and evaluation (pp. 288–317). The Guilford Press.

30.

Johnson

Tjosvold

Roseth

C. J.

(2019). Morton Deutsch: Celebrating his theorizing and research. Negotiation and Conflict Management Research, 12(1), 89–102. https://doi.org/10.1111/ncmr.12122

31.

Kelly

(2019). What’s the point? Program evaluation in small community development NGOs. Deakin University, Geelong.

32.

King

J. A.

(2007). Developing evaluation capacity through process use. New Directions for Evaluation, (116), 45–59. https://doi.org/10.1002/ev.242

33.

King

J. A.

Stevahn

(2013). Interactive evaluation practice: Mastering the interpersonal dynamics of program evaluation. Sage Publications. https://doi.org/10.4135/9781452269979

34.

King

J. A.

Volkov

B. B.

(2005). A framework for building evaluation capacity based on the experiences of three organizations. CURA Reporter, 3(35), 10–16.

35.

Lewin

(1935). A dynamic theory of personality. McGraw Hill.

36.

Lewin

(1948). Resolving social conflicts. Harper.

37.

Leviton

L. C.

(2014). Generative insights from the Eleanor Chelimsky Forum on Evaluation Theory and Practice. American Journal of Evaluation, 35(2), 244–249. https://doi.org/10.1177/1098214013503701

38.

Love

A. J.

(1991). Internal evaluation: Building organizations from within. Sage Publications.

39.

Mark

Donaldson

Campbell

(2011). Past, present, and possible futures of social psychology and evaluation. In Mark

Donaldson

Campbell

(Eds.), Social psychology and evaluation (pp. 4–28). Guilford Press.

40.

Mason

Hunt

(2018). So what do you do? Exploring evaluator descriptions of their work. American Journal of Evaluation, 40(3), 395–413. https://doi.org/10.1177/1098214018767049

41.

Mathison

(2011). Internal evaluation, historically speaking. New Directions for Evaluation, 2011(132), 13–23. https://doi.org/10.1002/ev.393

42.

Maton

(2017). Influencing social policy: Applied psychology serving the public interest. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199989973.001.0001

43.

Mayne

(2014). Issues in enhancing evaluation use. In Loud

M. L.

Mayne

(Eds.), Enhancing evaluation use: Insights from internal evaluation units (pp. 1–14). Sage. https://doi.org/10.4135/9781506335124.n1

44.

Mayne

(2017). Building evaluative culture in community services: Caring for evidence. Evaluation and Program Planning. https://doi.org/https://doi.org/10.1016/j.evalprogplan.2017.05.011

45.

Neigher

W. D.

Metlay

(1983). Values and methods: Evaluation and management perspectives. New Directions for Program Evaluation, 1983(20), 47–60. https://doi.org/10.1002/ev.1350

46.

O’Sullivan

R. G.

(2012). Collaborative evaluation within a framework of stakeholder-oriented evaluation approaches. Evaluation and Program Planning, 35, 518–522. https://doi.org/10.1016/j.evalprogplan.2011.12.005

47.

Ortega

R. M.

Coulborn Faller

(2011). Training child welfare workers from an intersectional cultural humility perspective: A paradigm shift. Child Welfare, 90(5), 27–49. https://doi.org/10.1037/e516652013-008

48.

Patton

(2008). Utilization-focused evaluation (4th ed.). Sage.

49.

Preskill

Boyle

(2008a). Insights into evaluation capacity building: Motivations, strategies, outcomes, and lessons learned. The Canadian Journal of Program Evaluation, 23, 147–174.

50.

Preskill

Boyle

(2008b). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443–459. https://doi.org/10.1177/1098214008324182

51.

Rogers

Gullickson

(2018). Evaluation champions: A literature review. Journal of Multidisciplinary Evaluation, 14(30), 46–63.

52.

Rogers

Kelly

L. M.

McCoy

(2019a). Evaluation literacy: Perspectives of internal evaluators in non-government organizations. Canadian Journal of Program Evaluation, 34(1). https://doi.org/10.3138/cjpe.42190

53.

Rogers

Kelly

L. M.

McCoy

(2019b). Pathways to becoming an internal evaluator: Perspectives from the Australian non-government sector. Evaluation and Program Planning, 74, 102–109. https://doi.org/10.1016/j.evalprogplan.2019.01.007

54.

Sonnichsen

R. C.

(2000). High impact internal evaluation: A practitioner’s guide to evaluating and consulting inside organizations (Kindle iOS). Sage. https://doi.org/10.4135/9781483328485

55.

Stevahn

(2013). Integrating co-operative community- based research (CBR) into doctoral leadership studies. Journal of Cooperative Studies, 42(2), 35–45.

56.

Stevahn

King

J. A.

(2005). Managing conflict constructively in program evaluation. Evaluation, 11, 415–427. https://doi.org/10.1177/1356389005059384

57.

Stevahn

King

J. A.

(2016). Facilitating interactive evaluation practice: Engaging stakeholders constructively. New Directions for Evaluation, (149), 67–80. https://doi.org/10.1002/ev.20180

58.

Stockdill

Baizerman

Compton

(2002). Toward a definition of the ECB process: A conversation with the ECB literature. New Directions for Evaluation, 2002(93), 7–26. https://doi.org/10.1002/ev.39

59.

Taylor-Powell

Boyd

H. H.

(2008). Evaluation capacity building in complex organizations. New Directions for Evaluation, 120(Winter), 55–69. https://doi.org/10.1002/ev.276

60.

Tindale

R. S.

Anderson

E. M.

(2002). Small group research and applied social psychology. In Tindale

R. S.

Heath

Edwards

Posavac

Bryant

Suarez-Balcazar

Henderson-King

Myers

(Eds.), Theory and research on small groups (pp. 1–8). Kluwer Academic Publishers. https://doi.org/10.1007/0-306-47144-2_1

61.

Tjosvold

Johnson

Sun

(2003). Can interpersonal competition be constructive within organizations? The Journal of Psychology, 137(1), 63–84. https://doi.org/10.1080/00223980309600600

62.

Tjosvold

Wong

A. S. H.

Feng Chen

N. Y.

(2014). Constructively managing conflicts in organizations. Annual Review of Organizational Psychology and Organizational Behavior, 1(1), 545–568. https://doi.org/10.1146/annurev-orgpsych-031413-091306

63.

Volkov

B. B.

(2011). Beyond being an evaluator: The multiplicity of roles of the internal evaluator. New Directions for Evaluation, 132(Winter), 25–42. https://doi.org/10.1002/ev.394

64.

Whitehall

A. K.

Hill

L. G.

Koehler

C. R.

(2012). A comparison of participant and practitioner beliefs about evaluation. American Journal of Evaluation, 33, 208–220. https://doi.org/10.1177/1098214011423803