The Prevalence of Evaluator Anxiety in Practice: An Empirical Examination

Abstract

Anxiety is a multifaceted force that can negatively impact the ability of evaluators to succeed in practice. In the evaluation literature, discussions concerning anxiety have primarily been limited to strategies to reduce stakeholder anxiety to encourage positive and productive working relationships with evaluators. This study was among the first to explicitly recognize and empirically assess the anxiety evaluators feel during practice. Using a mixed-methods design, a random sample of 109 American Evaluation Association members was surveyed. There was overwhelming agreement (96.30%) that evaluator anxiety is a commonly experienced phenomenon, often citing reasons such as lack of experience, imposter syndrome, unreasonable scope, and difficulties with stakeholder interactions as key contributors to increased anxiety. Evaluator anxiety varied across stages of the evaluation process, with some stages eliciting more anxiety than others. Further discussion and training concerning how to reduce evaluator anxiety could be useful in the preparation of evaluators.

Keywords

evaluator anxiety evaluator competencies interpersonal skills research on evaluation

Introduction

Anxiety is commonly defined as feelings of excessive worry and concern that negatively impact an individual's ability to function in daily life (Gale & Davidson, 2007). The detrimental effects of anxiety on working professionals have been studied across a wide array of disciplines that share many common elements with evaluation practice (Acker & Armenti, 2004; Crow, 1995; Holland, Lorenzi, & Hall, 2016; Lloyd et al., 2002).

One such detrimental effect is workplace performance anxiety which is expressed as nervousness or apprehension about participation in an activity conducted in front of a group of observers (Wilson & Roland, 2002). Workplace performance anxiety may arise during high-stakes negotiation processes, tough conversations with clients, or while presenting findings in front of an audience. In addition to workplace performance anxiety, working professionals may also experience social anxiety. Social anxiety arises from excessive fear or worries about how an individual is viewed by others and how these judgments are reflective of his or her self-worth and value (Zeidner & Matthews, 2005). Feelings of social anxiety among working professionals may result from having to interact with colleagues and clients outside of a performance-based setting.

As working professionals who are required to engage in negotiation, have tough conversations, interact with clients, and a variety of other activities acknowledged as anxiety-provoking, it is reasonable to posit evaluators will also be subject to anxiety in practice. However, the factors leading to anxiety for working professionals are context-specific—namely that they are likely to differ in their causes and consequences according to the nuances of the discipline at hand (Cheng & McCarthy, 2018). Therefore, specific factors leading to anxiety for evaluators will be unique to the profession of evaluation.

Anxiety Research in Evaluation

In the evaluation literature, discussions concerning anxiety have been primarily limited to strategies aimed at identifying and reducing stakeholder anxiety to encourage positive and productive working relationships (Donaldson et al., 2002). Stakeholder evaluation anxiety is defined as negative cognitions and emotions that arise in response to the belief that one's actions are under evaluative scrutiny (Wine, 1982). For example, stakeholders may be concerned about annual reviews of work quality or feel pressure to produce positive results (Cartwright & Cooper, 1997).

Conversely, there is a dearth of literature and research devoted to the study of evaluator anxiety. Evaluator anxiety is defined here as worry and apprehension about conducting the evaluation and interacting with colleagues and program staff, encompassing the reality that both performance and social anxiety are likely to be at play. Only one chapter we know of has explicitly referenced evaluator anxiety. Picciotto (2016) proposed evaluator anxiety often stems from the recognition that the results of the evaluation can never be asserted with perfect certainty as they are often shrouded in systemic and contextual factors that make the separation of truth from opinion a near-impossible task.

Evaluator Anxiety at Differing Stages of the Evaluation Process

Picciotto's work began the conversation on evaluator anxiety, but to the best of the authors’ knowledge, there is no research concerning the evaluation-specific factors leading to evaluator anxiety. To fill this lacuna, a useful way to organize these contextual factors is to consider the different challenges that may be encountered at each stage of the evaluation process. The stages of the evaluation process typically include identifying stakeholders, describing the program, focusing on the evaluation design, gathering credible evidence, justifying conclusions, and ensuring the use/sharing of lessons learned (Centers for Disease Control, 2018). There are numerous factors that may contribute to increased anxiety at each of these stages.

During the initial stages of the evaluation process (i.e., identifying stakeholders, describing the program, and focusing on the evaluation design) evaluators may become anxious at the ambiguity caused by not having clearly defined evaluation questions and methods. Uncertainty about how stakeholder relationships may develop can also provoke anxiety. Evaluators may worry that program leaders have expectations that exceed the monetary and personnel capacity of the evaluation team (Brown & Gerhardt, 2002). Evaluators may also be concerned that program leadership is unaware of what constitutes an evaluation and therefore have expectations that do not align with that of the evaluation plan. When approaching a new project, evaluators may also have doubts concerning their ability to conduct the evaluation. This fear may be intensified if they are new to the field, are working with a new group, or are using an unfamiliar methodology. Even more seasoned evaluators who have the necessary skillset may feel like “imposters” in the field and thus feel anxious about the process (Clance & O’Toole, 1987). In short, at the early stages of an evaluation, the presence of more unknowns than knowns may create excessive anxiety on the part of the evaluator (Carleton, 2016).

During the middle stages of the evaluation (i.e., gathering credible evidence) evaluator anxiety is likely to shift towards anxiety over the data collection process. Evaluators may feel anxious about receiving pushback from stakeholders when requesting access to relevant information or when beginning data collection (Taut & Brauns, 2003). This pushback could make data collection difficult, resulting in missed timelines and increased anxiety on the part of the evaluator about their reputation among stakeholders. Due to evaluators often being members of an out-group, program leaders may have a greater propensity towards viewing the data collection methods and the final evaluation results with suspicion (Brewer, 1999).

In the final stages of the evaluation (i.e., justifying conclusions and sharing lessons learned) fear concerning less than desirable evaluation findings is likely to be the primary source of evaluator anxiety. Even the most successful program is likely to have areas for improvement. Disclosing negative findings and making recommendations can be stress inducing and cause a great deal of anxiety on the part of the evaluator. Evaluators may feel pressure to produce evaluation results that will please program leadership and thus feel anxiety over having to have a tough conversation concerning less than desirable outcomes (Morris & Clark, 2013).

Evaluator Anxiety as a Function of Experience

Across stages, evaluation anxiety is also likely to be impacted as a function of age and experience. As with most professions, comfort increases with experience. Thus, it is likely that evaluators new to the field will experience the greatest levels of anxiety during the evaluation process (Moscovitch et al., 2009). Although newer evaluators may experience greater levels of distress and anxiety due to their relative lack of experience, this does not imply more seasoned evaluators are immune to these same feelings nor does it imply that these feelings do not impact their current practice.

Evaluator Anxiety as a Function of Value Constructions

Differing values held by evaluators and stakeholders may also contribute to heightened nervousness or apprehension for evaluators across the stages of the evaluation process. It has long been recognized that individuals are likely to feel uneasy when confronted with a situation that challenges, or stands in direct opposition to, their values (Heider, 1958; Smith & Campbell, 1973). These feelings of uneasiness will be heightened if the values called into question are of great importance and central to the identity of the individual (Jansz & Timmers, 2002; Norton et al., 2003). In April 2019, Michael Quinn Patton hosted a roundtable with evaluation scholars and practitioners to better understand the values most important to evaluators in their practice (Patton, 2021). Values centering around the principles of diversity, equity, and inclusion were found to be the most consistently mentioned and of the most important to evaluators. Evaluators also noted the importance of values surrounding social justice, evaluation use, learning from evaluation, evaluative thinking, and upholding moral and ethical principles. Thus, evaluators may experience higher levels of anxiety when their personal beliefs surrounding these key values conflict with those of their stakeholders. Conversely, evaluators may experience lower levels of anxiety when their personal values surrounding these key constructs align with those of their stakeholders.

Study Purpose

With the recent publication of the American Evaluation Association (AEA) evaluator competencies (American Evaluation Association, 2018; King & Stevahn, 2020), increasing attention has been paid to better understanding the professional, methodological, contextual, managerial, and interpersonal skillsets needed by evaluators, all of which have the potential to be impacted by anxiousness on the part of the evaluator. Therefore, knowing if, how, and why evaluator anxiety occurs during the various stages of the evaluation process is more crucial than ever to help prepare evaluators to encounter these feelings as well as equip them with strategies to ensure successful practice. Thus, the purpose of this study is four-fold: (1) understand the prevalence of evaluator anxiety in practice, (2) understand how, and why anxiety manifests for evaluators during the various stages of the evaluation process, (3) examine if experience level changes the degree of evaluator anxiety experienced, and (4) empirically test if the reasons purported to increase anxiety in the evaluation literature, do in fact increase the anxiety level of practicing evaluators. These purposes align to research questions 1, 1a, 1b, and 2, respectively.

Questions Investigated

The focal questions investigated in the study were:

Is anxiety a commonly experienced phenomenon during evaluation practice?

During what stages of the evaluation process are evaluators likely to experience feelings of anxiety?

Do new evaluators (< 5 years of field experience) differ in their mean levels of anxiety across stages compared to more seasoned evaluators (> 5 years of field experience)?

What factors may contribute to evaluators experiencing anxiety during the evaluation process?

Methodology

Design

A mixed-methods design was employed to answer the questions of interest. A survey comprising of quantitative and qualitative items was pilot tested prior to being administered to a random sample of American Evaluation Association members. The study was reviewed and approved by the Claremont Graduate University Human Subjects Institutional Review Board. All participants were provided with an electronic informed consent prior to participating in the study.

Sample

Pilot Study

Participants for the pilot study were recruited using convenience sampling from the researchers home academic institution (Claremont Graduate University). Twenty participants were recruited for the pilot study and 10 participants completed the survey in its entirety. After completion of the pilot test, the principal investigator debriefed with participants to refine items before the survey was distributed to members of the AEA.

Main Study

Participants for the main study were recruited using a simple random sample of the American Evaluation Association member database. A formal research request was submitted to the AEA Research Request Task Force. After receiving approval, the email addresses for 1,000 AEA members were chosen at random by the AEA Research Request Taskforce and provided to the principal investigator. AEA members were contacted twice over email from November to December 2020, using the Qualtrics web-based survey system. Inclusion criteria were that the participants had to be at least 18 years of age and a current or former practicing evaluator. One hundred and nine AEA members responded to the survey (10.9%). The respondent demographics as well as available comparisons to the AEA population are shown in Table 1 (Coryn et al., 2020). Of note, when respondents were asked to identify what evaluation contexts they were typically involved in, they were presented with a multi-select question, accounting for the reality that many evaluators may work in multiple contexts. Thus, the total percentage presented in Table 1 for Evaluation Context(s) exceeds 100%.

Table 1.

Demographic Characteristics.

Measure	AEA respondents (n = 10.9%)	AEA member population (%)
Gender
Male	15.3	24.1
Female	78.6	61.6
Prefer to self-describe	3.1	–
Prefer not to answer	3.0	–
Race/ethnicity
African American/Black	9.8	7.8
American Indian	2.9	1.0
Asian	4.9	6.0
Caucasian/White	67.7	53.5
Hawaiian or Pacific Islander	0.9	–
Hispanic or Latinx	6.9	4.5
Other/prefer not to answer	6.9
Age
Under 25	1.0	–
25–34	29.6	–
35–44	25.5	–
45–54	15.3	–
55–64	17.3	–
65–74	8.2	–
Over 75	1.0	–
Other/prefer not to answer	2.0	–
Length of time practicing evaluation
Less than a year	1.8	–
1–5 years	36.7	–
6–10 years	13.8	–
Over 10 years	36.7	–
Prefer not to answer	0.9	–
Job description
Independent consultant/external evaluator	50.0	–
Internal evaluator	28.6	–
Professor/teacher	7.1	–
Student worker	4.1	–
Other/prefer not to answer	10.2	–
Evaluation context(s)
Education	39.4	–
Public health/health care	33.9	–
Corporate/business sector	3.7	–
Government agency	30.3	–
Non-profit	53.2	–
Other/prefer not to answer	9.1	–

Instrumentation

The survey consisted of closed response and open-ended response items. The definition of evaluator anxiety used for this investigation (see Introduction: Anxiety Research in Evaluation) was first provided to participants. After defining evaluator anxiety, the survey then went through each stage of the CDC evaluation framework (2018), firstly providing the definition of key activities completed by the evaluator during each stage, and then asking respondents to rate the extent to which they experienced anxiety at each stage using a 5-point Likert scale ranging from “None, I do not feel anxious at this stage” to “Extreme—I feel very anxious at this stage.” Respondents indicating some degree of anxiety were subsequently prompted with an open-ended question asking them to describe what elicits feelings of anxiety at that stage. After going through each stage of the evaluation process, respondents were then asked to briefly list up to three factors that contribute to heightened feelings of anxiety generally in their evaluation practice. Next, respondents were prompted to consider their values stances surrounding key issues pertinent in the field of evaluation and how that might contribute to their feelings of anxiety during practice. Respondents were also asked to specifically rate on a 7-point Likert scale ranging from “Strongly Agree” to “Strongly Disagree” to what extent differences in specific values would heighten their evaluator anxiety. The survey concluded by obtaining key demographic information (age, gender, and race/ethnicity) as well as specific information on the conduct of evaluation practice such as what evaluation contexts they typically work in (health care, education, corporate, etc.), as well as years of experience in the field.

Results

Research Question 1

To answer the first research question, descriptive statistics were calculated to assess the frequency with which evaluators in the sample report experiencing feelings of anxiety. There was overwhelming agreement among respondents that anxiety was a commonly reported experience during evaluation practice. 96.30% (N = 104) of respondents answered “yes” to the question asking, “have you ever experienced feelings of nervousness/anxiety while conducting an evaluation?” To examine if evaluator anxiety differed across stages of the evaluation process, the mean levels of anxiety at each stage were calculated (Table 2). Evaluators reported being most anxious during Stage 1 of the evaluation process (M = 2.55, SD = 1.06) and least anxious during Stage 2 of the evaluation process (M = 1.68, SD = 0.83). Qualitative data gathered describing why evaluators feel anxious at each stage was also analyzed to triangulate the quantitative findings. Emergent themes as well as representative responses gathered from the open-ended response items describing why evaluators feel anxious at each stage are included in Table 2.

Table 2.

Anxiety as a Function of Stage of the Evaluation Process.

Evaluation stage	M	SD	Themes	Representative quotations
Stage 1: Engage stakeholders	2.53	1.06	Building rapport and understanding Interpersonal dynamics	“There is a bit of anxiety at the start of any new evaluation contract. There is a pressure to make quality impressions, get buy in for the work, and prove your worth.” “I’m engaging with new stakeholders, often from new organizations. More difficult to read new people and understand interpersonal/political dynamics. I find this part extremely anxiety producing.” “I sometimes don't have an existing relationship with the stakeholders, so this is the phase where I need to build rapport and trust. It can be anxiety inducing to think that the stakeholders might not like me or might be resistant to the evaluation process.”
			Understanding evaluation scope	“Getting my arms around the evaluation project/request; often I work with people who are not evaluators, and I have to translate their wants into something that is feasible and realistic, which sometimes requires difficult conversations.”
			Imposter syndrome	“I feel anxious that I’m not going to ask the “right” questions and that they will think I don't know what I’m doing.”
Stage 2: Describe the program	1.68	0.83	Poorly articulated program design	“Sometimes a program is not articulated well by the program implementer, leaving room for the evaluator to make assumptions or otherwise build an evaluation approach that may miss something down the road.”
			Concerns over bias	“Not wanting to impose my “personal interpretation” which may differ from the context organization due to socio-cultural factors.”
			Defensive program staff	“As an internal evaluator, some of the managers are defensive about how they have implemented their programs, are not transparent about how programs have been implemented.”
Stage 3: Focus the evaluation design	1.98	0.92	Fear of choosing wrong design	“Wanting to choose the best design without making mistakes or setting myself up for failure.” “Because I want to make sure the design captures everything that is needed for future analysis and is useful to the field of study.”
Stage 3: Focus the evaluation design	1.98	0.92	Feasibility and scope creep	“Landing on a design that is realistic, feasible and within the budget can, at times, raise uncomfortable conversations about what the client feels I should be able to provide vs. what is realistic based on my experience with the work. Some clients feel their budgets should ‘buy’ them more than they actually do.” “It is at this stage that scope creep begins to become apparent, or previously non-finalized understandings begin to make themselves evident.”
Stage 4: Gather credible evidence	2.02	0.97	Not feeling competent in data analysis	“In my experience so far, data is never as straightforward as it sounds like it should be. Difficult to find at all or to find in the correct units/format. Working with messy databases that lack documentation and/or for which extensive knowledge of program details is needed in order to avoid making logical errors.”
Stage 4: Gather credible evidence	2.02	0.97	Gathering data from reluctant groups	“Sometimes we need to gather data from people who are not fully “bought in” to the evaluation. They may not be interested in participating. We rely on them responding to surveys or collecting implementation data and it can be anxiety inducing to be repeatedly requesting information we may not get. I also am anxious that the people providing the information may be annoyed with my many requests.” “We all have different beliefs about what is ‘credible’ evidence.”
Stage 5: Justify conclusions	2.17	1.02	Questioning data credibility discussing negative findings	“Sometimes these claims about the program, especially if data/results say that it is not being implemented well or resulting in the outcomes as in the logic model, are met with push back. These discussions are normally delicate, and there needs to be careful consideration about how to present the information and lead the discussion. Anxiety results from the importance of these discussions and possible repercussions of them not going well.” “Stakeholders can be very defensive of their programs and not open to hearing constructive criticism. I find it is tough to relay results that are not positive while trying to maintain their satisfaction with my work.” “There is ambiguity in drawing conclusions, and I am generally the one who writes it all down. Printed words have power and there is anxiety in being sure that I have interpreted it correctly and that I have presented it fairly.”
Stage 5: Justify conclusions	2.17	1.02	Assuring conclusions are valid	“This is a big responsibility for evaluators. Do I have ample evidence or support to justify my conclusions? Am I introducing my own bias or opinion into it, particularly if I’m really integrated into a project? I want to make sure I’m doing my due diligence to ensure I’ve been able to justify my conclusions in a sound, objective way. Even just the perception of not doing so can be damaging.”
Stage 6: Ensure use and share lessons learned	1.88	1.01	Evaluation not benefitting groups equally	“Depending on the stakeholders involved and the amount of power that they wield, and who is funding the evaluation, many stakeholders don't benefit from the evaluation process. So it can be difficult sharing results when everyone doesn't benefit from them.” “I feel guilty that my evaluations end up being extractive, and that there is insufficient feedback to participants.”
			Presenting information	“Not everyone loves talking about data and evaluation. I get anxious thinking about how to best present data or engage with stakeholders about the evaluation so that this is a meaningful stage.”
			Lack of time and control to ensure use	“I don't feel I have control over whether or not the agency uses the results.” “Not enough time to adequately build relationships and influence around evaluation use.”

A repeated-measures ANOVA was conducted to assess if mean levels of anxiety differed across evaluation stages. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated, χ²(5) = 36.290, p < .001, and therefore, a Greenhouse–Geisser correction was used. There was a significant effect of the evaluation stage on evaluator anxiety level, F(4.258, 417.242) = 10.614, p < .001. Evaluators reported the highest levels of anxiety during Stage 1 of the evaluation process (M = 2.53, SD = 1.07), and the lowest anxiety levels during Stage 2 of the evaluation process (M = 1.68, SD = 0.84). Pairwise comparisons were calculated to determine which stages of the evaluation process elicited significantly different anxiety levels (Table 3).

Table 3.

Pairwise Comparisons for Mean Difference Anxiety Reported Across Evaluation Stage.

	M	SD	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5	Stage 6
Stage 1	2.53	1.07	–
Stage 2	1.68	.84	−0.85***	–
Stage 3	1.98	0.94	−0.55***	0.30**	–
Stage 4	2.02	0.97	−0.51**	0.34**	0.04	–
Stage 5	2.17	1.02	−0.35*	0.50***	0.19	0.15	–
Stage 6	1.88	1.01	−0.65***	0.20	−0.10	0.14	−0.29*	–

Note. *p < .05, **p < .01, ***p < .001.

To assess if new evaluators (< 5 years of field experience) differed in their mean levels of anxiety across stages of the evaluation process compared to more seasoned evaluators (> 5 years of field experience), an independent t-test was conducted. Although mean anxiety levels were higher for new evaluators (M = 2.15) compared to more seasoned evaluators (M = 1.94), the difference was not significant, t(91) = 1.87, p = .065.

Research Question 2

To explore what factors contribute to increased anxiety for evaluators in general when undertaking evaluation work, survey respondents were asked to briefly list up to three factors that have heightened their evaluator anxiety. Qualitative data gathered from these open-ended response items were collected, coded, and a content analysis was conducted. A general inductive approach was used to analyze the data, search for emergent themes, and describe the proposed reasons why evaluators may experience anxiety (Patton, 1990). Emergent themes as well as representative quotations detailing why evaluators generally feel anxious during practice are shown in Table 4. Themes are listed in accordance with their frequency, with the most (Imposter Syndrome/Lack of Experience) and least (Income Dependency) mentioned themes listed first and last, respectively. Of note, many of the themes listed in Table 4 mirror those reported by evaluators when describing their more specific experiences with anxiety at each stage of the evaluation process (Table 2). Overlapping themes are marked with an asterisk in Table 4.

Table 4.

Common Themes Cited for Increased Evaluator Anxiety.

Themes	Representative quotations
Imposter syndrome/lack of experience*	“I am only 5 years into my career, so there are tons of imposter syndrome … every project brings new experiences that can be a little scary.” “Not enough experience. No exemplar modeling in my training program (grad school).” “Lack of training and confidence in my early professional career, not sure if I am ‘doing it right.’” “Imposter syndrome, fear of not being seen as legitimate as a 30-year-old Black woman.” “I don't have enough experience, particularly with writing estimates and deciding how much my time is worth, and I’m always afraid I’ll do a poor job and have wasted someone's time or look incompetent.” “Lack of formal training. Lack of experience.” “I don't feel like I received adequate training before embarking on my career. I have a Master's degree in Program Evaluation, but only one of my courses was actually about evaluation. This means that I doubt my own skills and abilities and feel like I’m missing the necessary basic knowledge in the field.” “Insecurities about my ability to provide valuable feedback, especially when I’m still learning about the nuances of a particular area or issue.” “My unconventional path to becoming an evaluator; imposter syndrome.” “Constant awareness of all I don't know, working with more experienced evaluators who know more than I do and worrying about my skill level—is it enough?”
Stakeholder interactions (disagreement, conflict, and reaction to findings)*	“Diverging viewpoints on purpose among stakeholders. Disagreements amongst stakeholders on results.” “Conflict among stakeholders, sharing data that isn't positive.” “Stakeholder demeanor, unresponsive stakeholders.” “Presenting findings to individuals who are not interested in or are hostile to the findings.” “Disagreement on values, resistance to evaluation/data collection adoption.” “Presenting difficult results. Having a client who does not understand evaluation but believes s/he does.” “When there is conflict or poor communication with the funder or other partners anxiety is high. This can often happen as when you are an evaluator you don't always get to choose your team.”
Navigating politics	“Navigating the politics of funders, partners, and participants.” “Power dynamics and how to remain true to participants without constantly fighting funders.” “Organizational politics, and discovering and dispelling misconceptions of my field by those who don't do Eval work.”
Role uncertainty	“Fuzzy proposals/projects [and] uncertainty of the evaluators’ role.” “Contested or unclear/messy context where the focus or scope is hard to nail down.”
Lack of time/funding and unreasonable scope*	“Too-short timelines. Tight budgets.” “Minimal time to implement evaluations.” “Initial expectations to produce deliverables that exceed project resources and budget.” “Unreasonable expectations on what can be produced/achieved within resources available.” “Not having the time needed to achieve evaluate intentions or plans.” “Funding is always a concern that [causes] issues with implementers not doing what was promised.”
Income dependency	“When your evaluation feedback is negative or perceived to be negative, there can be anxiety linked to your source of income. Will you be contracted again or will others have a negative perception of the value of your work because others did not appreciate your conclusions? When your income depends upon your ability to obtain and maintain contracts, and even the income and well-being of others, including those you provide evaluation services, there can be considerable anxiety.”

Survey respondents were also asked to examine how their value constructions on issues deemed most pertinent to evaluators (see Background: Anxiety as Function of Value Constructions) would impact their feelings of anxiety in practice. There was majority agreement, with most respondents either strongly agreeing (N = 16) or somewhat agreeing (N = 52) that increased anxiety would result if they worked with program stakeholders that shared different values than their own. Differences in values regarding Commitment to Upholding Moral and Ethical Principles and Commitment to Upholding Diversity, Equity and Inclusion Principles were cited as causing the most anxiety among members of the sample (Table 5).

Table 5.

Anxiety as a Function of Differences in Value Constructions.

Value construction	M	SD
Commitment to upholding moral and ethical principles	5.89	1.50
Commitment to upholding diversity, equity, and inclusion principles	5.35	1.64
Importance of collaboration	5.22	1.71
Commitment to social justice	5.12	1.76
Importance of evaluation use	4.95	1.71
Importance of continuous improvement	4.91	1.77
Commitment to evaluation as a learning tool	4.89	1.69
Importance of thinking evaluatively	4.52	1.71

Discussion

As evidenced by the study findings, evaluator anxiety is commonly experienced during evaluation practice. 96.30% of evaluators reported having experienced anxiety during practice. Additionally, evaluators differed in their levels of anxiety across evaluation stages, with the highest anxiety levels reported in Stage 1 (Engage Stakeholders), Stage 5 (Justify Conclusions), and Stage 4 (Gather Credible Evidence), respectively. These stages were the most likely to lead to anxiety caused by imposter syndrome, discomfort with methodology, and apprehension about stakeholder interaction. Many of the themes associated with increased anxiety at a specific stage of the evaluation process were mirrored in the more general discussion of factors that increased anxiety across the evaluation process. Imposter syndrome, challenges with stakeholder interactions, and lack of time/funding and unreasonable scope were listed as factors associated with both a specific stage of the evaluation process as well as more general anxiety throughout the evaluation process, adding credence to the argument that these are the predominant set of variables contributing to evaluator anxiety. Three additional themes (navigating politics, role uncertainty, and income dependency) emerged as factors that elicited general anxiety throughout the evaluation process but were not associated with a particular stage, suggesting that they may be overarching factors that indirectly impact evaluators more than they directly affect a specific stage of the evaluation process.

The factors found to underpin evaluator anxiety suggest that many evaluators lack adequate training and preparation in many of the AEA Evaluator competency domains. Specifically, challenges with stakeholder interactions and navigating politics may both stem from inadequate training in the interpersonal domain, which focuses on building effective strategies to foster engagement, resolve conflict, and ensure effective communication. Feelings of role uncertainty and imposter syndrome may be due to inadequate training and knowledge of the professional practice domain, which focuses on understanding the distinct aspects of evaluation practice. Knowledge of this domain would aid evaluators in better understanding their role and feel confident they have the necessary skill set to meet the demands of conducting an evaluation. Finally, difficulties with unreasonable scope may reflect a lack of training in the planning and management domain, which focuses on ensuring work plans and timelines are reasonable and can be accomplished within the stated budget efficiently and effectively by the evaluation team. Taken together, the findings suggest that although the importance of these domains is emphasized by professional associations such as AEA, evaluators still struggle with their acquisition in practice, leading to heightened evaluator anxiety.

The study also found that there was not a significant difference in mean anxiety levels between those with more (> 5 years) and less (< 5 years) experience practicing evaluation. This may reflect the universal nature of evaluator anxiety regardless of experience, or the reality that imposter syndrome may still be felt and contribute to increased anxiety even among more seasoned evaluators. Conversely, due to the rather small sample size, the study may not have been sufficiently powered to detect such effects, or it may be the case that a smaller window of time for heightened anxiety exists at the start of an evaluator's career. By the fifth year of practice heightened anxiety due to unfamiliarity and inexperience may have already declined to the rate experienced by more seasoned evaluators.

The study also revealed that if values concerning upholding diversity, equity, and inclusion and other strongly held moral and ethical principles were called into question by program leadership and staff, evaluators reported that they would feel increased levels of anxiety, showcasing evaluators care deeply about these topics and are committed to defending issues of critical importance. Other issues such as the importance of continuous improvement, commitment to evaluation as a learning tool, importance of evaluation use, and thinking evaluatively were relatively less anxiety-inducing by evaluators. This may reflect a shift in the priorities and values of the evaluation community in response to recent world events and the increasing recognition of the role evaluation and evaluators can play in the growing social justice movement.

Strengths and Limitations

Knowing how anxiety is experienced by evaluators can serve to move the practice of evaluation forward in several ways. Firstly, since evaluator anxiety is an emerging phenomenon of interest and has not been studied extensively, the findings of the study are the first to confirm evaluator anxiety as a common experience for evaluators during practice. The recognition of evaluator anxiety as a commonly experienced state can help open the dialog to include discussions of anxiety to help normalize the experience among practitioners. However, as evaluator anxiety is a relatively new phenomenon that has yet to be examined, the instrumentation to assess evaluator anxiety did not draw upon a well-validated survey instrument and thus may have limited the study findings.

Secondly, the rich qualitative data collected provided invaluable insight into not only if evaluators are feeling anxious, but more importantly why they are feeling anxious. Nearly all respondents gave a detailed account of why they feel anxious in practice, hitting on many topics that are of increasing interest among evaluators today such as the detrimental effects of imposter syndrome (LaVelle et al., 2022), difficulties stemming from lack of experience faced by young or emerging evaluators (EvalPartners, 2018), and feeling a duty to uphold the principles of social justice. This highlights the intersections and synergy between the study of evaluator anxiety and other emerging topics in the evaluation literature and gives credence to evaluator anxiety being considered a crucial area of study in modern evaluation research.

Thirdly, with the recognition of evaluator anxiety as a tangible phenomenon, specific knowledge of when it emerges in the evaluation process, and an understanding of the key contributing factors to increased evaluator anxiety, efforts can be made to help evaluators understand and prepare for experiences of anxiety during practice. Currently, there is not a well-developed literature base concerning best practices to ensure evaluators feel well-equipped and confident. The recognition of evaluator anxiety takes the first step at expanding this literature base by acknowledging the negative experiences evaluators often must overcome during evaluation practice and understanding how the causes of their anxiety may manifest differently at each stage in the evaluation process. Of note, as the sample size of the study was small and comprised of mostly Caucasian and mostly female respondents, the responses gathered may not be reflective of the larger views of the evaluation community and further work needs to be done to develop this literature base. With a well-developed literature base, targeted evaluator training programs can be established to combat the detrimental effects of evaluator anxiety and ensure evaluators have the best chance at successful practice.

Finally, the discussion of anxiety for evaluators is timely. Given the onset and continuation of the COVID-19 global pandemic, evaluators have been faced with a barrage of challenges; often being asked to take on an increased number of tasks, learn new skills, navigate an unfamiliar virtual environment, all while operating with increasingly limited resources and personnel. Taken together, these factors are likely to heighten feelings of evaluator anxiety, thus making it essential that this phenomenon is better understood and recognized.

Future Directions

As among the first to empirically examine evaluator anxiety, this study lays a critical foundation for future investigation into this topic. Considering evaluation anxiety was found to differ across stages of the evaluation process, future studies could explore other factors that may similarly cause levels of evaluator anxiety to vary such as the role of the evaluator, level of organizational or institutional support, degree of professional training, and the evaluation context. For example, those working in an external evaluation capacity might have significantly more anxiety pertaining to income dependency than their internal evaluator counterparts, whereas internal evaluators may feel less apprehension and anxiety about stakeholder interactions due to familiarity than external evaluators. In addition, those with more formal evaluation training may feel more secure in their professional identity, decreasing feelings of imposter syndrome and associated anxiety.

As noted above, the sample size of the study was small and comprised of mostly Caucasian and mostly female respondents, thus subsequent research should represent the larger evaluation community. Subsequent studies might also explore strategies and effective principles to overcome evaluator anxiety, drawing upon methods evaluators have found useful in the past, as well as critical insights from other related fields of study. Additionally, with the recent publication of the AEA competencies (American Evaluation Association, 2018) there are many questions remaining about the application of the competencies in practice, as well as how to train evaluators to increase their likelihood of success in each domain. Understanding evaluator anxiety is one piece of this puzzle, but future studies could examine the individual experiences of practicing evaluators in greater detail with the hope of contributing to the ongoing professional development of the evaluation community, and better preparing the next generation of evaluators.

Concluding Remarks

As a pioneer empirical investigation into evaluator anxiety, this study established that evaluators do experience anxiety during practice, their level of anxiety differs according to the stage of the evaluation process, and that imposter syndrome, navigating stakeholder dynamics, and unreasonable evaluation scope were commonly cited reasons for increased anxiety. Future studies could expand on critical insights gleaned from this investigation to further explore and better understand the experiences of practicing evaluators.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jessica Renger

References

Acker

Armenti

(2004). Sleepless in academia. Gender and Education, 16(1), 3–24. http://doi.org/10.1080/0954025032000170309

American Evaluation Association (2018). American evaluation association evaluator competencies. https://www.eval.org/Portals/0/Docs/AEA%20Evaluator%20Competencies.pdf

Brewer

M. B.

(1999). The psychology of prejudice: Ingroup love and outgroup hate? Journal of Social Issues, 55(3), 429–444. http://doi.org/10.1111/0022-4537.00126

Brown

K. G.

Gerhardt

M. W.

(2002). Formative evaluation: An integrative practice model and case study. Personnel Psychology, 55(4), 951–983. http://doi.org/10.1111/j.1744-6570.2002.tb00137.x

Carleton

R. N.

(2016). Fear of the unknown: One fear to rule them all? Journal of Anxiety Disorders, 41, 5–21. http://doi.org/10.1016/j.janxdis.2016.03.011

Cartwright

Cooper

C. L.

(1997). Managing workplace stress (Vol. 1). Sage.

Centers for Disease Control (2018). Framework for Program Evaluation . https://www.cdc.gov/eval/steps/index.htm

Cheng

B. H.

McCarthy

J. M.

(2018). Understanding the dark and bright sides of anxiety: A theory of workplace anxiety. Journal of Applied Psychology, 103(5), 537. http://doi.org/10.1037/apl0000266

Clance

P. R.

O’Toole

M. A.

(1987). The imposter phenomenon: An internal barrier to empowerment and achievement. Women & Therapy, 6(3), 51–64. http://doi.org/10.1300/J015V06N03_05

10.

Coryn

C. L.

Becho

L. W.

Westine

C. D.

Mateu

P. F.

Abu-Obaid

R. N.

Hobson

K. A.

, … Ramlow

(2020). Material incentives and other potential factors associated with response rates to internet surveys of American Evaluation Association members: Findings from a randomized experiment. American Journal of Evaluation, 41(2), 277–296. http://doi.org/10.1177/1098214018818371

11.

Crow

(1995). Institutionalized competition and its effects on teamwork. The Journal for Quality and Participation, 18(3),46.

12.

Donaldson

S. I.

Gooler

L. E.

Scriven

(2002). Strategies for managing evaluation anxiety: Toward a psychology of program evaluation. American Journal of Evaluation, 23(3), 261–273. https://doi.org/10.1016/S1098-2140(02)00209-6

13.

EvalPartners (2018). What does EvalYouth Want to Achieve? https://www.evalpartners.org/evalyouth

14.

Gale

Davidson

(2007). Generalized anxiety disorder. Bmj, 334(7593), 579–581. http://doi.org/10.1136/bmj.39133.559282.BE

15.

Heider

(1958). The psychology of interpersonal relations. John Willey & Sons.

16.

Holland

Lorenzi

Hall

(2016). Performance anxiety in academia: Tensions within research assessment exercises in an age of austerity. Policy Futures in Education, 14(8), 1101–1116. http://doi.org/10.1177/1478210316664263

17.

Jansz

Timmers

(2002). Emotional dissonance: When the experience of an emotion jeopardizes an individual’s identity. Theory & Psychology, 12(1), 79–95. http://doi.org/10.1177/0959354302121005

18.

King

J. A.

Stevahn

(2020). Presenting the 2018 AEA evaluator competencies. New Directions for Evaluation, 2020(168), 49–61. http://doi.org/10.1002/ev.20435

19.

LaVelle

J. M.

Jones

N. D.

Donaldson

S. I.

(2022). Addressing the elephant in the room: Exploring the impostor phenomenon in evaluation. American Journal of Evaluation. https://doi.org/10.1177/10982140221075243

20.

Lloyd

King

Chenoweth

(2002). Social work, stress and burnout: A review. Journal of Mental Health, 11(3), 255–265. http://doi.org/10.1080/09638230020023642

21.

Morris

Clark

(2013). You want me to do what? Evaluators and the pressure to misrepresent findings. American Journal of Evaluation, 34(1), 57–70. http://doi.org/10.1177/1098214012457237

22.

Moscovitch

D. A.

Antony

M. M.

Swinson

R. P.

(2009). Exposure-based treatments for anxiety disorders: Theory and process. In Anthony

M. M.

Stein

M. B.

(Eds.), Oxford handbook of anxiety and related disorders (pp. 461–475). New York, NY: Oxford University Press.

23.

Norton

M. I.

Monin

Cooper

Hogg

M. A.

(2003). Vicarious dissonance: Attitude change from the inconsistency of others. Journal of Personality and Social Psychology, 85(1), 47. http://doi.org/10.1037/0022-3514.85.1.47

24.

Patton, M. Q. (2021). How rar dare an evaluator go toward saving the world?: Redux, update, and a reflective practice facilitation tool. American Journal of Evaluation, 42(2), 162–184. https://doi.org/10.1177/1098214020927095

25.

Patton

M. Q.

(1990). Qualitative evaluation and research methods. Sage Publications.

26.

Picciotto

(2016). Evaluator anxiety in an inequitable world. In M. M. Anthony & M. B. Stein (Eds.), Evaluation for an equitable society (pp. 217–235). Information Age Publishing.

27.

Smith

R. E.

Campbell

A. L.

(1973). Social anxiety and strain toward symmetry in dyadic attraction. Journal of Personality and Social Psychology, 28(1), 101. http://doi.org/10.1037/h0035589

28.

Taut

Brauns

(2003). Resistance to evaluation: A psychological perspective. Evaluation, 9(3), 247–264. https://doi.org/10.1177/13563890030093002

29.

Wilson

G. D.

Roland

(2002). Performance anxiety. The science and psychology of music performance: Creative strategies for teaching and learning (pp. 47–61). Oxford Press.

30.

Wine

J. D.

(1982). Evaluation anxiety: A cognitive-attentional construct. Series in Clinical & Community Psychology: Achievement, Stress, & Anxiety, 207–219.

31.

Zeidner

Matthews

(2005). Evaluation anxiety. In M. M. Anthony & M. B. Stein (Eds.), Handbook of competence and motivation (pp. 141–163). Oxford University Press.