The Role of Intuition in Evaluative Judgment and Decision

Abstract

Expert intuition is increasingly considered to be a valid form of knowledge, and research has proven its effectiveness in judgment and decision making in various fields. Theorists seem to recognize the contributions of intuition within evaluative practice, but it has never been well-documented. This article presents a study on expert intuition, addressing the manner in which intuition is developed, as well as how it contributes to producing judgments in the specific context of program evaluation. In-depth, in-person interviews were conducted with eight novice evaluators and eight experienced evaluators in order to assess the contributions of experience and expertise. Two key observations emerged from interview analyses. The first was that intuition is developed through a long, complex, and demanding process in which reflective analysis of experiments, successes, and failures play an essential role; the second was that the development of intuition is fostered by expertise and experience.

Keywords

decision making evaluation practice professional development theory practice relationship

It has been said that “when Michelangelo looked at a block of marble, he knew based on his intuition and experience what figure was imprisoned within” (Grinnell, 2009, p. 55). Expert intuition (Tversky & Kahneman, 1974) is increasingly considered to be a valid form of knowledge in psychology, cognitive science, business management, education, sports, health care, and engineering (Dane & Pratt, 2009; Gore & Conway, 2016; Hodgkinson et al., 2009; Kermarrec & Bossard, 2014; Klein, 2015; Okoli, Weller, & Watt, 2016; Sinclair & Ashkanasy, 2005). Research has also proven the effectiveness of expert intuition in judgment and decision making, namely in emergency contexts (Hodgkinson et al., 2009; King & Clark, 2002; Klein et al., 2010; Okoli & Watt, 2018; Rosciano et al., 2016; Vanlommel et al., 2017), where the reflective processes must be short lived because action must be taken quickly.

House (1977, p. 71) described evaluation practice as “an act of persuasion directed to a specific audience concerning the solution of a problem.” He added that the complexity of some situations can force evaluators to apply intuitive logic. Stake (1991, 2004) endorsed this stance, associating evaluative judgment with tacit knowledge as well as intuition, apprehension, and certain emotions. This blend provides a deep and rich understanding of any kind of program, and it is operationalized by the following two concepts: quality-as-measured and quality-as-experienced (intuitive; Stake & Schwandt, 2006). Although judgment is mostly intellectual, there are times when “well-educated” emotion and intuition, and more precisely expert intuition, transcend reasoning.

In his study on the influence of life experiences on the professional practice, Williams (2016, 2018) interviewed 29 experienced evaluators and concluded that all of them had values they associate with their professional evaluation practice, which they combine with varying levels of emphasis. One of the combinations was “balanced use of rationality with intuition (90%)” (cited in Williams, 2018, p. 134). The author illustrated this balance in an analysis of an expert evaluator’s process: “Smith clarifies his strong values glanced through a lifetime of experiences in terms of observations, reasonings, and intuitions which impact his criteria and methods for evaluating personally and professionally” (cited in Williams, 2018, p. 139).

Even though these theorists have recognized the contribution of intuition within evaluative practice, this contribution has never been documented with evidence. The study reported here, based on a postdoctoral study on expert intuition by Rahmanian et al. (2016), addresses the manner in which expert intuition is developed, as well as how it contributes to producing judgments in the specific context of program evaluation.

Theoretical Framework

The Concept of Intuition

Albert Einstein once said that “the intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honours the servant and has forgotten the gift” (cited in Klein, 2003, p. 3). The etymological roots of the word intuition come from the Latin word in-tuir, meaning “looking, regarding, or knowing from within” (Hodgkinson et al., 2008). Schwartz and Sharpe (2010) defined intuition as “thinking with feeling,” while Nobel Prize–winning economist Herbert A. Simon understood intuition as subconscious pattern recognition (cited in Frantz, 2003, p. 266). Chase and Simon (1973) studied intuition in the context of a chess game, referring to a form of tacit knowledge developed through years of dedication, hard work, and consistent and deliberate practice. They suggested that at least 10,000 hr of dedicated practice is required to attain the highest level of performance. Indeed, the more varied experience patterns people acquire over their lifetime, the more prestored patterns they have available to apply in new situations (Hoffrage & Marewski, 2015; Kermarrec & Bossard, 2014; Klein et al., 2010; Okoli et al., 2014; Rosen et al., 2010; Tissington & Flin, 2005; Zsambok, 1997). Gigerenzer (2008) referred to “a judgment that appears quickly in consciousness whose underlying reasons we are not fully aware of, and is strong enough to act upon” (p. 16). Regardless of the variations in these definitions, there is a growing consensus among theorists that intuitive judgment requires time to develop, incurs little or no information processing costs, and enables individuals to quickly integrate multiple justifications into their decisions with little mental energy (Hoffrage & Marewski, 2015).

Vaughan (1979) described intuition as knowing without knowing how, and Mukhopadhyay (2018) supported this opinion by adding that “intuitive process is non-conscious. However, the outcome is conscious” (p. 356). Nevertheless, intuition must include the following main characteristics: authenticated experience and expertise, a bank of lived experience promoting the development of patterns, and a feeling of déjà vu (Frantz, 2003; Klein, 2003; Mattsson & Fast-Berglund, 2016). Dörfler and Ackermann (2012) identified six characteristics of intuition. The first three relate to the process itself, while the last three relate to the outcome of the process. Intuition consists of instantaneous judgments, it involves unconscious information processing with minimal mental effort, and it follows an alogical process (i.e., it neither contradicts nor follows the rules of logic). The outcome of the intuitive process—the judgment—is usually tacit (it is difficult to describe how the judgment was formed), holistic (unconnected elements are integrated to generate more meaningful insights), and produces in professionals a sense of confidence about their judgment although they are unable to justify the chosen course(s) of action. According to House (2015), expert intuition is subject to two conditions: The first concerns the predictability of the environment, that is, relating the situation to a well-known experience, and the second is the opportunity to learn from these regularities, thereby feeding the bank of stored patterns.

Models

Evans (1989, 2003, 2009), Evans and Stanovich (2013), Kahneman (2000), and Kahneman and Frederick (2002) viewed intuitive decision-making processes in reference to System 1 and System 2 thinking. System 1 thinking, or intuitive processing, is unconscious, contextualized, and automatic, allowing for rapid execution that does not overburden a person’s central processing capacity. According to Klein (2003), intuitive decision making necessarily involves a familiar problem. In contrast, System 2 thinking, or analytic processing, is slow, sequential, conscious, and controlled. At the heart of these dual-process models lies the idea that human reasoning relies on two different types of thinking. Later, K. Stanovich (2011) divided System 2 thinking into algorithmic processing and reflective processing. According to K. Stanovich (2011), algorithmic processing is associated with the mental capacity that is measured by intelligence tests, while reflective processing describes what we would call critical thinking, or the “tendency to collect information before making up one’s mind…to calibrate to the degree of strength of one’s opinion on the evidence available…to think about future consequences before taking action” (p. 36). In short, System 2 thinking can override System 1 thinking in two situations: Either System 1 responses did not prove to be helpful or System 2 responses offer a better alternative. Kahneman (2011) noticed that people do not naturally engage in System 2 thinking despite the fact that deliberate thought is necessary for evaluating and managing risk. This remains true in the face of substantial research indicating that System 2 decreases biased judgments and that System 1 thinking often generates systematic errors. Risen (2016) developed the concept of acquiescence to describe situations where people maintain their intuitive judgment even when they are aware that it is incorrect.

Another theoretical framework is the recognition-primed decision-making model (RPDM) put forward by Klein et al. (1986), where decisions are directly derived from the recognition of critical information (cues) and prior experiential knowledge. Simply put, the RPDM describes how recognized patterns can be effectively used to solve current problems and suggests that intuition has a greater capacity for problem-solving than analysis because excessive deliberation tends to disrupt naturally flowing first impressions. The recognition/metacognitive (R/M) model subsequently developed by Cohen et al. (1996) takes into consideration the likelihood of encountering new events that can altogether defy existing knowledge. The R/M model proposes a two-tiered process: an activation stage where action plans are developed through pattern recognition and a critiquing and correcting stage where the workability of the outcome of the activation stage is quickly assessed and deliberately checked for faults.

Whereas the meta-analysis produced by Wang et al. (2017) strongly supported the view that intuition and analysis are two independent constructs, recent research suggests that intuition and analysis may be complementary rather than competitive, postulating that the expected answer produced by System 2 thinking can be cued by System 1 thinking. This way of operating generates two kinds of intuitive responses: heuristic responses, which are based on stereotypical associations, and logical responses, which are based on automatically activated elementary knowledge of logico-mathematical principles. This brings up the question of hybrid models (Bago & De Neys, 2017; Banks, 2017; De Neys, 2012; Epstein, 1994; Handley & Trippas, 2015; Johnson et al., 2016; Nakamura & Kawaguchi, 2016; Pennycook et al., 2014, 2015; Sloman, 1996; Smith & DeCoster, 2000; Thompson & Johnson, 2014; Trippas et al., 2016, 2017; Trippas & Handley, 2017). Bago et al. (2018) developed a hybrid dual-process model in which the faster System 1 thinking cues both heuristic, belief-based responses (stereotypes) and elementary logico-mathematical principles (base rates). Julmi (2019) also took this approach, adopting a parallel-competitive view based on growing evidence from neurological and experimental studies (Alós-Ferrer & Strack, 2014; Healey et al., 2015; Kuo et al., 2009; Lieberman, 2007). In short, the hybrid dual-process model should be retained as the approach offering the greatest validity.

Contribution to Decision Making and Judgments

House (2015, 2016) described intuition as “a natural thought process” (House, 2016, pp. 6–7). However, he acknowledged that it does not offer protection against bias since intuition is unconscious. King and Clark (2002) investigated the impact of intuition on nursing practice. Their results revealed that nurses solved difficult problems using both intuition and analytical styles. Baylor (2001) noted that novice nurses tended to display less confidence in the way they recalled and implemented their intuitive knowledge. To Sinclair (2010), intuition seemed to be more effective in complex decision situations because the amount of information was too huge to be processed consciously, while Rosciano et al. (2016) found that unconscious intuitive processes allowed nurses to synthesize information, giving them a sense of security, especially if their intuitive solution was supported by an improvement in the patient’s condition. In the field of management, Knemeyer and Naylor (2011) observed that managers could not rely solely on rational decision-making processes, leading them to combine intuition with systematic analysis, quantified data, and thoughtfulness.

Recently, Okoli et al. (2014), Okoli, Weller, and Watt (2016), and Okoli, Watt, et al. (2016) conducted a study involving 30 experienced firefighters across selected fire stations in the United Kingdom (n = 15) and Nigeria (n = 15) in order to develop a cognitive model describing how experts make important decisions within time constraints. The multistage information filtering and intuitive decision-making model works as follows. The process starts with an information search phase that consists of continuously scanning information feeds from internal and external sources to make sense of the event and then designing an action plan. Internal sources include memories of previously stored patterns, knowledge from prior training exercises, and relevant past incidents, while external sources are composed of cues associated with a particular incident and information from the public. In an emergency, this first phase can lead directly to a critical decision. In the second phase, information is filtered and the main cues that occur throughout the process are identified by distinguishing between relevant and irrelevant information. This phase feeds into the action plan, as do intuitive decisions that emerge from the first phase when there is an emergency. The last step, the decision-making phase (validation and implementation of the proposed action plans), usually takes the form of an individual validation through mental simulation and a group validation through quick consultation with peers, which the authors have termed intuition guided by analysis. Finally, the proposed action plan is implemented based on intuitive judgments made in an emergency (Phase 2) or when stakeholders are satisfied with the outcome of the validation process.

Over the last several years, there has been a growing consensus among researchers on the definition of intuition. Various models describing the process have been proposed, and some studies have been able to demonstrate how intuition influences decisions in the real world. The study reported here was a step in this direction.

Our Study

The current study was exploratory. In-depth, in-person interviews were conducted with eight novice evaluators (less than 2 years of professional experience) and eight experienced evaluators (recognized by their peers for their contributions to the field, expertise, publications, etc.) in order to rule out the contribution of experience. Research has indicated that the quality of intuitive decision making depends on having sufficient knowledge and domain-specific expertise (Betsch, 2008; Dane et al., 2011; Dane & Pratt, 2009; Hodgkinson et al., 2008; Reber et al., 2007; Sadler-Smith & Sparrow, 2008; Salas et al., 2010).

Comparison-focused sampling and group characteristic sampling between the eight novice and eight experienced evaluators were selected for this study (Patton, 2015). Volunteers and snowball sampling techniques were used to constitute this sample. The interview guide was developed in order to explore all concepts in the theoretical framework of the topic. In addition, researchers paid special attention to all items to ensure that they were singular, unambiguous, and corresponded to commonly used terms. Interviews were digitally recorded, and the data were compiled and analyzed using NVivo software (version 2011), differentiating the novice and expert groups. Finally, the researchers relied on the approach suggested by Yin (2011) to analyze and interpret the results, that is, compiling, dissembling, reassembling, interpreting, and concluding. To restate, our findings related to expert intuition development and the process underlying its use in an evaluation.

Intuition Development

Two major key observations can be extracted from the data: First, intuition is developed by means of a long, complex, and demanding process; and second, intuition requires expertise and professional experience to develop. This last point was confirmed by the testimony of both novice and expert evaluators. Novice evaluators aimed to acquire professional credibility, which is based on the use of rigorous methodology. From their point of view, there is no room for improvisation, and they rejected and ignored their intuition. The experts, for their part, remembered how little intuitive knowledge they had at the beginning of their careers.

But how does intuition develop? From the various statements collected, we were able to infer that intuition is built in several stages. The evaluators interviewed recognized the need to enter the field of evaluation with some kind of basic training, whether formal or informal, to adequately contribute to the accomplishment of various mandates; this is the first stage. Their participation in different mandates contributes to the development of expertise and professional experience, which is the second stage. Reflective analysis of experiments, successes, and failures plays an essential role in the development of intuition. Indeed, this analysis, which is strongly influenced by training, experimentation, and expertise, allows evaluators to gradually develop a bank of numerous, diversified scenarios, which is the last stage. This process is illustrated in Figure 1.

Figure 1.

Illustration of the developmental process of intuition.

Process Underlying the Use of Intuition

The following interview excerpt describes how one experienced evaluator uses intuition; this quote illustrates the main components of the processes used in applying intuition to evaluative judgment:

I used my intuition again to better identify opportunities and avoid pitfalls in specific evaluation situations and to find more creative ways to make evaluation useful in diverse environments. I tend to see intuition as a nearly instantaneous process, virtually automatic and subconscious, that interprets an evaluative situation based on my extensive experience of other evaluation situations and my knowledge of the evaluation field…. Although intuition (by evaluators, evaluation team, stakeholders, funder) can be a major source of blind spots, intuition is also crucial in recognizing blind spots, gaps, inconsistencies, and problems that may undermine the evaluation at any point in its planning, implementation, and reporting…. Sensation, perception, and intuition are required continuously and, in typical situations, they make survival and adjustment to our environment automatic and reduce our need for treating each event as something new…. Even so, as much as possible, I test my intuitions by checking them against the views of others, as well as data analysis…. In my evaluation practice, I always check my personal expert intuition against the expert intuitions of the members of my evaluation team, the steering committee, and key stakeholders.

The participants have generated the process underlying the use of intuition, which occurs in various phases: (1) After analyzing an actual situation, evaluators access their bank of lived experiences and proceed to match the event with recorded patterns (similarities, differences). This analysis occurs rapidly, automatically, and unconsciously (identifying elements that evaluators have seen before, elements that are different, etc.), which can give the impression that the assessment comes out of nowhere. (2) This reflective analysis provides the opportunity to review this first assessment: Should we continue to explore this avenue? Does this assessment, which seems to have emerged out of nowhere, deserve our attention? Analysis is inspired by the evaluators’ training, expertise, and past experiences, and it helps them to decide whether they should attempt to validate the initial assessment by collecting additional information, debriefing peers, and so on. (3) All participants in the study claimed that at this point it is essential to validate their intuition. They were clear about the fact that intuition can guide them, but that “an intuition that is not confirmed remains a hypothesis.” This practice is most suitable in the specific context of program evaluation because the situation or event being evaluated is not an emergency, as is often the case for other professionals such as firefighters. In addition, and this is essential, the evaluators took care to avoid the bias that would be introduced if they collected only information that would confirm their intuition. They were aware that they must justify their judgment by collecting data, analysis, and argumentation. Failure to do so could result in the questioning of their professional credibility. (4) Finally, this process allowed them to arrive at the final judgment. This process is illustrated in Figure 2.

Figure 2.

Process underlying the use of intuition.

To conclude, participants stated that “we must have the intelligence to use our intuition with discernment.” One of the participants stated:

Although I trust my own judgment as well as the process, I also know that expert intuition, even carefully developed, fails in unfamiliar situations and that it can result in overconfidence…. That’s why I work hard to develop strategies to consider the context, discipline my intuition, and confirm it using various sources and data.

Discussion

Mukhopadhyay (2018) and Julmi (2019) referred to intuition as being both a process and an outcome. By combining elements, intuition produces new and creative solutions, and it can be considered a unique human ability (Akinci & Sadler-Smith, 2012; Gobet & Chassy, 2009; Sinclair, 2010). That being said, it would be somewhat misleading to ignore the fact that the quality of a judgment involving intuition can only be as effective as the global thinking process upon which it was built (Baylor, 2001; Dane & Pratt, 2009; Eraut, 2004; Klein, 2008). Intuition also involves specific skills that are only acquired through practice and hands-on experience. Aside from experience and expertise, it is important to mention memory, which establishes a connection between information and past experience, validation with peers or additional data, and undoubtedly, reflective practice. In this sense, the final judgment requires both System 1 and System 2 thinking. Intuition contributes actively if it is used properly, but, on its own, it is an insufficient means of arriving at a final decision or judgment. Furthermore, it should be noted that our findings are in line with those presented by Okoli, Watt, et al. (2016).

However, House (2016) acknowledged that intuition does not offer protection against bias since it is unconscious. He suggested using reflective practice to remedy this undeniable weakness, considering that it is a superquality in the evaluation field that can be extremely useful. He concluded that “as professional evaluators, we evaluate naturally and employ methods to minimize biases that distort findings. Another step forward would be to become more reflective of our biases, cognitive frames, and thinking processes” (p. 10). This observation was shared by the participants in the study, who reported that “we can really trust ourselves when we can admit that we don’t understand, we need help, we need to validate something.” The reflections collected through the interviews supported House’s (2015) view that intuition contributes significantly to program evaluation practice.

Does intuition work better in some situations than in others? Most studies have revealed that intuition is particularly powerful in emergencies, while recognizing how difficult it is to prove the efficacy of intuition. Program evaluation contexts are usually deliberative (or thought based) and allow for a validation phase, except in conflict situations. If this is the case, evaluators are then at a crossroads: Although several scenarios can be envisaged, one will necessarily be more effective than the others. Evaluators must think fast, and intuition could guide them to the best solution in the circumstances. As for schools of thoughts, Williams (2018) interviewed evaluators from different horizons, and almost all of them admitted to using intuition; it seems to be a generic skill.

Finally, the idea of combined systems of thinking meets Amann’s (2017) concept of praxis (practical action), which linked knowledge and skills (Technè) with practical wisdom (Phronesis) in order to generate “the right way to do the right thing” (House, 2015; Schwartz & Sharpe, 2010). This concept is close to Aristotle’s view of ethics, which he defined as finding the right way to do the right thing in a particular circumstance, with a particular person, at a particular time, and this is what is ultimately desired. The proper use of intuition appears to contribute to decision making that demonstrates practical wisdom. Schwartz and Sharpe (2010), for example, considered that one of the virtues of the wise is knowing

how to make emotion an ally of reason, to rely on emotion to signal what a situation calls for, and to inform judgment without distorting it. He can feel, intuit, or ‘just know’ what the right thing to do is, enabling him to act quickly when timing matters. His emotions and intuitions are well educated. (p. 24)

Intuition becomes more and more relevant considering that stakeholders arrive with different, if not conflicting, agendas. Without being able to establish a correlation between intuition, practical wisdom, and ethics, these considerations remain valid, and they all contribute to the production of credible judgment.

Conclusion

Still in its infancy, the topic of intuition warrants further research in order to fully understand the “pattern recognition based on experience and learning that is especially useful in complex situations that require instantaneous actions or behaviours” (Redekop, 2009, p. 400). The current study was exploratory and has opened the door to the subject, leaving us with more questions than answers and leading us to future research. We compared novice and experienced evaluators since the literature related expertise and experience to the development of intuition. Could a comparison be undertaken with a quantitative or qualitative orientation? Would the choice of paradigms (positivist/naturalist) prove to be an influencing factor? Is the use of intuition likely to create bias? Does intuition impact the validity of the results analysis? Does intuition work better in some situations than in others? Even though evaluators feel that intuition improves their judgment, is this actually the case? Indeed, participant statements were based only on impressions and not documented facts. Finally, it is important to emphasize that although the evaluators interviewed reported using their intuition in their professional practice, not all of them were comfortable with this approach. While they had agreed to participate initially, some of them declined to participate after reading the questionnaire. Was it because they do not use intuition or because they were uncomfortable admitting that they do use it? Was it because they never thought about intuition and it does not interest them? We did not push the investigation after receiving a refusal, but this led us to conclude that even if the contributions of intuition are recognized in several types of professional evaluation practices, it is not suitable for every evaluator.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Marthe Hurteau

References

Akinci

Sadler-Smith

(2012). Intuition in management research: A historical review. International Journal of Management Reviews, 14(1), 104–122.

Alós-Ferrer

Strack

(2014). From dual processes to multiple selves: Implications for economic behavior. Journal of Economic Psychology, 41, 1–11.

Amann

(2017, April 6). Phronesis 2.0 and its impact on an executive learning [Paper presentation]. Presentation given during the research in a workshop , University of Twente, Enschede.

Bago

De Neys

(2017). Fast logic? Examining the time course assumption of dual process theory. Cognition, 158, 90–109.

Bago

Frey

Vidal

Houdé

Borst

De Neys

(2018). Fast and slow thinking: Electrophysiological evidence for early conflict sensitivity. Neuropsychologia, 117, 483–490.

Banks

(2017). Comparing dual process theories: Evidence from event-related potentials. In De Neys

(Ed.), Dual process theory 2.0. Routledge.

Baylor

(2001). A U-shaped model for the development of intuition by level of expertise. New Ideas Psychology, 19(3), 237–244.

Betsch

(2008). The nature of intuition and its neglect in research on judgment and decision making. In Plessner

Betsch

(Eds.), Intuition in judgment and decision making (pp. 3–22). Erlbaum.

Chase

W. G.

Simon

(1973). Perception in chess. Cognitive Psychology, 4(1), 55–81.

10.

Cohen

M. S.

Freeman

J. T.

Wolf

(1996). Meta-recognition in time-stressed decision-making: Recognizing, critiquing, and correcting. Human Factors, 38(2), 206–219.

11.

Dane

Baer

Pratt

M. G.

Oldham

G. R.

(2011). Rational versus intuitive problem solving: How thinking “off the beaten path” can stimulate creativity. Psychology of Aesthetics, Creativity, and the Arts, 5(1), 3–12.

12.

Dane

Pratt

M. G.

(2009). Conceptualizing and measuring intuition: A review of recent trends. In Hodgkinson

G. P.

Ford

J. K.

(Eds.), International review of industrial and organizational psychology (pp. 1–40). John Wiley & Sons Ltd.

13.

De Neys

(2012). Bias and conflict a case for logical intuitions. Perspectives on Psychological Science, 7 (1), 28–38.

14.

Dörfler

Ackermann

(2012). Understanding intuition: The case for two forms of intuition. Management Learning, 43(5), 545–564.

15.

Epstein

(1994). Integration of the cognitive and the psychodynamic unconscious. American Psychologist, 49(8), 709–724.

16.

Eraut

(2004). Informal learning in the workplace. Studies in Continuing Education, 26(2), 247–273.

17.

Evans

J. St. B. T.

(1989). Bias in human reasoning: Causes and consequences. Erlbaum.

18.

Evans

J. St. B. T.

(2003). In two minds: Dual process accounts of reasoning. Trends in Cognitive Sciences, 7, 454–459.

19.

Evans

J. St. B. T.

(2009). How many dual-process theories do we need: One, two or many? In Evans

J. St. B. T.

Frankish

(Eds.), In two minds: Dual processes and beyond (pp. 31–54). Oxford University Press.

20.

Evans

J. St. B. T.

Stanovich

K. E.

(2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223–241.

21.

Frantz

(2003). Herbert Simon. Artificial intelligence as a framework for understanding intuition. Journal of Economic Psychology, 24(2), 265–277.

22.

Gigerenzer

(2008). Gut feelings: Short cuts to better decision making. Penguin.

23.

Gobet

Chassy

(2009). Expertise and intuition: A tale of three theories. Minds and Machines, 19(2), 151–180.

24.

Gore

Conway

G. E.

(2016). Modeling and aiding intuition in organizational decision-making: A call for bridging academia and practice. Journal of Applied Research in Memory and Cognition, 5(3), 331–334.

25.

Grinnell

(2009). Everyday practice of science: Where intuition and passion meet objectivity and logic. University Press.

26.

Handley

S. J.

Trippas

(2015). Chapter two-dual processes and the interplay between knowledge and structure: A new parallel processing model. Psychology of Learning and Motivation, 62, 33–58.

27.

Healey

M. P.

Vuori

Hodgkinson

G. P.

(2015). When teams agree while disagreeing: Reflexion and reflection in shared cognition. Academy of Management Review, 40(3), 399–422.

28.

Hodgkinson

G. P.

Langan-Fox

Sadler-Smith

(2008). Intuition: A fundamental bridging construct in the behavioural sciences. British Journal of Psychology, 99, 1–27.

29.

Hodgkinson

G. P.

Sadler-Smith

Burke

L. A.

Claxton

Sparrow

P. R.

(2009). Intuition in organizations: Implications for strategic management. Long Range Planning, 42(3), 277–297.

30.

Hoffrage

Marewski

J. N.

(2015). Unveiling the lady in black: Modelling and aiding intuition. Journal of Applied Research in Memory and Cognition, 4(3), 145–163.

31.

House

E. R.

(1977). The logic of evaluative argument, Monograph no. 7 (pp. 67–96). Center for the Study of Evaluation, UCLA.

32.

House

E. R.

(2015). Evaluating: Values, biases, and practical wisdom. Information Age Publishing.

33.

House

E. R.

(2016, November 15). Reducing biases in evaluation [Paper presentation]. Presentation Given to Evaluators, Albuquerque, NM.

34.

Johnson

E. D.

Tubau

De Neys

(2016). The Doubting Systems 1: Evidence for automatic substitution sensitivity. Acta Psychologica, 164, 56–64.

35.

Julmi

(2019, March 9). When rational decision-making becomes irrational: A critical assessment and re-conceptualization of intuition effectiveness. Business Research. Advance online publication. https://doi.org/10.1007/s40685-019-0096-4

36.

Kahneman

(2000). A psychological point of view: Violations of rational rules as a diagnostic of mental processes. Behavioral and Brain Sciences, 23, 681–683.

37.

Kahneman

(2011). Thinking, fast and slow. Toronto: Anchor Canada a division of Random House of Canada Limited.

38.

Kahneman

Frederick

(2002). Representativeness revisited: Attribute substitution in intuitive judgment. In Gilovich

Griffin

Kahneman

(Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 49–81). Cambridge University Press.

39.

Kermarrec

Bossard

(2014). Defensive soccer players decision-making: A naturalistic study. Journal of Cognitive Engineering and Decision Making, 8(2), 187–199.

40.

King

Clark

J. M.

(2002). Intuition and the development of expertise in surgical ward and intensive care nurses. Journal of Advanced Nursing, 37(4), 322–329.

41.

Klein

(2003). The power of intuition: How to use your gut feelings to make better decisions at work. Doubleday.

42.

Klein

(2008). Naturalistic decision-making. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 456–460.

43.

Klein

(2015). Reflections on applications of naturalistic decision-making. Journal of Occupational and Organizational Psychology, 88(2), 382–386.

44.

Klein

Calderwood

Clinton-Cirocco

(1986). Rapid decision-making on the foreground. Human Factors and Ergonomics Society, 576–580.

45.

Klein

Calderwood

Clinton-Cirocco

(2010). Rapid decision-making on the fire ground: The original study plus a postscript. Journal of Cognitive Engineering and Decision Making, 4(3), 186–209.

46.

Knemeyer

A. M.

Naylor

R. W.

(2011). Using behavioral experiments to expand our horizons and deepen our understanding of logistics and supply chain decision-making. Journal of Business Logistics, 32(4), 296–302.

47.

Kuo

W.-J.

Sjöström

Chen

Y.-P.

Wang

Y.-H.

Huang

C.-Y.

(2009). Intuition and deliberation: Two systems for strategizing in the brain. Science, 324(5926), 519–522.

48.

Lieberman

M. D.

(2007). Social cognitive neuroscience: A review of core processes. Annual Review of Psychology, 58, 259–289.

49.

Mattsson

Fast-Berglund

(2016). How to support intuition in complex assembly? Procedia CIRP, 50, 624–628.

50.

Mukhopadhyay

(2018). Mechanism of intuition. Current Opinions in Neurological Science, 2(1), 356–361.

51.

Nakamura

Kawaguchi

(2016). People like logical truth: Testing the intuitive detection of logical value in basic propositions. PLos One, 11(12), e0169166.

52.

Okoli

Watt

(2018). Crisis decision-making: The overlap between intuitive and analytical strategies. Management Decision, 56(5), 1122–1134

53.

Okoli

Watt

Weller

Wong

W. B.

(2016). The role of expertise in dynamic risk assessment: A reflection of the problem-solving strategies used by experienced foreground commanders. Risk Management, 18(1), 4–25.

54.

Okoli

Weller

Watt

(2014). Eliciting experts’ knowledge in emergency response organizations. International Journal of Emergency Services, 3(2), 118–130.

55.

Okoli

Weller

Watt

(2016). Information processing and intuitive decision-making on the foreground: Towards a model of expert intuition. Cognition, Technology & Work, 18(1), 89–103.

56.

Patton

M. Q.

(2015). Qualitative research & evaluation methods (4th ed.). Sage.

57.

Pennycook

Fugelsang

J. A.

Koehler

D. J.

(2015). What makes us think? A three-stage dual-process model of analytic engagement. Cognitive Psychology, 80, 34–72.

58.

Pennycook

Trippas

Handley

S. J.

Thompson

V. A.

(2014). Base-rates: Both neglected and intuitive. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(2), 544–554.

59.

Rahmanian

Hurteau

Houle

Marchand

M-P

. (2016). The role of intuition in evaluative practice. Postdoctoral project in program evaluation. UQAM.

60.

Reber

Ruch-Monachon

M.-A.

Perrig

W. J.

(2007). Decomposing intuitive components in a conceptual problem solving task. Consciousness and Cognition, 16(2), 294–309.

61.

Redekop

B. W.

(2009). Common sense in philosophical and scientific perspective. Management Decision, 47(3), 399–412.

62.

Risen

J. L.

(2016). Believing what we don’t believe: Acquiescing to superstitious beliefs and other powerful intuitions. Psychological Review, 123, 183–207.

63.

Rosciano

Lindell

Bryer

DiMarco

(2016). Nurse practitioners’ use of intuition. The Journal for Nurse Practitioners, 12(8), 560–565.

64.

Rosen

M. A.

Shuffler

Salas

(2010). How experts make decisions: Beyond the JDM paradigm. Industrial and Organizational Psychology, 3(4), 438–442.

65.

Sadler-Smith

Sparrow

W. H.

(2008). Intuition in organizational decision making. In Hodgkinson

G. P.

Starbuck

W. H.

(Eds.), The Oxford handbook of organizational decision making (pp. 305–324). Oxford University Press.

66.

Salas

Rosen

M. A.

Granados

D. D.

(2010). Expertise-based intuition and decision making in organizations. Journal of Management, 36(4), 941–973.

67.

Schwartz

Sharpe

(2010). Practical wisdom. The right way to do the right thing. Penguin Group.

68.

Sinclair

(2010). Misconceptions about intuition. Psychological Inquiry, 21(4), 378–386.

69.

Sinclair

Ashkanasy

N. M.

(2005). Intuition: Myth or a decision-making tool? Managing Learning, 36(3), 353–370.

70.

Sloman

S. A.

(1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119(1), 3–22.

71.

Smith

E. R.

De Coster

J. M.

(2000). Dual-process models in social and cognitive psychology: Conceptual integration and links to underlying memory systems. Personality and Social Psychology Review, 4(2), 108–131.

72.

Stake

R. E.

(1991). Excerpts from: “Program evaluation, particularly responsive evaluation. Evaluation Practice, 12(1), 63–76.

73.

Stake

R. E.

(2004). Standards-based and responsive evaluation. Sage Publications.

74.

Stake

R. E.

Schwandt

T. A.

(2006). On discerning quality in evaluation. In Shaw

I. F.

Greene

J. C.

Mark

M. M.

(Eds.), The Sage handbook of evaluation (pp. 404–418). Sage Publications.

75.

Stanovich

(2011). Rationality and the reflective mind. Oxford.

76.

Thompson

V. A.

Johnson

S. C.

(2014). Conflict, metacognition, and analytic thinking. Dual Process Theories, 20(2), 215–244.

77.

Tissington

P. A.

Flin

(2005). Assessing risk in dynamic situations: Lessons from fire service operations. Risk Management, 7(4), 42–51.

78.

Trippas

Handley

S. J.

(2017). The parallel processing model of belief bias: Review and extensions. In De Neys

(Ed.), Dual process theory 2.0 (pp. 28–46). Routledge.

79.

Trippas

Handley

S. J.

Verde

M. F.

Morsanyi

(2016). Logic brightens my day: Evidence for implicit sensitivity to logical validity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42, 1448–1457.

80.

Trippas

Thompson

V. A.

Handley

S. J.

(2017). When fast logic meets slow belief: Evidence for a parallel-processing model of belief bias. Memory & Cognition, 45 (4), 539–552

81.

Tversky

Kahneman

(1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.

82.

Vanlommel

Van Gasse

Vanhoof

Van Petegem

(2017). Teachers’ decision-making: Data based or intuition driven? International Journal of Educational Research, 83, 75–83.

83.

Vaughan

F. E.

(1979). Awakening intuition. Anchor Press/Doubleday.

84.

Wang

Highhouse

Lake

C. J.

Petersen

N. L.

Rada

T. B.

(2017). Meta-analytic investigations of the relation between intuition and analysis. Journal of Behavioral Decision Making, 30(1), 15–25.

85.

Williams

D. D.

(2016). Seven North American evaluation pioneers. New Directions for Evaluation, 150, 112.

86.

Williams

D. D.

(2018). Twenty-nine evaluation lives. New Directions for Evaluation, 2018(157 ), 101–104.

87.

Yin

R. K.

(2011). Qualitative research from start to finish. Guilford Press.

88.

Zsambok

C. E.

(1997). Naturalistic decision-making: Where are we now? In Klein

G. A

Zsambok

C. E.

(Eds.), Naturalistic decision making (pp. 3–16). Erlbaum.