Toward Simple Representative Mathematical Models of Naturalistic Decision Making Through Fast-and-Frugal Heuristics

Abstract

Naturalistic decision making (NDM) describes how people make decisions with time pressure and other complexities in familiar and meaningful environments. The complexities of NDM, although essential for a thorough understanding of experts, have limited the use of NDM in decision support system (DSS) design and decision analysis. There have been attempts to computationally model NDM theories; however, to do so has required significant programming skill, resulting in opaque models while still leaving significant portions of the theories unrepresented. To provide a path toward simple, representative, mathematical models of NDM, we present a general mathematical form that can model most of the components of the judgment and decision-making strategies from the fast-and-frugal heuristics (FFH) program. As a case study, the NDM quick test process was transformed into representative FFH components and then into mathematical form. The results show that the quick test process is similar to the FFH strategy tallying, which itself is a type of fast-and-frugal tree. Although the mathematical form does not capture all the cognitive processes at work, it does provide a simple, representative form for use in DSS design and decision analysis. Moreover, the model’s basis in the extensive human-subjects, mathematical, and computational analyses completed by the FFH program provides another method for integrating FFH and NDM by providing (a) a framework for generating constructive questions about how NDM theories account for FFH components; (b) a basis for prescriptive NDM decision support tools that are easy to communicate, understand, and apply; and (c) a method for approximating experience, expertise, and time pressure.

Keywords

naturalistic decision making fast-and-frugal heuristics computational models mathematical models judgment decision making decision support

Introduction

Naturalistic decision making (NDM) focuses on understanding “how people make decisions in real-world contexts that are meaningful and familiar to them” (Lipshitz, Klein, Orasanu, & Salas, 2001, p. 332) and has provided new perspectives on how people actually make decisions in real-world settings. NDM expanded upon the classical decision-making process, which focused solely on the decision event, by including perception and recognition of situations and generation of appropriate responses, not just a choice from among given options (Klein, 2008; Klein & Calderwood, 1996; Klein, Calderwood, & Clinton-Cirocco, 1988; Lipshitz et al., 2001; Orasanu & Connolly, 1993). In a review of the contributions of NDM, Klein (2008) provided evidence that NDM spurred the development of cognitive field research and cognitive task analysis methods, guided methods for training decision making and related cognitive skills, and affected U.S. Army doctrine by the inclusion of intuitive and rapid decision making in the Army Field Manual (U.S. Department of the Army, 2014, chap. 9), among others.

Given this evidence, engineers, designers, and decision analysts interested in accurately modeling, simulating, or supporting real-world decision making should prefer NDM as a representative model of judgment and decision making rather than classical decision making. However, the complexities of algorithmically or computationally representing NDM have restricted its use in simulation and experimentation. Classical decision making with its probabilistic, weighted, linear-additive model is just simpler and easier to use despite its difficulty accounting for environmental and psychological issues (Katsikopoulos & Fasolo, 2006). For example, transforming NDM models, such as recognition-primed decision making (RPDM), into computational programs can require significant computational programming skill while still not implementing all aspects of the original theory (e.g., agent-based models; Fan, Sun, McNeese, & Yen, 2005; Nowroozi, Shiri, Aslanian, & Lucas, 2012; fuzzy logic; Ji et al., 2007; episodic recognition memory; Mueller, 2009; or system dynamics; Patterson, Fournier, Pierce, Winterbottom, & Tripp, 2009). Furthermore, the complexity and lack of transparency of the programs may limit their use in practice (e.g., Katsikopoulos, Pachur, Machery, & Wallin, 2008).

One of the most successful methods for developing decision support tools that are easy to communicate, understand, and apply has been the fast-and-frugal heuristics (FFH) program (Gigerenzer & Gaissmaier, 2011; Katsikopoulos et al., 2008). Similar to NDM, the FFH program has shown that people use the bounds on rationality of simplicity, speed, and frugality as a mechanism for simple, robust, and accurate strategies that adapt to the environment and ecology (e.g., ; Gigerenzer & Gaissmaier, 2011; Gigerenzer & Goldstein, 1996; Gigerenzer, Todd, & ABC Research Group, 1999; Todd, Gigerenzer, & ABC Research Group, 2012). The literature reviews by Katsikopoulos et al. (2008) and Gigerenzer and Gaissmaier (2011) show that professional decision making in the aviation, business, legal, and medical domains tend to match the predictions of the fast-and-frugal heuristics. Therefore, rather than using complex programs to model and support decision making, the FFH program has seen significant success with modeling, prescribing, and supporting the simple rules that decision makers already have in their “intuitive repertoire” (Katsikopoulos et al., 2008, p. 457).

The FFH program has also been explicitly called upon as having a “synergetic potential” for NDM (Keller, Cokely, Katsikopoulos, & Wegwarth, 2010, p. 261; see also Todd & Gigerenzer, 2001). One of the main critiques of NDM research is the perceived lack of scientific rigor caused by the lack of formalized and precise models, which has limited the potential for controlled human-subject experimentation and extensive testing and comparison of NDM theories (Keller et al., 2010; Lipshitz et al., 2001). The FFH program has attempted to avoid the issues caused by vague theories by prioritizing two tenants: the decomposition of heuristics into smaller “building blocks” (i.e., information search methods, discrimination rules, stopping criteria) and the definition of heuristics as precise, computational models (e.g., Gigerenzer & Gaissmaier, 2011; Katsikopoulos, 2011). This focus on decomposing and computationally defining heuristics has enabled the FFH program to perform extensive simulation testing and analysis of heuristics through both computational and human-subjects studies—often combining the two methods in one study (e.g., Katsikopoulos, Schooler, & Hertwig, 2010; Rieskamp and Hoffrage, 2008). The FFH studies have enabled researchers to answer both the descriptive question, “Which heuristics do people use in which situations?” and the prescriptive question, “When should people rely on a given heuristic rather than a complex strategy to make better judgments?” (Gigerenzer & Gaissmaier, 2011). By linking NDM theories to the FFH program, there is a potential to specify and test NDM models with greater precision so that these descriptive and prescriptive questions can begin to be answered for NDM theories as well.

To increase the prevalence of NDM theories among the engineering, design, and decision analysis communities and to help bridge the gap between NDM and FFH theories, this paper presents a method to transform components of NDM theories into simple mathematical models based on the FFH framework that are sufficiently representative without being perfectly descriptive. To explain further, the resultant mathematical model constructed in this paper for take-the-best (a well-studied FFH strategy) does not exactly match the descriptive process of the strategy presented in Gigerenzer and Goldstein (1996) but does result in the same decisions. Because the model is decision equivalent to previous implementations of take-the-best (e.g., Hogarth & Karelaia, 2005a; Katsikopoulos et al., 2010), no new studies are necessary to link the mathematical model to human-subjects studies in which take-the-best was shown predict human decisions (e.g., Rieskamp & Hoffrage, 2008). The mathematical model simply provides a new way of understanding and representing take-the-best.

This paper begins by introducing a new general mathematical form of judgment and decision making that can model most of the components and strategies within the well-studied FFH framework. Components are the building blocks of strategies that define, for example, how a strategy orders cues, estimates missing information, or estimates the utility of a cue value. Combining different variations of these components can represent most strategies within the FFH program. Most importantly, the mathematical form can be used to simulate the effect of experience, time pressure, and the fit of the strategy to the environment (i.e., ecological rationality; Todd et al., 2012). It is these shared contexts of expertise, time pressure, incomplete information, and ecological rationality that make this mathematical form of FFH strategies able to provide a path toward computational experimentation of NDM theories (Keller et al., 2010).

Following the description of the mathematical framework, we present an example of transforming a portion of an NDM theory into the framework, using the quick test process within the recognition/metarecognition (R/M) theory (Cohen, Freeman, & Wolf, 1996). The results show that the process of transforming an NDM theory through the lens of FFH components is beneficial itself in addition to the resulting mathematical model. The process shows that there are some components that the original quick test process does not define, thus suggesting new observations or experiments that can refine the understanding of the quick test process. The resulting mathematical model shows that the quick test process is very similar to the well-studied heuristic tallying, thus linking the quick test process to previously unrelated human-subject studies, computational experiments, and mathematical analyses of the tallying heuristic (for a review, see Katsikopoulos, 2011). Furthermore, since tallying has been shown to make identical decisions (or categorizations) as specific types of fast-and-frugal trees (Martignon, Katsikopoulos, & Woike, 2008), there is even further literature that can now be linked to the quick test process (Katsikopoulos et al., 2008; Luan, Schooler, & Gigerenzer, 2011).

Last, we explain how experience, expertise, and time pressure can be approximated using the mathematical models. Whereas NDM has extensively described the mechanisms of expert decision making and the causes of time pressure, the FFH program has focused on modeling and simulating expertise and time pressure to determine the effects that expertise or time pressure may, or may not, actually have on judgment and decision-making performance.

A General Mathematical Form of Fast-and-Frugal Judgment and Decision Making

To provide a potential avenue for the development of simple representations of NDM strategies, this paper presents a simple mathematical model that can represent almost any combination of many established FFH components. Multiple contributions to the FFH program have presented mathematical interpretations of specific strategies (e.g., Gigerenzer & Goldstein, 1996) or mathematical analyses (e.g., Hogarth & Karelaia, 2005b; Katsikopoulos, 2013). However, none have presented a single mathematical model of FFH judgment and decision making from which many strategies could be derived. The benefit of this general form is that NDM models can be converted to FFH components, each of which will have a corresponding mathematical representation. This section concludes with the conversion of the exemplar FFH decision-making strategy take-the-best into a mathematical model.

General Mathematical Form

There are five parts of the general mathematical form: task type, utility functions, incomplete information, estimates of missing values, and cue weights. The task type is the most fundamental and indicates whether the operator’s task is to make a judgment or a decision. Operators are situated within an environment which is represented by an m-by-n matrix, A^v, of alternatives and cues, respectively. Each entry of the matrix is a cue value ( $a_{i, j}^{v}$ ) representing the state of alternative i ∈{1, . . . , m} with respect to cue j ∈{1, . . . , n} in the cue’s natural or original sensed units. The goal of judgment is to evaluate information about a single alternative (i = 1) characterized by n cues to categorize the alternative using the criterion C. The goal of decision making is to evaluate information about multiple alternatives (i ≥ 2) characterized by n cues to select the alternative that maximizes the criterion C.

After the environment’s cue values are input to the operator, the cue values are converted to cue scores ( $a_{i, j}^{s}$ , which have units useful for the formal judgment and decision-making process) through utility functions ( $U_{j} : a_{i, j}^{v} \to a_{i, j}^{s}$ ). Additionally, individual cue values can be known (z_i_,j = 1) or unknown (z_i_,j = 0) such that the operator must estimate the missing cue values (e_j). After all the cue values have been converted or estimated, the operator combines them using cue weights (w_j) to determine the alternative’s criterion value, C.

The general mathematical form of judgment and decision making is

C_{i} = \sum_{j = 1}^{n} w_{j} \cdot [e_{j} + (U_{j} (a_{i, j}^{v}) - e_{j}) z_{i, j}] .

The general form in Equation 1 can model almost any combination of the FFH components for both judgment and decision making—cue weights, estimates of missing information, and utility functions—for m-alternatives, n-cues, and any distribution of incomplete information. The subscripts of the variables in Equation 1 are particularly important. The cue weights (w_j), estimates of missing information (e_j), and utility functions (U_j) are defined with respect to cues only, whereas there is a unique information state (z_i_,j) for each cue value ( $a_{i, j}^{v}$ ). The following subsections provide more background about the types of cue weights, estimates, and utility functions, described in Table 1.

Table 1:

Mathematical Components of Fast-and-Frugal Heuristics

Type	Mathematical Representation
Cue weights: The relative importance of cues (w_j)
Magnitude, ordered	v_j
Ordered, non-compensatory	$w_{j} > \sum_{k > j} w_{k}$
None or equal	1
Estimates: Estimating missing information (e_j)
Positive	$\max a_{j}^{s}$
Average	${\bar{a}}_{j}^{s}$
Median	${\tilde{a}}_{j}^{s}$
Negative	$\min a_{j}^{s}$
Utility function: Maps the state of the option in its natural units to a general utility in a range of [0,1] (U_j)
Exact	$a_{i, j}^{v}$
Binary, cutoff	$u_{a_{i, j}^{v}} (c_{j})$

Cue Weights

The cue weights within a task measure the relative importance of each cue for predicting the criterion. There are three aspects of cue weights (magnitude, order, and compensatoriness), and different judgment and decision-making strategies use and ignore different aspects. Magnitude is the numerical value of the importance of the cue (v_j), typically on a ratio scale of 0 to 1, although other scales are possible. There are many methods for getting the magnitude of cue weights, from eliciting weights from human subjects (e.g., Edwards & Fasolo, 2001) to calculating weights from information about the cue values and criterion scores (e.g., regression weights, ecological validity, success validity, or conditional validity, as described in Martignon & Hoffrage, 2002). Ordered cues are ranked and evaluated from most to least important. Whereas order can be determined from magnitude, some strategies are concerned only with order, not magnitude. Compensatoriness is the measure of whether a high score on a lower-ranked cue can compensate for a low score on a higher-ranked cue (Martignon & Hoffrage, 2002). If cues can compensate, the weights are defined as compensatory, whereas if cues cannot compensate, the weights are defined as noncompensatory.

Estimates of Missing Information

Garcia-Retamero and Rieskamp (2008, 2009) suggest that there are four ways that individuals estimate missing information: positive ( $\max a_{j}^{s}$ ), negative ( $\min a_{j}^{s}$ ), using the average of past observations for replacement ( ${\bar{a}}_{j}^{s}$ ), or using the most frequent observation of the available information as a placeholder ( ${\tilde{a}}_{j}^{s}$ ). The general form for the estimation process is provided by the cue state function, F_i_,j, described in Equation 2: When the cue value is unknown (z_i_,j = 0), the cue state is estimated as e_j as shown in Equation 3, and when the cue value is known (z_i_,j = 1), the cue state is the cue score determined by the utility function (U_j) as shown in Equation 4.

Cue State Function: F_{i, j} = e_{j} + (U_{j} (a_{i, j}^{v}) - e_{j}) z_{i, j} .

F_{i, j} (z_{i, j} = 0) = e_{j} .

F_{i, j} (z_{i, j} = 1) = U_{j} (a_{i, j}^{v}) .

The label of F_i_,j as the cue state function is an analogy to state functions that do not depend on the path by which a system arrived at its present state. Therefore, the cue state function only defines the state of the cue in an information-processing context, without reference to original cue value or whether the cue value is known or unknown. For example, a cue state can be 0 for two separate reasons: The cue value could be unknown and estimated as negative (0 or binary cues), or the cue value is known but the cue score is 0. Using the state function, an alternative to the general mathematical form in Equation 1 is

C_{i} = \sum_{j = 1}^{n} w_{j} \cdot F_{i, j} .

Utility Function

Though the general form in Equation 1 allows any utility function to be used, commonly studied strategies typically have one of two forms of the utility function: exact or binary cutoff. Exact utility functions use cue values as the cue scores because the cue values were already in units useful to the operator (Equation 6). Binary cutoff utility functions transform the cue values into binary cue scores (e.g., yes or no, positive or negative) based on a cutoff value, c_j (Equation 7). Binary cue scores are commonly used within FFH framework and heuristics have been shown to perform particularly well in binary cue environments (Hogarth & Karelaia, 2005a, 2005b; Katsikopoulos, 2013). Specifically, the binary utility function uses theunit step function, $u_{a_{i, j}^{v}} (c_{j})$ , such that if the value of the subscript (the cue value, $a_{i, j}^{v}$ ) is greater than, or equal to, the argument (the cutoff value, c_j) the result is 1, but if the value of the subscript is less than the argument, the result is 0. The cutoff value should be set by an analysis of the operator within context (this is particularly relevant given the NDM perspective); however, there are methods for analytically defining the cutoff value from cue value data (Slegers, Brake, & Doherty, 2000): the median-split model, which uses the median of the cue values ( ${\tilde{a}}_{j}^{v}$ ), and the δ model, which calculates the cutoff value by algorithmically determining when a cue discriminates.

Exact: U_{j} : a_{i, j}^{s} = a_{i, j}^{v} .

Binary: U_{j} : a_{i, j}^{s} = u_{a_{i, j}^{v}} (c_{j}) .

A Mathematical Model of Take-the-Best

One of the most studied FFH strategies is the take-the-best heuristic. In the exemplar case, take-the-best selects between two alternatives described by binary cue scores using three simple building blocks: (1) search through cues in order of importance, (2) stop the search when one cue discriminates, then (3) select the alternative that discriminates (Gigerenzer & Gaissmaier, 2011; Gigerenzer & Goldstein, 1996). A cue discriminates between two alternatives only if one alternative has a positive cue score (1) and the other alternative has a negative or unknown cue score (0).

The take-the-best strategy is the exemplar for one-reason heuristic decision making because it stops the search when one alternative discriminates and, therefore, no lower ranked cues can compensate and override the decision. This indicates that take-the-best uses ordered, noncompensatory weights (wj = 4^1–j). For the mathematical model of take-the-best in Equation 8, the utility function was set to binary (by definition, $U_{j} = u_{a_{i, j}^{v}} (c_{j})$ ) and uses the median-split method for the cutoff value ( $c_{j} = {\tilde{a}}_{j}^{v}$ ) because that method is commonly used when generating data sets or environments in which take-the-best is examined (e.g., Czerlinski, Gigerenzer, & Goldstein, 1999; Gigerenzer & Goldstein, 1996; Hogarth & Karelaia, 2006; Katsikopoulos, 2013; Katsikopoulos et al., 2010; Martignon & Hoffrage, 2002). The estimate of missing information was defined as negative (e_j = 0) because negative and unknown cue scores are equivalent with respect to the discrimination rule. The component mapping of take-the-best is shown in Table 2.

C_{i} = \sum_{j = 1}^{n} 4^{1 - j} \cdot u_{a_{i, j}^{v}} ({\tilde{a}}_{j}^{v}) \cdot z_{i, j} .

Table 2:

Mapping Take-the-Best, Tallying, and Quick Test Process to Components

Component	Take-the-Best	Tallying	Quick Test Process
Task type	Decision making	Judgment^a	Judgment
Cue weights ( $w_{j}$ )	Ordered, noncompensatory	Equal, none: 1	Equal, none: 1
Estimate of missing information ( $e_{j}$ )	Negative: 0	Negative: 0	Negative^b: 0
Utility function ( $U_{j}$ )	Binary: $u_{a_{i, j}^{v}} (c_{j})$	Binary: $u_{a_{i, j}^{v}} (c_{j})$	Binary: $u_{a_{i, j}^{v}} (c_{j})$
Cutoff value ( $c_{j}$ )	Median-split model: ${\tilde{a}}_{j}^{v}$ ${\tilde{a}}_{j}^{v}$	Median-split model: ${\tilde{a}}_{j}^{v}$	Unspecified^b: $c_{j}$

Tallying has been referred to as both a judgment (Gigerenzer & Gaissmaier, 2011) and decision-making strategy (Payne, Johnson, Bettman, & Coupey, 1990).

Unspecified in the model description.

The alternative i with the highest criterion C is the alternative that take-the-best would have selected by following its three simple building blocks. Additionally, although take-the-best was founded as a model of how people decide between two alternatives, note that Equation 8 can represent comparisons between m-alternatives (referred to as deterministic elimination by aspects [DEBA]; Hogarth and Karelaia, 2005a; Tversky, 1972).

Although take-the-best contains three simple building blocks, human-subjects studies have shown that the cue-wise search and one-reason decision rule embodied by take-the-best (and mathematically captured in Equation 8) match the observable information search process and decision outcomes of experts (e.g., Garcia-Retamero & Dhami, 2009; Gigerenzer & Gaissmaier, 2011), consumers (e.g., Kohli & Jedidi, 2007), and decision making with time pressure (e.g., Rieskamp & Hoffrage, 2008). From a prescriptive perspective, take-the-best has been shown to (a) predict more accurately than multiple linear regression models in various environments (Czerlinski et al., 1999), (b) perform particularly well in environments with high cue redundancy and high variability in cue weights (Gigerenzer & Gaissmaier, 2011), and (c) predict accurately even when there are very small training sets, that is, operator has very little experience with the situation, resulting in errors in the cue weights (Katsikopoulos et al., 2010). For a more complete review of take-the-best results, see Katsikopoulos (2011).

Building a Mathematical Model of the Quick Test Process

The R/M theory is an empirically based, two-tiered NDM model in which pattern matching and other methods are used to recognize a situation or aspects of a situation and then an optional process is used to critique and correct the recognition and its implications (Cohen et al., 1996). R/M attempts to account for the processes that decision makers use to understand and plan within novel and uncertain situations. The quick test process within R/M models the metarecognitional ability to determine when critiquing and correcting are worth performing and when the current plan is good enough (see Figure 1). Keller et al. (2010) originally suggested that the quick test process within R/M could be clarified using the FFH methodology. Since this clarification has yet to be completed, this section will show how the quick test process can be transformed into FFH components and a mathematical model. The resulting model shows that the quick test process is a form of the tallying heuristic, which itself is a form of fast-and-frugal trees as shown in Figure 1 (a strict pectinate, to be exact; Martignon et al., 2008). Therefore, by viewing the quick test process through the lens of FFH, it can be linked to a larger literature of theoretical, computational, and human-subjects studies.

Figure 1.

Summarized version of quick test process within the recognition/metarecognition model (adapted from Cohen, Freeman, & Wolf, 1996, Figure 3) and a fast-and-frugal tree representation.

Components of the Quick Test Process

As shown in Figure 1, the quick test process considers the following questions: (a) Do I have some time before I must commit to a decision? (b) Are the stakes of an error high? and (c) Is the situation unfamiliar, atypical? If the answer to all three questions is yes, then a process of further critical thinking begins to critique and correct the current plan. If the answer to any of the questions is no, then immediate action is taken based on the current plan.

Table 2 shows how the quick test process can be transformed to FFH components. At the most basic level, FFH can model judgment or decision making. Since the quick test process evaluates a single instance of the state of the world (a single alternative), it is a judgment model. If the quick test process were comparing different situations or courses of action in order to select one, it would be a decision-making model. The three questions can be represented as cues (time, stakes, and novelty) that are used to make a judgment about the criterion (should the critical thinking process begin?). The weight, or importance, of each cue is set equal because the order of the questions does not matter and each question is equally important. Since the questions require yes-or-no answers, it can be assumed that a cutoff value is being used to convert the cue values (e.g., seconds for the time cue) into a binary cue scores (yes or no).

Given the results in Table 2, it is clear that the quick test process is a form of the tallying heuristic, which is well studied within the FFH framework. Tallying ignores the relative importance of cues, does not order cues, counts the number of yes cues, and uses a threshold to make a judgment (Gigerenzer & Gaissmaier, 2011). Tallying has been shown to be a useful representation of tasks in which the information to be integrated is limited (Payne, Bettman, & Johnson, 1993) or “from diverse and incomparable sources” (Dawes, 1979, p. 574). As a prescriptive strategy, tallying has been shown to be useful for tasks ranging from eye exams for detecting stroke (Kattah, Talkad, Wang, Hsieh, & Newman-Toker, 2009) to avoiding avalanche accidents (McCammon & Hägeli, 2007).

It should also be noted that as shown in Figure 1, the quick test process can also be represented as a fast-and-frugal tree, of which tallying is a specific form. Fast-and-frugal tress are categorization trees in which there is one exit at each level of the tree (Martignon, Vitouch, Takezawa, & Forster, 2003). In terms of performance, fast-and-frugal trees have been shown to perform as well as advanced categorization methods (e.g., logistic regression) when there are low levels of prior information (Martignon et al., 2008). In prescription, fast-and-frugal trees have found increasing applications (e.g., medicine; Fischer et al., 2002; Green & Mehr, 1997) because they are easier to communicate and understand than more complex actuarial methods (Katsikopoulos et al., 2008).

Although all these transformations from the quick test process to FFH components were fairly straightforward, there are still important FFH components remaining undefined. The first need is a description of the utility function and cutoff value that maps cue values (time, stakes, and novelty) to cue scores (yes-or-no answers). For example, how much time must be available for there to be “some” time before having to commit to a decision? The second need is a description of how missing information about each of the cues would be estimated. For example, if the judge is unsure of how much time is available, does the judge estimate that there is enough available time or not enough available time? Generally, individuals treat missing information as negative when it is assumed that information was withheld or unavailable because it would lead to a negative evaluation otherwise, whereas individuals treat information as positive when positive information is more prevalent (Garcia-Retamero & Rieskamp, 2008, 2009).

In summary, there are two main benefits of transforming the NDM models, like the quick test process, into FFH components. First, the FFH perspective can suggest new observations or experiments that can more fully refine the NDM models (e.g., the utility function, cutoff value, and estimates of missing information). Neither of the two observational studies of battlefield commanders that provided the foundation for the R/M model specifically discussed what values of time, stakes, or novelty constituted a yes or no to the quick test process questions or how missing information about those values was estimated (Cohen et al., 1995; Cohen, Adelman, Tolcott, Bresnick, & Marvin, 1993). Second, almost all components of FFH models and the related strategies have been examined theoretically and through human-subjects and computational experiments.

Mathematical Form of the Quick Test Process

The mathematical representation of the quick test process is presented in Equation 9 with a flow chart representation of the math in Figure 2. The mathematical form is generated by inputting the definitions of the quick test process components from Table 2 into the general form in Equation 1. The judgment process described in Figure 2 estimates missing components where none were specified in the original model development: The estimates of missing information for each cue was defined as no (or negative), and the cutoff values for each cue’s utility function were 30 s for time, 4 for stakes, and 4 for novelty (assuming that the judges are asked to rate stakes and novelty on a 5-point scale). The incomplete information and cue values should be defined based on the environment being simulated. Because this is an abstract example, the values in Figure 2 were specified in order to highlight the various parts of the algorithm. For incomplete information, time and stakes were known (z_time = 1 and z_stakes = 1) whereas the novelty was unknown (z_novelty = 0); and for cue values, the time available was 45 s, stakes was a 1, and novelty was a 5.

C = \sum_{j = 1}^{3} e_{j} + (u_{a_{j}} (c_{j}) - e_{j}) z_{j} .

Figure 2.

Mathematical representation of the quick test process through transformation to fast-and-frugal heuristics components. See the text for more information about the example.

Quantitative Methods for Representing Expertise and Time Pressure

The previous sections have developed representative mathematical models of take-the-best and the quick test process (and by extension, tallying); however, no component was explicitly labeled expertise, experience, or time pressure—two very important contexts of NDM. Nevertheless, the FFH program upon which the general mathematical form was developed does have techniques for modeling and representing these two contexts. Generally, expertise is modeled through variations in the components and strategies of the model representing the operator (i.e., experts tend to use heuristics and focus on few important cues) and errors in the values of the components (i.e., experts have fewer errors in cue weights, cutoff values, or estimates of missing information). Time pressure is modeled through variations in incomplete information because of the time cost of acquiring and processing information. Additionally, time pressure can also be modeled through variations in components and strategies representing the operator (i.e., time pressure often results in the use of heuristics).

Experience and Expertise

There are two ways in which FFH research has represented experience and expertise in computational simulation: first, when selecting the components or strategies used to represent the operator, and second, when generating values for components. One very general synthesis of the FFH literature on strategy selection is that experts or decision makers with high time pressure, high information acquisition costs, information overload, or ill-structured environments tend to make decisions matching heuristic strategies (relying on fewer, more important attributes; e.g., take-the-best). Novices or decision makers with low time pressure and low information acquisition costs tend to make decisions matching normative strategies (relying on many attributes; e.g., a linear weighted additive model). For more thorough summaries of the FFH literature, see Katsikopoulos (2011), Gigerenzer and Gaissmaier (2011), and Todd et al. (2012).

Beyond strategy selection, FFH models and simulations approximate levels of experience and expertise by using different proportions of data sets—which represent the environment—when training the models. A novice may have very little experience with the environment and thus will base their judgments or decisions on only a few examples, whereas an expert will use information from many prior examples to make judgments and decisions. The relative level of expertise (or the proportion of the data set used to train models) affects the amount of error in defining four component values of the FFH models: the cue weights (w_j; e.g., Czerlinski et al., 1999; Gigerenzer & Goldstein, 1996), cue directions (the sign of the correlation between the cue and criterion; Katsikopoulos et al., 2010; von Helversen, Karlsson, Mata, & Wilke, 2013), estimates of missing information (e_j), and cutoff values (c_j). Therefore, experts are usually assumed to be “properly calibrated” in a modeling sense (Katsikopoulos et al., 2014, p. 155):

The decision maker . . . knows how to code the attributes [cues] a_j so that the weights w_j are positive and is able to order the w_j according to their magnitude. These assumptions are particularly plausible when the decision maker is familiar with the choice.

Time Pressure

Results from both the NDM and FFH programs support the bounded rationality perspective that people make decisions without gathering or processing all possible information (Simon, 1955). The most salient cause for making decisions with incomplete information is that not all information was available because it was either difficult to discern or not provided. These difficulties can create high information acquisition costs that are limited by time, money, or some other resource.

Within the FFH framework, time pressure has been shown to affect the types of components and strategies used by judges and decision makers. Rieskamp and Hoffrage (2008) empirically identified that when time pressure was high in terms of both opportunity cost and individual decision limits, people used heuristics as opposed to normative decision strategies. This finding confirmed previous findings that people focus their attention on more important information and thus use a more selective information search (e.g., Maule, 1994; Payne, Bettman, & Johnson, 1988; Rieskamp & Hoffrage, 1999). Therefore, time pressure, in addition to expertise, tends to result in the use of heuristic strategies that focus on a few important cues.

In a modeling sense, time pressure affects operator performance by reducing the amount of known information used when making the judgment or decision because there was not enough time to process all the information in the environment. Direct computational simulations of strategy performance with time limits (Payne, Johnson, Bettman, & Coupey, 1990) have used calculations of the amount of time required for each elementary information process of a strategy, for example, the time required to multiply a weight by a cue score or compare a cue value to a cutoff value (Bettman, Johnson, & Payne, 1990). Although we have not presented a method for measuring elementary information processes using the general mathematical form presented in this paper, we do believe it is possible and will be working to show it in future work.

Outside of using direct time estimations, mathematical models can approximate time pressure as its functional equivalent: incomplete information. Higher time pressure indicates less total information processed. Similar to studies of high time pressure, studies have shown that heuristics can perform well in environments with low total information (Garcia-Retamero & Rieskamp, 2009; Martignon & Hoffrage, 2002). These synchronous results support the general thesis of the FFH program that people use the bounds on rationality, such as lack of time and lack of information, as a mechanism for simple, robust, and accurate strategies that adapt to the environment and ecology (Gigerenzer et al., 1999; Gigerenzer & Gaissmaier, 2011; Gigerenzer & Goldstein, 1996; Todd et al., 2012). Recently, two additional measures of distributions of incomplete information have been added: information imbalance (the difference in available information between alternatives; Canellas, Feigh, & Chua, 2015) and complete attribute pairs (the number of cues in which information is known for all alternatives; Canellas & Feigh, 2014). Their studies have suggested that heuristics perform well with low total information by exploiting low levels of information imbalance and high levels of complete attribute pairs.

Conclusion

Engineers, designers, and decision analysts are dependent upon representative models of judgment and decision making when attempting to simulate, model, or support operators. The prevalence of classical decision-making models as a placeholder for operators is due, in part, to their being easier to simulate. In order for NDM, a much more accurate representation of real-world decision making, to become common in the engineering, design, and decision analysis communities, there must be simpler, representative, mathematical models of complex NDM models. The simpler models may not capture all of the rich and important processes discovered by NDM researchers, but, if representative, they can still capture the basic information-processing aspects of NDM.

We presented a method of generating simple, representative, mathematical models of NDM by transforming the NDM model into components based on the FFH program. Then we used a newly introduced general form of fast-and-frugal judgment and decision making to convert the components into mathematical representations. The general mathematical form was synthesized from the FFH program’s empirically derived conceptual (and sometimes mathematical) models of judgment and decision-making strategies with the addition of new, but necessary, components of incomplete information and estimates of missing information.

For the engineers, designers, and decision analysts, this method enables the modeling of a wide range of judgment and decision-making strategies quickly and easily—and transparently—in a single equation. Rather than implementing an algorithmic tool or specialty code, it is an equation that can be looked up and implemented. We envision it can be used to perform sensitivity studies on the robustness of overall designs to both heuristic and analytic judgment and decision-making methods.

The simple, representative, mathematical form also has multiple benefits for the NDM community through its foundation in the human-subjects, computational, and mathematical analyses completed by the FFH program: (a) a framework for generating constructive questions about how NDM models account for FFH components; (b) forming the basis for prescriptive NDM decision support tools that are easy to communicate, understand, and apply; and (c) providing a method for approximating experience, expertise, and time pressure. Although we have presented one method for generating a simple, representative model of NDM, more research is needed to distill other NDM theories (e.g., recognition-primed decision making; Klein, 1993) and characteristics (e.g., simulation–action matching decision rules; Lipshitz et al., 2001).

Footnotes

Acknowledgements

The authors thank Amy Pritchett and Brian German from the Georgia Institute of Technology for their motivating hypothesis that it might be possible to develop a general mathematical form of fast-and-frugal judgment and decision making. This work was supported by the Office of Naval Research Command Decision Making Program (CDM) under Contract No. N00014-13-1-0083 and the National Science Foundation Graduate Research Fellowship. The results do not reflect the official position of either agency.

Marc C. Canellas received a BS in mechanical engineering from the University of Missouri and a MS in aerospace engineering from the Georgia Institute of Technology. Currently, he is a PhD candidate in aerospace engineering at the Georgia Institute of Technology. His research interests include cognitive engineering, decision support for decision making with incomplete information, and robot law. He currently serves on Human-Robot Interaction Committee of the IEEE Standards Association’s Global Initiative for Ethical Considerations in the Design of Autonomous Systems.

Karen M. Feigh received a BS in aerospace engineering from the Georgia Institute of Technology, an MPhil in engineering from Cranfield University, UK, and a PhD in industrial and systems engineering from the Georgia Institute of Technology. Currently, she is an Associate Professor in the School of Aerospace Engineering at the Georgia Institute of Technology. Her research interests include cognitive engineering, design of decision support systems, human automation interaction, and behavioral modeling. Dr. Feigh serves on the National Research Council’s Aeronautics and Space Engineering Board (ASEB), and as the Associate Editor for the Journal of the American Helicopter Society and AIAA’s Journal of Aerospace Information Systems.

References

Bettman

J. R.

Johnson

E. J.

Payne

J. W.

(1990). A componential analysis of cognitive effort in choice. Organizational Behavior and Human Decision Processes, 45, 111–139.

Canellas

M. C.

Feigh

K. M.

(2014). Heuristic decision making with incomplete information: Conditions for ecological rationality. In 2014 IEEE International Conference on Systems, Man and Cybernetics (pp. 1963–1970). Piscataway, NJ: IEEE.

Canellas

M. C.

Feigh

K. M.

Chua

Z. K.

(2015). Accuracy and effort of decision-making strategies with incomplete information: Implications for decision support system design. IEEE Transactions on Human–Machine Systems, 45, 686–701.

Cohen

M. S.

Adelman

Tolcott

M. A.

Bresnick

T. A.

Marvin

F. F.

(1993). A cognitive framework for battlefield commanders’ situation assessment (Tech. Rep. 1). Fort Leavenworth, KS: U.S. Army Research Institute.

Cohen

M. S.

Freeman

J. T.

Wolf

(1996). Metarecognition in time-stressed decision making: Recognizing, critiquing, and correcting. Human Factors, 38, 206–219.

Cohen

M. S.

Thompson

B. B.

Adelman

Bresnick

T. A.

Tolcott

M. A.

Freeman

J. T.

(1995). Rapid capturing of battlefield mental models (Tech. Rep. 3). Fort Leavenworth, KS: U.S. Army Research Institute.

Czerlinski

Gigerenzer

Goldstein

D. G.

(1999). How good are simple heuristics? In Gigerenzer

Todd

P. M.

& ABC Research Group, Simple heuristics that make us smart (pp. 97–118). New York, NY: Oxford University Press.

Dawes

R. M.

(1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571–582.

Edwards

Fasolo

(2001). Decision technology. Annual Review of Psychology, 52, 581–606.

10.

Fan

Sun

McNeese

Yen

(2005). Extending the recognition-primed decision model to support human-agent collaboration. In Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (pp. 945–952). New York, NY: ACM.

11.

Fischer

J. E.

Steiner

Zucol

Berger

Martignon

Bossart

Altwegg

Nadal

(2002). Use of simple heuristics to target macrolide prescription in children with community acquired pneumonia. Archives of Pediatrics & Adolescent Medicine, 156, 1005–1008.

12.

Garcia-Retamero

Dhami

M. K.

(2009). Take-the-best in expert–novice decision strategies for residential burglary. Psychonomic Bulletin & Review, 16, 163–169.

13.

Garcia-Retamero

Rieskamp

(2008). Adaptive mechanisms for treating missing information: A simulation study. Psychological Record, 58, 547–568.

14.

Garcia-Retamero

Rieskamp

(2009). Do people treat missing information adaptively when making inferences? Quarterly Journal of Experimental Psychology, 62, 1991–2013.

15.

Gigerenzer

Gaissmaier

(2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482.

16.

Gigerenzer

Goldstein

D. G.

(1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.

17.

Gigerenzer

Todd

P. M.

ABC Research Group. (1999). Simple heuristics that make us smart. New York, NY: Oxford University Press.

18.

Green

Mehr

D. R.

(1997). What alters physicians’ decisions to admit to the coronary care unit? Journal of Family Practice, 45, 219–226.

19.

Hogarth

R. M.

Karelaia

(2005a). Ignoring information in binary choice with continuous variables: When is less more? Journal of Mathematical Psychology, 49, 115–124.

20.

Hogarth

R. M.

Karelaia

(2005b). Simple models for multiattribute choice with many alternatives: When it does and does not pay to face trade-offs with binary attributes. Management Science, 51, 1860–1872.

21.

Hogarth

R. M.

Karelaia

(2006). “Take-the-best” and other simple strategies: Why and when they work well with binary cues. Theory and Decision, 61, 205–249.

22.

Massanari

R. M.

Ager

Yen

Miller

R. E.

Ying

(2007). A fuzzy logic–based computational recognition-primed decision model. Information Sciences, 177, 4338–4353.

23.

Katsikopoulos

K. V.

(2011). Psychological heuristics for making inferences: Definition, performance, and the emerging theory and practice. Decision Analysis, 8, 10–29.

24.

Katsikopoulos

K. V.

(2013). Why do simple heuristics perform well in choices with binary attributes? Decision Analysis, 10, 327–340.

25.

Katsikopoulos

K. V.

Egozcue

Garcia

L. F.

(2014). Cumulative dominance in multi-attribute choice: Benefits and limits. EURO Journal on Decision Processes, 2, 153–163.

26.

Katsikopoulos

K. V.

Fasolo

(2006). New tools for decision analysts. IEEE Transactions on Systems, Man, and Cybernetics–Part A: Systems and Humans, 36, 960–967.

27.

Katsikopoulos

K. V.

Pachur

Machery

Wallin

(2008). From meehl to fast and frugal heuristics (and back): New insights into how to bridge the clinical–actuarial divide. Theory & Psychology, 18, 443–464.

28.

Katsikopoulos

K. V.

Schooler

L. J.

Hertwig

(2010). The robust beauty of ordinary information. Psychological Review, 117, 1259–1266.

29.

Kattah

J. C.

Talkad

A. V.

Wang

D. Z.

Hsieh

Y.-H.

Newman-Toker

D. E.

(2009). Hints to diagnose stroke in the acute vestibular syndrome three-step bedside oculomotor examination more sensitive than early mri diffusion-weighted imaging. Stroke, 40, 3504–3510.

30.

Keller

Cokely

E. T.

Katsikopoulos

K. V.

Wegwarth

(2010). Naturalistic heuristics for decision making. Journal of Cognitive Engineering and Decision Making, 4, 256–274.

31.

Klein

(2008). Naturalistic decision making. Human Factors, 50, 456–460.

32.

Klein

G. A.

(1993). A recognition-primed decision (RPD) model of rapid decision making. In Klein

G. A.

Orasanu

Calderwood

Zsambok

C. E.

(Eds.), Decision making in action: Models and methods (pp. 138–147). New York, NY: Ablex.

33.

Klein

G. A.

Calderwood

(1996). Investigations of naturalistic decision making and the recognition-primed decision model. Technical report, U.S. Army Research Institute for Behavioral and Social Sciences, Alexandria, VA.

34.

Klein

G. A.

Calderwood

Clinton-Cirocco

(1988). Rapid decision making on the fire ground (Technical Report No. AD-A199492). Retrieved from Defense Technical Information Center website http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA199492.

35.

Kohli

Jedidi

(2007). Representation and inference of lexicographic preference models and their variants. Marketing Science, 26, 380–399.

36.

Lipshitz

Klein

Orasanu

Salas

(2001). Taking stock of naturalistic decision making. Journal of Behavioral Decision Making, 14, 331–352.

37.

Luan

Schooler

L. J.

Gigerenzer

(2011). A signal-detection analysis of fast-and-frugal trees. Psychological Review, 118, 316–338.

38.

Martignon

Hoffrage

(2002). Fast, frugal, and fit: Simple heuristics for paired comparison. Theory and Decision, 52, 29–71.

39.

Martignon

Katsikopoulos

K. V.

Woike

J. K.

(2008). Categorization with limited resources: A family of simple heuristics. Journal of Mathematical Psychology, 52, 352–361.

40.

Martignon

Vitouch

Takezawa

Forster

M. R.

(2003). Naive and yet enlightened: From natural frequencies to fast and frugal decision trees. In Hardman

Macchi

(Eds.), Thinking: Psychological perspectives on reasoning, judgment and decision making (pp. 189–212). Chichester, UK: Wiley.

41.

Maule

A. J.

(1994). A componential investigation of the relation between structural modelling and cognitive accounts of human judgement. Acta Psychologica, 87, 199–216.

42.

McCammon

Hägeli

(2007). An evaluation of rule-based decision tools for travel in avalanche terrain. Cold Regions Science and Technology, 47, 193–206.

43.

Mueller

S. T.

(2009). A Bayesian recognitional decision model. Journal of Cognitive Engineering and Decision Making, 3, 111–130.

44.

Nowroozi

Shiri

M. E.

Aslanian

Lucas

(2012). A general computational recognition primed decision model with multi-agent rescue simulation benchmark. Information Sciences, 187, 52–71.

45.

Orasanu

Connolly

(1993). The reinvention of decision making. In Klein

G. A.

Orasanu

Calderwood

Zsambok

C. E.

(Eds.), Decision making in action: Models and methods (pp. 3–20). New York, NY: Ablex.

46.

Patterson

Fournier

Pierce

B. J.

Winterbottom

M. D.

Tripp

L. M.

(2009). System dynamics modeling of the time course of the recognition-primed decision model. Journal of Cognitive Engineering and Decision Making, 3, 253–279.

47.

Payne

J. W.

Bettman

J. R.

Johnson

E. J.

(1988). Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 534–552.

48.

Payne

J. W.

Bettman

J. R.

Johnson

E. J.

(1993). The adaptive decision maker. New York, NY: Cambridge University Press.

49.

Payne

J. W.

Johnson

Bettman

Coupey

(1990). Understanding contingent choice: A computer simulation approach. IEEE Transactions on Systems, Man and Cybernetics, 20, 296–309.

50.

Rieskamp

Hoffrage

(1999). When do people use simple heuristics, and how can we tell? In G. Gigerenzer, P. M. Todd, & ABC Research Group, Simple heuristics that make us smart (pp. 141–167). New York NY: Oxford University Press.

51.

Rieskamp

Hoffrage

(2008). Inferences under time pressure: How opportunity costs affect strategy selection. Acta Psychologica, 127, 258–276.

52.

Simon

H. A.

(1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69, 99–118.

53.

Slegers

D. W.

Brake

G. L.

Doherty

M. E.

(2000). Probabilistic mental models with continuous predictors. Organizational Behavior and Human Decision Processes, 81, 98–114.

54.

Todd

P. M.

Gigerenzer

(2001). Putting naturalistic decision making into the adaptive toolbox. Journal of Behavioral Decision Making, 14, 381–382.

55.

Todd

P. M.

Gigerenzer

ABC Research Group. (2012). Ecological rationality: Intelligence in the world. New York, NY: Oxford University Press.

56.

Tversky

(1972). Elimination by aspects: A theory of choice. Psychological Review, 79, 281–299.

57.

U.S. Department of the Army. (2014). FM 6-0, C1: Commander and staff organization and operations. Washington, DC: Author.

58.

von Helversen

Karlsson

Mata

Wilke

(2013). Why does cue polarity information provide benefits in inference problems? The role of strategy selection and knowledge of cue importance. Acta Psychologica, 144, 73–82.