Abstract
Judgment and decision-making research on discounting suggests that when humans are thinking about gains, they tend to prefer certain and immediate outcomes to uncertain and delayed outcomes. However, discounting has been studied primarily using monetary commodities and, until recently, by testing one feature of the binary forced-choice task at a time: delay, probability, or amount of money received/lost. The present research is the first test of a dual discounting task that combines probability and delay into a single, binary forced-choice task in a non-monetary loss context. The key findings, based on three studies, suggest that delay and probability discounting play a significant role in decisions including non-monetary loss commodities like plea bargaining. Future work should explore the boundary conditions of dual discounting based not only on the nature of the binary choice (probability and delay) but also on the nature of the commodity (amount, valence, and quantifiability).
Research on judgment and decision making is a wide tent, hosting a range of research paradigms grounded in social and cognitive psychology, economics, marketing, and other fields (Fiske & Taylor, 2013; Gilovich & Griffin, 2010). In everyday life, people make choices and decisions with short- and long-term consequences. Theoretical models in psychology and behavioral economics generally account for these choice behaviors, including heuristic (Ericson et al., 2015) and discounting models (Green & Myerson, 2004). Choosing between two options that differ on a single dimension is one such choice task and a relatively easy one: people likely prefer a parking ticket that is $20 rather than $40 and they prefer to pay a ticket in a month rather than immediately. In this instance, research on discounting measures decisions in such binary forced-choice preference tasks (e.g., Myerson & Green, 1995).
The pattern of preference for immediate as opposed to delayed outcomes is called delay discounting (also temporal discounting, e.g., Madden et al., 2003; Myerson et al., 2001). In classic discounting studies, the commodity is monetary gains (or losses): $10 now or $20 in 1 month (e.g., Green et al., 1994; Myerson & Green, 1995). In addition to delay until receipt (or loss), classic discounting studies demonstrate that probability of receipt (or loss) is another important feature of a preference task (e.g., Rachlin et al., 2000). For example, when provided a choice between $10 with 100% certainty of receipt and $20 with 60% certainty of receipt, humans tend to choose the certain option despite the fact that the choice with the highest expected value is the uncertain option (because expected value = Probability × Amount; so $10 × 1 < $20 × 0.60). The preference for certain as opposed to uncertain outcomes is called probability discounting (Estle et al., 2007; Myerson et al., 2001). In general, the body of research on discounting highlights three features of binary preference tasks: (a) commodity, (b) delay, and (c) probability.
The most common commodities used in discounting research are monetary gains and losses. However, there is evidence that monetary gains and losses are not discounted in exactly the same way. Estle et al. (2006) found that a lost commodity (for both delay and probability tasks) had smaller and less reliable effects on discounting than a gained commodity (termed the “sign effect”; see Frederick et al., 2002). Comparing within delay tasks, gained commodities were discounted significantly more steeply than lost commodities. That is, when the same amount of money was at stake, participants were more willing to accept some money now and write-off the potential for more in the future; however, when confronted with losing money, participants were (relative to gains) willing to put off the loss into the future despite the future loss being larger than the immediate loss. Estle et al. (2006, pp. 914) conclude from these findings that “different processes are involved in discounting positive and negative outcomes.” In addition, Green et al. (2014) found an effect of amount ($100,000 vs. $20) on discounting of gains but did not find the effect of amount on discounting of losses. Thus, even for the most commonly studied commodity (money), lost or negative outcomes do not show as reliable a set of findings as gained or positive outcomes.
Harris (2012) studied a variety of loss commodities to better understand the relative unreliability of losses. These studies revealed two different patterns based on category of losses: (a) monetary and property losses were put off in the future by most participants, and (b) non-monetary losses showed highly variable responses with two different modal strategies: getting it over with and putting it off for as long as possible. Harris explained the divergent findings based on loss commodity type by arguing that they may evoke different processes of decision making. Specifically, decisions concerning non-monetary losses may produce more affect-based “hot values” (Metcalfe & Mischel, 1999) and are more “affect-rich” (Rottenstreich & Hsee, 2001) than decisions about monetary or property losses. Harris tested this and found evidence that dread minimization explained why some participants preferred immediate non-monetary losses over delayed ones (counter to typical patterns with monetary losses).
Furthermore, an important and recent advance in discounting research was the integration of delay and probability preference tasks into a single experimental task (hereinafter referred to as the “dual discounting paradigm”). 1 This not only tests delay and probability discounting in a more complex binary task but also introduces the possibility of testing the interaction between delay and probability. Vanderveldt et al. (2015) performed two monetary discounting experiments that showed significant interactions between delay and probability for monetary gains. Cox and Dallery (2016) replicated Vanderveldt et al.’s (2015) findings for monetary gains and extended dual discounting to monetary losses, but to the best of our knowledge no subsequent research has used the dual discounting paradigm to explore the kind of non-monetary commodities that would be of interest to social psychologists in studying decision making.
In addition to allowing researchers to test the interaction between delay and probability, the dual discounting paradigm allows researchers to examine the unique effects of commodity amount (or severity) on delay and probability discounting. For example, the steepness of delay discounting seems to vary greatly based on how extreme the commodity is, whereas the steepness of probability discounting does not vary as greatly based on the same change in commodity (see Estle et al., 2006, Figure 2 vs. Figure 3). Researchers have demonstrated the unique effect of commodity amount on delay and probability discounting in separate tasks (e.g., Estle et al., 2006; Green et al., 2014), but with the two characteristics (i.e., probability and delay) in the same binary choice; the findings from Vanderveldt et al. (2015) and Cox & Dallery (2016) strongly suggest that delay and probability discounting may involve fundamentally different processes even without considering commodity amount.
The Vanderveldt et al. (2015) dual discounting paradigm allows researchers to directly compare the generalizability of the main effects of delay and probability discounting in a more ecologically valid choice context reminiscent of everyday life in which delay of an outcome often inherently reduces the likelihood of receipt. In addition, the dual discounting paradigm allows researchers to test the interaction of delay and probability features (i.e., their multiplicative effect). It is also possible to evaluate the impact of commodity amount (for monetary commodities) or consequentiality (for non-monetary commodities) on delay and probability. Although the use of the dual discounting paradigm is promising, it has only been replicated once for monetary rewards (gains), only recently applied to monetary losses, and has not been applied to a non-monetary context. We suggest that the legal context of plea bargaining represents a pertinent and novel non-monetary loss context in which to test the effects.
Dual Discounting in a Non-Monetary Loss Context: Plea Bargaining
In plea bargaining, the criminal defendant is offered a binary preference task: plead guilty now to crime X and receive sentence A with 100% certainty or go to trial eventually and face crime Y associated with sentence B with a chance C of losing. Importantly, plea bargain decisions contain the feature of delay: plea bargain now or go to trial later. Plea bargain decisions also contain the feature of probability: sign the plea agreement that the two attorneys have already agreed on, making it relatively certain, or go to trial, which is associated with a degree of uncertainty. The criminal sanction is the commodity in plea bargain decisions and is almost always associated with a loss of freedom, although there are different types and extremities of freedom loss (Costanzo & Krauss, 2017).
Although scholars have recognized the possibility of studying plea bargaining using a discounting framework (see, for example, Bibas, 2004; Wilford et al., 2019), most do not specifically point to dual discounting (but see Clatch, 2017). In addition, empirical legal scholars have extolled the utility of empirical discounting studies and acknowledged that the actual empirical work is “notoriously difficult” to conduct (Jolls et al., 1998). Thus, to our knowledge, no research has tested dual discounting in a loss context involving a non-monetary commodity, and we suggest that the structure of plea bargains offers a new and engaging context to do so. In addition, discounting research has utilized only continuous-variable commodities, and using an ordinal variable of criminal sanctions tests not only whether the type of variable (continuous vs. ordinal) affects decision-making processes but also whether the consequentiality of the commodity (i.e., loss of freedom) affects that process. In other words, do the features of delay and probability affect decision-making processes in the same way when the consequences of the choice are arguably weightier (loss of freedom vs. loss of money) and when the commodity is more difficult for decision-makers to quantify? It may be that discounting of an ordinal loss of serious non-monetary consequences is more likely to trigger affect-based decision making because participants are more fearful of making the wrong choice and are not able to reason through the decision by multiplying probability by a simple monetary sum to determine a mathematical subjective value.
In sum, delay and probability are naturally occurring situational features in plea bargaining, making it a conceptually appropriate context in which to examine the dual discounting paradigm. The three studies presented here have the overarching goal of testing the extent to which delay and probability discounting effects are (a) generalizable to a consequential non-monetary loss context like plea bargaining and (b) generalizable to commodities that are less easily numerically conceptualized. The goal of Study 1 was to develop a rank-ordered scale of criminal charges, criminal sentences, and criminal charge-sentence combinations, which is a necessary translational step before using the conventional discounting titration procedure (i.e., an adjusting-immediate-amount procedure; Holt et al., 2012) to experimentally test the impact of delay and probability on simulated plea bargain decisions.
Study 1
The goal of Study 1 was to operationalize and adapt the commodity of freedom (or lack thereof via criminal sentences) into a useable scale of criminal sanctions. Criminal sanctions are comprised of a criminal charge and a sentence, and neither of these components is a purely continuous, ratio variable: the criminal charges (e.g., misdemeanor) are categorical-ordinal and the criminal sentences involve elements of both categorical data (e.g., probation vs. jail time categories of punishment) as well as ratio data (e.g., 1-year probation vs. 3-year probation). Overall, the research question in Study 1 is whether the dual discounting paradigm is viable with a commodity of criminal charge-sentences in lieu of money.
The Adjusting-Immediate-Amount Titration Procedure
Rather than having participants sit through a long and boring study in which they run through every possible combination of choices (e.g., $10 now vs. $20 later, $10 now vs. $21 later, etc.), decision-making researchers commonly use the titration procedure because it uses the participants’ choices to shorten the procedure (e.g., Rachlin et al., 1991). Specifically, when confronted with $10 now versus $20 later, the participant may choose the immediate option, but when confronted with $11 now versus $20 later, the participant may flip, or switch, to choosing the delayed option. Logically, then, there is no reason to ask her whether she would also choose $20 later over $12 now, that is, self-evident. The difference between the last two immediate commodities (i.e., $11 – $10 = $1) becomes a means of calculating the participant’s indifference point, which is the point at which the two choices have equal subjective value to a given participant. This indifference point is presumed to be the average of the commodity in the last two certain options (e.g., average of $10 and $11). In discounting research, groups’ mean or median indifference points are then graphed to demonstrate that as delay increases (often across conditions, either between- or within-subjects), the indifference point decreases. That is, as the delay changes from 1 month ($10 now vs. $20 in 1 month) to 1 year ($10 now vs. $20 in 1 year), the participants, on average, will be expected to reach their indifference points at a lower monetary immediate value (e.g., $4) because the 1-year delay option even paired with $20 is not that attractive. In other words, participants would be willing to accept very little money now to not have to wait a fairly long time to receive $20. After graphing the indifference points, a hyperbola-like function describes the data of both gains and losses (e.g., Estle et al., 2006; Green et al., 2014).
To test whether probability and delay factor into plea bargain decision making like they do in monetary decision making, it is necessary to first create an ordinal scale of criminal punishments that includes both a charge (e.g., misdemeanor) and sentence (e.g., 100 hr community service) so that the adjusting-immediate-amount titration procedure is feasible. That is, when a participant chooses the certain-immediate option, the certain-immediate option is made less appealing to facilitate the choosing of the other option, that is, the uncertain-delayed option; and when a participant chooses the uncertain-delayed option, the certain-immediate option is made more appealing to facilitate the choosing of the other option, that is, the certain-immediate option. Note that the amount of the commodity of the certain-immediate option is what fluctuates in this execution of the adjusting-immediate-amount titration procedure.
More specifically, the purpose of the titration procedure is to isolate the indifference point, where theoretically, the two options are preferred equally. Once again, in the monetary context, the calculation of the indifference point is relatively simple: the average between the two immediate monetary values serves as the indifference point (Weigard et al., 2014). In the context of criminal sanctions, however, the indifference points cannot be calculated by a simple mean calculation. Rather, because the commodity scale is ordinal, the indifference point is also ordinal. Because the indifference point is ordinal, it is not possible to deduce the functional form of the indifference curves. In sum, the complexity of the commodity of criminal sanctions requires an initial study regarding the perceptions of leniency-harshness of criminal charges, criminal sentences, and criminal charge-sentence combinations. Study 1 thus examines the rank order of this new commodity, criminal sanctions, to determine whether we can use this ordinal commodity in the traditional titration procedure where the subsequent option is more or less appealing. The legal system relies on a ranking of criminal charges and sentences. Accordingly, we consulted two attorneys with a cumulative 40 years of experience in the criminal justice system to draft a list of realistic criminal sentences and charge-sentence combinations. These attorneys verified our assumptions about criminal charges and sentences. In fact, the U.S. criminal justice system functions on the assumption that criminal sanctions have varying severities (e.g., fines are less severe than probation; see Costanzo & Krauss, 2017).
Study 1 tests whether the legal system’s conceptual ordering matches a commonsense understanding. We predicted that participants would draw upon a pool of commonsense knowledge that matches our legally informed rankings of criminal charges and sentences. Specifically, this would mean that participants should (a) rank the criminal charges similarly, (b) rank the criminal sentences similarly, and (c) rank the charge-sentence combinations similarly to the legally informed rank-order. This study tests whether it is possible to convert loss of freedom, a commodity that varies on multiple dimensions (charge, sentence type, and sentence amount), into an ordinal scale to later test (in Studies 2 and 3) whether probability and delay affect the subjective value of plea bargain options.
Method
Participants
The participants were 569 individuals who participated through MTurk and were compensated with $0.50. 2 Data were excluded for 22 participants due to incomplete data on the ranking tasks leaving 547 participants in the final sample. In the Supplemental Material, a table (“SM 1”) shows the demographic break-down of Studies 1–3. Participants completed three ranking tasks, and both the task presentation order and order of response options were randomized.
Materials
SM 2A to 2C show screenshots of the Qualtrics ranking tasks with the left panel options shown in the conceptual (and hypothesized) order from least to most harsh. The right panel is the space to drop the options, and Qualtrics allows participants to move the options around after they have been dropped there. In addition, SM 2D shows that the options, once placed in the right panel, are numbered for clarity.
Participants also answered basic demographic questions about their race/ethnicity, gender, age, and experience in the criminal justice system. Experience in the criminal justice system was assessed by asking whether respondents have personally or vicariously had experience with the criminal justice system, using the following two questions: (a) Have you had any experience in the criminal justice system? and (b) Has someone close to you had any experience in the criminal justice system? These two questions were collapsed so that if a participant answered yes to either one, the participant was deemed to have criminal experience. They were collapsed because we wanted to know whether any criminal experience would inform participant rankings. The majority of discussion and analysis of the criminal experience items can be found in Table 1, Figure 1, and SM 3—SM 6.
Criminal Charge Rankings From Most Lenient to Most Harsh.
Note. Criminal experience was identified as a variable of concern because individuals with experience in the criminal justice system may have unique experience that affects their knowledge, attitudes, and perceptions about criminal sanctions. If people learn, firsthand or vicariously, how the system works and what charges and sentences are typical, it may change their rankings in either a more punitive or more lenient way. Accordingly, to determine if rankings differ as a function of criminal experience, we ran chi-square tests of independence comparing participant rankings on each criminal charge, sentence, and charge-sentence combinations. Those with criminal experience were expected to weight different features of the charge-sentence combination in light of their direct or indirect experience with the wide range of consequences associated with plea bargaining (e.g., social stigma associated with felony offenses). Rows labeled “CE” are for respondents self-reporting criminal experience (either direct or vicarious). Rows labeled “No CE” are for respondents self-reporting no criminal experience. For more information on the Friedman test see http://vassarstats.net/textbook/ch15a.html and https://onlinecourses.science.psu.edu/stat464/node/61 for calculations and examples. Table 1 shows that a larger proportion of participants with criminal experience ranked petty misdemeanor as the most lenient criminal charge (86%) compared with participants without criminal experience (74%); a larger proportion of participants with criminal experience ranked misdemeanor as the second most lenient criminal charge (82%) compared with participants without criminal experience (69%); a larger proportion of participants with criminal experience ranked gross misdemeanor as the third most lenient criminal charge (89%) compared with participants without criminal experience (75%); and a larger proportion of participants with criminal experience ranked felony as the harshest criminal charge (93%) compared with participants without criminal experience (85%). In sum, this
.05 ≤ p ≤ .1. *p < .05. **p < .01.

Charge-only rankings of all participants, criminal-experience participants, and no-criminal-experience participants.
Procedure
Participants were presented with three ranking tasks in which they were instructed to drag each of the charges, sentences, or charge-sentence combinations into a rank order with the highest rank number being associated with the harshest option (e.g., fourth in the charge-only task).
Data analysis
The three conceptual rank-orderings shown in Figure 1 and SM 3 and SM 4 reflect the assumption that lay people perceive the ranking of charges in generally the same way as the legal system ranks them. Figure 1 provides the frequency of participants that ranked each response option at each rank position for the full sample of participants (top panel), participants with criminal experience (middle panel), and participants without criminal experiences (bottom panel). As SM 2D shows there were three drag-and-drop ranking tasks. The Friedman test is a nonparametric alternative to a chi-square test that is useful in cases such as ours (see Hollander et al., 2013) where the intervals between scale points (ranks) are not equal (e.g., misdemeanors and felonies may not be equidistant from gross misdemeanors in terms of leniency/harshness). Accordingly, one Friedman test was conducted for each of the ranking tasks.
The null hypothesis was that rankings would be evenly distributed across the cells in the top panels of Figure 1, SM 3, and SM 4 for criminal charges, sentences, and charge-sentence combinations, respectively. More specifically, we tested whether the 16 cells in the top panel of Figure 1, the top panel of 144 cells in SM 3, and the top panel of 121 cells in SM 4, had frequencies that were evenly distributed.
Results
Conditional formatting was used in Figure 1, SM 3, and SM 4 to visually demonstrate which cells have higher relative frequencies: the higher the relative frequency, the redder the cell. Accordingly, the alternative hypotheses would predict a red line along the main diagonal of each figure.
Criminal charges
The Friedman test for criminal charges suggests that the descriptive, visual pattern in charge rankings seen along the main diagonal in Figure 1 (top panel) for all participants was not a matter of chance: rankings of the four charges were not equally distributed in the sample, χ2(3, N = 547) = 977.81, p < .001.
Table 1 presents participant rankings of criminal charges from most lenient (1) to most harsh (5); the body of the table shows the proportion of participants that ranked each criminal charge in each rank position. The pattern of results in Table 1 suggests that the criminal charge rankings of participants with criminal experience more closely matched the legal system’s ranking of criminal charges than the criminal charge rankings of participants without criminal experience.
Criminal sentences
The Friedman test for criminal sentences suggests that the descriptive, visual pattern in sentence rankings seen along the main diagonal in SM 3 for all participants (top panel) was not a matter of chance: rankings of the 12 sentences were not equally distributed in the sample, χ2(11, N = 547) = 3,600.65, p < .001. SM 3 shows the sentence-only rankings based on participants’ reported criminal experience (middle and bottom panels), and SM 5 shows no significant differences between participants with criminal experience and those without.
Criminal charge-sentences
The Friedman test for criminal charge-sentence combinations suggests that the pattern along the main diagonal in SM 4 for all participants (top panel) was not a matter of chance: rankings of the 11 charge-sentence combinations were not equally distributed in the sample, χ2(10, N = 547) = 2,803.72, p < .001.
Discussion
Study 1 represents a unique investigation, scaling a non-monetary commodity to be used in a discounting paradigm. Study 1 successfully demonstrated that criminal charges, criminal sentences, and criminal charge-sentence combinations could be rank-ordered to create a scale of criminal charge-sentence combinations, an ordinal commodity to be titrated to experimentally test the impact of delay and probability on plea bargain decisions. Based on the findings of Study 1, we used 7 of the 11 Study 1 charge-sentence combinations in Study 2. Figure 2 shows the final list of seven charge-sentences used in Study 2.

Final selection of seven charge-sentences for Studies 2 and 3.
The purpose of Study 2 was to test whether the pattern of dual discounting persists using a non-monetary loss commodity of criminal sanctions. We hypothesized that as delay until trial, probability of trial conviction, and post-conviction sentence increase, the severity of the charge-sentence associated with the plea at the indifference point increases. We expected that the longer a person has to wait for trial, the less appealing that option is (and the more appealing the plea option is) even if freedom is the potential outcome; and the greater the likelihood the person will be convicted at trial, the less appealing the trial is (and the more appealing the plea option is) even if freedom is a potential outcome at trial.
Study 2
Method
Study 2 was conducted in two sessions at least 3 days apart so that key dependent variables would be less likely affected by the collection of demographic and individual difference measures.
Participants
MTurk participants received $1 for Time 1 and $2 for Time 2. After removing IP duplicates, there were 395 MTurk participants with matched data from both Time 1 and Time 2. 3 A shirker analysis (Berinsky et al., 2014; SM 7) was conducted and 29 participants were excluded, leaving 366 participants. Another 33 participants were removed for not answering any of the plea bargain decision questions, leaving a total N=333. See SM 1 for demographic information.
Materials
At Time 1, demographic questions (race, gender, education, and age) and criminal experience questions were assessed. Two conceptually relevant individual differences were also included to be used as control variables: Eysenck et al. (1985) I-7 Impulsivity Scale and the System Justification Scale (Kay & Jost, 2003). Impulsivity has surfaced as a key individual difference of interest in the discounting literature (Myerson et al., 2017), and system justification scores may be related to willingness to acquiesce to accept a plea offer, which reduces the work for legal actors. The I-7 Impulsivity Scale’s response scale is “yes/no” and subscales of Impulsiveness (α = .87), Venturesomeness (α = .81), and Empathy (α = .91). The System Justification Scale is answered on a 9-point scale from 1 (strongly agree) to 9 (strongly disagree) and had a reliability of α = .67. Time 2 contained a vignette (SM 8) then a variety of binary, force-choice questions (see SM 9 for instructions that preceded the choices).
Design
This study had a 5 (Probability) × 3 (Post-Conviction Sentence) × 5 (Delay until Trial) mixed-factor design with two between-subjects variables: Probability of Conviction with five levels (5%, 25%, 50%, 67%, 95%) and Post-Conviction Sentence with three levels (1 month, 8 months, and 3 years). Post-Conviction Sentence was manipulated similar to how Estle et al. (2006) manipulated the uncertain/delayed amount of money offered (see pp. 917 describing the two delayed gain amounts of $200 and $40,000). The one within-subjects variable is Delay until Trial with five levels (6 months, 8 months, 10 months, 12 months, 15 months). Probability and Post-Conviction Sentence were fully crossed, and participants were randomly assigned to one of the 15 conditions. Each participant saw all five levels of Delay until Trial, and all five levels were presented in random order to reduce the chance of order effects. Figure 3 shows an example decision tree using the titration procedure, with the three design variables (probability, delay, and post-conviction sentence) staying constant within each decision tree but highlighted as a reminder that they change either between- or within-subjects.

Display logic decision tree for participant in 95% chance of conviction and 8-month post-conviction sentence condition, in 6-month delay.
The key outcome variable in this study was the subjective value of the trial option outcome, indicated by the indifference or tipping point of each participant within each decision tree. The ordinal subjective values are shown at each tipping point in Figure 3. Figure 3 represents one of the five levels of the within-subjects variable of Delay until Trial (within the 95% Chance of Conviction and 8-Month Post-Conviction Sentence Condition).
Procedure
Each participant read a vignette asking the participant to imagine driving down a residential street, accidentally hitting a child while driving, and then being charged with negligent driving (see SM 8). Next, the participant is given information about their choices: accept a plea bargain or go to trial (see SM 9).
Using the seven charge-sentence combinations developed in Study 1, the plea bargain option, which was the certain and immediate option, was altered (via the adjusting-immediate-amount procedure) to make the plea bargain more or less appealing depending on the participant’s initial choice. Specifically, as depicted in Figure 3, every participant saw the same first plea offer (i.e., charge and sentence), and if the participant accepted the plea bargain on the first option, the participant then saw the first option on the right-hand-side branch of the decision tree, which depicts an option with a less appealing plea bargain option. On the contrary, if the participant decided to go to trial on the first option, the participant then saw the first option on the left-hand-side branch of the decision tree, which depicts an option with a more appealing plea bargain option. This continued until either no more (or less) attractive option was available (if the participant chose “trial” or “plea” four times in a row) or participants hit an indifference point. The seven plea bargain options used in this study are shown in Figure 3.
Overview of data analysis
Based on participants’ decisions in the titration procedure, participants progressed to the next within-subjects delay condition after flipping from trial to plea or plea to trial (or choosing one option four times in a row). Figure 3 shows one delay decision tree, and each participant made at least two decisions (e.g., plea then trial P-T) and at most four decisions per decision tree (e.g., plea, plea, plea, trial; P-P-P-T), resulting in a total of eight possible response patterns to complete a decision tree: T-T-T-T, T-T-T-P, T-T-P, T-P, P-T, P-P-T, P-P-P-T, and P-P-P-P.
Based on these responses, we created Severity of Accepted Plea, an ordinal outcome variable with eight levels based on the eight possible response patterns. When a participant “flips,” their indifference point is theorized to be between the past two offered pleas; if a participant never “flips,” then the indifference point is somewhere past the range of pleas offered. Figure 3 shows the eight indifference points named based on participants’ responses. For example “SV from left = GM b/t 12m probation with 1,000.c.s. and 1,000.c.s.” stands for the severity when a participant rejected the offered plea deal of a gross misdemeanor with 12-month probation and 1,000 hr community service (i.e., chose trial) and then accepted a plea deal of a gross misdemeanor with 1,000 hr community service (see left two upper boxes of Figure 3).
Because this study used a mixed-factor design with both between- and within-subject independent factors and an ordinal outcome variable, ordinal logistic regression with mixed effects was used. Vanderveldt et al. (2015) did not control for the effects of demographic variables in discounting decisions. However, because of the decisional context we used, we tested the generalizability of discounting in a plea-bargaining context above and beyond demographic variables. Thus, the preliminary step was to enter control variables (gender, race, education, criminal experience, Impulsivity, and System Justification) into models for each response variable. Next, in Model 1, delay until trial (DUT), probability of trial conviction (PTC), and trial (post-conviction) sentence (TS) were entered into the models. Then, in Model 2, the three two-way interactions were entered, which Vanderveldt et al. (2015) found to be significant. Finally, in Model 3, we added in the three-way interaction into the models, which Vanderveldt et al. (2015) did not evaluate, but our methodology enabled.
We compared the Akaike information criterion (AIC) values of the models to determine which model had the best fit to predict plea bargain decisions and provide likelihood-ratio chi-squares for each model. The results are organized by response variable.
Results
It was predicted that participants’ indifference points would increase as all three experimental variables, delay until trial (DUT), probability of trial conviction (PTC), and trial (post-conviction) sentence (TS), increased. Indifference Point Plea Severity/Harshness (analogous to the subjective value from Vanderveldt et al., 2015) was first regressed on the demographic variables. However, once the three experimental predictors were entered into the model, the demographic coefficients became non-significant, and the AIC was lowered by excluding these demographic variables, AIC with the three significant demographic variables was 4,385.1 compared with 4,381.0; χ2(4) = 3.93, p > .1. Model 1 in Table 2 presents the coefficients, standard errors, and p-values for the three experimental predictors on Severity of Accepted Plea. The higher a participant’s Severity of Accepted Plea score, the more aversive the trial option was. PTC significantly affected the harshness of pleas accepted (β = 7.17, SE = 0.57, p < .001) with higher probabilities being associated with higher mean Severity (e.g., M = 2.5, SD = 2.40 for 5% probability of conviction and M = 5.7, SD = 3.31 for 95% probability of conviction; see SM 10). DUT significantly affected Severity (β = 0.09, SE = 0.03, p < .007) with higher delays being associated with higher mean Severity. TS also significantly affected Severity (β = 1.45, SE = 0.11, p < .001) with higher trial sentences being associated with higher mean Severity.
Indifference Point Plea Severity/Harshness Logistic Regressions.
Note. For the ordinal logistic regressions, the clmm() function in R was used with link = “logit.” The five levels of the within-subject independent variable (Delay) were nested in participants, and the random effects of participants was included with the term (1| Participant ID) in all models. Only the models of interest, with the experimental variables, are included in the table. Before the three experimental variables were entered in the model, Gender and Criminal Experience had coefficients (Bman = −0.19, Bgender.not.specified = 1.90, and B = −0.59) that produced significant t-values against the null hypothesis that the coefficients were zero, ps < .05; the coefficient associated with impulsivity (B = 0.01) was marginally significant, p = .06. The Bayes factors indicate that the data are 0.018 times more likely to occur under Model 1 than under Model 2 and 33.60 times more likely to occur under Model 2 than under Model 3. SE = standard error; CI = confidence interval; PTC = probability of trial conviction; TS = length of trial sentence if convicted; DUT = delay until trial.
.05 ≤ p ≤ .1. *p < .05. **p < .01. ***p < .001.
Model 2, also shown in Table 2, shows the coefficients, standard errors, and p-values for the three experimental predictors and the three two-way interactions. Adding the three two-way interactions increased goodness of fit, as indicated by a reduction in AIC from 4,381 to 4,356. In this better fitting model, χ2(3) = 30.34, p < .001, PTC significantly affected Severity (β = 4.09, SE = 1.18, p < .001), but DUT and TS became non-significant, ps > .1. The PTC × TS significantly affected Severity (β = 3.15, SE = 0.68, p < .001), and the DUT × TS significantly affected Severity (β = 0.065, SE = 0.03, p < .05), but the PTC × DUT interaction was only marginally significant (β = −0.2, SE = 0.12, p = .092). Figures 4 to 6 show the two-way interactions graphically for Severity.

Interaction of probability of trial conviction and trial sentence on severity of indifference point.

Interaction of delay until trial and trial sentence on severity of indifference point.

Interaction of probability of trial conviction and delay until trial on severity of indifference point.
Model 3, also shown in Table 2, shows the coefficients, standard errors, and p-values for the three experimental predictors, the three two-way interactions, and the three-way interaction. Adding the three-way interaction only increased the AIC from 4,356 to 4,358, χ2(1) = 0.39, p>.1, so further interpretations are made based on Model 2.
Discussion
The goal of Study 2 was to test our prediction that the non-monetary loss context would show a multiplicative relationship between delay and probability such that monetary dual discounting effects are generalizable to the non-monetary loss context of plea bargaining. That is, the longer the DUT, the less appealing trial is (and the more appealing plea is); and the higher the PTC, the less appealing trial is. Overall, our results from Model 2 demonstrate that PTC, DUT, and TS all influence plea bargain decisions, although not always to the expected extent or in the expected direction. Specifically, all three predictors featured in at least one of the two significant two-way interactions. The expected PTC × TS was obtained such that higher amounts (i.e., more severe Trial Sentences) evinced steeper relationships between PTC and Plea Severity. However, the influence of DUT did not show the predicted positive relationship between delay and outcome variable based on monetary gains (e.g., Vanderveldt et al., 2015) and monetary losses (Cox & Dallery, 2016): rather, there was almost no detectable influence of DUT.
The PTC × TS interaction in Figure 4 shows a line for each trial sentence with PTC on the x-axis. Note that the slopes of the lines increase as trial sentences increase. The interaction is reflected in the fact that the rate of increase in Plea Severity as a function of probability of conviction is greater for longer trial sentences such as 36 months (green line) than for shorter sentences such as 1 month (red line).
The DUT × TS interaction in Figure 5 shows a line for each TS with DUT on the x-axis. The (green) line associated with the 36-month TS is the only line with a positive slope. Participants confronted with a 36-month TS were more willing to accept a plea when delay until trial increased. This suggests that when the TS is especially long, participants may perceive longer delays as more attractive, which implies that they may want to postpone steep losses.
The marginally (p = .09) significant PTC × DUT interaction in Figure 6 shows many nearly overlapping delay lines. These nearly overlapping lines show that PTC explains more of the outcomes measure’s variance than DUT. The findings tend to suggest that despite a marginally significant PTC × DUT interaction, there is very little evidence of a multiplicative PTC × DUT discounting akin to monetary outcomes in dual discounting tasks (Cox & Dallery, 2016; Vanderveldt et al., 2015).
Study 3
The purpose of Study 3 was to determine whether the lack of a multiplicative relationship between Delay until Trial and Probability of Conviction in Study 2 was due to (a) the ambiguity of the decision maker’s location during the delay period, (b) measurement differences (i.e., the ordinality of the outcome measure and range restriction of the delay variable in Study 2), and/or (c) qualitative differences between monetary and non-monetary loss commodities (see Harris, 2012).
Method
Participants
MTurk participants received $1 for their participation. After removing 63 participants suspected of VPN-use to hide their lack of presence in the United States, there were 524 MTurk participants. A shirker analysis (Berinsky et al., 2014) was conducted and 126 participants were excluded, leaving a total of 398 participants. SMs 11 to 13 show the shirker analyses (Berinsky et al., 2014) based on the same scenario and attention check questions from Study 2.
Materials and procedure
The online survey contained the same vignette/scenario as Study 2 (SM 8), and legal decision-making questions using the same description of the plea and trial options as Study 2 (SM 9). The only procedural differences between the legal decision-making questions of Study 2 and Study 3 are that (a) every participant had to make four decisions before exiting the decision tree (Figure 7) rather than a range of 2 to 4 decisions (Figure 3) and (b) the plea options were simplified to only vary based on length of time in jail, making the key outcome variable continuous. The plea options all had the same charge (gross misdemeanor) and the criminal sentence only contained jail time, rather than alternative punishments like community service and probation. A question asking participants whether they drive was added to the survey to determine if participants’ responses to the plea bargain decisions were different among drivers and non-drivers. 4

Study 3 decision tree with subjective values.
Design
This study had a 4 (Probability) × 3 (Waiting Location) × 5 (Delay until Trial) mixed-factor design with two between-subjects variables: Probability of Conviction with four levels (5%, 50%, 95%, 99%) and Waiting Location with three levels (Jail, Bail, Ambiguous). The “Ambiguous” condition contained exactly the same language as Study 2 (i.e., the scenario left ambiguous where participants waited for trial). The Jail and Bail conditions contained added language in the vignette describing specifically where participants were waiting (see SM 14). The one within-subjects variable is Delay until Trial with five levels (1 day, 1 week, 1 month, 6 months, 1 year). Probability and Waiting Location were fully crossed, and participants were randomly assigned to one of the 12 conditions. Each participant saw all five levels of Delay until Trial, presented in random order to reduce the chance of order effects.
The key outcome variable in this study was the Subjective Value of the trial option outcome, indicated by the indifference point of each participant within each decision tree, which are shown at each tipping point in Figure 7.
Measures
Subjective value
Based on a series of four choices between plea bargains and trial (within a delay-specific decision tree), participants’ subjective value is obtained. Figure 7 shows the range of subjective values (32 to 88 days in jail). Because this study used the adjusting-immediate-amount procedure, the plea option’s jail amount increased when participants chose the plea (to incentivize “flipping” to trial). In this sense, subjective value in a loss context can be construed as the level of subjective aversion to the trial loss.
Impulsivity
The Impulsivity subscale of the I-7 Impulsivity Scale was measured and had a reliability of α = .84.
Contemplative emotions
Molouki et al. (2019) found higher contemplation emotions (i.e., emotions experienced while considering a distant event) for losses relative to gains and that contemplation emotions mediated the relationship between their key independent variables and delay discounting. Accordingly, we adapted these measures and asked the extent of positive and negative contemplative emotions participants felt about waiting for trial (the delayed outcome) as well as during trial (see SM 15 for the items). To get a general measure of negative contemplative emotions toward trial, the four questions were summed with negative emotions assigned negative values and positive emotions were positive.
Hypotheses
To determine whether the differences in measurement (ordinality to continuous and less range restriction in Delay) made a difference in the outcome of the multiplicative relationship between Delay and Probability, a mixed-effects linear regression was run in only the Ambiguous condition (the condition most analogous to Study 2). It was hypothesized that the measurement differences would not make a difference, such that in the Ambiguous condition, delay and probability would not be multiplicatively related (i.e., the two-way interaction would not be significant).
To determine whether the ambiguity of Study 2’s scenario regarding whether participants were expected to wait in jail or were out on bail until trial influenced the multiplicative relationship between Delay and Probability, a mixed-effects linear regression with all three experimental variables was conducted. It was hypothesized that participants in the Jail condition would be more likely to evince a stronger effect of delay and thus reveal a significant multiplicative effect between Delay and Probability relative to the Bail and Ambiguous conditions. 5 Being in jail is an aversive condition and so, the longer the pre-trial jail time, the more eager the person may be to seek a way out via a plea deal.
Results
Measurement differences between Studies 2 and 3 did not reveal multiplicative relationship
Table 3 presents the coefficients, standard errors, and p-values for the regression model with two continuous predictors DUT and PTC within the Ambiguous Location condition. As predicted, the mixed-effects linear regression revealed no significant two-way interaction between DUT and PTC when it was ambiguous where the participant would wait for trial during the delay (β = −0.08, SE = 0.06, p > .1). Figure 8 (right panel) shows the Subjective Value of trial as a function of DUT with relatively flat and consistent rates of increase across the four PTCs.
Study 3’s Mixed-Effects Regression Results (Hypothesis 1).
Note. Presented here are the results for a mixed-effects linear regression model based on two experimental variables (Delay until Trial and Probability of Conviction) and the key outcome measure of Subjective Value. As predicted, the measurement changes from Studies 2 to 3 did not cause a change in statistical significance of the interaction term. SE = standard error; CI = confidence interval; DUT = delay until trial; PTC = Probability of Trial Conviction.
.05 ≤ p ≤ .1. ***p < .001.

Study 3’s three-way interaction.
Consistent with Study 2, however, there was a significant main effect of PTC (β = 26.34, SE = 4.49, p < .001). There was also a significant main effect of DUT (β = 0.08, SE = 0.04, p = .05).
In jail: Multiplicative effect of probability and delay
Table 4 presents the coefficients, standard errors, and p-values for the full model with two continuous predictors DUT and PTC and one categorical predictor (Location). Model 1 shows the results when Location’s Ambiguous condition is the reference category, and Model 2 shows the results when Location’s Bail condition is the reference category. Both mixed-effects linear regression models revealed the predicted three-way interaction between DUT, PTC, and Waiting Location (Model 1, β = −0.38, SE = 0.09, p < .001; Model 2, β = −0.35, SE = 0.09, p < .001). Figure 8 shows the Subjective Value of trial as a function of DUT. The significant three-way interaction is reflected in the fact that for the Jail condition (left panel), the rate of increase in the Subjective Value as a function of DUT is greater for lower probabilities of conviction such as 5% (M = 49.13, SD = 21.44 at DUT of 1 day and M = 68.93, SD = 23.23 at DUT of 1 year) than for higher probabilities of conviction such as 99% (M = 68.89, SD = 21.06 at DUT of 1 day and M = 67.88, SD = 22.26 at DUT of 1 year). However, this pattern is not found in the Bail and Ambiguity conditions (middle and right panels).
Study 3 Mixed-Effects Regression Results (Hypothesis 2).
Note. Presented here are the results for two mixed-effects linear regression models based on all three experimental variables (Waiting Location, Delay until Trial, and Probability of Conviction) and the key outcome measure of Subjective Value. Model 1 shows the results when Waiting Location’s Ambiguous condition was the reference category, and Model 2 shows the results when Waiting Location’s Bail condition was the reference category. As predicted, the Jail condition was the only Waiting Location condition with the predicted PTC × DUT interaction. SE = standard error; CI = confidence interval; DUT = delay until trial; PTC = Probability of Trial Conviction.
.05 ≤ p ≤ .1. *p < .05. ***p < .001.
Discussion
Study 3 demonstrates that Study 2’s ordinal outcome measure and the fact that DUT had a narrower range were not causes of the only marginally significant PTC by DUT interaction. In addition, Study 3 shows that the multiplicative effect of PTC and DUT exists when participants awaited trial in Jail, but not when participants were out on Bail or when location was left Ambiguous. In the Bail and Ambiguous Location conditions, only PTC significantly affected participants’ plea bargain decision making.
In the Jail condition, on the contrary, PTC and DUT had a multiplicative effect on participants’ subjective values. Figure 9 shows the patterns of delay and probability discounting for the participants in the Jail conditions. Notably, the direction of delay discounting in Figure 9 (top panel) is different than what is predicted by monetary gains (e.g., Vanderveldt et al., 2015) and monetary losses (Cox & Dallery, 2016). Specifically, the findings from the monetary studies show that as delay increases, subjective value decreases. However, in Study 3, the lines are heterogenous with two of the four lines (purple and black) increasing (i.e., as delay increases, subjective value increases). The relatively flat line of 99% PTC demonstrates a ceiling effect analogous to the floor effects in delay discounting of monetary commodities (e.g., Cox & Dallery, 2016; Vanderveldt et al., 2015). Traditional monetary findings show that participants prefer delayed monetary losses to immediate ones (seen in their decreasing curves, e.g., Cox & Dallery, 2016, Figure 2). In the current study, for PTCs of 5% and 50%, however, as delay increased, so too did subjective value (though not monotonically). This suggests that as DUT increased, participants were more willing to accept pleas associated with longer jail sentences.

Multiplicative effect of DUT and PTC on subject value.
Figure 9 (bottom panel) shows shallow, decreasing curves for delays 1 month or less. Delays of 6 months and 1 year, however, do not show an expected pattern of probability discounting. In particular, as Odds Against increases along the x-axis from 0.05 to 1 and 19 (PTC of 95%, 50%, and 5%, respectively), there is a notable difference in subjective values. Strikingly, participants who were more likely to win at trial were more likely to take a plea compared with participants who had worse chances of winning at trial, but only when their wait until trial was long (i.e., at least 6 months).
This result may be deemed a “rational” choice, since even the harshest plea deal offered was 87 days in jail (close to 3 months in jail), whereas to wait until trial, they would have to wait 6 months in jail to be (possibly) vindicated. This suggests that jail time, rather than exercising one’s trial right for the possibility of avoiding future jail time and a conviction on one’s criminal record, was the key determinant.
One way to examine the plausibility of this interpretation is to examine whether differences in impulsivity and negative emotions offer insight into the obtained patterns of discounting. The logic of the account offered is based on the idea that less impulsive people will be more deliberative than individuals above the median in impulsivity who will not be able to tolerate longer wait times, and/or that negative contemplative emotions capture the aversiveness of being in jail and push people toward a deal that will allow them to leave jail sooner rather than later. We assessed the impact of the Impulsivity subscale (Eysenck et al., 1985) and extent of Contemplative Emotions (Molouki et al., 2019) toward Trial on discounting patterns. Figure 10 shows that both individuals with low impulsivity and individuals with very negative Contemplative Emotions toward trial (but not their counterparts) exhibited a notable difference in subjective values for low-PTC-high-DUT conditions.

Multiplicative effect of DUT and PTC on subject value based on individual differences.
Overall, Study 3 clarifies that plea bargain decisions about loss of freedom are different from monetary commodities in at least two important ways: (a) they are not significantly affected by delay unless the delay is particularly arduous (i.e., the decision-maker is waiting in jail), and (b) when plea bargain decisions are affected by delay (and probability), the relationship between delay and decisions is indeed consistent with other non-monetary losses (see Harris, 2012).
General Discussion
Our central research question is whether the features of delay and probability affect decision-making processes in the same way when the commodity is a consequential non-monetary loss that is more difficult for decision-makers to quantify and place on a mental number line.
Studies 2 and 3 first showed that regardless of whether the outcome variable was ordinal or continuous, the feature of Probability affects decision making in the same way as with monetary commodities (gains or losses): as the Probability of loss of freedom at trial increases, participants were more likely to accept harsher pleas to avoid trial (i.e., the subjective aversive value of trial increased). In addition, Study 2 showed a significant interaction between Probability and Amount (i.e., Trial Sentence). Interestingly, Green et al. (2014) reported no interactions between probability and amount for monetary losses, whereas the present findings based on a non-monetary loss are more consistent with findings on monetary gains. Specifically, at least for Probability, large Amounts produced steeper probability discounting than small Amounts, and here a Trial Sentence of 36 months produced steeper probability discounting than a 1-month Trial Sentence.
Second, regarding delay, in Study 2, Delay’s only significant influence on decisions was interactively with Amount (i.e., trial sentence); again, different from monetary losses (Green et al., 2014). Typical interaction effects between Delay and Amount with monetary gains show steeper delay discounting at smaller amounts, whereas in Study 2, the reverse was found: the 36-month Trial Sentence showed steeper delay discounting than a 1-month trial sentence. In addition, Study 3 clarifies that the interactive influence of Delay and Amount on plea bargain decision making is not the only delay effect in plea bargain decisions. Delay also had an interactive relationship with Probability in plea bargain decisions (as seen in dual discounting studies of monetary gains and monetary losses), conditioning on an especially aversive experience of delay until trial (i.e., waiting in jail for trial). Importantly, however, the direction of the delay effect is the reverse of that found in monetary losses and gains.
Consistent with Harris’s (2012) non-monetary losses study, people preferred the immediate option (a plea) when trial was associated with longer delays. This may partly have to do with uncertainty over whether they would receive credit for time served. The other part, however, seems to be a sense that waiting for trial in jail when likely to win at trial (making credit for time served moot) feels aversive enough to be worthy of a sense of dread. Evidence for this interpretation comes from the fact that 50% and 95% chances of winning at trial and long Delays (6 months or 1 year) diverged from patterns in other conditions; that is, they were associated with much higher plea acceptance (i.e., trial aversion). As expected, this pattern of divergence was especially pronounced in participants with low impulsivity, consistent with Myerson et al. (2017), or strong negative contemplation emotions, consistent with the dread minimization account presented by Harris (2012) to explain why non-monetary losses are different from monetary losses.
Third, the present studies add important support to the notion that not all losses are created equal (Harris, 2012), which suggests different processes from gains as well as monetary losses. Dread, contemplative emotions, and impulsivity are all promising ways forward to understanding the mechanisms of different types of discounting. Some researchers have suggested that there are three types of discounting: discounting of delayed gains, discounting of probabilistic gains, and discounting of losses (Green et al., 2014). The present research suggests that “discounting of losses” should at least be sub-divided into discounting of delayed monetary-property losses and discounting of delayed non-monetary losses.
Implications for Future Decision-Making Research
Essential to the success of Studies 2 and 3 was Study 1, which showed that criminal sanctions, composed of a criminal charge and criminal sentence, can be rank-ordered, producing an ordinal scale of criminal sanctions. This was crucial to establish because only commodities that are (at least) rank-ordered can be used as a commodity in a traditional adjusting-immediate-amount procedure, which relies on relative attractiveness of commodities to locate preference tipping points. The three studies together suggest that as long as researchers can establish a commodity’s ordinality, a discounting paradigm can be used to study decisions. Establishing ordinality may be especially useful with multi-faceted commodities like health outcomes. For example, exercising might immediately present both rewards (e.g., adrenaline rush) and losses (e.g., discomfort and strain) as well as delayed gains (e.g., increased strength or weight loss) and losses (e.g., less time to spend doing other things).
Importantly, the present effort also demonstrates the promise of using the dual discounting paradigm to study more complex, ecologically valid decisions. Research that isolates delay and probability discounting into their separate experimental tasks may be overestimating the impact of delay in decision making. In fact, even when studying monetary gains, the effect of delay can be fragile, or weaker, when using the dual discounting paradigm compared with simple delay-only discounting studies (see the marginally significant effect of delay in Vanderveldt et al.’s [2015] Study 1 followed by the significant effect in Study 2). Especially if one considers the fact that two options almost never have the same probability in real-world choices and an outcome’s delay (in isolation) may itself be a basis for its uncertainty, it is important for more dual discounting work to be done so that researchers can be confident in the generalizability of the effects they study.
Finally, it should be noted that plea bargaining is a complex system of human interaction between the criminal defendant, defense counsel, and prosecutor, and the criminal defendant’s human propensity to value certainty over uncertainty and immediacy over delay, making plea bargain decision making psychologically interesting to researchers in the judgment and decision-making tradition. According to Wilford et al. (2019), such research on the cognitive processes that influence plea decision making is still in its “infancy.” While the present research provides important insights into the plea-bargaining process, it is but one step forward to understanding what might influence criminal defendants’ decisions to forego their Constitutional right to a criminal trial. Although the present work cannot confidently be used to evaluate real-world plea-bargaining procedures, it offers methodological innovations that can be used in future studies with an eye toward application in many areas of decision making.
Supplemental Material
Clatch_Online_Appendix – Supplemental material for Plea Bargaining: A Test of Dual Discounting Preferences for Non-Monetary Losses
Supplemental material, Clatch_Online_Appendix for Plea Bargaining: A Test of Dual Discounting Preferences for Non-Monetary Losses by Lauren Clatch and Eugene Borgida in Personality and Social Psychology Bulletin
Supplemental Material
Supp_Materials.5.1.20 – Supplemental material for Plea Bargaining: A Test of Dual Discounting Preferences for Non-Monetary Losses
Supplemental material, Supp_Materials.5.1.20 for Plea Bargaining: A Test of Dual Discounting Preferences for Non-Monetary Losses by Lauren Clatch and Eugene Borgida in Personality and Social Psychology Bulletin
Footnotes
Acknowledgements
The authors are solely responsible for the research that was conducted and reported herein. They thank Andrew Sell, Alicia Hofelich Mohr, and Abbey Hammell for providing invaluable support in constructing the survey instruments’ complex display logic as well as JaneAnne Murray for helping design realistic survey materials. The authors are also grateful to Chris Federico for providing comments on an earlier draft and to Katerina Marcoulides and Alicia Hofelich Mohr for their feedback on mathematical modeling. They also thank anonymous Reviewer 2 for an abundance of astute comments that were liberally included in the current version of the article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Lauren Clatch was supported by a Graduate School Fellowship from the University of Minnesota and a 2017 Grant-in-Aid from the American Psychology-Law Society.
Supplemental Material
Supplemental material is available online with this article.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
