The Impact of Ambiguity-induced Error in Offender Decision-making: Evidence from the Field

Deterrence and Ambiguity

Deterrence is typically conceptualized as an information-based process of threat communication (Gibbs 1968; Zimring and Hawkins 1973). Geerken and Gove (1975) formalized the logic of deterrence as a social psychological theory which ushered in the importance of perceived beliefs about the costs and risks of punishment. The authors note (p. 503, emphasis added): “[un]like classical economists’ assumptions that men accurately perceive rewards and costs and then act, they expect a range of accuracy in prediction of actual rewards and costs.” Embedded in this explanation is the idea that individuals are prone to error and uncertainty in their subjective beliefs about the consequences of engaging in criminal behavior. Thus, for decades criminologists have been attentive to the study of deterrence through the perceptual properties of punishment (Paternoster et al. 1983, 1985; Nagin 2013; Waldo and Chiricos 1972).

Behavioral sciences and behavioral economics, too, have long considered the implications of uncertainty about subjective beliefs, drawing a distinction between risk and uncertainty surrounding this risk, dating back to Knight (1921). Risk is commonly conceptualized as a perceived probability of an event, and ambiguity is the uncertainty surrounding that assessment of probability (i.e., “second order probability”; Camerer and Weber 1992). For instance, consider the example of flipping a fair coin and observing heads. This likelihood is governed by a single parameter—the probability of heads, p. Assuming you believe our assertion that the coin is fair, then you can conclude that the first order parameter, p = .5. The subjective belief that the coin is truly fair, meaning you are certain p = .5, implies that ambiguity, the second order parameter, is zero; any lack of confidence about the coin’s probability of landing on heads in expectation induces ambiguity.

In an early experiment demonstrating a paradoxical and consistent violation of expected utility theory, Ellsberg (1961) found that individuals consistently prefer simple bets over those that are complicated by ambiguity. In his “one-urn paradox,” individuals were told that an urn contained 90 balls, 30 of which were red, and the remaining 60 were a mix of black and yellow. Asked to choose between a bet to win a prize by drawing a red ball, or an alternative bet to draw a black ball, most chose the first bet. When asked to choose between a bet where they would win a prize by drawing a non-black ball, or an alternative to draw a non-red ball, most chose the second bet. These choices are inconsistent. The result from the first scenario implies that the largest fraction of balls is red, which also implies the belief that the largest fraction of balls is not black (i.e., red or yellow). This combination of preferences defies the transitivity assumption underlying rational choice through the demonstration of ambiguity aversion, the preference for clearly provided information (e.g., the proportion of red balls), even when the expected outcome is the same.

Machina and Siniscalchi (2014) review the substantial evidence regarding ambiguity aversion on decision-making across multiple domains including public policy, finance, and consumer choice. The implications for the role of ambiguity in punishment and deterrence, which by nature involve fuzzy beliefs about punishment risks, are also potentially numerous. For instance, Loughran et al. (2011) posit ambiguity aversion as a mechanism for Sherman’s (1990) idea of generating deterrence from rotating police crackdowns, whereby an otherwise fixed level of certainty of apprehension became variable. More recently, in a set of experiments contrasting expected utility and prospect theory Pickett and colleagues (2020) did not detect a change in attractiveness of a criminal opportunity due to the introduction of ambiguity in arrest risk.

An assumption, often implicit, underlying much of the theoretical rationale for ambiguity as a mechanism of deterrence, such as with rotating police crackdowns, is that individuals are always uniformly ambiguity averse. That is, less information about that risk yields more deterrent value for any level of perceived apprehension. However, as pointed out by others (Bleichrodt, Courbage, and Rey 2019; Kocher, Lahno, and Trautmann 2018) this idea is, in fact, inconsistent with Ellsberg’s (1961) original conceptualization. In particular, Kocher et al. (2018) note that in situations with highly probable losses, individuals should in fact be ambiguity-seeking. ² For instance, Casey and Scholz (1991a, 1991b) tested the effect of ambiguity on tax compliance and evasion using a vignette study among student subjects in which the precision of the probability of detection varied across conditions. The experiments yielded evidence of boundary effects, where participants demonstrated ambiguity aversion and higher compliance as the probability of detection neared zero (the lower boundary) but tended to become ambiguity-seeking and less compliant when the ambiguous estimate approached more certain detection (the upper boundary). This logic implies that in situations where the probability of detection is very close to 1 (i.e., individuals are facing a near-certain loss), the injection of ambiguity may in fact be a catalyst, not a deterrent.

24/7 Sobriety as a Field Test of Ambiguity

Perhaps the most difficult barrier to studying the effects of ambiguity in the decision-making process is properly measuring it. In their review of ambiguity in experimental economics, Krahnen, Ockenfels, and Wilde (2014: 8, emphasis added) describe the challenge of moving from the laboratory to field thusly: “A main challenge for theoretical and empirical studies is the issue of how ambiguity can be defined and how it can be captured. So far, it is largely unclear how to operationalize the concept of ambiguity in the context of real economic applications.” Since its introduction into the literature on offender decision-making, criminologists have also struggled with this exact problem (Loughran et al. 2011; Pickett et al. 2015; Pickett & Bushway 2015). ³ Prior studies of offending decisions in the presence of ambiguity are limited in that ambiguity is studied either in an artificial lab setting, or in the context of financial decisions. While experimental evaluations of ambiguity in a lab or survey experimental setting allow for direct tests of theoretical constructs with high internal validity, the tradeoff is that those estimates are contrived and do not reflect “real world” decision-making. As such, scholars have stressed the importance of tests based on natural rather than contrived events (Baillon et al. 2018; Camerer 1998; Camerer and Weber 1992; Ellsberg 2011).

The implementation of the 24/7 Sobriety Program in South Dakota and Montana presents an opportunity to measure the impact of naturally occurring state-level variation in ambiguity on decisions across otherwise similar programs. We employ quasi-experimental evaluation methods to evaluate the importance of ambiguity in situ. An experiment randomly varying the definition of a violation that leads to incarceration within a single program contemporaneously would provide the strongest causal evidence but is ethically and legally tenuous. Another study doing so in a simulated environment would have strong internal validity, but potentially weaker external validity. Taken together, evidence from the field and evidence from lab settings are complementary, and both types of studies contribute to the evidence base on decision-making.

The 24/7 program is a particularly apt setting for the study of ambiguity. The original South Dakota 24/7 program was designed to directly reduce alcohol and other drug consumption through high-frequency alcohol and other drug testing with the threat of brief incarceration for failing a test or missing a test without an excuse. ⁴ The 24/7 program is grounded in the notion of certainty of detection for violation of probation conditions approaching unity, or in the words of program advocates, “[t]he consistency and predictability of punishments make the consequences of bad behavior clear to the offender, reinforcing the need to make better decisions and change behavior” (Swift Certain Fair Resource Center, n.d.).

Beginning in 2010, Montana replicated 24/7 following a similar path from small pilot toward eventual statewide implementation. Montana’s program was intended to mirror South Dakota (Wickum 2017). Both states’ 24/7 programs combine high-frequency alcohol monitoring via breathalyzer or a Secure Continuous Remote Alcohol Monitor (SCRAM) bracelet, with the threat of certain and immediate but brief incarceration—typically 12 to 48 hours without monetary penalty or any record on criminal history—for participants that are found to drink alcohol or skip a test unexcused while assigned to the program. All participants are provided an orientation session where program rules and expectations are reviewed. In both states, a judge may assign individuals with any offense that is associated with alcohol misuse to participate in the program.

Our study focuses primarily on participants in both states that are monitored using twice-daily breathalyzer tests at a court-assigned central location in each county. Participants submit to a preliminary test via a breath alcohol test device that meets the National Highway Traffic Safety Administration evidentiary standard for false negatives and false positives. Confirmatory testing of potential violations further reduces the chance of false positives. In all jurisdictions, breathalyzer participants must travel to a testing location where the test is administered during a fixed window each morning and evening, though the window varies slightly from county-to-county.

A Theoretical Model of the Certainty Effect and Ambiguity

The realization of punishment in the criminal justice system is often multifarious, meted out over the experience of arrest, pretrial detention, and sentence. Each node adds complexity to the perception of deterrence. Loughran (2019) advises that before investigating systematic departures from rationality predicted by behavioral economics, a necessary condition a priori must be a fully specified rational model (see Camerer and Loewenstein [2004] for further motivation of this point). Nagin (2013) describes how the certainty of punishment can be formally conceptualized:

[T]he certainty of punishment is conceptually and mathematically the product of a series of conditional probabilities—the probability of apprehension given commission of a crime, the probability of prosecution given apprehension, the probability of conviction given prosecution, and the probability of sanction given conviction

The 24/7 programs in South Dakota and Montana present a unique opportunity to study ambiguity in that, by design of the intervention, ambiguity can be isolated to a single parameter in the chain of events Nagin describes. The program design of 24/7 has several features which allow us to simplify one’s expectation about punishment likelihood. First, given the nature of the testing, the probability of apprehension approaches unity (i.e., participants are tested twice per day, approximately every twelve hours). In both states, all failures result in punishment. That is, there is no uncertainty regarding punishment conditional on failing a test. ⁵ The probabilities associated with the steps between apprehension and conviction are effectively invariant at 1. These properties make the dimensions of this ostensibly complex probability of receiving punishment reducible to a single parameter, specifically, the probability of failure given testing.

In South Dakota, participants are informed that any detected level of alcohol results in a violation, thus conditional on drinking, the probability of failure equals 1 with certainty (i.e., no ambiguity). Conversely in Montana, where there is a more lenient violation policy, participants are informed that the violation threshold is 0.02 mg/l, so very close to zero but not zero. While this conveys information about the risk of detection, it also induces ambiguity about whether or not drinking will result in a failed test and subsequent punishment. In other words, when individuals decide to drink in Montana, the perception of punishment risk is more ambiguous than in South Dakota. This “noise” around the failure likelihood is potential variance in subjective belief about risk, which captures ambiguity as described by Camerer and Weber (1992). This variability between states is the condition which our empirical analysis aims to leverage.

There are several key features of this design that are attractive both empirically and theoretically. First, in any traditional setting, it would be difficult to isolate the (risk) certainty effect from the additional effect of ambiguity, as the two factors are easily conflated, especially with regard to the closeness to the boundary, which would simultaneously affect both. Hence, this major structural challenge of disentangling the risk certainty effect and ambiguity effect is one that our design can uniquely parameterize.

Second, the sharp delineations of testing and punishment certainty put forth to 24/7 participants circumvents other key complications inherent to most perceptual deterrence studies including the questions about the credibility of elicited subjective beliefs, both in general (Dominitz and Manski 1997; Hurd 2009) and specifically related to deterrence (Loughran, Paternoster, and Thomas 2014), as well as key disagreements in the literature regarding the relationship between objective and subjective sanctions (Apel 2013; Kleck et al. 2005; Pogarsky et al. 2018). More specifically, program participants experience a structure that sets most key first-order parameters to be certain or near-certain, including probability of detection and probability of sanction given detection. Participants are explicitly provided information on the other key parameter—the BAC level determining violation—and we observe their behavioral response to the threat of real punitive repercussions, rather than elicited perceptions.

Hypothesis

We define the hypothesis of interest to explain the potential impact of ambiguity when participants face certain detection in both states, and there is ambiguity in punishment in Montana but not South Dakota:

Hypothesis: If participants are ambiguity-seeking, ambiguity positively influences their decision to drink beyond the threshold limit, P(failure)_MT > P(failure)_SD.

This hypothesis implies that participants make riskier choices in the presence of ambiguity when facing near-certain detection and punishment, which is an idea informed by insights from the behavioral economics literature. In a variety of contexts, individuals prefer uncertainty when facing losses (Kocher et al. 2018; Machina and Siniscalchi 2014). Further, evidence suggests ambiguity-seeking behavior and lower compliance when perceived likelihood is near the upper boundary when facing likely loss (Casey and Scholz 1991b; Kocher et al. 2018). An empirical finding against this hypothesis would provide evidence to support the practical intent of the program designers in Montana, who sought to minimize violations by preventing intoxication rather than imposing abstention. If participants are ambiguity averse or neutral, their propensity to drink beyond the threshold limit will be reduced or will be unaffected, and we would expect violation rates in Montana to be equal to or less than those in South Dakota, that is P(failure)_MT ≤ P(failure)_SD.

Analytic Plan

To establish credible counterfactuals (Rosenbaum and Rubin 1985) in a real community supervision setting, we employ two alternative methods to test the robustness of our findings to modeling choices, Mahalanobis distance matching (MDM) and doubly-robust inverse propensity score weighting (IPW). Each method requires a set of observable characteristics with which to establish balance between the treated group and a comparison group drawn from the untreated sample. We evaluated a broad set of theoretically informed candidate measures that may be related to program assignment or violations (Apel and Sweeten 2010): participant demographics associated with differential offending risk (gender and age), criminal history record (the count of prior arrests, separate indicators of prior arrests for violent crime, weapons charges, or drug charges), and community characteristics capturing sociodemographic and socioeconomic variation (a binary metropolitan area indicator, a binary for counties with Native American population centers, poverty rate, median household income, and unemployment rate), as well as and alcohol availability as measured by the density of on and off-premises alcohol retailers in each participant’s county. The inclusion of uninformative or uncorrelated controls can both lead to relatively poor balance and excessive variability in propensity scores (Brookhart et al. 2006; Wyss et al. 2013) and can compound omitted variable bias (Pearl 2000). Additionally, some measures such as unemployment rate, median household income, and poverty rate are highly collinear. To mitigate the risk of assignment model-induced bias, we exclude the Native American population center indicator, median household income, and on-premises retail alcohol outlet density from our final model specification.

We then estimate probability of violation under ambiguity in Montana as compared to the certainty of violation (and subsequent sanction) in South Dakota with the MDM and IPW models. For each time duration, we estimate the average treatment effect on treated (ATT) of ambiguity on our outcome of interest, the probability of violation in the first 30, 90, or 180 days of program participation. In the matching model, treated Montana cases are matched to their nearest neighbor via Mahalanobis-distance one-to-one with replacement to untreated cases in South Dakota.

The MDM matching approach is an alternative estimation strategy to the propensity score-based method that is more commonplace in criminological studies. Pearl (2009) described the potential for bias in point estimates induced by propensity score matching. King and Neilsen (2019) demonstrate conditions under which this concern, as well as a loss of efficiency in standard errors, are empirically evident. The MDM matching estimator also allows for a sensitivity analysis to evaluate how much hidden bias can be present before the qualitative conclusions of the study begin to change (Rosenbaum 2002, 2005). We also implement analytic standard errors proposed by Abadie and Imbens (2006) that are consistent under a range of underlying data distributions. We further present results that leverage a doubly-robust inverse propensity score-weighted model relying on a logit selection model as an alternative estimation strategy to show that our results are consistent across alternative estimators. ⁷ The doubly-robust IPW estimator is unbiased if either the propensity score selection model or the model estimating impact is correctly specified (Funk et al. 2011; Huber 1973; Kang and Schafer 2007).

Finally, while there is no direct test of the unconfoundedness assumption required for causal inference from both MDM and IPW, we present two sensitivity analyses to test the robustness of our findings. We first quantify the sensitivity of observed treatment effect to hidden bias by calculating a Γ statistic advised by Rosenbaum (2002). Γ is a crucial element of a selection on observables analysis because it demonstrates how much hidden bias would have to be present in the omitted selection mechanisms to negate the estimated relationship. In our case, the Γ statistic shows how much bias from unmeasured factors it would take to negate the relationship we observe between the ambiguity condition and the probability of a violation.

We also evaluate program violations for a separate sample of 24/7 participants who are monitored via ankle-worn secure remote continuous alcohol monitoring (SCRAM) devices, rather than the breathalyzer devices that are at the core of this study. Differences in the way that the device detects the presence of alcohol transdermally require a different procedure for determining a program violation. Detection of a violation for the SCRAM version of the program is administered by SCRAM systems in a single central location, and the process does not vary across states—participants receive the same information, face the same threat of sanction, and pay similarly higher daily fees than in-person breathalyzer participants. Thus, the remote alcohol monitor-based program should be virtually identical across states. Central to the validity of our main findings, the differential ambiguity condition does not exist. So, if violation rates are equivalent among SCRAM participants in the two states, it bolsters our claim that the settings are comparable across states and that we have isolated the effect of ambiguity on probability of violations using breathalyzer participants.

Comparison between Montana and South Dakota 24/7 Participants

Fifty-nine percent of Montanans violated by failing an alcohol test in their first 30 days under supervision, compared to 30 percent in South Dakota. Among those who fail, average time to failure is much shorter in the presence of ambiguity, 40 days in Montana versus 109 days in South Dakota. Table 1 displays the unweighted pretreatment characteristics of our analytic sample of DUI-2 24/7 participants in Montana and South Dakota to the characteristics in South Dakota after weighting. Of the nine control variables included in the selection model, balance exists in the unadjusted data among four variables based on the 20 percent standardized bias threshold, and none are beyond the 50 percent threshold Cohen (2013) defined as a “moderate” difference. Roughly 72 percent of observations are male and the average age is 33.5 years. Among the sample, participants in Montana were more likely to have been arrested for a violent crime, drug crime, and weapons crime, though only the difference in violent crime is statistically significant in the unweighted sample. Montana participants also reside in areas with marginally greater alcohol retailer density and higher poverty rates. These potential risk factors suggest that participants in Montana may be on average at higher risk of misconduct, including program violations. Both models were balanced over all model covariates and all cases in Montana—no standardized effect size for any covariate exceeded |20 percent| in the MDM or IPW model.

OPEN IN VIEWER

Table 1.

Unweighted Sample Characteristics and Standardized Percentage Bias after Matching and Weighting.

	Montana (BAC ≥ 0.02)	South Dakota (BAC > 0)
Matching Variables	Raw	Raw	Percentage Bias	MDM	Percentage Bias	IPW	Percentage Bias
Individual Characteristics
Male	0.719	0.733	−3.1	0.726	−1.6	0.708	2.4
Age	33.55	34.88	−11.5	33.30	2.2	33.38	14.5
Prior Arrest History
Count of Prior Arrests	2.888	3.462	−22.3	2.978	−3.5	2.961	−3.1
Violent Crime Prior	0.319	0.227	20.7	0.317	0.5	0.329	−1.5
Drug Crime Prior	0.290	0.265	5.6	0.290	0	0.296	−1.2
Weapons Crime Prior	0.043	0.024	10.2	0.043	0	0.356	3.7
County Characteristics
Metropolitan Area	0.362	0.515	−31.2	0.364	−0.5	0.343	3.9
Poverty Rate (percent)	14.83	13.07	45.4	14.33	12.9	16.04	−19.3
Liquor Store Density	0.779	1.025	−41.8	0.780	−0.2	0.712	13.0

*p < 0.05, **p < 0.01, ***p < 0.001

Impact of Ambiguity in the Alcohol Threshold on the Violation Rate

Table 2 presents the output of the MDM and IPW models predicting violation as a function of the ambiguity introduced by a non-zero violation threshold in Montana. The matching and weighting models produce nearly identical results. The MDM estimates the average causal effect of ambiguity on violations among those facing the higher ambiguity condition in Montana to be a 27.1 percentage point increase, 95 percent CI [0.195, 0.347] in the probability of violation, which is a 46 percent reduction in the compliance rate. These estimates do not differ statistically from those produced by the IPW model, for which the point estimate is marginally larger (ATT = 0.284, 95 percent CI [0.233, 0.336]). The higher probability of violation in Montana remains significant through 180 days in both models, though magnitude declines to 26–28 percentage points at 90 days and 18–21 percentage points at 180 days. This decrease may be evidence of experiential learning (e.g., through observation of other participants over time) or evidence of differential risk between states among those who are assigned to the program for longer periods. Since the mean and median time on the program in South Dakota is longer than in Montana, those assigned for more than 180 days in Montana may be a higher risk group than those assigned to the program for the same length of time in South Dakota. Nevertheless, impact on violations remains substantively large.

We find that greater ambiguity overwhelms the small difference in defined failure BAC threshold, consistent with our alternative hypothesis. This suggests that ambiguity is a salient feature of the decision to engage in prohibited behavior, and its effect on violations is large and meaningful. We estimate the net effect of ambiguity introduced by the seemingly small policy choice to slightly increase the BAC threshold to roughly double the probability of a violation after 30 days on the program and to increase the probability of a violation by approximately 20 percent at the 180-day mark among the higher-risk pool of long-term participants.

OPEN IN VIEWER

Table 2.

MDM and IPW ATT Estimates of Violation Rates by Ambiguity Condition.

Threshold	Model	n	Violation Rate		Ambiguity Impact (ATT)	95 percent Confidence Interval
			≥.02 BAC	>0 BAC
30 days	MDM		0.588	0.317	0.271	[0.195, 0.347]
	IPW	3,814	0.588	0.304	0.284	[0.233, 0.336]
90 days	MDM		0.588	0.556	0.257	[0.206, 0.308]
	IPW	3,368	0.861	0.582	0.279	[0.238, 0.321]
180 days	MDM		0.861	0.751	0.214	[0.169, 0.259]
	IPW	3,006	0.965	0.784	0.181	[0.154, 0.208]

*p < 0.05, **p < 0.01, ***p < 0.001; MDM estimated with analytical standard errors based on Abadie and Imbens (2006); IPW estimated via bootstrap to account for propensity weights (Lunceford and Davidian 2004).

Sensitivity to Unmeasured Confounders

The most important assumption of causal interpretation of estimates generated by MDM and IPW is that, after conditioning on observed confounders, no unmeasured confounder exists (Angrist and Pischke 2009; Rosenbaum 2002). In this specific context, unobserved differences between the 24/7-eligible populations in Montana and South Dakota, which have similar but not identical demographic characteristics, laws, and criminal justice processes may confound our estimates if they are correlated with differential selection into the program in either state. We further constrain our analysis to examine only the probability of a first violation among first-time 24/7 participants since the deterrent effect of the program could be differentially affected by the experience of punishment after a violation in the two states. To assess the sensitivity of our findings to this type of hidden bias in program assignment, we perform the sensitivity analysis prescribed by Rosenbaum (2002, 2005) using the Mantel-Haenszel Γ test (Becker and Caliendo 2007). Importantly, benchmarking our estimates using Γ allows us to assess the sensitivity of our estimates to hidden bias.

Based on the 30-day threshold MSM estimator, at Γ = 2.27 our estimate is statistically significant at p < .05. A value of Γ near 1 means a study is potentially very sensitive to hidden bias. Our obtained test statistic implies that some missing confounding factor would need to be at least 2.27 times more common in Montana to explain the higher rate of violations in the state. In light of the documentation of the two contexts from qualitative assessments based on field observations, interviews, and document reviews (Midgette 2014; Midgette & Kilmer 2021), in our view it is unlikely that there exists a factor that differentiates selection into Montana’s program that can be simultaneously so influential and hidden.

We further examined the sensitivity of our results using SCRAM-based method of alcohol supervision to interrogate the key assumption that program and environmental conditions are equivalent across states. If there were important differences in the program or other risk factors associated with drinking that led to higher risk in Montana than South Dakota, we would expect to see a difference in both breathalyzer and SCRAM violation rates. Using the same Mahalanobis matching method, among DUI-2 24/7 participants on SCRAM we find that the 30-day violation rate among 463 SCRAM participants in Montana is statistically indistinguishable (p = 0.443) from the 1,537 participants in South Dakota. This is consistent with the assumption that the states and programs are equivalent but for the ambiguity condition (see Figure 1). ⁸

OPEN IN VIEWER

Figure 1.

Comparison of 30-day violation rates by state and alcohol supervision technology.