The limitations of relative plausibility theory

The domain(s) of inquiry

A&P compare their relative plausibility theory with what they consider the conventional probabilistic account. The latter includes two, quite distinct theories: a theory of inference, based on processes (not necessarily calculations) conforming to the rules of mathematical probability, such as Bayes’s Rule (hereafter, ‘Bayesian inference’); and a theory of the burdens of proof based on conventional probabilistic decision theory (hereafter, ‘decision theory’). As often done before, A&P proceed as if these two are inextricably connected (they are not) and as if the former only makes sense as involving serial updating calculations (it does not). Besides the mischaracterisation of Bayesian inference models, this ignores the fact, noted by several scholars, that decision theory can be used as a model of proof standards regardless of how fact-finders reach their assessments of likelihood that a claim is true. ² So long as they can capture their overall assessment of the comparative strength of the evidence in the form of an approximate odds ratio (e.g., the evidence favours the claimant over the defendant by a ratio of at least n:1), decision theory can be employed, assuming that other objections to such a theory can be overcome. One can give, and I have given, a qualified endorsement both to the decision-theoretic model of standards of proof and to relative plausibility as a theory of factual inference—without inconsistency. A&P’s presentation of their theory as providing the key to both domains of inquiry, however, places a greater burden on their theory than it can bear.

The task at hand

A&P characterise their task as that of articulating the best explanation of juridical fact-finding. At other times, they describe their enterprise as descriptive, as if an explanation is a description. I think of these as different. The best description of the orbit of a planet may be that it moves in an ellipse around the sun, but the best explanation of that movement would involve recourse to a theory of gravity. Differentiating their theory from the so-called ‘story model’, A&P say their theory is not a ‘psychological’ theory of behaviour, thus opening a potential gap between accurate description and whatever it is that they are doing (see, e.g. Allen and Pardo, 2019: 17 n 86). Acknowledging that their theory does not account for all the details of the various litigation systems within even the American branch of the common-law tradition, they maintain that their theory is an effort ‘to capture to the extent possible the essence of the object under inquiry’, that is, juridical proof (see Allen and Pardo, 2019: 7). As juridical proof is a purposive activity, a variety of normative issues inevitably pervade such a theory of ‘essence’, and indeed A&P compare the competing theories in terms of how they fit the goals of adjudication. For this reason, I think they are actually offering an interpretation of the practices that constitute the object of inquiry, for which both descriptive accuracy and normative appeal have a role. ³

In any event, A&P emphasise many ways in which their account is consistent with prevailing litigation practices. I mention this only because, when addressing other scholars’ claims that aspects of probabilistic theory are consistent with legal norms, A&P are inclined to be dismissive, saying things like ‘consistency is not enough’. ⁴ One cannot have it both ways: either consistency (or lack thereof) counts in favour of (or against) the descriptive component of an interpretation, or it does not. The answer is that is does, though of course that is not the whole story. ⁵ To be sure, from the way A&P articulate this particular objection, I suspect A&P think they are addressing mere claims that legal norms are logically consistent with a probabilistic theory. However, the theorists they are criticising make no such limited claims. ⁶

Plausibility and probability

Use of the decision-theoretic model of standards of proof requires an interpretation of probability that is applicable to uncertainty about the ultimate issues that are the subject of fact-finding. A&P’s discussion makes it clear that they think there is none. Their initial discussion acknowledges only two interpretations of probability potentially applicable to this context: objective frequentist and subjective. A&P rightly reject a frequentist interpretation, which only a few scholars from the law and economics tradition seem to employ. They also reject subjective probability as too unconstrained by the truth because it permits any ‘coherent’ set of probabilities (see Allen and Pardo, 2019: 11–12). Later, however, they consider what I call epistemic probability—the degree of rational belief or rational justification of belief—and reject it as well. I argue—and A&P seem to accept—that essentially the same factors that determine the degree of plausibility of an explanatory hypothesis help to determine its epistemic probability; that various legal norms help to channel a fact-finder’s subjective probability toward epistemic probability; that if a fact-finder perceives a difference between his or her subjective probability and the epistemic probability, the former will be adjusted accordingly. Because of such factors, there is something decidedly more objective in epistemic probability than a mere coherence requirement. Nevertheless, A&P dismiss this, noting a fact that I have acknowledged: the only available measure of the fact-finder’s epistemic probability is the fact-finder’s inevitably subjective judgment about it. They seem to think that this reduces epistemic probability to mere coherence constraints. ⁷ It does not.

A&P state that ‘[i]t is, of course, possible to ask fact-finders to attach numbers to the conclusion of their abductive inferences. But we think these numbers would be purely epiphenomenal…[T]he transmutation of explanations into probability statements would be necessary only when the standards of proof are conceptualised as probabilistic thresholds (which we maintain they are not and should not be)’ (Allen and Pardo, 2019: 21 n 112). What is at stake here? One possibility is a variance between the explanatory theory and a Bayesian theory that arises because the latter may embrace more information than the former. The likelihood ratio for the competing claims of the parties, relative to all the admitted evidence and uncontested facts, seems to capture the information provided by an explanatory inquiry. But one also requires prior odds on those competing claims in order to determine posterior odds (see Nance, 2016: 84–88). The prior odds are provided by the uncontested factual background of the case assessed in terms of the background beliefs of the fact-finders (see Nance, 2016: 95–101). The explanatory theory ignores such prior odds and their impact, or else arbitrarily assumes that they are always 1:1. Insofar as this replicates the experimentally demonstrated tendency of individuals to underweight the significance of priors, the explanatory theory may be descriptively accurate (on this point), assuming that the experimental results are replicated in adjudicated cases. But that limits its utility as a theory of a system the main goal of which, A&P agree, is accuracy. This is a weakness if we take their theory as interpretive.

The problem of explanatory hypotheses not advanced by a party

In the present version of the theory, A&P allow that fact-finders may (and sometimes do) infer an explanation that the parties do not themselves advance. ⁸ Moreover, in order to decide the case, fact-finders will need to aggregate that third account with one or more non-negligible accounts offered by the party it favors. ⁹ Thus, if the plaintiff asserts, ‘The light was red,’ the defendant asserts, ‘The light was green,’ the jury determines that an additional plausible explanation of the evidence is that the light was yellow, and the jury determines that a yellow light negates one of the elements of the plaintiff’s claim, then the jury compares the plausibility of ‘The light was red’ to the plausibility of ‘The light was either green or yellow’ (see Allen and Pardo, 2019: 25 n 141).

Presumably the same is true for a fourth hypothesis, a fifth hypothesis, and so on. Unless this sort of aggregation is limited by some principle, which A&P have not yet been willing to endorse, their theory collapses into the probabilistic model. This is because A&P’s theory would require that the fact-finder compare the disjunction of all non-negligible explanations favouring one side to the disjunction of all non-negligible explanations favouring the other side, whether such explanations are advanced by the parties or not. To the extent that the plausibility of these disjunctions is functionally equivalent to the probability thereof (about which, see above), the jury is doing the same thing as required under the probabilistic model.

A&P’s response to this difficulty (putting aside the issue of the functional equivalence of plausibility and probability) is to observe that the law and their theory both leave it to the parties to determine what theories to advance at trial, observing that ‘[i]nvoking the full range of possibilities will often not be to a party’s advantage at trial’ (Allen and Pardo, 2019: 26). I do not doubt the accuracy of their observation. But it is inapposite: the question at hand is not what explanations the parties will offer, but rather what explanations the fact-finder is supposed to consider in order to decide whether the standard of proof has been met.

The higher standards of proof

A&P defend their articulations of the higher standards of proof (such as ‘clear and convincing evidence’ and ‘beyond a reasonable doubt’) as non-circular because, for example, a standard legally expressed in terms of ‘reasonable doubt’ is to be understood under their theory in terms ‘plausible explanation’ (Allen and Pardo, 2019: 28). But the question is whether this advances our understanding discernibly. It may well be that ‘doubt’ is best understood in explanatory terms, but that still begs an obvious and persistent question: How strong must one’s doubt be before it becomes a reasonable one? This question is not illuminated by being told a doubt is strong enough to be reasonable when the pro-defence explanation is ‘plausible’. That simply produces the same question in different terms: how viable must an explanation be in order to be plausible? One advantage of the decision-theoretic model is that it provides an answer to this ‘how much’ kind of question and explains the reason for selected it. Nothing equivalent appears in A&P’s theory. Curiously, they seem to think that this is a descriptive/explanatory advantage of their theory because it reproduces (though it does not explain) the legal standard’s ambiguity and vagueness. ¹⁰ One rightly expects more of their theory.

A&P also defend the comparative nature of their interpretations of the higher standards of proof. In the case of the criminal law standard, for example, they admit that their version of this standard is not comparative in the obvious sense, but they remind us nevertheless that, in order to apply this standard, the fact-finder still will have to compare the plausibility of competing explanations (Allen and Pardo, 2019: 28–29). Perhaps so but the same can be said of decisions pursuant to a probabilistic standard, which undermines this claimed advantage of their theory. Moreover, probabilistic thresholds actually have an advantage in regard to their comparative quality. A test that requires (for example) the fact-finder to assess whether the prosecution’s case is at least ten times more likely to be true than false is discernibly more comparative than a test that requires the fact-finder to assess whether the prosecution’s case is plausible and, if so, to decide for the prosecution unless the defence’s case is also plausible.

The importance of weight

In one part of their paper, A&P provide a critique of my theory of proof burdens in an effort to support their theory by comparison. But they do not address my theory as a whole. Instead, they discuss only the three respects in which they regard my theory as diverging from the conventional probabilistic account. One of these, my use of the idea of epistemic probability, has been addressed above. Another is based on a simple misreading of my work, and I relegate my response to a footnote. ¹¹

I turn now to the third, their comments on the principal contribution I hope to have made in my book, one concerning what I called ‘Keynesian weight’, which is a measure of the completeness of the evidence presented to the fact-finder. Why they consider it necessary to criticise my theory of weight is not clear. I have emphasised that, even though I think decision theory provides a better interpretation of standards of proof than relative plausibility, my theory of weight is compatible with, and would constitute a useful addition to, their theory of standards of proof, should it prove to be the better account of such standards (see Nance, 2016: 83, 101–103). Are we then to infer that A&P do not think it important for the law to regulate evidential completeness?

A&P accurately describe the basic idea of Keynesian weight as a separate proof requirement, one articulated and administered almost entirely by legislatures and the judiciary. They note that I develop how such a requirement is effectuated by a great variety of doctrinal tools, including discovery requirements and attendant sanctions, admissibility rules, and rulings on the burden of production (Allen and Pardo, 2019: 44–45). (The list of tools I discussed also includes statutes of limitation, statutes of frauds, rules requiring recording of police interrogations, corroboration rules, and rules permitting adverse inferences against a party for withholding or destroying evidence (see Nance, 2016: 186–243).) But in their critique, A&P focus almost entirely on the issue of the formal burden of production. ¹² This is misleading because the formal burden of production is not a doctrinal tool on which I rely strongly, let alone exclusively, and for good reason. As I have acknowledged, conventional doctrinal formulations do not explicitly recognise the role of weight (by any name) in the context of the burden of production. My point about rulings applying the doctrine is that sometimes courts deciding whether to let a case go to trial make a point of noting the absence from court of specific evidence (even though that evidence is available to both sides and thus no inference against either side can be drawn from their failure to present it). Naturally, courts’ decisions are explained by reference to the conventional doctrine, so it is not surprising that A&P can make the case that the results in those cases can be and were justified without recourse to the idea of weight. ¹³

One might think that A&P focus only on this category of admittedly hard-to-decipher cases because they are the only ones directly involving the topic of the burdens of proof. But no, they also discuss, albeit briefly, the role of admissibility rules. Here again, however, they target a very minor part of my discussion, namely my argument regarding the Chambers case and its exercise of constitutional authority to override rules that otherwise would exclude evidence (see Allen and Pardo, 2019: 48). In doing so, they pass by the principal arguments I make about the operation of ordinary admissibility rules, which appear elsewhere in the book under the apparently easy-to-miss heading, ‘Admissibility Rules’ (see Nance, 2016: 195–201). Even in regard to Chambers, they miss the point of my discussion. I discussed that case merely to illustrate the diversity of doctrinal tools that can be used to augment Keynesian weight, whether they are articulated in such terms or not. A&P reject my illustration because the Court did not justify its decision in terms of weight and because the case ‘is not about a judge overriding party choice by declaring one side’s evidence to be incomplete’ (Allen and Pardo, 2019: 48). The standard they thereby impute to my theory is imprecise: the weight concern is whether the evidence as a whole is (reasonably) complete, not necessarily whether what a particular party presents at trial is complete. The obstacle in the way of such completeness can be rules just as much as party choice. ¹⁴ In any event, A&P neglect what I thought was interesting about the case: the fact that the Court conspicuously noted both the inadequacy of the prosecution’s investigation of the case (where appropriate augmentation of weight might have terminated the prosecution before it even began) and the effect of the mechanical application of the state’s evidence rules in depriving the fact-finder of a full account of what happened.

Finally, A&P invoke a separate parameter, one not strictly required by either their theory of inference or their theory of proof standards, namely, a seemingly strict reliance on the parties, with the benefit of discovery, to choose what evidence to present and how to present it (see Allen and Pardo, 2019: 15 n 69). Based on this, they argue their theory better fits extant practice than my theory of weight, which involves significant potential judicial intervention. ¹⁵ Of course, any theory of adversarial litigation must incorporate the idea of presumed party control, and mine does (see Nance, 2019: 186–188). But for a theory to fit the facts, it must also explain the panoply of legal interventions observed in adversarial presentations. That my theory does as well, while theirs does not. Indeed, A&P express scepticism about such interventions on the ground that they are unlikely to improve accuracy, at least in civil cases. ¹⁶ Perhaps so, but neither a descriptive/explanatory theory nor an interpretive one can ignore such interventions simply because they are arguably unwise.