Abstract
In this article, professors Dawid, Faigman, and Fienberg seek to “begin to remedy” (p. 361) the disconnect between science, as it is practiced and understood by scientists, and its legal use in the courtroom. Although purportedly speaking the same language, scientists and jurists often appear to be in dire need of translators. Since expert testimony has become a mainstay of both civil and criminal litigation, this failure to communicate creates conundra in which jurists seek testimony that scientists are in no way prepared to make. Statistics to the rescue! The authors frame the issue as one of perspective: Science infers the effects of causes, while law infers the causes of effects. Perhaps, the authors posit, an understanding of the statistical concepts underlying the notions of cause and effect will help law understand what it can and cannot expect from valid scientific testimony. Too rarely do statisticians weigh in on this debate. This article, therefore, is a welcome foray into the fray about what counts as valid scientific evidence.
In their quest, professors Dawid, Faigman, and Fienberg make a number of crucial points. First, there is a “yawning disconnect between how law defines expert proof and the ability of scientists to supply such proof” (p. 382). The primary issue the authors identify is a failure to grapple with reasoning from group data to the individual case. As examples of where the courts go astray, the authors proffer the legal concepts of “differential etiology” in medical causation cases (p. 363), 1 pattern recognition in criminal identification cases (p. 364), 2 the opinion rule, in which experts are permitted to testify from the individual effect to the general cause (p. 374), 3 and class certification in employment discrimination cases (p. 379). 4 In each of these instances, the law’s “unrealistic expectations” about what scientific proof can offer makes for illogical decisions.
Second, the authors propose educating courts (and presumably the lawyers who appear before them) about the distinction between “the effects of causes” and the “causes of effects” (p. 382). 5 Making this distinction, the authors assert, would help the courts in the above examples to reason more logically about the testimony offered by experts. Rather than conflating effects of causes and causes of effects, the authors would have the courts demand valid scientific proof of each aspect (p. 383). 6
The third important point the authors make is that when the courts do try to use statistical concepts in evaluating the validity of scientific testimony, they make a muddle of it. The primary example here is the widespread judicial requirement that epidemiological testimony demonstrate a relative risk (RR) of two before admitting it as proof of general causation. This requirement confuses the legal requirement that proof of causation be more probable than not with a doubling of the risk (p. 374). This is wrong, the authors explain, because of the difficulty of reasoning from the general to the specific. As epidemiologists have tried to explain to the courts, any increase in group risk from exposure to a chemical (i.e., any RR > 1) may be attributable to the cause of the effect experienced by the individuals within the group. 7
The authors have certainly made their case that courts are far from adept at reasoning about causation. But how does this article help the courts to perform their reasoning task? The authors discuss two possible solutions, the odds ratio, and the probability of causation. The odds ratio at first blush appears to be a good solution, because it is symmetrical between cause and effect, and can be estimated from both prospective and retrospective studies (p. 373). 8 This is in contrast to the concept of RR, which measures only the effect of the cause (p. 374), 9 while legal interest is primarily in the causes of effects. But odds ratio is not a great solution either, the authors acknowledge, because “as was clear in the work of Galton” the regression of X on Y is not obtainable from the regression of Y on X.” In other words, it isn’t really worth a lawyer’s time to figure out what this means because it doesn’t actually get us any closer to understanding the relationships and differences between “effects of causes” and “causes of effects.” So what does?
The authors propose a concept that they term “Probability of Causation” (p. 377). Following several pages of elegant equations and tables, however, the authors are forced to acknowledge that “even if we start with the best possible information (perfect experimental results) about the effects of causes, and use all relevant auxiliary information, we need to apply subtle logic to make inferences about the causes of effects (which will still, necessarily, remain imprecisely determined)” (p. 377). Well, in an alternate universe those factors might exist in concert, but in our own less perfect world, they are rather scarce. What real-life legal case has perfect experiments that the expert can rely on? When is “all relevant auxiliary” information known or available? Subtle logic? And what judge will permit testimony that is “imprecisely determined” (speculation, anyone)?
The sad fact is that “perfect experimental results” are rarely obtained in science and are almost never available for courtroom testimony. The real problem for scientific experts and the lawyers who employ them is how to make sense of the imperfect studies that do exist. Relevant auxiliary information may be unknown or unavailable for courtroom presentation and possibly not even guessed at. As for subtle logic, some jurists are better at this than others, but almost everyone could use some guidelines as to what this means and how to employ it. The reason that many judges require an RR of 2 for admissibility is not because they have thought through the meaning of RR in the particular case, but because they don’t have to. RR ≥ 2 is a bright-line rule that judges can apply reflexively (if mistakenly). 10
Perhaps what the authors are suggesting is that a statistician be permitted to testify about the factors that go into the probability of causation, without actually saying what that probability is—something like eyewitness experts, who identify factors that may affect the accuracy of eyewitness testimony, without actually opining on the particular eyewitness’s credibility. Similarly, a statistician could set up one of the authors’ tables, explain the factors going into each box, and the formula to assess the evidence, and let the jury have at it. The problem with this idea is that, as a practical matter, courts are loathe to admit eyewitness expert testimony, excluding it (incorrectly in my opinion) as invading the province of the jury.
So what have the authors achieved for their considerable efforts? The most valuable contribution of this article is opening the conversation about causal reasoning from a statistical point of view. As the authors note, it is a conversation that statisticians have too rarely joined, and one in which their input is pivotal. Scientists themselves often poorly understand the statistics that they use in designing and interpreting experiments. Judges and lawyers are even more at sea, preferring to use statistics as bright-line rules rather than as ways to reason about problems of causation. The authors understand that “any redress of the challenges we have attempted to identify here will take the concerted effort of both scientists and legal scholars” (p. 384). The real conversation, as the authors conclude, must be about where expert testimony can actually be helpful to the jury. That is, if the most scientists—including statisticians—validly can offer the jury is testimony about general causation (the effects of causes) and if, indeed there is “little scientific evidence to bear” on the question of individual causation (the causes of effects) perhaps the courts should not be demanding this kind of testimony from them. Instead of demanding from experts what they cannot validly provide, perhaps the court should be guiding the jury by instructing them about the “subtle reasoning” that is required to make the leap from general to individual causation. This is a singularly important point.