Disagreement and epistemic arguments for democracy

1. Introduction

Oregon Ballot Measure 73 increased minimum sentences for certain repeated sex crimes and repeated drunk-driving offenses. Before the 2010 election in which Measure 73 appeared on the ballot, two dozen Oregon citizens were selected at random to serve on a review panel, which heard expert testimony and weighed arguments for and against the proposal. ¹ After deliberating they drafted a statement summarizing their conclusions, which was published and included in voters’ pamphlets. Voters saw how many panelists supported the measure and how many opposed.

The Citizens’ Initiative Review is now part of the referendum process in Oregon. Why submit ballot initiatives to this pre-referendum audit? Perhaps because we care about the quality of referendum outcomes (Gastil et al., 2007). We may fear that voters will not reach reasonable judgments on measures about which they know little and have spent little time deliberating. Since citizen panelists acquire information and deliberate about proposals, we might hope that their conclusions are likely to be sensitive to the reasons for and against proposals. When voters can see the panelists’ summaries of their conclusions and reasoning, their voting decisions are more likely to inherit this sensitivity, or so we might hope.

This line of thinking exemplifies an increasingly popular approach in democratic theory, according to which deliberative, democratic decision-making has ‘epistemic’ properties. According to this view, decisions reached through deliberative forms of democratic decision-making will be not merely fair, but also reasonable, according to some procedure-independent standard. ² For ‘epistemic democrats’, this alleged tendency is part of what justifies deliberative democratic procedures, part of what underwrites their legitimacy or explains their value relative to alternative procedures. What there is to be said for deliberative democratic procedures derives in part from what there is to be said for the quality of their decisions, according to this view.

I will argue that epistemic justifications for deliberative democratic procedures conflict with two widely shared intuitions about the nature and relevance of disagreement in politics. First, we ordinarily think that given the sources of political disagreements, we cannot expect disagreement among ostensibly rational citizens to disappear once decision procedures issue their verdicts, in the way that we might hope for the conclusions of well-designed scientific procedures of inquiry to produce consensus among rational observers. If an account of democracy implies that, with the right democratic procedure in place, the political judgments of rational observers would converge to a consensus merely upon observing the procedure’s decisions, then the account is, if not ipso facto implausible, at least deserving of special scrutiny. I will treat respect for this intuition as a constraint on adequate justifications, which for ease of reference I will call the non-convergence constraint. I will show that if an epistemic justification for a decision procedure is sound, then, contrary to ordinary intuition, there is a feasible decision procedure the verdicts of which demand not simply obedience, but the same sort of credence from rational observers as a well-designed scientific study.

Second, we ordinarily think that justifications for decision procedures should not presuppose substantive agreements on the outcomes of these procedures. Since Oregon’s citizens disagreed over the merits of Measure 73, a good justification for introducing the Citizens’ Initiative Review should not be premised on an assumption about the merits of Measure 73. Any such justification would be ‘sectarian’, convincing only for like-minded citizens. If the justification were addressed to individuals who disagree over this premise, it would appear question-begging. I will call this constraint on what counts as an adequate justification the constraint on evidence. ³

Epistemic justifications for deliberative democratic procedures violate the non-convergence constraint because if the justifications were sound, democratic decisions would provide evidence for and against different political judgments, and with enough independent decisions on an issue the evidence would become overwhelming and beliefs would converge to a consensus. Epistemic justifications violate the constraint on evidence because, in order for these arguments to be convincing, democracy and scientific inquiry must admit analogous forms of justification, but they do not. Just as the justification of a method of scientific inquiry does not presuppose the hypotheses that the method aims to test, so, likewise, must the justification of democracy’s epistemic properties refrain from controversial assumptions about what the outcomes of the democratic process should be, if it is to satisfy the constraint on evidence. But, I argue, the analogy fails: redeeming democracy’s tendency to yield the right decisions would require controversial assumptions about which decisions are right.

The critical discussion of epistemic justifications of deliberative democratic procedures is not intended to undercut the ideal of deliberative democracy – much less democracy, needless to say. There are other strategies for justifying deliberative democratic procedures that can be deployed alongside or instead of epistemic justifications. The aim is simply to highlight an incompatibility between epistemic justifications and common intuitions about the nature and relevance of political disagreement for the justification of decision procedures.

Section 2 describes in more detail these constraints on justifications as well as what is meant by an epistemic justification for deliberative democratic procedures. Sections 3 and 4 explain why epistemic justifications run afoul of the non-convergence constraint and the constraint on evidence, respectively. Section 5 concludes, and an appendix contains formal statements of some of the definitions and arguments.

2. Definitions

By an epistemic justification of deliberative democratic procedures, I will mean, roughly, a justification that rests partly on the premise that deliberative democratic procedures have some tendency to yield the right decisions. This gloss captures the basic, common feature of various accounts of ‘epistemic democracy’ that one finds in the recent literature. ⁴

The reference to the ‘right’ decisions is a placeholder and meant to be interpreted loosely for the purposes of classifying justifications as ‘epistemic’. Particular epistemic justifications might be concerned with more narrowly defined properties of decisions, such as their compliance with justice, economic efficiency, congruence with the general will, or something narrower still. What matters for my purposes, as will become clear below, is that the standard for ‘right’ decisions is such that citizens regularly find themselves disagreeing over which decisions are right.

What does it mean to say that decisions made by deliberative democratic procedures ‘tend’ to be right or, in a similar vein, ‘track’ truths about justice or the common good? Following Estlund (2008) and Landemore (2012), I will say that a procedure has a minimal tendency to produce right decisions if it does so more reliably than choosing at random.

More precisely, when there are two possible outcomes, a random procedure is defined as one that selects each outcome with a probability of .5 and a better-than-random decision procedure is defined as one under which the probability that an option will be selected, given that it is the right decision, is greater than .5. Generalizing this definition to cases involving more than two possible outcomes is unnecessary. It suffices for my purposes if I can show in the two-option case that epistemic arguments are inconsistent with how we ordinarily think about the nature and relevance of political disagreement. ⁵

Thus, we can say that the majority opinion of the Citizens’ Initiative Review was better than random if, conditional on approval of Measure 73 being the right decision, the majority of panel members was more likely to approve it than not, and conditional on rejection being the right decision, the majority was more likely to reject it than not. ⁶

Random procedures may not be politically viable options. But even if it was not politically feasible to decide on Measure 73 by flipping a coin, it impugns the rationality of the Citizens’ Initiative Review if its verdict enjoys no greater presumption of being right than a decision made by flipping a coin. Epistemic democrats (and deliberative democrats more generally) typically wish to claim that decisions made through deliberative democratic procedures enjoy some presumption of reasonableness or rationality. ⁷ Such a presumption would be unwarranted if these decisions were no more reliable indicators of which decision is right than the flip of a fair coin. In what follows, my focus will therefore be on epistemic justifications that claim that deliberative democratic procedures are at least better than random.

One need not be any kind of Platonist in order to entertain epistemic justifications. All that is required is that there is some desired procedure-independent property, so that one can intelligibly ask whether a procedure tends to yield decisions with the desired property. For example, perhaps what is important is that decisions on laws and policies be faithful to culturally specific understandings of freedom and dignity. Provided that one can make sense of the probabilities of policies, conditional on policies being supported or condemned by these culturally specific values, one can define random and better-than-random decision procedures.

What makes epistemic justifications problematic is not that they presuppose the existence of procedure-independent standards for evaluating collective decisions. Anyone who forms a judgment about the merits of a decision before learning its political fate concedes as much. The problem, I will argue, is that people commonly disagree over what the relevant standards are. If they agree on the relevant standards, they disagree over how particular decisions measure up against those standards. Epistemic justifications, I will argue, conflict with two common intuitions about the nature and relevance of this disagreement, which I express in the form of two constraints on what can be considered a plausible and adequate justification of a decision procedure.

The non-convergence constraint is motivated by an intuition about the nature of persisting disagreements among ostensibly rational citizens and the limits of political procedures. The intuition is that there is no feasible decision procedure such that we could expect disagreements to disappear once the procedure has issued its verdict, in the way that we might hope for the conclusions of well-designed scientific procedures of inquiry to produce consensus among rational observers. The non-convergence constraint is the assumption that a satisfactory account of democracy should respect this intuition.

The constraint on evidence eliminates justifications of decision procedures that presuppose substantive agreements on the outcomes of these procedures. A satisfactory argument for subjecting Ballot Measure 73 to the Citizens’ Initiative Review should be premised neither on the assumption that it was right to approve the measure nor on the assumption that it was right to reject the measure. Relying on a premise about how the issue should be decided would be either question-begging, if addressed to all citizens, some of whom would reject the premise, or sectarian, if addressed only to like-minded citizens who accept the premise.

Two metaphors may be helpful in this context. One might start from assumptions about which decisions are right and then, on the basis of empirical evidence or theoretical speculation, defend the conclusion that deliberative democratic institutions tend to produce these desired decisions. From this perspective, these institutions are the solution to a problem of political engineering: we want to show that they are effective instruments for achieving what we already know to be the right result.

Alternatively, one might start from a state of ignorance or agnosticism about what count as the right decisions, but defend a decision procedure as a reliable method of figuring out which decisions are worth taking. This perspective treats democratic procedures as analogous to well-designed scientific research. A particular research design can be defended as a fallible, but reliable method of determining the truth on some question, and this defense need not presuppose the answer to the question under investigation. Like the selection of a method of inference or research design, we might hope to defend deliberative democratic procedures not by establishing their tendency to produce a particular result to our liking, but rather by identifying the properties in virtue of which they tend to produce reasonable decisions, whatever decisions are in fact reasonable.

The constraint on evidence clearly rules out the ‘political engineering’ perspective. But the epistemic arguments for democracy that one finds in the recent literature resemble not those of an engineer with a predetermined vision of what outcomes should result from democratic procedures, but rather those of a researcher defending a particular method of inquiry. They seek to defend the epistemic virtues of a deliberative democratic process without recourse to assumptions about what the specific outcomes of that process should be. Instead of such assumptions, they point to formal properties of deliberation and democratic voting rules, the epistemic benefits of diversity of perspective, and seemingly minimal assumptions about citizens’ cognitive abilities.

To sum up, epistemic arguments claim that deliberative democratic procedures are ‘better than random’, as defined above. The nature of political disagreement is plausibly taken to impose two constraints on how we justify decision procedures, which I labeled the ‘non-convergence constraint’ and the ‘constraint on evidence’. We might hope that modest epistemic arguments, which claim only that deliberative democratic procedures are better than random, can satisfy the non-convergence constraint. In addition, we might hope to satisfy the constraint on evidence by developing epistemic justifications analogous to the justifications of methods of scientific inquiry.

Whether we should accept the validity of these constraints is a question outside the scope of the article. If my argument is right, then epistemic democrats will want to reject them and explain why the underlying intuitions about political disagreements are unfounded. But I think that the intuitions behind the constraints are widely enough accepted that it is worth asking whether they are consistent with epistemic arguments for democracy. The next two sections argue that epistemic arguments can satisfy neither constraint.

3. Convergence toward consensus

The democratic pedigree of a law may obligate citizens to obey, but it does not compel citizens to concede the law’s merits. The mere revelation that a majority approved Measure 73 did not automatically render opposition to the measure irrational, or so we are ordinarily inclined to think. This is true in virtue of the nature of political disagreement, not a shortcoming specific to democratic procedures. There is no feasible political procedure such that merely observing its decisions would compel the judgments of rational observers to converge to a consensus. I will argue that epistemic justifications of deliberative democratic procedures are hard to square with this ordinary intuition.

Defenses of majority rule that appeal to Condorcet’s jury theorem afford an especially clear and illustrative example of how this intuition conflicts with the logic of epistemic justifications. Assume that each referendum voter was more likely to support Measure 73 than not, if passing the measure was the right decision, and more likely to oppose it than not, if rejecting it was the right decision; that is, assume that each voter’s judgment was more reliable than a coin flip. Assume, further, that each voter’s judgment was probabilistically independent of the next voter’s. The jury theorem states that under these assumptions, it becomes increasingly likely, and approaches a certainty, that a majority of the referendum voters would support the right decision as the number of referendum voters increases. In the case of the Oregon referendum, in which more than a million voters participated, the assumptions imply that, if approving Measure 73 was the right decision, it was almost certain to be approved, and if rejecting Measure 73 was the right decision, it was almost certain to be rejected. They imply that on this matter the majority opinion was nearly infallible.

The problem with the jury theorem is that it proves too much to be a plausible basis for an epistemic defense of majority rule. Its immodest conclusion not only makes it a dubious resource for defending the epistemic value of majority rule under realistic conditions. It also means that any appeal to the jury theorem will conflict with the ordinary intuition that someone who dissents from majority opinions is not necessarily irrational. Given normal electorate sizes, accepting the jury theorem’s assumptions means crediting majority opinions with near infallibility. A rational observer cannot credit the majority with a nearly oracular infallibility and at the same time disagree with its opinion, unless she is all but certain of her prior opinion, which is rarely a justified stance in politics. Appeals to the jury theorem do not satisfy the non-convergence constraint.

Many epistemic democrats make no appeal to the jury theorem, and they are often content to credit democratic procedures with a more modest epistemic value. One might think that epistemic arguments that disavow the jury theorem and aim for more modest conclusions could be reconciled with the non-convergence constraint. The weakest epistemic claim one could make on behalf of a decision procedure, when the decision is binary, is the claim that it is just barely better than random, that is, just slightly more reliable than the flip of a fair coin. I want to argue that even epistemic arguments that make no appeal to the jury theorem and claim only that a democratic procedure is better than random may still run afoul of the non-convergence constraint. For once we grant that a decision procedure has some epistemic value, however modest, we have implicitly conceded that there also exists, at least in theory and often in practice, another decision procedure, derived from the original, that would possess near infallibility. This derivative procedure consists in the repeated, independent application of the original procedure to the same question. Upon observing the outcomes of this preposterously accurate derivative procedure, the political judgments of rational observers would quickly converge to a consensus.

For example, suppose that we wanted to argue that the citizen panels created by the Oregon Citizens’ Initiative Review are better than random, perhaps just barely so. The explanation might appeal not to the jury theorem, but rather to the deliberative aspects of their decision-making process – panelists hear testimony on the proposed initiatives, deliberate together, and each side must draft a statement of their reasons for their respective positions. These considerations might be thought to underwrite confidence that the citizens’ panel is more reliable than a coin flip: conditional on the passage of a ballot initiative being the right decision, the citizen panel is slightly more likely to support it than not, and conditional on rejection being the right decision, the panel is slightly more likely to oppose it than not.

Now imagine that instead of convening just one of these panels to review Measure 73, the state convenes thousands of panels. Each deliberates in isolation from the next and forms a verdict which is more reliable than a coin flip and probabilistically independent of the next panel’s verdict, conditional on the facts being judged. If enough panels were convened, they would provide in the aggregate a nearly infallible indication of whether it was right to pass Measure 73. If passing the measure is the right decision, then the true probability of each panel supporting the measure exceeds .5, because each panel is better than random. Hence, as the number of panels increases, it becomes increasingly certain that the actual fraction of approving verdicts will exceed .5. Thus, with many panels, it is all but certain that a majority of the panels will support Measure 73, if passing the measure is the right decision, and all but certain that a majority will oppose Measure 73, if rejecting the measure is the right decision. Were many panels to be convened simultaneously, there would be little basis for disagreement over the measure once the panels’ verdicts were revealed and everyone received the nearly infallible signal of its merits.

Thus, if an epistemic justification for a decision procedure such as Oregon’s Citizens’ Initiative Review is sound, then all that would stand in the way of achieving political consensus is the practical difficulty of executing the procedure repeatedly and independently. This conclusion follows even if the decision procedure is just barely better than random and just barely responsive to facts about which decisions should be taken. Epistemic justifications assert, in effect, that the stochastic process that has political decisions as its outcomes also has these facts as parameters. With enough observed realizations of the stochastic process we can identify those parameters conclusively. Political disagreements result merely from small samples.

Note, moreover, that if an epistemic argument for a procedure is sound, the point is not merely that there is a long-run tendency toward convergence, such that consensus is achievable in the limit, but never in a finite time. Consensus could be achieved in whatever span of time is needed to execute the decision procedure. For example, if all citizens recognize that the verdict of a citizens’ panel is a better-than-random indicator of whether an initiative deserves to be passed, then they might be in deep disagreement over the initiative on Friday evening, convene several thousand panels, allow these panels to deliberate in isolation from each other over the weekend, and then, upon learning the panels’ verdicts on Monday morning, they would all be faced with a mountain of evidence pointing toward one conclusion. If most of the better-than-random panels favored the initiative, that would be overwhelming evidence that approving the initiative is the right decision, and if most of the panels opposed the measure, that would be overwhelming evidence that rejecting the measure is the right decision.

It might sound strange to describe the panels’ verdicts as ‘evidence’ for normative judgments. We normally think that what count as reasons for a judgment about something like minimum sentencing guidelines are claims about fairness, moral desert, deterrent effects, the weight to give to each of these considerations, and so on. Whether Oregon’s Measure 73 should be passed was not a question that could be settled by empirical observation. But we normally hold this view because we assume that these normative judgments do not have empirically testable implications. Epistemic justifications contradict this assumption. If a large number of citizen panels are convened and each citizen panel is better than random, then the judgment that Measure 73 should be passed implies that with a probability close to 1, the fraction of panels supporting the measure will exceed .5. The negation of this judgment (the judgment that the measure should not be passed) implies that the fraction of panels supporting the measure will exceed .5 with a probability close to 0. We do not normally think of normative judgments having empirically testable implications, let alone such sharp implications, for the probabilities of observable events. But if they do– and epistemic arguments entail that they do– then the occurrence or nonoccurrence of those observable events should be treated as evidence for or against the normative judgments.

Two rational observers might still hold conflicting opinions about the measure even after each acknowledges that the panels’ verdicts strongly indicate that, say, the measure should be rejected. The supporter of the measure might have other, prior reasons for his opinion which still recommend it. But, if he is not rationally justified in an attitude of absolute certitude, then the confrontation with overwhelming evidence against his opinion must diminish his confidence in it, and compel him to put more stock in the possibility that he is wrong. In addition, if he must continue to adjust his confidence in this way, as more and more evidence accumulates, then eventually, as the number of panels and the strength of their combined evidence grow, he will have to conclude that his prior opinion is probably wrong and the opinion of most of the panels is probably correct. ⁸

Estlund (2008) considers the objection that if majorities were highly reliable epistemic authorities on the kinds of questions that arise in politics, then minorities would be, contrary to ordinary intuition, rationally compelled to defer to the moral judgments of majorities. ⁹ As a response to the objection, he suggests drawing a distinction between, on the one hand, concluding that the moral judgment of a reliable expert is ‘probably correct’ and, on the other, ‘accepting’ this judgment. One may, he argues, consistently maintain that ‘the expert is almost certainly correct’, even if one is ‘not prepared to share in the expert’s judgment’ (Estlund, 2008: 105). If the distinction holds up, then it may block the reductio ad absurdum according to which if majorities are highly reliable epistemic authorities, then an individual, upon learning that he is in the minority, must concede that the majority’s judgment is probably correct and, therefore, must accept the majority’s judgment in place of his prior judgment. The distinction blocks this last step of the reductio. But the distinction does not block the first step, which issues in what will strike many as a sufficiently absurd conclusion for the purposes of a reductio: upon learning the majority judgment, a member of the minority must concede that it is probably correct if he believes that the majority was a sufficiently reliable epistemic authority. Perhaps this concession is not the same thing as accepting the majority’s judgment. But the idea that rational observers of majority rule must all concede that its verdicts are probably correct remains counterintuitive, and it arguably speaks against epistemic defenses of majority rule if they entail this conclusion.

We can only get to this conclusion if the judgments of the epistemic authority in question are sufficiently reliable indicators of the truth. If a decision procedure is just slightly better than random, then observers of its decision can continue to disagree, not just in the sense that they continue to accept conflicting judgments about which decision should have been taken, but also (insofar as this is not redundant) in the sense that they disagree about which of these judgments is likely to be correct. But my claim is that if a decision procedure is better than random, then there exists, in theory and often in practice, a feasible derivative procedure that consists in repeated, independent executions of the original. By merely observing the outcome of this derivative procedure, observers would receive a nearly infallible indication of which decision is right. If we get a modest amount of information from the verdict of a single deliberative citizens’ panel, then we should be able to get a lot of information from convening thousands of independent panels simultaneously. Rational observers would converge in their assessments of which side of the issue is likely correct. It is implausible that there exists such a quick and easy method to generate this kind of consensus on typical political questions, but the claim that each citizens’ panel is better than random implies just that.

I conclude this section by considering an explanation proposed by Goodin (2003) for how rational dissent from majority opinions might be reconciled with Condorcet’s jury theorem, which, as explained above, credits majorities with extreme accuracy. Goodin distinguishes between nonevaluative judgments of shared facts, on the one hand, and evaluative assessments, on the other. We have good reasons to revise our beliefs about shared facts upon learning others’ beliefs about these facts, but no comparable reasons to revise our evaluative assessments upon learning others’ evaluative assessments. The distinction, Goodin suggests, allows us to reconcile rational, persistent opposition to majorities with the epistemic value of majority rule. ‘The epistemic power of majorities, when dealing with intersubjectively shared facts, is what underwrites the rationality of majority rule. Their lack of any epistemic authority, when it comes to matters of evaluations, is what underwrites the rationality of persisting opposition’ (Goodin, 2003: 145).

One might take Goodin to be suggesting that the rationality of majority rule can be defended only when we can view majority decisions as reflecting purely factual judgments, without any evaluative assessments mixed in, as we might (ideally) view the verdicts reached by juries. But a restriction to such contexts would render epistemic defenses of majority rule uninteresting. It is hard to think of any consequential political problems for which all disagreements about the appropriate course of action are simply disagreements about factual questions, with evaluative assessments playing no role.

I interpret Goodin’s argument, instead, along the following lines. Majority decisions do not force adjustments to observers’ beliefs because these decisions inevitably reflect evaluative assessments. But since citizens’ factual beliefs reliably track the truth, it is unlikely that the majority decision rests simply on erroneous factual judgments. So jury-theorem-like arguments can rule out at least one potential source of irrationality in majority opinions.

But even if the theorem’s assumptions hold true for factual questions, regular factual errors may still undermine the rationality of majority rule. As an example, suppose that on the question of whether the new minimum sentences would deter crime, each voter in the Oregon referendum formed beliefs that were correct with a probability of .55 and probabilistically independent, conditional on the truth. Suppose that 10 percent of voters invariably vote for increasing minimum sentences, no matter their beliefs about deterrent effects, because they believe that justice requires more stringent sentences. Assume that everyone else votes for the increased sentences if, and only if, they believe that they will have a deterrent effect. (Note that I make no assumption about what is the ‘correct’ way of voting, which would require evaluative assessments.) Assume that increasing the sentences would not have a deterrent effect. Then, we would expect a majority to support the increased sentences, despite also expecting a (different) majority to form the correct belief about their inefficacy as deterrents: 10 percent always support the increase and we would expect 45 percent of the remaining 90 percent of voters to conclude, wrongly, that increased sentences would have a deterrent effect and to support the increase as a result of this false belief. Thus, while we would expect a majority of 55 percent to form a correct belief about the absence of deterrent effects, we would also expect a majority of 50.5 percent to support the increase, with most of them doing so on account of a false belief about its deterrent effects. If the number of voters is large enough, then with a probability close to 1 the actual number will be close to the expected value. A majority is sure to support the increased sentences even though they have no deterrent effects, and most of the majority will do so because of false beliefs about their deterrent effects. Thus, even if we are prepared to grant the jury theorem’s assumptions in the domain of purely factual questions, the assumptions do not ‘underwrite the rationality of majority rule’.

There may be other viable explanations for how rational individuals could dissent from the verdicts of what they recognize as a nearly infallible epistemic authority, but I leave their investigation as a future task.

4. The constraint on evidence

What of the other constraint, which rules out justifications of democratic procedures that presuppose substantive agreement on their outcomes? For illustrative purposes, let us start again with Condorcet’s jury theorem. An appeal to the theorem may seem on the face of it to satisfy this constraint, since it does not invoke any disputed assumptions about which decisions are right. It assumes only that democratic citizens pick the better of two alternatives more reliably than a coin flip.

But we should demand evidence for this assumption, and the demand for evidence cannot be met without presupposing disputed political judgments. To see why we should demand evidence, consider the Oregon referendum from the perspective of a member of the minority. If an observer of the Oregon referendum grants that the assumptions of the jury theorem were met, then, as explained above, he must conclude that approval of Measure 73 was almost certainly the right decision.

But he might just as well invoke the logic of the jury theorem to draw an inference in the opposite direction, from the observed outcome to a conclusion about individual competence. A member of the minority could evaluate the observed referendum outcome in light of an assumption about which of the two alternatives was the right decision. Conditional on approval of Measure 73 being the wrong decision, the observed referendum outcome was extremely improbable if individual voters were competent judges of the matter. ¹⁰ For members of dissenting minorities, the jury theorem can just as well explain why members of opposed majorities are unlikely to be reliable as it can explain why the majority opinion is likely to be right. Condorcet’s modus ponens is, as it were, Plato’s modus tollens.

If the jury theorem is to be part of an explanation of the epistemic value of democratic institutions, one that is meant to be convincing even for citizens who often find themselves in the minority, then what is needed is independent, compelling evidence of the reliability of the typical citizen’s judgments. The evidence must be strong enough that it compels us to acknowledge the epistemic value of even those disagreeable majority decisions that we would otherwise treat as symptoms of voters’ unreliability.

The challenge is to produce this evidence without violating the constraint on evidence. Suppose one wanted to gather direct empirical evidence that someone’s judgments about such things as minimum sentencing guidelines are probabilistically dependent on whether it is right to impose the sentencing guidelines. That requires identifying particular instances in which it is right to impose a given minimum sentence, so that one can assess the probability that the individual believes it is right to do so. That is, identifying evidence of reliability presupposes that one can already identify which decisions are right in particular instances, but appeals to such claims are what the constraint on evidence rules out.

The problem may be most vivid when the facts being judged are moral facts or intertwined with moral facts, but even nonmoral facts are often the subject of deep disagreement in politics. For example, there would be no way to gather uncontroversial evidence that voters form reliably accurate judgments about the effects of fiscal policies on economic growth. Evidence of reliability would be evidence that voters' judgments about a given policy are probabilistically dependent on truths about the policy’s effects. But since there is no consensus on their effects, there can be no consensus on what counts as evidence that voters’ judgments about these effects are reliably accurate.

The jury theorem offers an account of the logic of majority rule that is free of any controversial assumptions about which decisions are right. But its assumption of individual competence should not be accepted without evidence, and any evidence would appear to presuppose just such controversial assumptions.

The problem is not unique to the jury theorem. Suppose one wished to argue that when citizens engage in the right form of deliberation prior to reaching a collective decision, their collective decisions reliably track facts about which decisions are right. That might sound to some like a plausible consequence of deliberation, but what can one say to skeptics? Without disputed assumptions about which collective decisions are right, trying to meet this skepticism with empirical evidence would amount to searching for evidence of a correlation between two variables, one of which is impossible to measure. Validating the reliability of democratic procedures, as methods of figuring out the right answers to political problems, would seem to presuppose that we already know the answers. ¹¹

Instead of trying to find evidence of a probabilistic relationship between the decisions that democratic citizens support and the quality of those decisions, one might look for evidence that democratic citizens (under actual or counterfactual, improved conditions) form reliably accurate judgments on factual questions whose answers can be uncontroversially determined and are relevant to decision-making. We might, for example, ask whether citizens who have engaged in proper deliberation over trade policy tend to form reliable judgments about the existence of import tariffs, subsidies, the provisions of trade agreements, and other mundane facts which can be uncontroversially established.

But if people who all agree on these facts can still rationally disagree over the merits of some proposed trade policy (for example, something like US tariffs on Chinese solar panels, as retaliation for the Chinese government’s subsidies of the same), then successfully documenting that democratic decision-makers form accurate beliefs on these mundane issues will not establish the epistemic democrat’s desired conclusion. If instead of these mundane factual questions, we consider decision-makers’ beliefs about factual determinations that would significantly reduce or even settle disagreements about the merits of policies (for example, whether free trade invariably contributes to social welfare, rightly understood), then the putative facts are unlikely to be objects of uncontroversial consensus. ¹²

A reasonable suspicion with the discussion so far is that this quandary resembles a common and entirely surmountable challenge to empirical inquiry. Perhaps epistemic democrats are in no worse a position than scientists who routinely need to validate the reliability of some method of measurement even when the existence or values of the quantities to be measured are unknown. Since it is possible to redeem the epistemic properties of thermometers and particle accelerators, perhaps the prospects for epistemic arguments for democracy are not so bleak after all.

Consider, for example, the use in economics of satellite images of luminosity as a measure of economic output (Henderson et al., 2011). If reliable, this indirect measure of economic output would be useful for measuring output in countries in which government records are not considered trustworthy or in subnational units where no records are kept. To determine the accuracy of the measure for these cases, in which there is no way of directly measuring its correlation with actual economic output, researchers consider how well it correlates with economic output in countries where independent and reliable measures of economic output are available.

Epistemic democrats might consider an analogous strategy (compare Estlund, 2008: 170). Disagreement may be widespread and deeply rooted in democracies, but some matters are plausibly not subject to any disagreement among ‘right-thinking’ or ‘reasonable’ people. All right-thinking people can agree, for example, that slavery is unjust. We might, therefore, reformulate the constraint on evidence as a weaker, but more sensible constraint: justifications of democratic procedures should not presuppose any substantive agreements on the outcomes of democratic procedures, except possibly in those cases in which all right-thinking people would agree anyway.

Suppose we have some list of points on which such agreement can be expected: famines should be avoided, wars of aggression are wrong, and economic growth is, all else being equal, a good thing. Grant, for the sake of argument, that we can even expect agreement on which policies contribute to famines or economic growth, which wars count as wars of aggression, and so on. With such a list, we could test for probabilistic dependence between democratic decisions on these issues and facts about which decisions should be taken on these issues. Perhaps tax cuts are less likely to meet with democratic approval when they would trigger disastrous government shutdowns, democracies are less likely to declare war when self-defense is not a valid pretext, and so on, and any ‘reasonable’ or ‘right-thinking’ person can appreciate the evidence for these probabilistic relationships.

With this evidence, we might think to develop an epistemic argument for democratic procedures in much the way that social scientists might defend the use of luminosity as a measure of economic output. In each case, the goal is to defend the conclusion that two variables (luminosity and economic output, and democratic decisions and the facts about which decisions should be taken) are, in general, probabilistically dependent, despite the fact that one of the variables (economic output or the facts about which decisions should be taken) can be uncontroversially ‘observed’ in only a subset of cases. If there is a way to make the inference work in social science, then there should be a way to make it work for epistemic democracy, or so the thought might go.

In the economic measurement example, the inference is defensible only if we think that whatever the relationship between luminosity and economic output, it does not depend on whatever factors explain why we do not have access to independent measures of economic output. For the inference to work, what explains why data on economic output are missing cannot also be an unobserved determinant of the strength of the relationship between luminosity and economic output. Suppose you thought that output in some sector of the economy is both hard to measure and also requires little electricity. Then you would not be warranted in using luminosity as a measure of economic output when direct measurements on the latter are missing; the fact that they are missing would mean that the unobserved economic output is of the kind that does not correlate well with luminosity anyway. For the inference to be valid, we must be willing to assume that the places where direct data on economic output are available and the places where this data are missing are similar with respect to the factors determining the relationship between luminosity and economic output.

An analogous assumption is needed if epistemic arguments are to rest on an analogous inference, but the assumption that epistemic democrats need is unwarranted. Suppose that we verify that when it comes to the issues on which all right-thinking people support the same decisions, democratic procedures perform better than random. If we wish to infer from this fact that democratic procedures are, in general, better than random, then we have to assume that the issues on which right-thinking people unanimously agree and those on which they do not are similar with respect to the factors determining the outcomes of democratic procedures.

But that assumption is implausible. Here is one reason the assumption might be false: when it comes to issues on which there is general agreement among right-thinking people, (almost) everyone benefits from making the right decision, but issues on which there is no such agreement tend to be issues on which majorities just as often benefit as lose out from doing the right thing. These patterns conspire to make majority opinions responsive to the right reasons in the first set of cases, but not the second. An inference from the first to the second domain would be unjustified. Here is another possible story: the issues on which we find general agreement are cognitively undemanding, but the issues on which right-thinking people disagree require people to think slowly and overcome common, predictable forms of irrationality. In the latter cases, majorities are more often wrong than right. Once again, extrapolating from one domain to the other is unwarranted.

It is not difficult to think up variations on these stories that would defeat the strategy of extrapolation under consideration. It is hard even to imagine how the presence or lack of (reasonable) disagreement within each domain could be uncorrelated with the factors that determine majority decisions and hence determine the epistemic value of majority rule in each domain. But without some plausible story along these lines, there is no reason to think that the epistemic properties of democracy within one domain are suggestive of its epistemic value in the other.

5. Conclusion

Epistemic arguments for democracy sit uneasily with two widespread views about the nature and significance of disagreement in democratic societies. First, if an epistemic argument for a democratic procedure is sound, then there would exist nearly infallible democratic procedures, capable of producing overwhelming evidence for judgments about which decisions are right. The judgments of rational observers of these nearly infallible procedures would rapidly converge to a consensus. Epistemic arguments, therefore, run afoul of what I have called the ‘non-convergence constraint’ on justifications of democratic procedures. Second, any convincing epistemic argument for democratic procedures would have to presuppose answers to divisive political questions. Everyone could accept an epistemic argument for democracy only if they already found themselves in substantive agreement about which collective decisions should be taken. Epistemic arguments, therefore, run afoul of what I called the ‘constraint on evidence’.

Here I would make two observations. First, these critiques extend to parallel epistemic arguments for nondemocratic procedures. They apply, in particular, to rationalizations of political exclusion and rule by elites, insofar as the power of elites is defended as a means of ensuring good policy-making and insofar as there is the relevant kind of disagreement about what constitutes good policy. Moreover, the constraint on evidence and the non-convergence constraint would be just as difficult to reconcile with charges that democracies reliably make bad decisions as with claims that they reliably make good decisions.

Second, I have not defended the validity of these constraints, although their intuitive plausibility and widespread acceptance are the motivation for the article. Thus, for all I claim to have shown, one might still justify democratic institutions as instruments for achieving desirable, but controversial social policy, or even as instruments for realizing the platform of one’s preferred party. Such ‘sectarian’ or ‘partisan’ justifications of democratic institutions are incompatible with the constraint on evidence. But insofar as I have simply taken the constraint for granted, I have not given the reader any explanation for why sectarian or partisan justifications might be deemed unsatisfying.

What I hope to have shown in this article is, instead, that the strategy of justification found in the recent literature on epistemic democracy is not a genuine alternative to the sectarian strategy. Epistemic democrats try to defend the epistemic properties of democracy without invoking any substantive, controversial assumptions about what makes for good outcomes, and without committing themselves to the view that disagreement in democratic societies is merely the product of citizens’ irrationality or ignorance of the epistemic properties of democracy. The arguments of this article suggest that they are bound to fail on both counts.