Abstract
Baron and Jost (this issue, p. 292) present three critiques of our meta-analysis demonstrating similar levels of partisan bias in liberals and conservatives: (a) that the studies we examined were biased toward finding symmetrical bias among liberals and conservatives, (b) that the studies we examined do not measure partisan bias but rather rational Bayesian updating, and (c) that social psychology is not biased in favor of liberals but rather toward creating false equivalencies. We respond in turn that (a) the included studies covered a wide variety of issues at the core of contemporary political conflict and fairly compared bias by establishing conditions under which both liberals and conservatives would have similar motivations and opportunities to demonstrate bias; (b) we carefully selected studies that were least vulnerable to Bayesian counterexplanation, and most scientists and laypeople consider these studies demonstrations of bias; and (c) there is reason to be vigilant about liberal bias in social psychology, but this does not preclude concerns about other possible biases, all of which threaten good science. We close with recommendations for future research and urge researchers to move beyond broad generalizations of political differences that are insensitive to time and context.
When we decided to conduct a meta-analysis to examine susceptibility to partisan bias across the political spectrum (Ditto et al., this issue, p. 273), we expected that whatever we found would be contentious, celebrated by some and challenged by others. Debates about psychological theory and methods can be contentious enough, of course, but our topic’s unavoidable entanglement with politics, at a time when politics itself is so deeply contentious, led us to gird ourselves for a heated debate about the validity and meaning of our findings, however they turned out.
Because of this, our approach has been deliberately agnostic. We approached this project from the outset as an empirical test of two competing hypotheses: the asymmetry hypothesis (predicting greater partisan bias in conservatives than in liberals) and the symmetry hypothesis (predicting similar levels of partisan bias in liberals and conservatives). We had no a priori preference or expectation for either hypothesis and did our best to fairly review the existing literature—which supports both positions. In fact, as much as we would like readers to view our agnosticism on the symmetry question as noble scientific restraint, much of it flowed from genuine uncertainty about what our meta-analysis might find. Our own lab has previously published findings generally consistent with both the asymmetry position (Liu & Ditto, 2013; Wojcik, Hovasapian, Graham, Motyl, & Ditto, 2015) and the symmetry position (Uhlmann, Pizarro, Tannenbaum, & Ditto, 2009).
Our methodological strategy was targeted. We examined one prototypical form of partisan bias: the tendency to evaluate otherwise identical information more favorably when it supports rather than challenges one’s political affinities. We chose this focus because experiments documenting this judgment pattern are common in the empirical literature and because it captures a familiar form of bias frequently bemoaned in everyday political discourse (i.e., “hypocritically” being more charitable to people, policies, and information on one’s own side of the political aisle than on the other). We restricted our analysis to those studies in which only the strongest inferences about bias could be drawn, that is, studies in which both liberal and conservative participants were asked to make targeted evaluative judgments about virtually identical pieces of information carefully manipulated to be either politically congenial or politically uncongenial.
Our study uncovered evidence consistent with the symmetry hypothesis. Across 51 studies varying widely in the political topics they examined and the specific methods they used, both liberal and conservative participants showed a robust tendency to find otherwise identical information more valid and compelling when it confirmed rather than challenged their political affinities. This tendency was no more pronounced on one side of the political aisle than on the other.
In short, we are confident in the conceptualization and conduct of our meta-analysis and that the reported results accurately capture the extant experimental literature on this particular form of partisan bias. We are equally confident, however, that our findings do not settle the ideological symmetry debate and that many fascinating questions regarding the relative judgmental virtues of liberals and conservatives remain to be addressed.
In that spirit, we appreciate the vigorous critique of Baron and Jost (2019). We find their analysis and conclusions flawed in a number of ways but believe that their comments serve well to highlight crucial methodological and conceptual issues that need to be further debated and clarified, and we welcome the chance to start that process here.
We address each of Baron and Jost’s critiques and then turn our attention to future research directions. We discuss what Baron and Jost referred to in their Supplemental Material as “anomalous outcomes” of our analyses in our own Supplemental Material, available online. A key theme of our response is the importance of moving beyond the framing of the ideological symmetry debate as a unidimensional question of whether one side of the political aisle is more biased than the other. That framing is seductive, and we admit that our own writing falls prey to it as well, but it promotes a perspective that is both overly political and insufficiently nuanced as psychological analysis. Instead, we suggest a more multidimensional path forward, one that recognizes that bias can occur at multiple points in the information-processing sequence, that “liberal” and “conservative” are broad self-descriptions that contain multiple and distinct political and psychological elements, and that contextual factors play a crucial role in how, when, and in whom political biases manifest themselves.
Addressing Critiques
Baron and Jost present three primary critiques of our meta-analysis: (a) that the studies we examined are not representative of political judgments in the real world but rather were biased toward finding symmetrical bias among liberals and conservatives, (b) that the studies we examined do not actually measure partisan bias but rather demonstrate rational Bayesian updating, and (c) that the fields of social and political psychology are not biased in favor of more positive views of liberals but rather toward constructing a false narrative of equivalence between liberals and conservatives. We address each critique in turn.
Is bias research biased?
Baron and Jost claim that the studies we examined involved topics that are not “statistically representative of the entire population of ideological differences in public opinion” (p. 296). Instead, they argue that the researchers who performed the studies were motivated to select topics that would produce symmetrical patterns of bias. Although speculative, these points are worth considering.
First, Baron and Jost correctly note that the overwhelming majority of the studies included in our meta-analysis were not designed to test the symmetry-asymmetry question. Most did not even report separate results for liberals and conservatives (we had to e-mail the authors for the necessary data). It is possible that researchers disinterested in comparing the two sides’ proclivity toward biased judgment would gravitate, either intentionally or inadvertently, toward topics that they believed would evoke equal bias on both sides. It seems equally plausible, however, that predominantly liberal researchers might gravitate toward topics that would downplay liberal bias, either because of an active desire to portray conservatives unfavorably or more likely because conservative bias would be easier to recognize because of the difficulty people have in detecting their own judgmental shortcomings (Pronin, 2007). 1 Of course, any suggestion that the body of research we examined was systematically biased toward either symmetry or asymmetry is mere conjecture. One advantage of a meta-analysis is the ability to derive conclusions from the work of multiple researchers who are unlikely to all share a consistent set of motivations or methodological blind spots.
In our view, that the majority of the included studies were conducted without our core hypotheses in mind is a strength of our meta-analysis rather than a weakness. The questionable research practices that lie at the very heart of our field’s current replicability crisis arise when researchers have favored hypotheses that influence their design and analysis choices (e.g., Simmons, Nelson, & Simonsohn, 2011). Because very few of the included studies were conducted with the goal of comparing the quality of liberal and conservative judgments, concerns about directional motives of researchers are minimized.
But even if a researcher were explicitly interested in comparing the magnitude of bias in liberals and conservatives, a fair test of that question would have to establish conditions under which both sides have similar motivations and equal opportunity to demonstrate bias. We have little doubt that researchers selected topics for which they thought liberals and conservatives were likely to have similar but oppositional preferences (e.g., for vs. against capital punishment), but symmetrical preferences need not manifest as symmetrical bias. Democrats and Republicans could have similar proclivities to favor their own political parties, but when presented with a particular policy, Democrats could evaluate the policy based entirely on the specific content and stated consequences of the policy, whereas Republicans could rely heavily on party cues. It was just this sort of effect—differential bias in response to conditions with similar political significance for each group—that our meta-analysis was intended to isolate and examine.
The strategy Baron and Jost offer to address their concerns is to examine a “sufficiently large” sample of topics such that those issues will be representative of all political topics. We doubt that it is practical or even possible to define the entire population of political opinions or to specify a statistically representative sample of those opinions that would be resilient over even relatively short periods of time and that all or even most psychologists (and political scientists) would agree on. Our meta-analysis included judgments about many of the most contentious topics in modern political discourse, including capital punishment, gun control, abortion, welfare, health care, global warming, same-sex marriage, affirmative action, immigration, education, abstinence education, tax policies, presidential behavior, outsourcing, campaign tactics, and medical marijuana (see Table 1 in Ditto et al., 2019). We have no evidence to suggest that this list is representative of all possible political topics, but it hardly seems like what Baron and Jost describe as “a very small and unrepresentative set of issues that were handpicked in part to avoid ideological asymmetries” (p. 296). As we note in our original article, there is good reason to believe that the magnitude of partisan bias—both overall and its relative strength in liberals and conservatives—differs across political topics and over time, and these differences can offer important clues about determinants and boundary conditions (Crawford, 2012; Federico & Malka, 2018). But given the robustness of our findings across the wide range of topics examined, we believe the most scientifically reasonable position at this point in time is that ideological symmetry looks more like a feature of this particular form of partisan bias than a bug. 2
Is partisan bias rational?
Baron and Jost also claim that the studies in our meta-analysis do not demonstrate partisan bias at all but rather are simply evidence of rational (Bayesian) belief updating. According to Baron and Jost, it is “perfectly rational to evaluate new information on the basis of prior beliefs” (p. 294), and so the results from the studies we examined can be seen as irrelevant to issues of partisan bias.
Disentangling bias from rationality (both conceptually and empirically) is an old and venerable issue in psychology, one that researchers studying perceptual and judgmental biases have wrestled with for decades (see Ditto, 2009, for a detailed treatment of this history, but also Erdelyi, 1974; Miller & Ross, 1975; Nisbett & Ross, 1980; Tetlock & Levi, 1982). The issue has also received considerable attention in the specific context of political judgment and decision making (e.g., Bullock, 2009; Gerber & Green, 1999; Kahan, 2016; Taber, Cann, & Kucsova, 2009; Taber & Lodge, 2006). Therefore, Baron and Jost were right to raise this interpretational ambiguity as an issue to be considered in evaluating the results of our meta-analysis. That is why we raised it throughout our article as well. 3 In fact, rather than “ignoring this crucial caveat” as Baron and Jost claim, our meta-analytic approach was structured explicitly to address and minimize this very problem. If we failed to explain sufficiently how we addressed this problem in our original article, let us try to clarify that here.
Bayesian reasoning is a normative account of how beliefs should be updated in response to new information. Its central insight is that the updating of beliefs based on new information must be considered in the context of prior beliefs. According to Bayes’s theorem, this is accomplished by multiplying one’s prior probability that a given belief is correct by the likelihood ratio associated with the new information (which can be roughly equated with the perceived validity of the new information) to generate a new belief (the posterior probability in Bayesian terms). Baron and Jost argue on the basis of this logic that the pattern of results we forwarded as evidence of partisan bias is simply evidence of rational information processing. If a scientific study (or any other new piece of information) supports your prior beliefs about a topic, then it is rational to assume that it is a valid piece of information, but if that information contradicts your prior beliefs, then it is rational to assume that it is an invalid piece of information.
This kind of normative counterexplanation (not always strictly Bayesian but always involving a rationality-based analysis) has historically been invoked to explain data ostensibly demonstrating various types of biased judgment, including motivated reasoning (Ditto, 2009). Individual studies have effectively refuted normative accounts of motivated reasoning using a variety of methodological strategies, including equating prior beliefs across conditions (Ditto & Lopez, 1992; Ditto, Munro, Apanovich, Scepansky, & Lockhart, 2003; Ditto, Scepansky, Munro, Apanovitch, & Lockhart, 1998) and demonstrating that—inconsistent with an account based on cold, rational information processing—the tendency to derogate the validity of information that challenges one’s beliefs or desires is mediated by affect (Ditto et al., 2003; Munro & Ditto, 1997). Because this kind of tight methodological control is not possible when meta-analyzing existing data, however, we took a different approach. We used two inclusion criteria to restrict the studies we examined to those that were least amenable to a rational belief-updating account.
First, we restricted our analysis to studies in which the measure of partisan bias involved a specific assessment of the validity/quality of the new information. A frequently overlooked nuance of Bayes’s theorem is that it directly accounts for the effect of priors on the posterior probability, but it is silent on how the likelihood ratio of new information should be determined. In other words, it is perfectly rational for one’s belief in some proposition after exposure to new information to be influenced by one’s prior level of belief in that proposition (see Note 2 from Ditto et al., 2019, p. 287). This rational belief-updating process (which involves the effect of prior beliefs on postinformation belief—also known as posterior probability) should not be confused, however, with the less rational process of using consistency with one’s priors to assess the validity of the new information (i.e., the likelihood ratio; Kahan, 2016). There are independent truth-convergent criteria for evaluating the validity of information, and an individual who fails to incorporate these criteria into their judgments cannot be said to be acting in a fully rational fashion (Kahan, 2016). An individual who judged the validity of new information solely on the basis of its fit with prior beliefs would disregard high-validity information that contradicted those beliefs, information that should, rationally, lead them to update those beliefs. In the extreme, this would produce a completely closed cognitive system in which people accept as true any information that reinforced current beliefs without a mechanism for credible belief-inconsistent information to alter those beliefs (in Piagetian terms, all assimilation with no accommodation). Over time, this would lead to more extreme or rigid beliefs rather than more correct ones. 4
The methodological implication of this normative analysis is that the more directly the dependent measure used in a given study captures specific validity evaluations of new information (judgments about the likelihood ratio) rather than general belief updating (judgments about the posterior probability), the less susceptible the findings are to normative counterexplanation.
In arguing their point, Baron and Jost quote the following from our article: Studies needed to measure participants’ evaluation of the validity, quality, or acceptance of the matched politically congenial and politically uncongenial information. Examples of information evaluation measures included ratings of a scientific study’s methodological quality, approval or disapproval of a political actor’s behavior, and endorsement of specific policy proposals presented in the stimulus materials. (Ditto et al., 2019, p. 277)
However, they fail to point out the next line stating that “studies were not included if their only evaluation measure was endorsement of a general political attitude (e.g., attitude toward capital punishment after reading a study on capital punishment) given the vulnerability of general attitudinal measures to normative counterexplanation” (p. 277).
We built this distinction between the measurement of general beliefs versus specific validity judgments into the fabric of our meta-analytic approach. To minimize the susceptibility of our findings to normative counterexplanation, we excluded studies that measured only general beliefs about a topic after exposure to new (politically congenial or uncongenial) information, restricting our analysis to studies in which participants judged the quality or validity of that new information. If individuals evaluate the same new information less favorably when it has politically uncongenial implications than when it has politically congenial implications, then a fully rational account is hard to maintain. 5
This is particularly evident if we also consider a second inclusion criterion of our meta-analysis: The evaluated information had to be virtually identical in both the congenial and uncongenial conditions (see the Manipulation of Political Congeniality section in Ditto et al., 2019, pp. 276−277). In other words, the studies we examined in our meta-analysis all show the same basic effect: Political partisans evaluate the exact same study methods, policy, or behavior differently depending on its political implications. There are two key reasons to interpret this pattern as evidence of bias.
First, the field has consistently accepted such findings as evidence of biased judgment. The matched-information design is arguably the most common design used in the judgment and decision-making literature. Research on framing effects (Kahneman & Tversky, 1984; Tversky & Kahneman, 1981), mental accounting (Kahneman & Tversky, 1984; Thaler, 1999), and omission bias (Spranca, Minsk, & Baron, 1991) all report differential judgments of identical information after decision-irrelevant manipulations as evidence of nonnormative judgment. More poignantly, within stereotyping and prejudice research, the evidence forwarded for the existence of racial, gender, and ethnic bias is often the differential evaluation of identical information (e.g., identical behavior, essays, test performances, job resumes, medical records) depending on the demographic characteristics of the actor (e.g., Bertrand & Mullainathan, 2004; Darley & Gross, 1983; Duncan, 1976; Goldberg, 1968; Green et al., 2007). If an individual evaluates the identical essay as lower in quality when written by a female than by a male, a fully rational account of this difference is again hard to justify.
Second, everyday people agree with experimental psychologists that findings such as ours are evidence of bias. In the published version of his Nobel Prize acceptance address, Kahneman (2003) suggests that one way he and Tversky approached questions of rationality was to rely on subjective assessments of decision makers themselves (p. 702). Presented with the famous Asian disease problem (Tversky & Kahneman, 1981), for example, people generally recognize that whether identical numerical outcomes are framed in terms of gains or losses should not rationally affect how those outcomes are evaluated.
To examine how laypeople perceive the experimental evidence in our meta-analysis, we presented 401 U.S. Amazon Mechanical Turk workers with one of four different types of experimental results and asked whether those results demonstrated bias (full details of the methods and results of this study are presented in the Supplemental Material). The four results were as follows: (a) evaluating the methods of a research study more favorably when it supports rather than opposes your political views (e.g., Lord, Ross, & Lepper, 1979); (b) approving of a policy more when the policy is proposed by one’s own political party than by the opposing party (e.g., Cohen, 2003); (c) using more lenient standards to evaluate the behavior of a member of one’s political in-group than one’s political out-group (e.g., Kahan, Hoffman, Braman, & Evans, 2012); and (d) evaluating the same essay more positively when it was written by a male than a female (e.g., Goldberg, 1968).
In each case, more than 90% of participants viewed the result in question as evidence of biased judgment. 6 This was as true of the scientific methodology example (98%) as of the essay evaluation example (95%), and no differences were found across self-reported ideology. Thus, both judgment and decision-making researchers and the very people who display these patterns of judgment seem to agree that matched-information designs reveal patterns of judgment that are best characterized as bias. Moreover, as we allude to both in our original article and earlier in this response, the form of partisan bias that is the focus of our meta-analysis is very much the kind of bias that both lay and professional political observers seem to complain about most vociferously. The rampant “whataboutism” heard in everyday political discourse on both sides of the aisle (“What about Obama/Trump? You didn’t complain when he went golfing!”) is grounded in the differential treatment of similar behavior. Thus, this particular form of bias is an important contributor to real-world political conflict.
Still, it is important to reiterate the genuine challenges of normative analysis and what can and cannot be expected of everyday lay judgment. In some situations, particularly situations in which individuals have little knowledge or expertise, it is reasonable for people to use their priors to evaluate new information. Policy assessments (e.g., Cohen, 2003) are potentially vulnerable to this counterexplanation (see Ditto et al., 2019, p. 284, last para.). Faced with evaluating an unfamiliar policy, it seems a reasonable heuristic strategy to rely on the endorsement of a politician or party one trusts.
However, for many studies included in our meta-analysis, a normative account must be stretched to breaking to explain the observed patterns of judgment. Consider the following three examples:
The same ballot mark is judged as a legitimate vote if cast for one’s own candidate but as illegitimate if cast for an opponent (Kopko, McKinnon, Budziak, Devine, & Nawara, 2011).
The same dirty campaign trick is judged as more wrong if used by the opposing political party than by one’s own party (Claassen & Ensley, 2016).
The identical behavior observed in a video of a protest is seen as more closely fitting the legal definition of intimidation if the protest is for a cause the participant opposes rather than supports (Kahan et al., 2012).
In each of these cases, the combination of a vague and general prior belief (essentially boiling down to “my side is composed of better people than the other side”) and a very targeted and specific evaluative judgment make a normative account implausible. A determined Bayesian might argue that people typically expect candidates of their favored party to get more votes than their opponents, but it is much more difficult to defend the rationality of judging the exact same “hanging chad” as a valid vote if it was thought to be for one’s favored candidate but invalid if cast for a disfavored one. Likewise, one might reasonably believe that members of the opposing political party are less ethical than members of one’s own party (and thus more prone to dirty campaign tricks), but it is much less reasonable to then judge the identical dirty trick as dirtier if committed by a political foe than a political ally.
In summary, the history of bias research reveals that normative counterexplanations are hard to shake completely (see Ditto et al., 2019, p. 285, second para.). This is particularly true in the context of a meta-analysis in which the researcher cannot exert direct control over the methods used in the included studies. Normative analysis itself can also be challenging and complex, with researchers frequently disagreeing about what is rational and why. Baron and Jost argue that the research included in our meta-analysis is irrelevant to issues of political bias and thus to the question of ideological symmetry. We believe instead that there is important information to be gained about individuals’ susceptibility to partisan bias by carefully mining the existing experimental literature and that dismissing research examining differential reactions to politically congenial versus politically uncongenial information is overly restrictive. Meta-analysis, like any particular research strategy, has both strengths and limitations. The goal of our project was to embrace those strengths (e.g., the ability to examine judgments about a wide range of political topics on the basis of data collected from different populations, methods, and researchers) while working to minimize the concomitant interpretational limitations (e.g., by carefully selecting studies least susceptible to Bayesian and other normative counterexplanations). We believe we struck this balance well and that the findings we report add an important piece to the puzzle of partisan bias.
Is social psychology liberally biased?
Baron and Jost also devote a section of their response to disputing the notion that liberal bias is a problem in social psychological research and argue instead that a more severe threat to the field is the pressure to create false equivalencies between the left and right. Liberal bias in psychology was not the focus of our meta-analysis. However, we did raise the possibility that the field’s emphasis on conservative bias may reflect in part the challenges a liberal-dominated field would have in detecting the flaws of liberal thinking. So, let us address Baron and Jost’s comments with three brief points.
First, there is little dispute that the field of social psychology is composed overwhelmingly of political liberals (Inbar & Lammers, 2012; von Hippel & Buss, 2017). Given decades of accumulated evidence documenting various forms of implicit and explicit in-group favoritism (including the research reviewed in our meta-analysis), the simple fact that our field is composed almost exclusively of individuals of any single political persuasion calls for vigilance regarding the potential this creates for bias to creep into the research process (Duarte et al., 2015).
Second, data suggest the field may be vulnerable to liberal bias. A recent analysis of a large sample of social psychology abstracts showed both that conservatives were described in more negative terms than liberals and that conservatives and conservatism were portrayed as more in need of explanation than liberals and liberalism (Eitan et al., 2018). This builds on research from the 1970s and 1980s showing that liberal psychologists rated a manuscript as higher in quality and more publishable when the results were favorable rather than unfavorable toward a leftist group (Abramowitz, Gomes, & Abramowitz, 1975) and that institutional review boards approved research proposals less often when they hypothesized discrimination against White men than discrimination against ethnic minorities (Ceci, Peters, & Plotkin, 1985). Perhaps most telling, the majority of social psychologists explicitly reported that they would discriminate against conservatives (at least somewhat) in graduate admissions, faculty hiring, manuscript reviews, and grant reviews (Inbar & Lammers, 2012). 7
Finally, that social psychology as a field should be concerned about the potential for liberal bias in no way mitigates concerns about the potential for other types of bias to influence our practices and conclusions. A bias toward finding equivalence between liberals and conservatives—if it exists—would be as problematic as a bias against liberals or conservatives (see Ditto et al., 2019, p. 284). Although we put little stock in the use of an anecdote by Baron and Jost to support their argument (see p. 299), we also suspect that some researchers may indeed be more comfortable finding equivalence rather than difference across political and demographic groups, particularly on such valued characteristics as rationality or intelligence. A preference for patterns of equivalence across groups may even be one way that liberal bias manifests itself (Winegard, Clark, Hasty, & Baumeister, 2018).
We reiterate our position that bias of any kind poses a threat to scientific validity and that the new ethos of methodological rigor taking hold in the field can be as helpful in the realm of political psychology as in all other areas of psychological science (and science in general for that matter).
Future Research
Baron and Jost clearly disagree with us about the value of our meta-analytic findings. We have now both made our case on that score, and readers can decide what arguments they find most compelling. At this point, the most fruitful path forward is to think through how future research might advance our understanding of political reasoning, not only to further adjudicate the relation between political orientation and judgmental bias but also to clarify crucial conceptual and methodological issues that can impede or facilitate research progress.
One path that we believe will not be productive for future research is a focus on documenting the relative accuracy of liberal and conservative beliefs. An important subtext that runs throughout Baron and Jost’s response is that liberals can be said to be less biased than conservatives based on the fact that liberal beliefs are closer to factual truth than conservative beliefs. Even those who have sympathy for some variant of this argument can recognize the significant challenges of pursuing it empirically. More importantly, documenting the accuracy of political beliefs offers little insight into psychological process. Knowing that one side’s beliefs are more accurate than the other side’s beliefs tells us precious little about why those differences in accuracy exist.
This point is well illustrated by considering two hypothetical ways that liberals and conservatives might develop differential factual beliefs about a political topic. One possibility is that both liberals and conservatives base their judgments on an identical body of information but process that identical information in different ways and thus reach different conclusions. This is the possibility tested by the studies included in our meta-analysis. The studies all examine partisan bias under controlled experimental circumstances in which differences in prior information are controlled by confronting participants with closely matched scenarios differing only in the partisan attachments they evoke. Concerns about potential normative counterexplanations point out the limitations of the studies in achieving this ideal informational equivalence, but to the extent that the studies successfully minimize the influence of prior beliefs and other confounding factors, any differences in beliefs are attributable solely to biased information processing.
But there is another obvious way that liberals and conservatives could develop different beliefs. Even if political partisans processed information in a completely unbiased manner, their beliefs could still differ because the information bases they derive those beliefs from differ. Such informational differences could theoretically account for differential beliefs without the need to assume any differential (i.e., biased) processing of information once that information is received. Liberals and conservatives could have different beliefs solely because their beliefs are based on different bodies of information.
What is important to note here is that documenting that liberals and conservatives have different beliefs tells us nothing about which of the two accounts above (or some other) explains those differences. Differential beliefs could result from differential processing of the same information or similar processing of differential information, or some hybrid account. (The most plausible account in our minds based on the available data is that the differential beliefs of liberals and conservatives are a function of both biased processing and selective exposure tendencies.)
The same analysis can be extended to a situation in which one set of differential beliefs (e.g., the ones held by liberals) is shown to be more factually accurate than the other. Documenting that the beliefs of liberals and conservatives differ in accuracy does nothing to disentangle whether those differences are explained by differences in the degree of bias the two sides show in their processing of information, differences in the extent to which the two sides expose themselves selectively to belief-supportive information, or even perhaps whether the two sides differ neither in biased processing nor selective exposure tendencies but that the information presented by one side’s media is more accurate than the information presented by the other side’s media. 8
A similar multifaceted approach can be directed toward isolating contributors to biased processing itself. Political events occur daily, even hourly, but elections—the only events that galvanize many people’s attention on political considerations—occur only every few years. For that reason, memory processes (e.g., “Did the economy do well or poorly during Obama’s time in office?”) likely play an important role in many real-world political judgments. It is possible that memory reconstruction provides more fertile ground for partisan biases to emerge (Frenda, Knowles, Saletan, & Loftus, 2013) than the online processing of political information explored by the experiments included in our meta-analysis. The overall pattern of partisan bias we examined—more favorable judgments of politically congenial than politically uncongenial information—can also be separated into two components: the uncritical acceptance of politically congenial information and the defensive rejection of politically uncongenial information (see Garrett & Stroud, 2014, for a similar approach to selective exposure processes). These two components could be fully or partially dependent on different psychological mechanisms. Teasing apart this distinction was beyond the scope of our meta-analysis, but future research should explore these processes independently to identify both common and unique contributors to each.
There are two crucial points we wish to make here. The first is that accuracy and rationality are different things—accuracy refers to judgment outcomes, whereas rationality refers to judgment processes—and exploring the factual accuracy of judgments will do little to uncover the psychological processes that contribute to the different (and likely even differentially accurate) factual worlds inhabited by liberals and conservatives. This is not to say that exploring judgments in which normative criteria apply cannot be a helpful research strategy. For example, presenting participants with outcome tables from which an objective correlation coefficient could be calculated has been used to document how greater cognitive sophistication can encourage rather than mitigate partisan bias (Kahan, Peters, Dawson, & Slovic, 2017).
In the same vein, Baron and Jost point to the potential utility of studies examining sensitivity to argument quality (Stanovich & West, 1997, 1998). Research on motivated reasoning outside of a political context has used this technique to demonstrate biased processing, showing that people are more sensitive to the quality of arguments for nonpreferred judgment conclusions than for preferred ones (Ditto et al., 1998). 9 In both of these examples, however, it is important to recognize that the behavior of participants is being compared using standards of rationality (whether a given piece of information is used appropriately according to normative or even mathematical standards of judgment), not standards of accuracy (whether the belief participants ultimately express matches some standard of independently verifiable truth). Even comparing the accuracy of political beliefs “issue by issue,” as Baron and Jost suggest, would tell us very little about which of the many different psychological processes described above might have contributed to any inaccuracy we might find.
The second and equally important point we wish to make here, however, is that distinguishing inaccuracy from bias makes clear that partisan bias is not one thing but many. The “alternative facts” possessed by liberals and conservatives may differ (both from each other and/or from reality) because of the biased interpretation of political information, biased memory for political information, biased exposure to political information, and so on. Liberals and conservatives could differ in their susceptibility to all, some, or none of these biases, and research is needed to carefully explore each of these unique psychological processes as a potential contributor to the growing fact gap between liberals and conservatives.
It is similarly beneficial to recognize the complexity of other aspects of the ideological symmetry issue as well. A recent review makes a powerful case for the “contingent, contextual” relation between individual difference factors and political attitudes (Federico & Malka, 2018). For example, considerable data suggest that the needs for security and certainty that Jost and others (Jost, 2017; Jost, Glaser, Kruglanski, & Sulloway, 2003) take as primary evidence for ideological asymmetry are associated more reliably with social conservatism than economic conservatism (Feldman & Johnston, 2014; Janoff-Bulman, Sheikh, & Baldacci, 2008; Johnston & Wronski, 2015). The relation between these needs and the endorsement of a particular political ideology is also contingent on national context (Federico & Malka, 2018). For example, research in Western countries finds that variables related to the needs for security and certainty show the expected positive relation with conservative ideology, but in formerly Communist countries (which until relatively recently were dominated by governments espousing egalitarian ideologies) the data show these needs to be most pronounced in individuals who associate themselves with left-wing rather than right-wing politics (Malka, Soto, Inzlicht, & Lelkes, 2014; Thorisdottir, Jost, Liviatan, & Shrout, 2007).
All of this complexity suggests that future research will move forward best when it moves beyond conceptualizing partisan bias as a unitary construct with a context-independent and historically invariant relation to political orientation. There are many varieties of partisan bias to be explored, many political inclinations underlying the liberal-conservative continuum to be disentangled, and many contextual factors that could exacerbate or mitigate partisan attachments and animosities and thus alter how, when, and in whom partisan bias reveals itself most strongly. Pursuing this complexity could potentially result in a more flattering portrait of some political ideologies than others, or the picture that emerges may be more mixed and conditional. But whatever the data hold, the image will be a psychological one, rich in nuance, varied in tint and hue, rather than a political one, colored only in shades of red or blue.
Final Remarks
To us, it seems ironic and more than a little bewildering that social psychologists are drifting into this relativistic view of morality and politics just as authoritarian conservatism (and illiberal hostility to democratic norms) seem to be reaching new heights of popularity and brazenness not only in Trump’s America but also in Erdogan’s Turkey, Orban’s Hungary, and Netanyahu’s Israel. (Baron & Jost, 2019, p. 293)
Baron and Jost are clearly troubled by recent political developments in the United States and worldwide. We are too. We fail to see the irony, however, in separating our pursuit of scientific understanding from our feelings as concerned citizens of both the United States and the world.
The struggle to maintain scientific objectivity is particularly crucial for political psychology (Tetlock, 1994), and all of us who study it must work, at every step throughout the research process, to maintain as clear a firewall as we can manage between the prescriptive world of politics and the descriptive world of psychological science. Baron and Jost contend that our research suggests a “relativistic view of morality and politics.” But any equivalence we argue for is psychological not moral. Our findings have nothing to do with who is currently the leader of Turkey, Hungary, Israel, or the United States. It is similarly a mistake if readers take the findings of our meta-analysis as a condemnation of political liberalism or as a vindication of political conservatism. Our research should be viewed through a purely descriptive lens.
It should surprise no reader of our current exchange with Baron and Jost that we emerge from it convinced of the quality of our methods and the value of our findings. One goal of this response, however, was to be more precise about the nature of our claims and to situate our findings in a broader consideration of the complex psychological, social, and historical dynamics of partisan reasoning. If our original article encouraged a more expansive interpretation of our findings than is warranted, we hope to have modulated that interpretation here.
We would be similarly unsurprised if, after reading our response to theirs, Baron and Jost remain convinced of the incisiveness of their critique and maintain their conviction that political conservatism is associated with biased reasoning and inaccurate beliefs. Our hope, however, is that this exchange may lead Baron and Jost to modulate their claims as well. Research documenting particular ideological asymmetries in motivated social cognition is compelling, but so too is a growing body of research questioning the generality of these asymmetries (e.g., Brandt, Reyna, Chambers, Crawford, & Wetherell, 2014; Brandt, Wetherell, & Reyna, 2014; Collins, Crawford, & Brandt, 2017; Conway et al., 2016; Conway, Houck, Gornick, & Repke, 2017; Federico & Malka, 2018; Frimer, Skitka, & Motyl, 2017; Malka, Lelkes, & Holzer, 2017; Nisbet, Cooper, & Garrett, 2015; Pennycook & Rand, 2019; Van Hiel, Onraet, & De Pauw, 2010; Washburn & Skitka, 2018). We believe strongly that a key impediment to future research progress, an impediment that our original article contributed to unfortunately, is to continue to frame the ideological symmetry issue as a simple question of which side, liberals or conservatives, is more biased. Research will progress most effectively once proponents of both the symmetry and asymmetry positions recognize the multifaceted, context-sensitive, and historically bounded nature of partisan bias.
When two sets of researchers offer what seem to be conflicting views of a phenomenon, the question that inevitably emerges is which of the two is right. But the question we entertained in our original article, and have yet to revisit in this one, is what if we both are right? What if there is good empirical evidence for both of our contentions: that conservatism is associated with a relatively simple and rigid thinking style, but also that, when placed under careful laboratory conditions, liberals and conservatives show similar levels of partisan bias?
We discussed several potential resolutions in our original article, but let us expand on one here that we mentioned previously only in passing. What if bias is not the sole province of the simpleminded? A key assumption underlying Baron and Jost’s position is that the cognitive style of conservatives makes them more susceptible to partisan bias, or stated in the opposite way, that the more systematic thinking style of liberals protects them from it. But this view of integratively simple thinking as cognitively inferior has been questioned (e.g., Kahan, 2016; Tetlock, Armor, & Peterson, 1994; Tetlock & Tyler, 1996). Motivated reasoning involves both systematic and heuristic reasoning processes (Chaiken, Giner-Sorolla, & Chen, 1996; Ditto & Lopez, 1992; Ditto et al., 1998; Giner-Sorolla & Chaiken, 1997), and a growing body of research suggests that greater cognitive sophistication and expertise often predicts greater levels of political bias, not less (Kahan, 2013; Kahan et al., 2017; Liu & Ditto, 2013; Nyhan, Reifler, & Ubel, 2013; Taber & Lodge, 2006; Vallone, Ross, & Lepper, 1985). Cognitive sophistication may allow people to more skillfully argue for their preferred conclusions, thus improving their ability to convince others—and themselves—that their beliefs are correct (Mercier & Sperber, 2011).
It is possible then that even if liberals have a more thoughtful cognitive style than conservatives, it may offer them little protection against biased judgment, and in some cases, could actually operate in service of it. If true, the inconsistency between our findings and Jost’s body of work begins to dissolve, and the focus of future research shifts toward exploring whether and how bias might operate differently in people high and low in cognitive reflection (or under conditions that promote or degrade an individual’s capacity for effortful cognitive analysis).
In our view, the most important point from our research is that people across the political spectrum are vulnerable to partisan bias. This fits with a wealth of research suggesting that motivated reasoning and in-group favoritism are not mere bugs in our cognitive system but rather fundamental features of human thought (Brown & Kobayashi, 2002; Kunda, 1990; Mercier & Sperber, 2011; Sedikides, Gaertner, & Vevea, 2005; Stanovich, West, & Toplak, 2013; Tajfel, 1970). It also suggests that vigilance regarding one’s own susceptibility to partisan favoritism is an appropriate epistemic stance for anyone engaged in the political arena.
In the current political climate, it is tempting for liberals, and perhaps especially liberal scientists, to see themselves (on the basis of an implicit connection between thoughtfulness and protection from bias) as a rational baseline to be compared with conservative irrationality (e.g., Eitan et al., 2018). If our field’s current reproducibility crisis has taught us anything, however, it is the vulnerability of even smart, experienced, and well-intentioned individuals to biased reasoning.
Supplemental Material
Ditto_Supplemental_Material – Supplemental material for Partisan Bias and Its Discontents
Supplemental material, Ditto_Supplemental_Material for Partisan Bias and Its Discontents by Peter H. Ditto, Cory J. Clark, Brittany S. Liu, Sean P. Wojcik, Eric E. Chen, Rebecca H. Grady, Jared B. Celniker and Joanne F. Zinger in Perspectives on Psychological Science
Footnotes
Action Editor
Brad J. Bushman served as action editor and June Gruber served as interim editor-in-chief for this article.
Author Contributions
P. H. Ditto and C. J. Clark contributed equally to this article. All the authors approved the final manuscript for submission.
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
