Abstract
Both liberals and conservatives accuse their political opponents of partisan bias, but is there empirical evidence that one side of the political aisle is indeed more biased than the other? To address this question, we meta-analyzed the results of 51 experimental studies, involving over 18,000 participants, that examined one form of partisan bias—the tendency to evaluate otherwise identical information more favorably when it supports one’s political beliefs or allegiances than when it challenges those beliefs or allegiances. Two hypotheses based on previous literature were tested: an asymmetry hypothesis (predicting greater partisan bias in conservatives than in liberals) and a symmetry hypothesis (predicting equal levels of partisan bias in liberals and conservatives). Mean overall partisan bias was robust (r = .245), and there was strong support for the symmetry hypothesis: Liberals (r = .235) and conservatives (r = .255) showed no difference in mean levels of bias across studies. Moderator analyses reveal this pattern to be consistent across a number of different methodological variations and political topics. Implications of the current findings for the ongoing ideological symmetry debate and the role of partisan bias in scientific discourse and political conflict are discussed.
We asked 951 U.S. visitors to the website YourMorals.org (http://www.yourmorals.org) how well they thought the term “biased” described the average Democrat and the average Republican. Respondents describing themselves as Democrats saw the average Republican as substantially more biased than the average Democrat. Republican respondents expressed the mirror image belief that the average Democrat was substantially more biased than the average Republican (see Fig. 1). 1

Mean level of agreement with how well the term “biased” describes the average Democrat and the average Republican, presented separately for Democratic and Republican participants (N = 951). Error bars indicate ±1 SE.
This finding should be unsurprising to even a casual observer of contemporary U.S. politics. A few hours watching cable news or reading accounts of political events on any of hundreds of partisan websites will reveal a pervasive narrative in which political allies are characterized as rational, informed, and reasonable, whereas political opponents are described as irrational “low-information voters” blinded by partisan bias. These recriminations are distinctly mutual, to the point that politicians and pundits from both the left and right rely on the same colorful phrases to capture how the other side is “drinking the Kool-Aid” (e.g., Huffington, 2002; O’Reilly, 2005) or suffering from one form or another of “derangement syndrome” (Horowitz, 2008; Krauthammer, 2003; Raimondo, 2016).
In this article, we take such reciprocal accusations of partisan bias as our starting point and examine whether either side’s accusations are correct. Is there empirical evidence to support the contention that one side of the political aisle is more biased than the other? Or is partisan bias a bipartisan problem, or perhaps little problem at all?
Assessing the magnitude of partisan bias across the political spectrum is a challenging task, ill-suited to examination in a single survey or experiment. Consequently, we report a targeted meta-analytic comparison of the magnitude of one particular variety of partisan bias in liberals and conservatives—the tendency to evaluate otherwise identical information more favorably when it supports one’s political beliefs or allegiances than when it challenges those beliefs or allegiances—examining results from 51 different experimental tests involving more than 18,000 participants.
Defining Partisan Bias
At the broadest level, partisan bias refers to a general tendency for people to think or act in ways that unwittingly favor their own political group or cast their own ideologically based beliefs in a favorable light. Politically involved individuals, of course, hold many beliefs that favor their chosen political party or ideology, and many engage in actions deliberately intended to promote the political groups they identify with and the political beliefs they hold. Our focus is on cases in which this favoritism is less conscious and intentional, such that people are generally unaware that their political affinities have affected their judgments or behavior. This kind of partisan bias can take many forms and can occur at multiple levels of the information processing sequence, including selectively exposing oneself to information that supports one’s own political group or views (e.g., Iyengar & Hahn, 2009; Stroud, 2008), selectively remembering information that supports one’s own political group or views (e.g., Frenda, Knowles, Saletan, & Loftus, 2013), and, most prototypically, selectively evaluating information in ways that support one’s own political group or views (e.g., Lord, Ross, & Lepper, 1979).
In real-world political discourse, partisan bias is often labeled as hypocrisy, in that the individual applies different (and harsher) standards when evaluating the judgments and behavior of political opponents than when evaluating similar or identical judgments or behaviors displayed by political allies. Analogously, the classic approach to empirical examination of partisan bias is to ask participants to evaluate “matched” information: information that is as identical as possible in every way except that in one case it favors the participant’s political affinities (politically congenial information), and in the other it challenges those affinities (politically uncongenial information). For example, Lord et al. (1979) recruited participants with strong attitudes either in support of or in opposition to capital punishment and asked them to rate the methodological quality of fictitious but realistic empirical studies examining whether the death penalty deters homicide. Two versions of the studies were created: one with results supporting the deterrent efficacy of capital punishment, and one with results showing that capital punishment actually increased rather than decreased homicide rates. On average, participants (regardless of whether they supported capital punishment or opposed it) rated the studies as better quality research when the results supported their views on the efficacy of capital punishment than when they challenged those views, despite the fact that the methodologies of the studies were held constant across conditions and only the results were altered. Likewise, Cohen (2003) presented participants self-identifying as Democrats or Republicans with identical welfare policies that were said to be strongly supported by either the majority of congressional Democrats or the majority of congressional Republicans. Both Democratic and Republican participants expressed more positive views of the identical policy when it was ostensibly supported by members of their own party than by members of the opposition party.
These studies rely on a logic for demonstrating bias that is ubiquitous, albeit typically implicit, in psychological research and grounded in the logic of expected utility theory (von Neumann & Morgenstern, 1944). According to Kahneman and Tversky (1984; see also Keys & Schwartz, 2007), a fundamental axiom of all analyses of rational choice is the principle of invariance: Judgments should not be affected by trivial changes (i.e., those irrelevant to the decision) in the way information is presented. If a decision about otherwise identical alternatives is affected by whether those alternatives are presented in terms of the number of lives lost versus the number saved (Tversky & Kahneman, 1981), or if a judgment about otherwise identical behavior differs on the basis of whether it was enacted by an African American person or by a White person (Duncan, 1976), then some deviation from rationality (i.e., bias) is implied. Analogously, if the identical scientific study or policy proposal is evaluated differently depending on whether it reflects positively on liberals or conservatives, partisan bias is implied. The magnitude of that bias (i.e., the divergence between how that study or policy is evaluated when it is politically congenial vs. politically uncongenial) can be measured to determine whether the bias is significantly more pronounced for participants on the political left or the political right.
The simplicity of this analysis is belied by the fact that applying the logic of invariance in actual empirical studies faces a number of challenges (Kahan, 2016; Keys & Schwartz, 2007). Information supplied to participants must be experimentally manipulated and carefully matched to rule out inadvertent informational differences between conditions. This minimizes the possibility that the manipulated information (e.g., frame, race, politics) itself conveys relevant information that could plausibly account for differential judgments from a Bayesian or related normative perspective. Still, the difficulty of ruling out counterexplanations that are based on rational cognitive factors such as expectations (“priors” in Bayesian terms) has vexed research on motivated perception and reasoning for decades (Ditto, 2009; Erdelyi, 1974; Miller & Ross, 1975; Nisbett & Ross, 1980; Tetlock & Levi, 1982). Such counterexplanations are notoriously difficult to rule out completely, but their plausibility is reduced to the extent that (a) the politically congenial information and politically uncongenial information presented to participants are matched in every way possible except for their agreement with a participant’s political beliefs or allegiances, and (b) participants’ evaluations are specifically focused on the validity or quality of the matched information provided rather than on a general assessment of the information’s conclusion (“updating” in Bayesian terms).
Evidence for Asymmetrical Partisan Bias
Interest in locating bias along the political spectrum has deep roots in psychology, stretching back at least to work by Adorno and colleagues on the authoritarian personality (Adorno, Frenkel-Brunswik, Levinson, & Sanford, 1950). Although the specifics have evolved over the years (Altemeyer, 1981, 1996), the essential thesis of this research tradition is that deep-seated conflicts (psychodynamic and/or interpersonal) predispose some people to extreme “conservative” views characterized by conventionalism, antipathy toward minority groups, a preference for strong authoritarian leaders, and rigid black-or-white, good-or-bad thinking.
Recent research in political psychology has updated and reinforced this notion that conservative political views are tied to biased thinking and, in particular, resistance to novel or threatening information. For example, political conservatism has been described as a form of motivated social cognition associated with a host of personal dispositions related to resistance to change (dogmatism; low levels of openness to experience; and high need for order, structure, and closure; Jost, Glaser, Kruglanski, & Sulloway, 2003). Other work has found associations between conservatism and threat sensitivity (Hibbing, Smith, & Alford, 2014; Lilienfeld & Latzman, 2014), avoidant search strategies (Shook & Fazio, 2009), shallow system 1 thinking (Eidelman, Crandall, Goodman, & Blanchar, 2012), valuation of group loyalty (Graham et al., 2013), and self-enhancement motivation (Wojcik, Hovasapian, Graham, Motyl, & Ditto, 2015). All of these factors could plausibly manifest themselves as a stronger tendency among political conservatives than among political liberals to favor information that supports rather than challenges their political affinities.
Evidence for Symmetrical Partisan Bias
No analogous research tradition has championed a hypothesis of greater bias in liberals than in conservatives. There is, however, considerable theory and data to suggest that conservatives do not have a monopoly on bias. The psychological literature is replete with examples of motivated reasoning, particularly in the form of self- and group-enhancing biases, and these biases have been found in a multitude of different populations and contexts (Alicke, 1985; Billig & Tajfel, 1973; Darley & Gross, 1983; Ditto, 2009; Hastorf & Cantril, 1954; Kunda, 1990; Mercier & Sperber, 2011). There is little reason to expect political liberalism to provide immunity against motivated reasoning and some reason to expect that political and moral judgments in general may be particularly vulnerable to motivational and affective influence (Ditto, Pizarro, & Tannenbaum, 2009; Haidt, 2001).
More specifically, just as the “rigidity-of-the-right” hypothesis underlying work on the authoritarian personality was challenged almost immediately by arguments that extreme ideologues at both ends of the political spectrum tend toward cognitive inflexibility (Rokeach, 1956; Shils, 1954; Taylor, 1960), recent research confirms that many tendencies often viewed as particularly characteristic of conservative thought are found in liberals too, if you look in the right place. The central theme of this work is that all people are motivated to defend core beliefs and moral commitments, but because beliefs, commitments, and moral sensitivities differ across the political spectrum (e.g., Graham et al., 2013), similar motivations will lead liberals and conservatives to direct bias and intolerance toward different topics and targets (e.g., Brandt, Reyna, Chambers, Crawford, & Wetherell, 2014). This analysis suggests that, in terms of any specific political judgment, either liberal or conservative bias could be magnified depending on how that judgment affects each side’s core commitments (Crawford, 2012, 2014), but that if judgments were aggregated across politically relevant topics, both sides would reveal an equal proclivity to bend information in their political favor. Implicit in this analysis is that academic psychology’s particular focus on bias in political conservatives is partly a function of the blind spots (Pronin, 2007) of a scientific discipline that is overwhelmingly composed of political liberals (Duarte et al., 2015; Inbar & Lammers, 2012).
The Current Study
How bias is distributed across the political spectrum is clearly a matter of current empirical debate. Two different hypotheses can be supported by evidence in the literature: an asymmetry hypothesis that predicts greater partisan bias in conservatives than liberals and a symmetry hypothesis that predicts that levels of partisan bias will not differ between liberals and conservatives. It is also possible, of course, that partisan bias could be greater in liberals than in conservatives even though this hypothesis has not received extensive attention in the literature. The current study seeks to evaluate these hypotheses in a targeted meta-analytic comparison of the magnitude of one prototypical form of partisan bias among liberals and conservatives in the United States.
We selected meta-analysis as our approach to take advantage of the wealth of data on partisan bias that have already been collected. Meta-analysis also allows us to examine partisan biases across studies using differing operationalizations of acceptance of or resistance to political information, left versus right political orientation, and judgments about a variety of political topics. Given the challenges of differentiating partisan bias from some form of rational belief updating, we restricted our analysis to studies in which the strongest inferences about bias can be made: experimental studies similar to those conducted by Lord and colleagues (1979) and Cohen (2003) that used matched-information designs to explore partisan biases in the processing of politically congenial and politically uncongenial information. These studies come from many different labs, including scholars who support both the symmetry and asymmetry perspectives. Our goal is to provide a thorough representation of the extant psychological research regarding susceptibility to partisan bias in liberals and conservatives.
Method
We conducted literature searches using PsycINFO, Psych Articles, and Worldwide Poli Sci databases. We searched for the following terms anywhere in the main text: “bias* assim*,” “confirm* bias*,” “my* bias*,” “bias* evaluat*,” “motiv* reason*,” and “motiv* skeptic*.” We also searched for the reverse construction of each term (e.g., “assim* bias*”). We included the term polit* in each search to limit our results to studies with political content. An initial search was conducted in October 2012 and was updated in October 2014 and December 2016. In an effort to locate studies that fit our inclusion criteria but were not published or did not fall under our literature search terms, we performed a search of the Social Science Research Network (an online repository that contains both published and unpublished works), emailed the Society for Personality and Social Psychology listserv and well-known researchers in the field requesting articles fitting our criteria, and searched the reference lists of articles that fit our inclusion criteria. Two additional articles were suggested by one of the reviewers of the initial version of this article. These searches returned a total of more than 1,500 articles, book chapters, and dissertations.
After an initial culling of articles that were clearly inappropriate for inclusion (e.g., nonempirical pieces), each remaining article was evaluated by at least two members of our research team to determine whether it met four inclusion criteria. In rare instances of disagreement, decisions were resolved through discussion with the whole group.
Inclusion criteria
The four criteria for a study to be included in our analysis were the manipulation of political congeniality, the measurement of left-right political orientation, a measure of information evaluation, and a sample composed of U.S. participants.
Manipulation of political congeniality
Included studies had to manipulate (either within or between subjects) whether participants were presented with stimuli that either (a) supported or opposed their political beliefs (e.g., Lord et al., 1979) or (b) associated a particular policy or behavior with the participant’s own party or the opposing political party (e.g., Cohen, 2003). Political-congeniality manipulations included fictional scientific studies with results supporting either liberal or conservative beliefs, examples of similar behavior demonstrated by liberal or conservative actors, and identical policies endorsed by Democratic or Republican politicians. We excluded studies in which the manipulated information was only loosely matched, such as studies presenting participants with persuasive essays for liberal and conservative positions that differed substantially in their content (e.g., Taber & Lodge, 2006).
Measure of left-right political orientation
Included studies had to measure participants’ self-reported placement on a left/liberal to right/conservative dimension of political orientation. Variations included measures of liberal-conservative ideology, Democratic or Republican party affiliation, and endorsement of specific attitudes with a clear left-right dimension (e.g., in favor of or against gun control). We did not include studies that measured only personality dimensions associated with political ideology (e.g., right-wing authoritarianism) or that equated conservative ideology with prejudicial attitudes (e.g., toward African Americans). Studies were also excluded if they included only one ideological group (e.g., conservatives only); deriving estimates of bias from liberals and conservatives evaluating the same closely matched stimuli most effectively leverages the power of matched information designs to isolate and compare the magnitude of partisan bias.
Information evaluation measure
Studies needed to measure participants’ evaluation of the validity, quality, or acceptance of the matched politically congenial and politically uncongenial information. Examples of information-evaluation measures included ratings of a scientific study’s methodological quality, approval or disapproval of a political actor’s behavior, and endorsement of specific policy proposals presented in the stimulus materials. Studies were not included if their only evaluation measure was endorsement of a general political attitude (e.g., attitude toward capital punishment after reading a study on capital punishment) given the vulnerability of general attitudinal measures to normative counterexplanation. 2
U.S. sample
Although we have no reason to doubt the generality of political bias, our particular interest is on liberal-conservative differences in the context of U.S. politics. Because of this focus and the difficulties of defining liberal and conservative in different national contexts, we included only studies with participants from the United States.
Of the articles evaluated, 48 included data that met all four inclusion criteria. Because the majority of qualifying articles were interested in documenting the existence of partisan bias in general rather than cross-ideological comparisons of bias, only 11 of 48 articles included enough information to calculate separate liberal and conservative effect sizes. For the remaining articles, we contacted authors and asked them to provide additional analyses or to provide data that we could use to perform these analyses ourselves. For 10 articles, the relevant data were no longer available or the authors did not respond to our requests. For articles with multiple studies, each unique sample was counted as an individual study and contributed one effect size in the main analyses. If a study included judgments about multiple topics manipulated between subjects (i.e., some participants responded to materials about gun control and others responded to materials about capital punishment; e.g., MacCoun & Paletz, 2009), effect sizes for each topic were entered as a separate “study.” Our final sample included effect sizes from 51 studies culled from 38 articles, with a total of 18,815 participants (for the full list of included studies, see Table 1).
Effect-Size Estimates and Study Characteristics for All Studies
Note: Values in parentheses indicate the number of participants. Positive roverall values indicate greater bias; positive rdiff values indicate conservatives show more bias than liberals. Pol = political orientation; P = party; C = composite; I = ideology; A = issue attitude; Man = manipulation type; So = source; Co = content; Sam = sample type; Rep = representative; Stu = students; Ad = adults; C = community; Ol = online; Bur = bureaucrats; May = U.S. mayors; Info = information type; S = scientific; NS = nonscientific.
These studies are unpublished.
p < .05. **p < .01. ***p < .001.
Primary analyses
Among the 51 studies included, the statistical tests used to assess the differential evaluation of politically congenial and politically uncongenial information varied a great deal. Reported statistics included ts or Fs with their degrees of freedom; βs with their standard errors; χ2s with their sample sizes; and means, standard deviations, and their sample sizes. For each study, we used the available statistics to compute a Pearson’s r effect size for overall partisan bias (roverall). Positive values reflect the degree to which both liberal and conservative participants responded more positively to politically congenial information than to politically uncongenial information. We examined support for the symmetry versus asymmetry hypotheses in two ways. First, we calculated separate partisan-bias effect sizes for liberals and conservatives for each study (rlib and rcon). 3 Positive rlib values indicate that liberals evaluated liberal-friendly stimuli more positively than they did conservative-friendly stimuli. Likewise, positive rcon values indicate that conservatives evaluated conservative-friendly stimuli more positively than they did liberal-friendly stimuli. Second, we calculated an rdiff effect size from each study reflecting the degree to which rcon and rlib differ within each study. We assigned positive rdiff values to indicate that rcon was greater than rlib in a given study (and negative rdiff values to indicate that rlib was greater than rcon) in line with the asymmetry hypothesis described above. All aggregate r effect sizes were computed with the Comprehensive Meta-Analysis software (Ver. 3.0; Biostat, Inc., Englewood, NJ), which converts r effect sizes to Fisher z values, and were analyzed using random-effects models.
Moderator analyses
Because of the relatively modest number of studies included in our analysis, we limited our examination of potential moderators to five. The moderators we chose to examine were three common methodological variations found in existing studies (the nature of the manipulation, the nature of the political-orientation measure, and the nature of the sample) and two additional variables we suspected might moderate the magnitude of partisan-bias effects. At least two members of our research team coded each study for each moderator.
Political-congeniality manipulation
We coded for whether the manipulation of political information entailed varying the content or the source of the stimuli being evaluated. For instance, Lord et al. (1979) manipulated the content of the political information by showing participants evidence that either supported or challenged the effectiveness of capital punishment. Cohen (2003) manipulated the source of political information when he showed participants the same welfare policy but varied whether that policy was endorsed by congressional Democrats or congressional Republicans.
Political orientation measure
Measures of political orientation were coded for whether they were based on liberal-conservative ideology, Democratic-Republican party affiliation, or liberal-conservative position on an issue-specific attitude.
Sample
We coded for whether the sample was drawn from a student population, a convenience sample of adults online, or a nationally representative sample.
Type of information
We coded for whether the information was presented in the form of scientific data (e.g., Lord et al., 1979) or nonscientific information such as a description of a specific policy (e.g., Cohen, 2003) or the behavior of a political actor (e.g., Crawford, 2012).
Political topic
We coded for the specific topic represented in the political-congeniality manipulation. Among the 51 studies, six political topics were used in three or more studies, allowing us to aggregate and compare their results: capital punishment (k = 6), presidential behavior (k = 5), 4 welfare policy (k = 4), environmental policy (k = 4), abortion (k = 3), and gun control (k = 3). 5
Results
Table 1 presents mean effect sizes for overall partisan bias (roverall), partisan bias separately for liberals and conservatives (rlib, rcon), and the relative magnitude of liberal and conservative partisan bias (rdiff) for all 51 studies. Table 1 also shows how each study was coded on the five moderator variables.
Overall partisan bias ranged from rs = .001 to .696; some studies showed very little partisan bias and others showed a great deal of bias. There was also a substantial range of effect sizes for rlib, rcon, and rdiff, indicating that studies ranged from showing substantially greater bias for liberals than for conservatives to showing substantially greater bias for conservatives than for liberals.
Table 2 displays aggregated r effect-size analyses for the main hypotheses with random-effects models. There was a statistically significant small- to medium-sized mean effect of overall partisan bias (roverall = .245, p < .001; 95% confidence interval, or CI = [.208, .280]), which suggests that people in general showed a clear tendency to evaluate politically congenial stimuli more favorably than similarly structured politically uncongenial stimuli.
Mean Effect Size Estimates Across All Studies for Overall Partisan Bias, Difference in Partisan Bias Between Liberals and Conservatives, Liberal Partisan Bias, and Conservative Partisan Bias
Note: Positive roverall, rlib, and rcon values indicate that participants demonstrate bias; positive rdiff values indicate that conservatives show more bias than liberals. The Q statistic (also known as Cochrane’s Q) is the weighted sum of squared differences between the observed effects and the weighted average effect. Tau, used for computing prediction intervals (PIs), is an estimate of the standard deviation of the distribution of true effect sizes, assuming that those effect sizes are normally distributed. The 95% PI gives the range in which the point estimate of 95% of all true effects (including those from both previously completed and future studies) are expected to fall, assuming that true effect sizes are normally distributed through the domain (Borenstein, Hedges, Higgins, & Rothstein, 2009). CI = confidence interval; QW = within-studies heterogeneity.
p < .001.
The average effect sizes for rlib and rcon differed significantly from zero, indicating that liberal and conservative participants were both biased in favor of information that supported their particular political beliefs and allegiances. The results provided support for the symmetry hypothesis: The mean levels of liberal and conservative bias were very similar in magnitude (rlib = .235, 95% CI = [.192, .276]; rcon = .255, 95% CI = [.205, .304]) and the aggregate rdiff effect size across all 51 studies was extremely small and was not significantly different from zero (rdiff = .009, p = .55, 95% CI = [−.020, .038]; see Table 2), indicating no difference in degree of bias between liberals and conservatives. In other words, whether partisan bias was aggregated separately for liberals and conservatives or compared within each study and then aggregated, our results suggest that liberals and conservatives were both significantly biased in favor of information that supported their ideological beliefs and groups, and the two groups were biased to very similar degrees.
Moderator analyses
There was significant heterogeneity within roverall and rdiff effect sizes (see results of QW tests for homogeneity in Table 2), so we tested whether any of our coded variables moderated our main findings. These moderator analyses should be interpreted cautiously, however, because the relatively small number of studies examined in subgroups creates the possibility of confounding among the moderators (e.g., many of the studies examining a particular political topic may also rely on a particular methodological approach).
Overall, none of our analyses revealed statistically significant differences for any of our moderator variables for either overall magnitude of partisan bias (roverall), magnitude of bias in liberals and conservatives separately (rlib, rcon), or the relative magnitude of bias in liberals and conservatives (rdiff). It is noteworthy that the overall partisan-bias effect was significant for every subgroup for all five moderator variables examined. All statistics for the moderator analyses are reported in Table 3.
Moderator Analyses for Partisan-Bias Effect-Size Estimates
Note: Results are for random-effect moderator analyses. Positive roverall values indicate greater overall partisan bias; positive rdiff values indicate that conservatives show more bias than liberals. Moderator analyses were performed on roverall and rdiff, but liberal and conservative partisan bias (rlib and rcon, respectively) are also shown for reference. QB = between-studies heterogeneity.
p < .10. *p < .05. **p < .01. ***p < .001.
Although we found no significant moderators in our analysis, the prediction intervals associated with our mean effect sizes (presented in Table 2) suggest that the true effects of partisan bias—for liberals, for conservatives, and for both groups combined—are likely to vary widely from study to study, such that true effects range from nonexistent (very close to zero) to fairly large. Furthermore, the true effects for the difference between conservatives and liberals are also likely to vary, ranging from liberals being slightly more biased than conservatives to conservatives being slightly more biased than liberals. These wide prediction intervals underscore the fact that moderators of these effects are likely to exist even though we were not able to identify these moderators in our study.
Publication bias
We addressed the possibility of publication bias in multiple ways. First, we sought out and included both published (k = 42) and unpublished studies (k = 9) of partisan bias.
Second, we looked at whether publication in a peer-reviewed source moderated effect size. Published studies showed a larger mean partisan-bias effect size (roverall = .266, p < .001) than did unpublished studies (roverall = .139, p = .003; QB = 6.35, p = .012), but the mean effect sizes in both sets of studies were significantly greater than zero. Moreover, our primary interest in this project was not whether overall bias exists, but rather the relative magnitude of bias in conservatives and liberals. In this case, publication status did not moderate results. Conservatives and liberals showed equivalent levels of relative bias in both published (rdiff = −.001, p = .95) and unpublished studies (rdiff = .054, p = .11; QB = 2.15, p = .14).
Third, we used funnel plots to visually assess publication bias by plotting Fisher’s transformation of the effect size for each study on the horizontal axis against the natural log of its sample size on the vertical axis, and we used linear regression to test the slope through the points in the funnel plot (Sterne, Becker, & Egger, 2005). Symmetrical funnel plots with a nonsignificant slope indicate that publication bias is not an issue. Asymmetry in the funnel plot with a negative slope indicates publication bias because studies with small sample sizes showing null or negative effects are absent from the sample of studies. There was no evidence of publication bias for either overall partisan bias (roverall β = −.12, p = .42) or the relative degree of bias in conservatives and liberals (rdiff β = −.01, p = .97).
Discussion
The clearest finding from this meta-analysis was the robustness of partisan bias. A tendency for participants to find otherwise identical information more valid and compelling when it confirmed rather than challenged their political affinities was found across a wide range of studies using different kinds of samples, different operationalizations of political orientation and political congeniality, and across multiple political topics. The mean effect for overall partisan bias was modest in size, but statistically significant partisan-bias effects were found in 39 of 51 samples and in every subgroup compared in our moderator analyses. That is, the tendency to evaluate politically congenial information more charitably than politically uncongenial information was found whether the study manipulated congeniality via the source of the information or its content; whether political orientation was operationalized as ideology, party affiliation, or a specific attitude about a particular political issue; whether the sample was composed of students, adults opting into an online study, or a representative sample of U.S. citizens; whether the information evaluated was scientific or nonscientific; and across several different politically charged topics. None of this should be surprising given the extensive body of research confirming a pervasive human tendency toward motivated reasoning and self- and group enhancement (Brown & Kobayashi, 2002; Kunda, 1990; Mercier & Sperber, 2011; Sedikides, Gaertner, & Vevea, 2005; Stanovich, West, & Toplak, 2013). People are less skeptical consumers of information that they want to believe than of information that they do not want to believe (Ditto & Lopez, 1992), and this pattern is as evident in the political realm as it is in other realms of life that evoke strong emotions, preferences and social allegiances.
The question of ideological symmetry
A corollary of the general robustness of partisan bias was specific support for the symmetry hypothesis. Our meta-analysis contributes to a long-standing and ongoing debate regarding the psychological similarities and differences between people occupying the left and right ends of the ideological spectrum (Adorno et al., 1950; Brandt et al., 2014; Crawford, 2017; Jost, 2017; Jost et al., 2003; Rokeach, 1956). Contrary to the view that political conservatives are particularly prone to defensiveness and cognitive rigidity (Adorno et al., 1950; Jost et al., 2003), our analysis found that when partisan bias was aggregated across studies, topics, and methodological details, both liberals and conservatives were biased in favor of information that confirmed their political beliefs, and the two groups were biased to very similar degrees.
Given the pervasiveness of motivated reasoning and the strong tribal animosities between left and right that have long characterized U.S. politics, it might seem odd to expect people on one side of the political divide to be substantially more or less evenhanded in their judgments than people on the other side. And yet there is a large and growing body of literature, including considerable experimental work, associating political conservatism with a broad array of motivational orientations suggestive of cognitive rigidity and resistance to negative or threatening information (Hibbing et al., 2014; Jost, 2017). This work is compelling, but it is important to note that these studies focus their comparisons on individual differences in general motivational proclivities (e.g., need for order, tolerance for ambiguity), whereas our meta-analysis examined specific judgment outcomes (e.g., the differential evaluation of politically congenial and politically uncongenial information). As such, the two sets of studies do not directly contradict each other, but the question clearly arises as to why the differential motivational tendencies of liberals and conservatives documented in past research were not found to manifest themselves in differential susceptibility to partisan bias in our meta-analysis.
One possibility is that the asymmetrical psychological propensities of liberals and conservatives have their primary impact not on susceptibility to bias in general but rather on the topics about which the two groups are likely to be biased (Brandt et al., 2014; Crawford, 2012, 2014). Greater commitment to attitude positions is associated with more selective processing and resistance to persuasion (Krosnick, 1988; Pomerantz, Chaiken, & Tordesillas, 1995; Zuwerink & Devine, 1996), and moral commitments may be particularly potent in rousing psychological defenses (Mullen & Skitka, 2006; Skitka, Bauman, & Sargis, 2005). By this account, conservatives’ heightened discomfort with uncertainty and threat might reveal itself, not in more biased processing of information about any political topic, but rather in relatively pronounced bias about information that threatens or assuages those (or other) particularly conservative concerns. A recent study, for example, found political conservatism to be associated with greater credulity to information about personal or societal risks (e.g., attacks by terrorists or sharks) but not personal or societal benefits (e.g., the health advantages of carrots or cats; Fessler, Pisor, & Holbrook, 2017). Analogously, liberals by this account might be expected to show particularly biased responses to information bearing on their core concerns about protection for vulnerable groups and societal inequality. Uhlmann, Pizarro, Tannenbaum, and Ditto (2009), for example, found political conservatives to be unaffected by the race of an individual to be sacrificed in a moral dilemma, whereas liberals did discriminate on the basis of race: Liberals were significantly less likely to sacrifice an individual with a stereotypically African American name than a stereotypically White name (for similar findings, see Norton, Vandello, & Darley, 2004). 6 Our meta-analysis found only nonsignificant differences in bias across political topics, but future research with greater statistical power and topics chosen to map onto the known psychological and moral sensitivities of liberals and conservatives (e.g., Graham et al., 2013; Jost et al., 2003) would be a more compelling test of the topic-specific bias hypothesis.
Another possibility is that the psychological differences between liberals and conservatives have their effects on aspects of the information processing sequence other than the biased evaluation of political information. The studies examined in our meta-analysis all confronted participants with information that either supported or challenged their political beliefs, a “strong situation” (Mischel, 1977) likely to evoke motivated responding in most or all people, and one that precludes the choice generally available in the natural environment to direct one’s attention toward or away from particular kinds of information. It is possible, then, that the choice of what information to seek out or avoid is where conservatives’ relative reticence toward novel and threatening information has its effects, rather than how that information is processed once it is confronted. Research in the selective-exposure tradition has produced several studies suggesting that the tendency to preferentially seek out information that supports rather than challenges political views is more pronounced in conservatives than in liberals (Barberá, Jost, Nagler, Tucker, & Bonneau, 2015; Messing & Westwood, 2014; Nam, Jost, & Van Bavel, 2013; Rodriguez, Moskowitz, Salem, & Ditto, 2017). It is also true, however, that several studies have revealed no political differences in selective-exposure tendencies (Collins, Crawford, & Brandt, 2017; Frimer, Skitka, & Motyl, 2017; Iyengar & Hahn, 2009; Stroud, 2008), and a few have suggested greater selective exposure among liberals than among conservatives (Bakshy, Messing, & Adamic, 2015; Knobloch-Westerwick & Meng, 2009).
Much like the pattern seen in our meta-analysis, the literature on selective exposure reveals considerable variability across studies in the relative magnitude of bias in liberals and conservatives; the clearest conclusion to be drawn from the extant data concerns the proneness of both sides to favor politically congenial over politically uncongenial information. Research on political selective exposure, however, is a step ahead of work on the biased processing of political information in its recognition of important boundary conditions and contextual influences on political bias such as information utility and attitude importance (Garrett & Stroud, 2014; Knobloch-Westerwick & Kleinman, 2012; Knobloch-Westerwick & Meng, 2009). Similar contextual factors have been found to moderate motivated-reasoning processes outside of the political domain (Kunda, 1990; Lerner & Tetlock, 1999; Neuberg & Fiske, 1987), and exploring their operation in political motivated reasoning has the potential to clarify predictions regarding when and in whom partisan bias is most likely to be found, including variability over time and political climate (Federico & Malka, 2018).
Of course, further research is needed to thoroughly investigate all of the speculation above. This research would ideally include new experimental studies (e.g., comparing the magnitude of partisan bias across topics that differ in attitude importance or moral conviction for liberals and conservatives), longitudinal studies (where data are available) to track changes in political congeniality biases over time and historical context, as well as additional meta-analyses (e.g., comparing selective exposure tendencies in liberals and conservatives). The swelling body of research examining the psychological underpinnings of liberalism and conservatism should be particularly helpful in generating testable hypotheses.
There are almost certainly both important symmetries and important asymmetries between liberal and conservative psychology, and research exploring this complicated web of commonality is inaptly characterized as pursuing “Swiss-style neutrality” or some kind of false moral equivalency between liberal and conservative ideology (Jost, 2017). Different psychological processes contribute to different manifestations of bias, and there are complexities to political ideology that belie the simple unidimensional (liberal-conservative) characterization relied on here (e.g., Crawford, Jussim, Cain, & Cohen, 2013; Iyer, Koleva, Graham, Ditto, & Haidt, 2012; Malka & Soto, 2015). All this complexity must be considered in any comprehensive treatment of the ideological symmetry question, and given that complexity, a simple portrait of the psychological superiority of one ideology over another seems unlikely to emerge. Moreover, psychological comparisons are completely independent of, and in no way preclude, thoughtful assessments of the superiority or inferiority of political ideologies at a social, economic, or moral level. Psychological equivalency does not imply moral equivalency, despite a fundamental human tendency to conflate descriptive evaluations with prescriptive ones (Ditto & Liu, 2016; Hume, 1740/1985; Liu & Ditto, 2013). Political psychologists, ourselves included, face a unique challenge, highlighted ironically by the findings of pervasive partisan bias presented here, to prevent our own political views from influencing how we conduct and interpret our research. We agree with Jost (2017) that a preference for finding commonalities between ideologies is no less problematic than a preference for showing one particular ideology to be psychologically (or morally) superior to others, and we encourage all researchers interested in partisan bias to take every precaution to avoid falling prey to the very phenomenon we seek to understand.
Limitations
Our meta-analysis was more targeted than some because of our desire to focus on studies that provide the most compelling evidence for partisan bias: experimental studies using a matched information design to examine differential evaluation of politically congenial and politically uncongenial information (Kahan, 2016). We could have cast our net more broadly to include studies using correlational data or other experimental designs or examining other kinds of partisan biases (e.g., selective exposure, hostile media bias). Instead, we felt that given the long-established difficulties of disentangling motivated bias from normative decision processes (Ditto, 2009; Kahan, 2016; Tetlock & Levi, 1982), focusing only on the highest quality studies as a first step would provide the most accurate and modest yardstick to compare bias across groups.
This does not mean, however, that bias is always the sole explanation for differences found in studies with carefully matched stimulus materials. For example, several studies included in our meta-analysis demonstrate significant differences in how positively an identical policy was evaluated, even when the only difference between conditions was a single word indicating whether one’s own party or the opposing party endorsed that policy (e.g., Malka & Lelkes, 2010). At one level this can be construed as bias: A person favors the very same policy that they would have rejected if the other party had proposed it. But party labels can also be thought of as cues, and favoring a policy supported by people one agrees with on many other issues can be thought of as a sensible heuristic strategy rather than a bias (Bullock, 2011; Leeper & Slothuus, 2014).
This interpretational ambiguity, of course, is just one example of the formidable challenge of ruling out normative counterexplanation that transcends the study of bias in political judgment. In our meta-analysis, studies that manipulate the political content of information rather than its source are (arguably) less vulnerable to this ambiguity, and our analysis shows the mean effect of partisan bias to be equally strong in the former (r = .236) and in the latter (r = .251). But ultimately, there is an empirical catch-22 at the heart of all research on motivated reasoning. Because contextual information must be manipulated to produce differential motivations to accept or reject a given piece of target information, the informational differences between conditions that are a necessary part of the motivational manipulation are always a potential cause of any differential judgments between those same conditions. As long as information is used to manipulate motivation, the entanglement between the two (and the potential confounding that inevitably results) will always persist, at least to some degree.
Minimizing the plausibility of normative explanations for putative bias effects is important in scientific research, and restricting our meta-analysis to only the most carefully designed experiments was our attempt to do that here. But it is important to recognize that in the real world of politics, as in virtually every real-world situation, prior belief and motivated bias are naturally confounded (Ditto, Munro, Apanovich, Scepansky, & Lockhart, 2003), and both are likely to play a role in partisan resistance to politically challenging information. When confronted with the latest Republican tax-reform plan, for example, most Democrats approach that plan both expecting it to be bad policy (based on prior information to which they have been exposed about the ineffectiveness of tax cuts, almost certainly shaped by selective-exposure tendencies) and motivated to perceive it as bad policy, either because aspects of the policy offend their moral sensitivities or because of their general antipathy toward the Republicans who proposed it. This natural coalition of belief and motivation may help to explain why the bias we observe under tightly controlled experimental conditions seems so subtle compared with the seemingly blatant hypocrisy people often perceive in their real-world political antagonists.
Another key limitation of our study was our decision to treat political orientation dichotomously rather than continuously. This decision flowed primarily from our focus on matched information designs in which political congeniality was defined by whether information confirms or challenges participants’ existing political views or allegiances, making the inclusion of individuals with moderate or politically independent views in continuous analyses problematic. Included studies also used varied operationalizations of left versus right ideology, many measuring or reporting it only dichotomously, so adopting a dichotomous approach allowed us to include the maximum number of studies in our analyses. Still, our approach of comparing the magnitude of liberal and conservative bias in reactions to information manipulated to either challenge or support partisan beliefs raises important issues about the equivalency of stimulus materials across experimental conditions (for examination of one such issue, see the Supplemental Material available online) as well as the extent to which our liberal and conservative samples were equally extreme in their ideological commitments. Future work should consider how to best gauge bias across the continuous spectrum of ideology, most critically for the ability to evaluate what is likely to be an important role for ideological extremity in fomenting partisan bias.
Finally, it is important to consider whether the political views of researchers may have influenced the sample of studies available for our meta-analysis, especially in a field so disproportionally composed of individuals whose sympathies lie with one particular political perspective (Inbar & Lammers, 2012). In most meta-analyses, the file-drawer problem is a straightforward matter of gauging the extent to which null results are underrepresented in the published data. The current case is more complicated in that (a) our primary result of interest is a null finding (no difference in magnitude of bias between liberals and conservatives) and (b) it is plausible to consider whether a particular pattern of affirmative results—those showing strong liberal bias—might be underrepresented in the literature as well.
First, we made active attempts to uncover and include data from unpublished sources and conducted moderator analyses comparing the relative effect size of conservative and liberal bias in published and unpublished studies, which revealed no significant differences. Second, suppression of evidence of liberal bias (either active or passive) seems unlikely in that very few of the studies included in our meta-analysis were specifically focused on comparing the magnitude of liberal and conservative bias; most did not even report the relevant data or comparisons. Still, we should note again that although we can find no evidence that the strength of liberal bias was underestimated in the current study, research on partisan bias is naturally fraught with the potential for that same partisan bias to influence the research process at multiple levels, from study design and construction of stimulus materials to the analysis and reporting of relevant data. New methods being promoted to enhance the reproducibility of empirical findings in the field of psychology (e.g., Cumming, 2014; Simmons, Nelson, & Simonsohn, 2011) should help combat all forms of research bias, including those flowing from researchers’ political commitments (Ditto, Wojcik, Chen, Grady, & Ringel, 2015).
Conclusion
It is common in political discourse to hear politicians and pundits contrast the biased opinions of their political opponents with their own side’s impartial view of the facts. Our meta-analysis suggests instead that partisan bias is a bipartisan problem and that we may simply recognize bias in others better than we see it in ourselves (Pronin, 2007). This same myopia toward our own side’s biases may also help explain why a field dominated by liberal researchers has been so much more focused on the biased perceptions of the political right than the political left. This meta-analysis raised more questions than it answered in terms of the specific determinants of partisan bias, and future research may suggest that our assessment of the magnitude of bias in each side may be imprecise (see the confidence and prediction intervals in Table 2) or historically variable. What is most clear from the data, however, is that both liberals and conservatives show a consistent tendency to be less skeptical consumers of information that supports than challenges their political beliefs. The fact that neither side is immune to partisan bias may be the more important point than whether one side falls prey to it slightly more than the other.
Using different standards to evaluate information when it supports your political views than when it challenges them represents an obvious problem in terms of normative standards of judgment. Still, it can be argued that in terms of individual self-interest, a tendency to adjust one’s political views to fit with norms of important social or cultural groups makes good sense (Kahan, 2013). But partisan bias represents a practical problem as well. It is increasingly clear in contemporary U.S. politics that liberals and conservatives often hold dramatically different factual beliefs about key political issues (Frankovic, 2016; Rampell, 2016). The processing biases documented in our meta-analysis, particularly in conjunction with partisan selective-exposure effects, are likely to be an important contributor to these “alternative facts” by leading political partisans to readily accept “facts” that support their side’s positions rather than to carefully scrutinize them. These differences in factual belief can in turn contribute to political conflict and governmental dysfunction by making compromise and negotiation more difficult and fueling corrosive political stereotypes of the other side as deluded, hypocritical, or just plain dumb (Ditto & Liu, 2016; Kennedy & Pronin, 2008).
One solution many in the academy might suggest is the provision of clear scientific data to provide impartial answers to disputed questions of fact and to use as a foundation for evidence-based policy prescriptions. Our data, however, present a potential obstacle for this proposed solution in that our moderator analyses revealed that political partisans responded to information composed of scientific data in just as biased a fashion as they responded to nonscientific arguments. Rather than being the final arbiter of truth—the impartial political referee that many people seem to crave—empirical data may simply provide “grist for a motivated cognitive mill” (Ditto & Lopez, 1992, p. 579). Together with a growing body of evidence suggesting that increased knowledge and expertise in a topic area exacerbates rather than ameliorates political bias (Kahan et al., 2012; Liu & Ditto, 2013; Taber & Lodge, 2006), the prognosis for eradicating partisan bias with harder data and better education does not seem particularly rosy. 7
Sophisticated strategies informed by psychological science need to be developed to combat our political prejudices (e.g., Feinberg & Willer, 2013; Fernbach, Rogers, Fox, & Sloman, 2013) and to begin to build a less polarized, more civil, and more evidence-based political culture. The evidence available right now, both scientific and anecdotal, suggests that this will not be easy. But a crucial first step is to recognize our collective vulnerability to perceiving the world in ways that validate our political affinities.
Supplemental Material
Ditto_Supplemental_Analyses – Supplemental material for At Least Bias Is Bipartisan: A Meta-Analytic Comparison of Partisan Bias in Liberals and Conservatives
Supplemental material, Ditto_Supplemental_Analyses for At Least Bias Is Bipartisan: A Meta-Analytic Comparison of Partisan Bias in Liberals and Conservatives by Peter H. Ditto, Brittany S. Liu, Cory J. Clark, Sean P. Wojcik, Eric E. Chen, Rebecca H. Grady, Jared B. Celniker, and Joanne F. Zinger in Perspectives on Psychological Science
Supplemental Material
DittoSupplemental_figure_1_copy – Supplemental material for At Least Bias Is Bipartisan: A Meta-Analytic Comparison of Partisan Bias in Liberals and Conservatives
Supplemental material, DittoSupplemental_figure_1_copy for At Least Bias Is Bipartisan: A Meta-Analytic Comparison of Partisan Bias in Liberals and Conservatives by Peter H. Ditto, Brittany S. Liu, Cory J. Clark, Sean P. Wojcik, Eric E. Chen, Rebecca H. Grady, Jared B. Celniker, and Joanne F. Zinger in Perspectives on Psychological Science
Footnotes
Action Editor
Brad J. Bushman served as action editor and Robert J. Sternberg served as editor-in-chief for this article.
Author Contributions
All the authors contributed to finding articles, evaluating whether studies fit inclusion criteria, extracting data, computing effect sizes, and coding moderators. Analyses were performed by R. H. Grady and E. E. Chen. J. F. Zinger served primarily as a meta-analysis consultant. P. H. Ditto oversaw the entire research project and drafted the manuscript, and all the authors provided critical revisions. All the authors approved the final manuscript for submission.
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
