Partisan Evaluation of Partisan Information

Abstract

One recent and conspicuous change in the U.S. media landscape has been the shift toward more markedly partisan news content. At the same time, data suggest that the media audience has become more polarized across a wide array of controversial and politicized issues. Recruiting from a group of highly polarized opponents of childhood vaccinations, this study employed a 3 (content bias) × 2 (partisan vs. neutral participants) × 2 (information source) experimental design to examine audience perceptions of information bias. The data supported an expected hostile media perception in the case of “fair and balanced” information, but different patterns in the other bias conditions suggest that content variables can sometimes disarm defensive processing.

Keywords

hostile media effect partisanship partisan media contrast effects vaccinations

Partisans, people with strong opinions and/or high involvement in a contested issue, are prone to perceive news coverage of that issue as biased, and biased in a disagreeable direction. This so-called hostile media effect has been demonstrated in scores of studies of “neutral” news: Partisans of all stripes perceive an unfavorable slant in the same content that disinterested observers see as fair and balanced.

When this hostile media perception was first empirically tested more than 25 years ago (Vallone, Ross, & Lepper, 1985), balanced news was arguably the norm. From the thud of the morning newspaper on the front porch to Walter Cronkite’s closing words on the CBS evening news, people had subsisted over many decades on a relatively homogeneous diet of what the news industry called a “fair and objective” representation of events.

But the traditional news monolith has undergone profound changes, including a shift away from the posture of objective or balanced journalism toward a markedly partisan press. Democrats describe the most popular cable news channel, Fox News, as a “. . . communications arm of the Republican party” (CBSNews, 2009) while MSNBC is seen as “awash in progressive politics” (Kornacki, 2011). Even traditional mainstream media are less staunchly committed to the ideal of objectivity and observations of “the rise in media polarization” have become commonplace (Harwood, 2009).

In addition to changes in content are changes in choice. The proliferation of cable and later online news and information sources has given the media audience a vastly increased selection from which to choose. One reflection of increased news media choice is the general decline of broadcast news viewership while cable and online news continue to gain viewers (PEW, 2010). With the relative ease of a TV remote or computer mouse people can access information from a very broad array of sources (Prior, 2007).

Alongside these shifts in content and access are signs that the media audience is becoming increasingly partisan (Abramowitz & Saunders, 2006). Empirical evidence, for example, demonstrates that exposure to opinionated news content promotes greater perceived distance between conservatives and liberals (Jamieson & Cappella, 2008). And increasing division is true not only on the political-ideology spectrum but across a wide range of controversial topics where specific issue publics congregate (Kim, 2009).

In the traditional system of generic content and minimal choice, readers from all points of view were exposed to the same version of the news. For two decades Walter Cronkite’s signature nightly sign off, “And that’s the way it is,” symbolized the uniform approach to news (Baym, 2010). But current trends in news slant and access make it increasingly likely that opinionated individuals will encounter news that is either more consistent with, or at times contrary to, their own points of view. And this increase in the quantity and availability of partisan news begs a critical question: How is such information received and evaluated by the audience, particularly a partisan audience? Do people with a point of view see news as fair and balanced only when it is actually slanted in their favor? Do they consider “disagreeable” news as hopelessly biased and not worth their attention?

Underlying partisan evaluations is a broader model of biased information processing—biased to maintain preexisting beliefs. One component of such processing, well supported in contemporary research, is the notion of selective exposure to like-minded content (e.g., Kim, 2007; Stroud, 2011), suggesting that congenial encounters are the most common kind. Hence, we are compelled to ask whether the rise of a partisan press, and the resulting ease with which issue publics can selectively choose attitude-consistent content, will diminish critical thinking. Do partisans still distinguish among source characteristics or do they process agreeable content with an uncritical eye?

Even though the changes in content and access described above suggest such encounters are becoming more commonplace, data on partisans’ processing of strategically slanted or blatantly biased information is scarce. Numerous hostile-media-effect studies have focused on partisan evaluations of “neutral” news (e.g., Gunther & Schmitt, 2004; Vallone et al., 1985), but only a few have examined partisan reactions to partisan content. Coe et al. (2008), for example, found relative¹ (though not absolute) hostile media effects when they exposed college students to programming from CNN, Fox News, and The Daily Show (see also Feldman, 2011) and Gunther et al. (Gunther, Christen, Liebhart, & Chia, 2001) found similar results with animal rights activists. In the study reported here we sought to observe partisan evaluations more closely by (a) recruiting highly involved partisans, (b) controlling the stimuli by constructing slanted content, and (c) manipulating source factors known to influence hostile media perceptions.

To examine the interactions between a partisan audience and partisan content we recruited (along with control participants) a group of individuals deeply opposed to childhood vaccinations—an issue that has received major media attention. These participants were exposed to content manipulated to be favorable, neutral, or unfavorable toward vaccinations and presented as either a USAToday news article or a student essay. Participants then answered a battery of evaluation items, including questions about persuasiveness and bias.

Hostile Media Perception

The hostile media perception (aka hostile media effect) is usually defined as a contrast bias—a deviation of judgment in which a partisan individual perceives or evaluates media content to be further away, in terms of valence, from his or her own point of view. An age-old vexation for journalists, the hostile media effect received its first empirical demonstration in a study showing that Israeli and Palestinian partisans each perceived the same news broadcasts about Middle East conflict to be biased in favor of the other side (Vallone et al., 1985). Subsequent research has examined partisans’ perceptions of media bias across a variety of controversial issues, including genetically modified food (Gunther & Liebhart, 2006; Gunther, Miller, & Liebhart, 2009), sports reporting (Arpan & Raney, 2003), Israeli settlements in Gaza (Tsfati & Cohen, 2005), research on primates (Gunther & Chia, 2001), Korean security laws (Choi, Yang, & Chang, 2009), gambling in Singapore (Chia, Yong, Wong, & Koh, 2007) and Columbian politics (Rojas, 2010). Partisan perceptions of hostile bias appear to be a robust finding across a wide range of issues, partisan groups, and media channels. Thus, in an aggregate view we expected a global hostile media perception, a main effect for partisanship in content evaluations across all conditions.

As noted above, tests of the hostile media effect typically employ stimulus material that is objectively neutral or balanced. Evidence for the contrast bias is found when partisans’ subjective evaluations interpret the content as not at all neutral, but rather biased or slanted in a “disagreeable” direction. The most persuasive data show partisans from both sides of an issue judging neutral content to be slanted toward the opposing side.

Neutral stimulus content is the standard for hostile media effect studies. But in one exception, participants who either supported or opposed the use of primates in laboratory research were presented with pairs of slanted news articles, one pair slanted in the favorable direction and the other pair opposed. The data revealed a “relative” hostile media effect; for example, opponents of primate research saw the proresearch stories as significantly more proresearch than did research supporters. While the latter group did not evaluate the content as hostile, they did see it as significantly closer to neutral and this pattern is theoretically consistent with hostile-media-effect expectations. The slanted content caused an overall adjustment up or down on the bias scale, but the relative group differences remained the same (Gunther, Christen, Liebhart, & Chia, 2001). In this test we fully expected that the content manipulations would produce corresponding differences in perceived slant, but we wondered, as a research question, whether the partisan group would see significantly larger differences between content versions than would the neutral group.

Among several theoretical explanations for perceptions of disagreeable media bias is that it may be an effect particular to mass media. One such conjecture is the reach hypothesis, namely, partisans’ concern that information in mass media will reach a wide audience and may have an undesirable influence on less knowledgeable and more vulnerable others. As a result, these highly involved, partisan individuals will shift into a defensive processing mode in which they are keenly attentive to unfavorable content, or more likely to interpret it that way, and thus more prone to see the content as biased. A 2004 study tested the reach hypothesis by presenting partisans with content in either a news-article or student-essay format. The high-reach article generated the expected perceptions of hostile media bias, but in the low-reach essay condition that perception disappeared (Gunther & Schmitt, 2004). In a follow-up study, reach and source—either student or journalist—were both seen to have independent effects (Gunther & Liebhart, 2006) and several other experiments also lend support to the reach hypothesis (Christen & Huberty, 2007; Gunther et al., 2009). (For the sake of consistency we will refer to this factor, where both reach and/or source may play a role in shaping perceptions of content bias, as the reach hypothesis.)

Because the reach variable appears to turn the hostile media effect on and off, we reasoned that crossing the article/essay manipulation with three levels of slant—favorable, balanced,² and unfavorable information on the topic of childhood vaccinations—would provide a novel platform for examining hostile-media-effect processes in the context of objectively slanted news content. First, we formulated an aggregate-level hypothesis focusing on the main effect of reach.

Hypothesis 1: A high-reach news article will produce significantly stronger judgments of provaccination bias than a low-reach student essay.

Then, to learn whether the contrast bias follows the same patterns when the stimulus information is balanced, or slanted in a favorable or unfavorable direction, we turned to the individual content conditions.

Balanced content

We expected balanced or neutral information to produce findings consistent with past research. Thus, the first hypothesis proposed that information in a news-article context would be seen as truly hostile; the second one predicted the expected article-versus-essay difference:

Hypothesis 2a: Compared to nonpartisans’ evaluations, antivaccination partisans will perceive balanced information presented as a news article to be biased in favor of the provaccination position.

Hypothesis 2b: Antivaccination partisans will perceive balanced information presented as a student essay to be relatively less biased in the provaccination direction compared to the news article condition.

Provaccination content

Since vaccination opponents will see provaccination content as inherently disagreeable, we expected them to evaluate that content as more extreme than would nonpartisan respondents and also to exhibit the classic hostile media effect pattern in terms of article-versus-essay differences.

Hypothesis 3a: Compared to nonpartisans’ evaluations, antivaccination partisans will perceive provaccination information presented as a news article to be more biased in favor of the provaccination position.

Hypothesis 3b: Antivaccination partisans will perceive provaccination information to have a stronger provaccination bias in the news article versus the student essay conditions.

Antivaccination content

It seems reasonable to expect a consistent pattern across the three content conditions. If the news article puts partisans into a defensive frame of mind, then regardless of actual slant they will be likely to see more disagreeable, or less agreeable, content than those partisans in the student essay condition. However, it is also plausible that antivaccination stimulus material may disengage the mechanisms that normally produce a hostile media perception. If partisans are reading information they see as supporting their own point of view, they are likely to accept it at face value and not shift into a defensive processing mode. As a result, they will not be sensitized to the potential for undesirable influences on others, and thus, the implications of the reach of the message become inconsequential. That is, partisans will not be worried about undesirable influences on a neutral audience and, thus, will not be on the lookout for the things that might seduce that audience toward an undesirable position.

Because the reasons above would predict different outcomes, we asked as a research question whether partisans in the antivaccination condition would perceive bias differences as a function of reach.

Since questions about bias might be considered leading, we also examined perceptions of information persuasiveness. We expected that partisans’ views of persuasiveness would follow the same general patterns as perceived bias and, based on past research (Gunther & Schmitt, 2004), that partisans would perceive content to be more persuasive in an undesirable direction compared to nonpartisans. We also expected that partisans would see the student essay as ostensibly lacking manipulative intent and effective reach and thus less persuasive than the news article but that nonpartisans would make no such distinctions.

Hypothesis 4a: Compared to nonpartisans, antivaccination partisans will perceive content to be more persuasive in a disagreeable direction.

Hypothesis 4b: Antivaccination partisans will perceive content to be relatively more persuasive in a disagreeable direction in the news article than in the student essay conditions; nonpartisans will see no article-versus-essay differences in persuasiveness.

One additional factor crucial to hostile-media-effect research is the identification of partisans. Some past research has reported mixed evidence for the contrast effect (e.g., Giner-Sorolla & Chaiken, 1994), and the authors speculated that their student sample did not capture a sufficient level of involvement. Other research indicates that people who lean toward one side or another of a controversial issue, but are not deeply committed, are more likely to demonstrate assimilation—seeing content as more favorable to one’s own view—than contrast (Gunther & Christen, 2002). Hence, we sought to recruit partisan participants who would be strongly opposed to childhood vaccinations.

Study Context

Vaccination against diseases began in the late 18th century and, from the beginning, opponents of immunization programs argued that they may be ineffective or even dangerous and that mandatory vaccination violates individual rights. In 1998, the well-respected British medical journal The Lancet published an article by Dr. Andrew Wakefield and colleagues suggesting a possible link between the MMR (measles, mumps, rubella) vaccine and autism spectrum disorders. Subsequent widespread media coverage raised the visibility of the issue by orders of magnitude and vaccination rates dropped sharply in the United Kingdom, Ireland, and other areas. Wakefield’s article was widely criticized and eventually retracted. But concern about childhood vaccination persists. Opponents argue that vaccines may contain harmful compounds that put children at risk and that vaccination should be optional. Supporters cite social benefits such as the eradication of smallpox, polio, and other serious communicable diseases and argue that failure to vaccinate children puts large populations at risk.

Numerous well-organized groups of people, united in their partisan opposition to current vaccination practices, can be found online.

Method

We tackled the research questions and hypotheses with an online survey instrument containing an embedded experiment.

Participants

As a first step in finding participants who would be highly involved in the childhood vaccinations controversy, we contacted a large organization with several thousand online subscribers. The group’s stated mission is to oppose the use of vaccines containing mercury and to provide “information and alternatives to reduce the risk of autism and related health disorders.” This organization agreed to circulate, via its e-mail list, an invitation to members. The invitation included a link to the online instrument and a pledge to donate US$2.50 to the parent organization for every completed survey. The invitation went out on October 26, 2009, and we received over 200 responses within 3 days. Of these, 175 contained complete data for our purposes.

We used a set of initial measures to ensure that respondents were indeed opposed to current forms of childhood vaccinations. Participants were asked to use a 11-point Likert-type scale ranging from 0 (strongly disagree) to 10 (strongly agree) to respond to a battery of attitude and opinion statements, including “Parents should be able to choose whether or not their children are vaccinated,” “Complications from childhood vaccinations can lead to autism,” “Scientists have enough evidence to say that vaccinations do not lead to autism,” and “Administering multiple vaccinations at the same time increases a child’s risk of having an adverse reaction.” A final item in this section asked about support for childhood vaccination, followed by a 0 to 10 scale where 0 was strongly oppose and 10 was strongly support, with 5 as the neutral midpoint.

Childhood vaccination is a complex issue, with concerns ranging from the state of scientific evidence and risks of multiple injections to the autism link and parental rights. To verify that respondents from this partisan group had a shared set of beliefs about vaccination issues and the relationship between autism and vaccines, we used a cultural consensus statistical analysis (Romney, Batchelder, & Weller, 1987). This approach employs factor analysis to examine the degree to which a single factor (e.g., beliefs about childhood vaccination risks) explains the agreement between participants’ responses to the five initial attitude questions. This analysis was conducted using the software package UCINet (Borgatti, Everett, & Freeman, 2002). The high factor loadings on the first factor compared to the second factor (12:1), combined with the fact that there were no negative competence scores (indicating that all the participants’ responses were positively correlated with the group’s average response), suggest that this is a homogeneous group with respect to their beliefs about the relationship between vaccines and autism and toward vaccinations in general.³

The instrument included a measure of reading time for the stimulus that allowed us to cull respondents who spent too little time (less than 30 seconds) to adequately digest the material. Thus, our final N for these analyses was 160 antivaccination partisans.

To include a nonpartisan control group we also recruited 369 students from several large undergraduate classes at a major Midwestern university. These students filled out virtually the same online instrument, and we used the “Complications from childhood vaccinations” item to identify a subset of 154 who could be described as genuinely nonpartisan or neutral; only students who chose the midpoint answer of 5 (neither agree or disagree) were included in the nonpartisan sample. As with partisans, we also dropped student respondents (77) who spent too little time on the stimulus page. To account for potential group differences we controlled for age, religious guidance, conservative ideology, and household/family income in our analyses.

Procedure

As noted earlier, we tested our hypotheses using a 2 (source: article vs. essay) × 3 (slant: pro-/neutral/antivaccination) × 2 (preexisting attitude: antivaccination vs. neutral) fully counterbalanced design. Source and slant were manipulated conditions; preexisting opinion was a measured variable.

Following initial attitude measures, respondents were randomized to one of the six stimulus conditions. For the slant manipulation we produced three versions of an information passage titled “Vaccination-Autism Link: The Debate Lives On.” Each version consisted of a different array of facts and arguments concerning childhood vaccination and related autism issues.

To create these stimuli we worked with a basic paragraph-by-paragraph template derived from existing news articles. Then, to produce a provaccination version we presented more facts and arguments in favor of vaccinations and omitted some that were opposed. Thus, a typical paragraph in the provaccination condition read:

Linda Lange, a registered nurse who chose to vaccinate all three of her children, is concerned about the rise in parents’ choosing to not vaccinate their children. “If parents choose not to vaccinate, they have a huge responsibility to the greater community to take precautions,” she said. “They might have to change their lifestyle.”

The antivaccination version followed the same template but with the slant of content reversed, deleting arguments in favor of vaccination while including more arguments opposed:

Parent Robert Taylor remains skeptical about the seemingly large number of vaccinations that are currently recommended by health officials. “My grandparents had a lot of these diseases and they’re fine,” Taylor said. “I think these childhood diseases are there for a reason. They’re there to build up the immune system.”

The neutral or “balanced” version contained an even distribution of pro- and antivaccination information, including both of the paragraphs represented above, plus paragraphs that adopted a middle-of-the-road position such as this one:

It is clear that there are no easy answers—the vaccination debate is an issue that will persist, with the science community and parents searching for answers.

In this way the stimulus content was kept to essentially the same structure in all three versions. Lengths of the pro- and anti-versions were virtually the same (928 vs. 922 words, respectively). The balanced version was somewhat longer (1,101 words), a result of our intention to balance pro and con material. All content for the initial drafts was taken from existing news articles and related material found online. We then employed feedback from a panel of 12 disinterested judges to achieve the desired level of balance or slant in these three versions. Analysis of variance, followed by Bonferroni corrections for multiple comparisons, confirmed that the three content conditions were perceived as significantly different in slant in the expected directions, both for the partisan (F(2, 155) = 150.6, p < .001) and the student sample (F(2, 148) = 49.5, p < .001).

The reach manipulation was accomplished by describing the information either as a news article or a college student’s composition. The high-reach article version was preceded by a statement saying it had appeared in USAToday and had been “carried nationwide by the Associated Press wire service.” A blue USAToday logo appeared at the top of the page along with a newspaper-style headline and byline followed by text set in typical newspaper format. The low-reach student version was introduced as an essay “written by a senior at the University of XXX for a composition class” and appeared in the format one would expect of a student assignment written in a typical word-processing program. In each slant condition the content in the article and essay versions was identical. An independent-samples t test confirmed that people in the news article condition estimated a significantly larger audience (t(303) = 4.57, p < .001).

Measurement

After stimulus exposure respondents answered a series of questions relating to their perceptions of bias in the article or essay: “Would you say that the portrayal of the vaccination issue in this article/essay was strictly neutral, or was it biased in favor of one side or the other?” and “Would you say that the portrayal of opponents of vaccinations in this article/essay was strictly neutral, or was it biased for or against them?” A third, parallel question referred to the portrayal of supporters. And finally we asked, “Would you say that the journalist/student responsible for this article/essay was strictly neutral, or was he biased in favor of or against vaccinations?” All questions were asked using a 11-point scale, with 0 labeled strongly biased against vaccination and 10 indicating strongly biased in favor of vaccinations (the “opponents” item was reverse-coded). The midpoint of 5 was labeled strictly neutral. The bias index demonstrated good reliability for both groups (adult partisans: M = 6.27, SD = 2.15, alpha = .87; student nonpartisans: M = 5.39, SD = 1.67, alpha = .84).

In addition to the bias measures we asked four questions about persuasiveness (adapted from Gunther & Schmitt, 2004) of the article/essay using the 11-point scales with appropriate anchors. These included (a) overall persuasiveness, (b) whether the arguments are stronger on one side than the other, (c) whether the evidence leans more toward one side than the other, and (d) how persuasive the article/essay would be for a neutral reader. These items also produced good reliability for both groups (adult partisans: M = 5.57, SD = 2.46, alpha = .92; student nonpartisans: M = 5.34, SD = 2.318, alpha = .95). A final section included standard demographic items: household/family income, age, political ideology, and religiosity.

Analysis

Main effects for partisanship, content, and reach were tested with an analysis of variance. In cases where we compared the partisan and nonpartisan samples we employed ANCOVA (analysis of covariance) controlling for age, political ideology, religiosity, and income. For content-specific analyses several planned comparisons were made using the Bonferroni correction. Due to the directional nature of our hypotheses, all analyses reported one-tailed significance levels.

Results

Global Effects

We began with a look at the very large picture, for we expected an overall main effect for partisanship in which antivaccination partisans perceive stimulus information across all conditions to be relatively skewed in the provaccination direction. The pattern of results in Figure 1 appears to support this expectation and statistical tests bear it out. When compared to the scale midpoint, the mean partisan evaluation of vaccination-related content (M = 6.27, SD = 2.15) fell significantly on the disagreeable side of neutral (t(155) = 7.36, p <.001). In addition, ANCOVA confirmed that, across all conditions, these antivaccination partisans rated the content as significantly different from the nonpartisan control group (F(1, 266) = 10.51, p < .001, partial η² = .040). Figure 2 illustrates the comparable pattern for nonpartisans; specifically the antivaccination partisans rated the aggregate content as more provaccination (M = 6.49, SE = 0.17) than did the nonpartisans (M = 5.48, SE = 0.17).

Figure 1.

The impact of content and reach on partisans’ perceptions of bias

Figure 2.

The impact of content and reach on nonpartisans’ perceptions of bias

A second big-picture question concerned overall evaluations of content. As noted earlier, both groups perceived the content to be slanted in the directions we intended. However, separate analyses revealed that the partisan group saw significantly larger differences between content versions than did the nonpartisan group. This difference can be seen in the relatively flatter slope for the nonpartisan group in Figure 3 and it is affirmed statistically in an ANCOVA test of the Content × Partisan Group interaction (F(2, 266) = 12.41, p < .001, partial η² = .090). This pattern suggests that, relative to their less interested and involved counterparts, partisans are prone to see content differences as more polarized. Specifically, in the provaccination condition, partisans viewed the content as significantly more provaccine (M = 9.01, SE = 0.25) compared to nonpartisans (M = 6.82, SE = .23, F(1, 251) = 32.22, p < .001, partial η² = .114). In the neutral condition, the difference between partisans’ (M = 5.86, SE = 0.23) and nonpartisans’ (M = 5.39, SE = 0.22) assessments of hostility dropped slightly below the significance threshold (F(1, 251) = 1.50, p = .11) and became wholly nonsignificant in the case of the antivaccination condition (partisans: M = 4.60, SE = 0.23; nonpartisans: M = 4.22, SE = 0.23, F(1, 251) = 0.97, ns).

Figure 3.

The interaction of content and partisanship on perceptions of bias

Our specific research hypotheses begin with Hypothesis 1, the main effect of reach. The omnibus ANCOVA revealed a significant reach effect (F(1, 266) = 3.05, p < .05, partial η² = .012) but analyzing by groups showed that this effect was exhibited by the partisan group only. In the partisan sample, collapsing for content, the high-reach news article (M = 6.83, SE = 0.20) produced significantly stronger judgments of provaccination bias than did the low-reach student essay (M = 6.15, SE = 0.22) even though content was identical (F(1, 251) = 8.14, p < .01, partial η² = .031). For nonpartisans, by contrast, the article versus essay judgments (M = 5.42 and SE = 0.20, and M = 5.53 and SE = 0.20, respectively) were not different (F(1, 251) = 0.64, ns). Hence, this overall picture indicates that partisan evaluations were highly sensitive to reach, while neutral participants were not.

These general findings are instructive, but, as expected, analyzing the three content conditions separately reveals a more complex pattern.

Effects by content condition

Hypothesis 2a, the basic hostile media perception, supposes that antivaccination partisans will perceive balanced information presented as a news article to be biased toward the provaccination position. In most tests of the hostile media effect, two partisan groups’ evaluations of content are weighed against one another; significant differences in opposing directions are taken as evidence that these partisans are perceiving the same content in different (and hostile) ways. Here, because we were in pursuit of additional questions, we recruited partisans from just one side of the vaccination issue and compared their evaluations to those of the neutral, disinterested group. The result, accounting for controls, affirmed the hostile media perception: The balanced news story was read as substantially more biased in the provaccination direction by antivaccination partisans (M = 6.40, SE = 0.28) compared to nonpartisan control participants (M = 5.49, SE = 0.30, F(1, 251) = 4.00, p < .05, partial η² = .016).

Hypothesis 2b proposed that antivaccination partisans will perceive balanced information presented as a student essay to be relatively less biased in the provaccination direction compared to the news article condition. The data supported this conjecture also. The balanced article (M = 6.40, SE = 0.28) was perceived as more biased in favor of vaccination than the balanced essay (M = 5.33, SE = 0.32, F(1, 251) = 8.06, p <.01, η² = 0.031). This result, consistent with past findings, suggests that the effects of broad reach or a journalist author, or both, significantly increase the hostile media perception.

Hypothesis 3a predicted that antivaccination partisans would see the provaccination article as more strongly slanted in the pro direction than would nonpartisans. The data showed dramatic support for this relative difference. Mean score for the partisan sample (M = 9.44, SE = 0.37) was almost three scale points higher than for nonpartisans (M = 6.57, SE = 0.31, F(1, 251) = 32.18, p < .001, η² = .114).

Hypothesis 3b, predicting that antivaccination partisans would perceive provaccination information to have a stronger provaccination bias in the news article than the student essay conditions, was also supported. The provaccination article, with a mean score close to the high end of the 10-point scale (M = 9.44, SE = 0.37), was perceived as significantly more biased toward vaccination than the provaccination essay (M = 8.58, SE = 0.30, F(1, 251) = 3.54, p < .05, η² = .014).

However, for our research question regarding the antivaccination condition, these data failed to reveal a similar article-versus-essay pattern. There was no significant difference in perceived bias between the antivaccination article (M = 4.65, SE = 0.30) and essay (M = 4.55, SE = 0.31, F(1, 251) = 0.067, ns). In addition, and perhaps as an accompaniment to the disabling of any hostile-media-effect mechanism, partisans’ perception of bias (M = 4.65, SE = 0.30) were not different from those of nonpartisans (M = 4.21, SE = 0.29, F(1, 251) = 0.91, ns).

Finally, estimates of persuasiveness generally paralleled the bias judgments, but with some instructive differences. To test H4a, we examined persuasiveness for article and essay together and found that partisans saw both the provaccination (M = 8.53, SE = 0.33) and neutral content (M = 6.03, SE = 0.31) as more persuasive in a provaccination direction than did nonpartisans (M = 7.06, SE = 0.30, F(1, 251) = 8.59, p < .01, and M = 4.88, SE = 0.29, F(1, 251) = 5.27, p < .05, respectively). Interestingly, antivaccination content was seen as significantly more persuasive by nonpartisans (M = 3.02, SE = 0.30) than by the antivaccination partisans (M = 4.04, SE = 0.31, F(1, 251) = 4.13, p < .05). Although partisans see less persuasiveness in this condition, it is actually consistent with the underlying rationale of H4a since, on this side of the neutral midpoint, persuasiveness is going in a desirable rather than disagreeable direction.

H4b, proposing that partisans would consider the news article more persuasive than the student essay, received partial support. Antivaccination partisans’ evaluations were significantly different only in the neutral condition, where they found the news article more persuasive in an undesirable, provaccination direction (M = 6.49, SE = 0.37) than the student essay (M = 5.58, SE = 0.41, F(1, 251) = 3.50, p < .05). As expected, nonpartisans saw no article-versus-essay differences by condition.

Discussion

A significant aspect of these data lies in the perceptions of bias stemming from the content manipulations and their juxtaposition with reach. However, several global observations can be drawn from the aggregated results. For one, partisan perceptions of content bias were clearly skewed in an unfavorable direction. But these biased perceptions of bias were not uniform across the three slant conditions; instead, as the significant partisanship-by-content interaction reveals, partisans’ perceptions of bias increasingly diverged from those of nonpartisans as they moved from congenial to neutral to disagreeable content conditions. This finding suggests that partisans will see unfavorably slanted content as even more polarized (and perhaps polarizing) than it actually is.

A second broad finding stems from the main effect for reach, an effect apparent for partisans but not for the control participants. Partisans appear to be sensitive to something that neutral individuals are not. The sense of broad reach and/or a distrust of the journalist source is affecting their information processing, evaluations, and judgments. Nonpartisans, by contrast, show no sensitivity to these article-versus-essay differences in any of the content conditions.

But more revealing were the particular condition results. The balanced version produced data precisely in line with theoretical expectations and past results—a hostile media perception in the news article condition but a virtually neutral judgment in the essay condition. Does the same mechanism operate when content is patently slanted in an unfavorable direction? Or does disagreeable information override any sensitivity to reach, producing the same hostile judgment irrespective of the source? These data clearly favor the former conjecture.

Perhaps most interesting and important, however, was that in the antivaccination condition partisans’ sensitivity to article-versus-essay differences apparently disappeared; the reach manipulation produced no differences in bias judgments. These results suggest that, insofar as response to the reach variable gives us a reliable indicator of a contrast bias, defensive processing or other hostile-media-effect mechanisms were shut off in the antivaccination content condition.⁴ There were also no significant differences between partisan and nonpartisan evaluations, further reinforcing the notion that favorably slanted content muted the defensive judgment bias.

The pattern of results for persuasiveness invites several additional inferences. For one, in the disagreeable provaccination condition, and more notably for “neutral” content (content they see as unfavorably slanted), partisans report significantly more persuasiveness than do nonpartisans. We speculate, in line with past arguments (Gunther & Schmitt, 2004), that this perception of undesirable persuasive influence is one engine driving the hostile media perception—that partisan concerns about influence on others initiate or exacerbate the defensive processing that leads to perceptions of hostile bias.

It is illuminating, then, to find that partisans consider the antivaccination content—content that is more consistent with and pleasing to their own position—to be significantly less persuasive than do nonpartisans. At first glance one would expect partisans to regard congenial and like-minded information to be more persuasive. But the result here looks like a familiar page from the first-person-effect literature. When content is seen as positive or beneficial or “good to be influenced by” (Gunther & Mundy, 1993), people are less likely to see influence on others, presumably because, for optimistic bias or ego enhancement reasons, they think less discerning others will not see the logic and good sense as clearly as they do themselves. And, at the same time, a partisan respondent may not regard the information as personally persuasive because he or she already knows about and agrees with this position.

A final note on persuasiveness: In all conditions source only matters to partisans, and only when the content is neutral or balanced. Why do partisans find the neutral news article substantially more persuasive than the identical content presented as a student essay? Two complementary explanations appear plausible. One, unlike objectively biased content that presents an easy basis for judgments, balanced or neutral content is more difficult to assess, and thus, partisans are more attentive to source heuristics, such as the dubious journalist source, for a judgment cue. Two, because the neutral condition presents a more ambiguous array of content, it is easier to selectively evaluate the ambiguous elements. And because partisans in the high-reach, news-article condition have shifted into a more defensive processing mode, more of those persuasive elements would appear to be going in an unfavorable direction. This evaluation bias is consistent with the selective categorization process found in previous research (Gunther & Liebhart, 2006; Schmitt, Gunther, & Liebhart, 2004).

As always with quasi experiments like this one, the partisan and nonpartisan comparison findings must be viewed with some caution. This important but unavoidable design problem is the lack of random assignment when one is studying naturally occurring groups. It is a chronic problem but a necessary one if we are to operationalize the true meaning of high-involvement partisanship, a variable very difficult to manipulate in a laboratory setting. But we have tried to mitigate this problem to some degree by measuring observable group differences and controlling for them in the relevant analyses.

A related issue in the hostile-media-effect literature is that the distinction between involvement and partisanship is not well defined. While our focus is on partisan opinion, it is possible that involvement differences might explain the partisan versus nonpartisan outcomes. However, stimulus reading time may be one indicator of interest and involvement and that measure revealed no difference between groups. That said, the distinction is important and conceptual clarification would be a useful contribution to future research.

A second potential problem lies in the possibility of confounds in the content manipulations. Although we focused on varying the slant in these three versions of the stimulus, and the manipulation checks verified those differences, it is possible that other factors also varied between conditions. We tried to minimize this difficulty by following a paragraph-by-paragraph template in all three versions, but we cannot rule out the possibility of other differences.

The reach variable also presented us with somewhat ambiguous data. We used this manipulation because it has been a reliable predictor of the hostile media perception in past research. And in that role it proved a useful device for testing our hypotheses: We could see a distinctive article-versus-essay difference in the conditions that appeared to produce perceptions of hostile bias, but no difference where that contrast bias was absent. What remains unclear is whether this result is due to separate, independent effects of reach and source or a combination of both.

Caveats aside, this research design has opened a window on several of the many interesting questions unanswered in research on the hostile media perception. One, the manipulations of content slant allowed us to consider global evaluations across conditions, along with main effects of content and reach. Two, providing partisans with information slanted in different directions revealed that the hostile media perception is not always the same when the stimulus content is not “balanced and neutral.” Three, the control group allowed us to compare the judgments of partisans to a disinterested, nonpartisan group that provided an effective baseline for gauging the actual extent of the contrast bias. Four, data from individuals highly involved in the childhood vaccination controversy provided important insights into a consequential public health issue and how partisans on that issue may process information differently depending on the context and type of content they encounter. Five, crossing content differences with a reach manipulation gave us an innovative way to measure, rather than simply infer, actual evaluation mechanisms.

And indeed, these partisan individuals demonstrated that some conditions appear to disarm the defensive processing that leads to the hostile media effect. But perhaps the most important observation is that these data tell us about more than just the hostile media effect, or its absence. Partisans are demonstrating in their different responses to different types of source and content, a larger picture of biased information processing. This processing appears motivated to maintain preexisting beliefs, either by means of defensive responses when the real or perceived slant conflicts with their views or uncritical acceptance when the content is agreeable.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported in part by grants from the UW Robert Wood Johnson Foundation Health and Society Scholars program and the UW Graduate School to the first author.

Notes

Bios

Albert C. Gunther (PhD, Stanford University, 1987) is a professor in the School of Journalism and Mass Communication at the University of Wisconsin–Madison. His research interests focus on the psychology of the mass media audience—often in the context of scientific controversies or public health. He publishes in most major communication journals and has received funding from numerous sources, including the USDA, NSF, NIH, and the Robert Wood Johnson Foundation. He teaches courses in journalism writing and graduate seminars in theory and research methods.

Stephanie Edgerly is a doctoral student in the School of Journalism and Mass Communication at the University of Wisconsin–Madison. Her research explores the effects of the new media environment in providing citizens with new opportunities for political engagement. She is particularly interested in the interplay between individual differences, messages, and environment on political understanding and decision making.

Heather Akin is a doctoral candidate in the Department of Life Sciences Communication/Mass Communications at the University of Wisconsin–Madison. Her research focuses on environmental and health communication, social movements, and how media are used to shape values and engage in political and scientific discourse in East and Southeast Asia. She has presented research at the International Communication Association (ICA), Association for Education in Journalism and Mass Communication (AEJMC), and Midwest Association for Public Opinion Research (MAPOR) conferences.

James A. Broesch (PhD, Emory University, 2010) is a Robert Wood Johnson Foundation Health & Society scholar at the University of Wisconsin–Madison in the Department of Population Health Sciences. He is an anthropologist whose work focuses on the transmission and acquisition of culture, and the intersection between culture and health. As an RWJF-HSS fellow, he is working on incorporating methods and theory from anthropology to operationalize and measure culture in population health research. His prior research has been published in major anthropology and psychology journals.

References

Abramowitz

A. I.

Saunders

K. L.

(2006). Exploring the bases of partisanship in the American electorate: Social identity vs. ideology. Political Research Quarterly, 59, 175-187.

Arpan

L. A.

Raney

A. A.

(2003). An experimental investigation of news source and the hostile media effect. Journalism and Mass Communication Quarterly, 80(2), 265-281.

Baym

(2010). From Cronkite to Colbert: The evolution of broadcast news. London, UK: Paradigm Publishers.

Borgatti

S. P.

Everett

M. G.

Freeman

L. C.

(2002). UCINET for Windows: Software for social network analysis. Harvard, MA: Analytic Technologies.

CBSNews. (2009). Retrieved from http://www.cbsnews.com/stories/2009/10/23/eveningnews/main5415921.shtml

Chia

S. C.

Yong

S. Y.

Wong

Z. W.

Koh

W. L.

(2007). Personal bias or government bias? Testing the hostile media effect in a regulated press system. International Journal of Public Opinion Research, 19(3), 313-330.

Choi

Yang

Chang

J. J. C.

(2009). Elaboration of the hostile media phenomenon: The roles of involvement, media skepticism, congruency of perceived media influence, and perceived opinion climate. Communication Research, 36(1), 54-75.

Christen

C. T.

Huberty

K. E.

(2007). Media reach, media influence: The effects of local, national and Internet news on public opinion inferences. Journalism and Mass Communication Quarterly, 84(2), 315-334.

Coe

Tewksbury

Bond

B. J.

Drogos

K. L.

Porter

R. W.

Yahn

Zhang

(2008). Hostile news: Partisan use and perceptions of cable news programming. Journal of Communication, 58(2), 201-219.

10.

Feldman

(2011). Partisan differences in opinionated news perceptions: A test of the hostile media effect. Political Behavior, 33(3), 407-432.

11.

Giner-Sorolla

Chaiken

(1994). The causes of hostile media judgment. Journal of Experimental Social Psychology, 30, 165-180.

12.

Gunther

A. C.

Chia

S. C.-Y.

(2001). Predicting pluralistic ignorance: The hostile media effect and its consequences. Journalism and Mass Communication Quarterly, 78(4), 689-701.

13.

Gunther

A. C.

Christen

C. T.

(2002). Projection or persuasive press? Contrary effects of personal opinion and perceived news coverage on estimates of public opinion. Journal of Communication, 52(1), 177-195.

14.

Gunther

A. C.

Christen

C. T.

Liebhart

J. L.

Chia

S. C.-Y.

(2001). Congenial public, contrary press and biased estimates of the climate of opinion. Public Opinion Quarterly, 65, 295-320.

15.

Gunther

A. C.

Liebhart

J. L.

(2006). Broad reach or biased source? Decomposing the hostile media effect. Journal of Communication, 56(3), 449-466.

16.

Gunther

A. C.

Miller

Liebhart

(2009). Assimilation and contrast in a test of the hostile media effect. Communication Research, 36(6), 747-764.

17.

Gunther

A. C.

Mundy

(1993). Biased optimism and the third-person effect. Journalism Quarterly, 70, 58-67.

18.

Gunther

A. C.

Schmitt

K. M.

(2004). Mapping boundaries of the hostile media effect. Journal of Communication, 54, 55-70.

19.

Harwood

(2009, November 2). “If Fox is partisan, it is not alone.” The New York Times.

20.

Jamieson

K. H.

Cappella

J. N.

(2008). Echo chamber: Rush Limbaugh and the conservative media establishment. Oxford, UK: Oxford University Press.

21.

Kim

Y. M.

(2007). How intrinsic and extrinsic motivations interact on selectivity: Investigating the moderating effects of situational information processing goals on issue publics’ web behavior. Communication Research, 34(2), 185-211.

22.

Kim

Y. M.

(2009). Issue publics in the new information environment. Selectivity, domain specificity, and extremity. Communication Research, 36(2), 254-284.

23.

Kornacki

(2011, January 21). Is Olbermann the victim of his own success? Salon.com.

24.

PEW. (2010, September 12). Americans spending more time following the news. Retrieved from http://people-press.org/report/652/

25.

Prior

(2007). Post-broadcast democracy: How media choice increases inequality in political involvement and polarizes elections. New York, NY: Cambridge University Press.

26.

Rojas

(2010). “Corrective” actions in the public sphere: How perceptions of media and media effects shape political behaviors. International Journal of Public Opinion Research, 22(3), 343-363.

27.

Romney

A. K.

Batchelder

W. H.

Weller

S. C.

(1987). Recent applications of cultural consensus theory. American Behavioral Scientist, 31(2), 163-177.

28.

Stroud

N. J.

(2011). Niche news. The politics of news choice. New York, NY: Oxford University Press.

29.

Tsfati

Cohen

(2005). The influence of presumed media influence on democratic legitimacy: The case of Gaza settlers. Communication Research, 32, 794-821.

30.

Vallone

R. P.

Ross

Lepper

M. R.

(1985). The hostile media phenomenon: Biased perception and perceptions of media bias in coverage of the Beirut massacre. Journal of Personality and Social Psychology, 49(3), 577-585.

31.

Weller

S. C.

(2007). Cultural consensus theory: Applications and frequently asked questions. Field Methods, 19(4), 339-368.