The effects of media narratives about failures and discoveries in science on beliefs about and support for science

Abstract

This study examines the effects of exposure to media narratives about science on perceptions pertaining to the reliability of science, including trust, beliefs, and support for science. In an experiment (n = 4497), participants were randomly assigned to read stories representing ecologically valid media narratives: the honorable quest, counterfeit quest, crisis or broken, and problem explored. Exposure to stories highlighting problems reduced trust in scientists and induced negative beliefs about scientists, with more extensive effects among those exposed to the “crisis/broken” accounts and fewer for those exposed to “counterfeit” and “problem explored” stories. In the “crisis/broken” and “problem explored” conditions, we identified a three-way interaction in which those with higher trust who considered the problem-focused stories to be representative of science were more likely to believe science is self-correcting and those with lower trust who perceived the stories to be representative were less likely to report that belief. Support for funding science was not affected by the stories. This study demonstrates the detrimental consequences of media failure to accurately communicate the scientific process, and provides evidence for ways for scientists and journalists to improve science communication, while acknowledging the need for changes in structural incentives to obtain such a goal.

Keywords

media narratives reproducibility crisis science communication trust in scientists

1. The nature of media of science

Much of the public’s knowledge about science comes not from perusing scientific journals but from accounts conveyed by media (National Science Foundation, 2018) and by the stories they tell (Kaplan and Dahlstrom, 2017). Through agenda setting, media tell us what scientific topics to think about (McCombs and Shaw, 1972); through framing, they instruct us in how to think about them (Nisbet, 2010). By framing “stories . . . as relevant to the everyday lives of individual audience members, [they] can affect people’s beliefs about science . . .” (Hwang and Southwell, 2009). By increasing embrace of medical advances, such as vaccines, but also by enhancing support for federal funding of scientific research (Carter and Bélanger, 2005; Gauchat, 2012; Haerlin and Parr, 1999; Rogers et al., 2007), media stories also can improve science’s well-being (Wilholt, 2013) as well as individual lives. Although confidence in science remains high (Krause et al., 2019), media coverage can both positively (Hilgard and Jamieson, 2017) and negatively (Jasanoff, 1997) affect trust in it and in the scientific community. In this study, we specifically focus on media narratives that bring forward the question of the reliability of science as a source for information about the world (Oreskes, 2019).

There are multiple ways to define narratives. In this study, we follow the broad definition of narratives as stories containing information about setting, characters, and their motivations (Braddock and Dillard, 2016), “a representation of connected events and characters that has an identifiable structure, is bounded in space and time, and contains implicit or explicit messages about the topic being addressed” (Kreuter et al., 2007: 222). Since our understanding of the world comes in part through narratives, and exposure to and engagement with a narrative can produce narrative-consistent beliefs (Braddock and Dillard, 2016), it is important that media narratives about science accurately reflect its practices and norms.

Both experiments (Cappella and Jamieson, 1996) and surveys (Avery, 2009; Morris, 2007) have documented the effects of media coverage on trust in, beliefs about, and support for politicians and governmental institutions. In the scientific domain, however, conclusions about the existence of comparable media effects have been based on inferences either from the content analysis of patterns of media coverage (Boykoff, 2008; Jamieson, 2018) or on survey data (Anderson et al., 2014; Hmielowski et al., 2014) showing co-variance between the existence of certain content and changes in public trust in scientists. Missing is an experimental assessment of exposure to science narratives, in general, and focused, in particular, on media narratives about its reliability, failures, and corrective efforts. Specifically, this study investigates media narratives that could shape perceptions around the reliability of science (Oreskes, 2019). It is the goal of this study to fill this gap by examining the effects of four media narratives about science on perceptions pertaining to the reliability of science and scientists as custodians of knowledge and truth (Nisbet et al., 2002).

2. The potential effects of science narratives

The narratives that media convey about the reliability of science include four of interest here (Jamieson, 2018). The “honorable quest” story chronicles a scientific discovery in which the hero scientist has produced reliable and consequential knowledge through a journey yielding knowledge, often cast as discovery. By contrast, the “counterfeit quest” story recounts a specific kind of retraction of a published work, one not based in innocent error but in guileful conduct. Here, the scientist is not a hero but a poseur who has engaged in a dishonorable or counterfeit quest, gulled the custodians of knowledge into certifying a resulting finding unjustified by the evidence, been caught and eventually punished. The third narrative type indicts science and scientists, in general, for their failure to address a problem in its conduct. In it, science is portrayed in “crisis” or as “broken.” The final narrative focuses on scientists exploring and hence potentially remedying one of the problems focal to the crisis or broken narrative. Among the topics on which the latter two concentrate are failures to reproduce key findings, the rise in numbers of publication that have been retracted, failures of peer review and concerns about fraudulent conduct by scientists. Our purpose here is exploring whether exposure to these narratives affects trust in science and scientists and willingness to increase scientific funding.

The honorable quest or discovery narrative features terms such as “advance,” “path-breaking,” and “breakthrough” and tells the story of scientists who have advanced knowledge through a finding cast as new and important, as a discovery. These stories rarely acknowledge dead ends or false starts, and often fail to emphasize the need for additional ongoing research (Jamieson, 2018). Like the honorable quest narrative, the counterfeit quest story focuses on individuals. But where the former celebrates an advance by a scientist, the latter tells the story of a scientist or team of scientists whose journey to “discovery” and a resulting finding have been found wanting and purged from the scholarly record through retraction.

A counterfeit narrative assumes the existence of an earlier discovery story. So, for example, upon publication of Andrew Wakefield’s subsequently discredited research linking the Measles, Mumps, and Rubella (MMR) vaccination and autism, “discovery” headlines proclaimed, “Doctors link autism to MMR vaccination” (Laurance, 1998). But when evidence of error and deception surfaced, counterfeit quest stories announced, “Retracted autism study an ‘elaborate fraud,’ British journal finds” (CNN Wire Staff, 2011). As in the retraction of other widely publicized but spurious findings, Wakefield’s wrongdoing and ultimate punishment were chronicled. “British medical council bars doctor who linked vaccine with autism,” (Burns, 2010) noted one headline. A comparable pattern characterized the discrediting of the pluripotent stem cell findings of Haruko Obokata and her colleagues (Hilgard and Jamieson, 2017), and the retracted work of Anil Potti on microarray genetic analysis to create personalized cancer treatment (Federal Register, 2015).

The “science is broken/in crisis” narrative concentrates not on individual scientists but on broader and more systemic problems in a specific scientific discipline or in science writ large. Like the honorable question narrative, this one also recounts discovery, but here the discovery is of a problem that science as an institution or collective community has ignored or downplayed. These narratives call attention to a problem and assert that is persistent, pervasive, and costly (Jamieson, 2018). Prominent instances of the narrative include the Economist’s “Trouble at the lab: Scientists like to think of science as self-correcting. To an alarming degree, it is not” (The Economist, 2013) and the Los Angeles Times’ “Science has lost its way, at a big cost to humanity” (Hiltzik, 2013).

Driving the crisis or broken narrative are problems such as reports of unsuccessful efforts to replicate findings in psychology (Open Science Collaboration, 2015) and preclinical cancer research (Errington et al., 2014), a rise in the number of retractions (Marcus and Oransky, 2017), failures of peer review (Ferguson et al., 2014), misuse of statistics (Gelman and Loken, 2014), and multiple, highly publicized case in which supposed scientific achievements have been decertified, some of them for fraudulent behavior.

The news coverage that followed the scientific failures detailed earlier has the potential to deteriorate trust in science and scientists. In essence, the narratives around scientific failures and the ways science respond to them communicate messages about the reliability of science (Oreskes, 2019). We focus on beliefs about both the promise of science to advance humanity (science as beneficial) and about scientists’ trustworthiness (or in the case of negative depictions of science, potential reservations about scientists’ competence, integrity, and benevolence, see Jamieson, 2018).

3. Recontextualizing scientific failures: The norm of self-correction

The crisis or broken narrative is problematic for the communication of the reliability of science for a number of reasons. By ignoring efforts to address the identified problems, it invites an inaccurate inference about the nature of scientific inquiry. At the heart of the scientific endeavor is a culture of transparency and self-correction, characteristics that differentiate it from dogma and some other ways of knowing (Binder et al., 2016; Oreskes, 2019). If identifying and publicizing a problem in science is evidence of these norms in action, then it is inapt to use this evidence as the basis for characterizing the state of science as “rigor mortis” (Harris, 2017) or a rise in the number of retractions as a “retraction epidemic” (Fang et al., 2012) or failures to replicate as a “replication crisis” (Hendriks et al., 2020), without also indicating that the identification of the problem is a first stage in the process of correcting it and a sign of science’s health not its demise (Oreskes, 2019). Because detection of mistakes and errors is a central part of a healthy scientific process (Jamieson, 2017), journalists could treat a retraction not only as evidence of a first stage in science’s process of self-correction, but also of its transparency norm at work (Alberts et al., 2015). In this view, retractions are a signal that science is working as it should (Marcus and Oransky, 2017). So too is identification a systemic problem. When a media piece points to efforts by scientists to understand the nature and scope of an identified problem, we treat it as an instance of the “problem explored” narrative.

While the “problem explored” narrative better reflects the notion that science is engaging in the process of self-correction than does the crisis or broken story, it was not found in prior research (Jamieson, 2018) or in our own content analysis to be a prevalent narrative in media coverage of science (see Supplemental material). Instead, Jamieson (2018) found that most media accounts about scientific problems fail to include a statement about active measures taken by the scientific community to address them. While most coverage is dedicated to honorable quest depictions of scientific discovery (see Supplemental material), when a mistake or a fraud is discovered and put on the agenda, the journalistic focus on negative information makes “science is self-correcting” a less appealing headline than “science is broken” or “science is in crisis” (Hilgard and Jamieson, 2017). Yet, many have argued that the former is a more accurate representation of the nature and function of science (Alberts et al., 2015; Shiffrin et al., 2018).

To sum, while scholars have argued that media narratives about science can shape public perceptions about science and scientists (Kaplan and Dahlstrom, 2017), experimental evidence remains lacking. In this study, we examine the effects of exposure to specific narratives about scientific activity that were identified in prior research to be prevalent in American news media (Jamieson, 2018). We argue that the common narratives used to discuss scientific failures, by either individual exemplars (counterfeit quest) or systemic problems (science is in crisis), fail to communicate scientific norms of continuing exploration, scrutiny, and skepticism. In the next section, we propose hypotheses for their expected effects, and suggest that a narrative structure focused on how scientific failures are identified and corrected could alleviate the detrimental effects of the failure stories and thus improve science communication.

4. This study

Since the honorable quest or discovery stories celebrate the success of scientific efforts and the contribution of scientists to the development of knowledge and innovation (Jamieson, 2018), we expected exposure to them to yield the most positive perceptions about science. Due to their focus on scientific failures, we expected each of the problem stories to yield more negative perceptions than exposure to no stories about science (control). As for the differences between problem stories, we expected narratives focusing on the individual scientist (i.e. quest and counterfeit quest) to affect perceptions of science to a lesser extent than the crisis narrative (which focuses on problems in science, in general) because media accounts focused on a specific person’s experience have been found to elicit attributions of responsibility to the individual rather than to an institution such as government (Iyengar, 1994, 1996). Finally, we expected the problem explored narrative to ameliorate the negative effect of the crisis or broken focus by supplementing it with evidence that scientists are exploring the problem. As a result, we hypothesized that

H1. Exposure to narratives about science will influence trust in scientists, with the honorable quest or discovery stories yielding the highest trust and the failure narratives the lowest.

H2. Exposure to narratives about science will influence beliefs about scientists, with the honorable quest or discovery stories yielding a stronger agreement with the argument that science is beneficial and self-correcting and the failure narratives the weakest agreement with the statements.

H3. Exposure to narratives about science will influence support for funding of science, with the honorable quest or discovery stories yielding the strongest support and the failure narratives the weakest support.

Finally, we expected audiences’ interpretations of whether coverage of scientific problems was evidence that science is self-correcting to be influenced by readers’ preexisting attitudes and beliefs about science. According to Fishbein and Ajzen (1972), beliefs are subjects’ perceptions that an object or person has certain characteristics, qualities, or attributes, while attitudes include a crucial component of affect and valence (e.g. favorable–unfavorable evaluations). In this study, we focus on two specific beliefs—that scientists are reliable authorities (Binder et al., 2016) and the belief that the stories read could shed light on broader phenomena in real-world science (Brosius & Bathelt, 1994). According to Jamieson et al., (2019), the trustworthiness of science could be defined as the belief that scientists are competent (able to successfully perform their complex tasks), benevolent (working toward social good and not driven by competing interests), and of high integrity (honoring scientific norms of independence, scrutiny, and collaboration), and could, therefore, yield reliable scientific findings.

As for representativeness, media accounts often employ a type of information focusing on illustrating vivid individual cases (exemplars) to represent broad topics. The effectiveness of exemplars depends, in part, on the audience’s tendency to generalize from the specific exemplar features to a potentially larger aggregate population of similar exemplars (Zillmann, 2006). In our context, the effectiveness of the stories about individual scientists will depend, in part, on whether or not readers of the stories will generalize from the examples to a broader pattern of behavior among scientists in the real world. Since two of the three problem-based stories do acknowledge that those who wronged science were caught and punished (in the case of counterfeit) or that problems are being taken seriously and probed (in the case of problem explored), we hypothesize (H4) that on average, when reading these two kinds of problem stories, perceived representativeness will increase the belief that science is self-correcting. Since the crisis narrative focuses solely on problems in science, we do not expect the same finding among readers of that narrative. In addition, we expect perceptions of representativeness to interact with trust in scientists. Specifically, in H4, we posit that

H4a. The effect of exposure to narratives on belief that science is self-correcting will be moderated by perceived-representativeness and trust, such that those who perceive counterfeit and problem explored stories as representative will be more likely than those who do not see them as representative to believe that science is self-correcting.

Even if one perceives the stories read to be representative of science, they will not necessarily embrace the belief that science is self-correcting. For one, scientists can identify mistakes and errors in their own work or in the work of others and not pursue its correction. In order for corrective efforts to be effective and useful, one need to believe scientists are trustworthy, competent, and benevolent, and will, therefore, dedicate themselves to improving the scientific endeavor (Jamieson, 2018). We, therefore, expect readers who perceive the counterfeit and problem explored stories as representative but have lower levels of trust will be less likely to conclude that scientists who catch errors and fraud work toward correcting them:

H4b. The effect of representativeness on the belief that science is self-correcting will be moderated by trust, such that high levels of trust will strengthen the effect of perceived-representativeness, while lower trust will reverse it.

As detailed in the “Method” section, each condition randomly assigned participants to one of three stories representing the condition. The full texts can be found in the Supplemental material. The narratives created for this experiment were edited from actual media pieces, standardized for length, and were reliably categorized into the expected conditions by three independent coders (Krippendorff’s alpha of .85). To test the effects of exposure on perceptions, we conducted ANOVA tests, using Tukey contrasts for post hoc comparisons. Since each condition was represented by three individual stories (each participant read only one), we first compared the collapsed conditions (averaged across the three stories) and then separately compared the 13 stories (12 experimental conditions and control) to examine whether any produced different effects than others within condition.

5. Method

Participants and procedure

We randomly assigned 4497 Americans (2462 females, 3034 Whites, 552 African-Americans, 911 other) between the ages of 18 and 81 (M = 45.71, SD = 13.59)^1,2 from a Survey Sampling International (SSI) online panel³ to one of five conditions—four treatment conditions and one control.⁴ In each of the treatment conditions, participants read an article about science cast in one of the following four narratives: quest or discovery (condition 1), counterfeit quest (condition 2), “science is broken/in crisis” (condition 3), or “problem explored” (condition 4). Those in the control read about baseball. As a measure of attention, participants answered a question about the article’s topic. Those who failed the attention check were omitted (83.2% completed the study). After exposure to condition, participants answered a questionnaire and demographic data were collected. All procedures were approved by the University of Pennsylvania’s Institutional Review Board.

A common challenge to the internal validity of message effects studies, known as the “case-category confound” (Jackson, 1992) is that if studies use only one story to represent each condition, it is impossible to know whether the effects found were the result of the intended manipulation, or some specific features of that one story. To minimize the possibility that responses to a single story were idiosyncratic, three stories (labeled a, b, and c below) that fit each narrative were created from actual media accounts. Each participant read one media piece. Length ranged from 117 to 169 words.⁵ Because the aim of this study is to test the potential effects of exposure to ecologically valid media narratives about science, the stimuli examined in this study rely on edited instances located in media that are consistent with the four story lines whose existence was supported by content analysis. (For method used for supporting content analysis see the Supplemental material). To reduce a potential threat to validity due to case-category confound (Jackson, 1992), we used the following three stories per condition:

Condition 1: Quest or discovery narrative. These stories described a discovery in (a) immunotherapy to treat leukemia, (b) treatment of diabetes using a vaccine, or (c) astrophysics in the form of ghost particles in a distant galaxy. Each explained the discovery and noted its importance. The headlines of the articles (whose texts can be found in the Supplemental material) were (1) in cancer treatment, science produces a breakthrough; (2) diabetes breakthrough: Common vaccine can improve blood sugar levels long-term; and (3) in cosmic first, scientists detect “ghost particles” from a distant galaxy.

Condition 2: Counterfeit quest narrative. These pieces described the discrediting and retraction of widely publicized work in the areas of (a) tailored cancer treatments (Potti), (b) pluripotent stem cells (Obokata), or (c) eating behavior (Wansink), and reported the negative consequences for the researcher. The headlines were (1) how bright promise in cancer testing fell apart; (2) stem cell scientists implicated in scandal; and (3) a Cornell scientists’ downfall.

Condition 3: “Science is broken/in crisis” narrative. These stories included information on lack of replication of key studies, concerns about peer review, evidence of increasing numbers of retractions, and worries that the incentive structures in science are leading to corner-cutting or misconduct. The first story described the failure to replicate 47 “landmarks in the science of cancer,” widespread statistical mistakes, failures in peer review, and pressures that “push scientists to cut corners.” The second cited three instances in which fraudulent scientific research published in top journals was retracted, noted the frequency of retraction, specified harm done by one of the retracted articles, and noted the need for the US Office of Research Integrity to sanction some researchers for misconduct. The third story reported an “alarming increase in the number of retractions,” noted the frequency with which misconduct was a potential cause of retractions, and indicted the “perverse incentives that drive scientists to make sloppy mistakes or knowingly publish false data.” The headlines were (1) scientists like to think of science as self-correcting. To an alarming degree, it is not; (2) what is behind big science frauds? and (3) misconduct widespread in retracted science papers, study finds.

Condition 4: “Problem explored” narrative. Each described replication problems in psychology and efforts by scientists or projects to determine whether replications recreated under rigorous conditions produced the same results as the original studies. As in the crisis condition, these stories focused on a systemic problem in scientific work and not the wrongdoing of an individual (the focus of the counterfeit condition). The exploratory efforts captured in this condition were conducted by teams of scientists. The headlines were (1) psychologists explore ways to increase the reliability of psychology studies; (2) science has its problems, but the web could be the fix; and (3) psychology under scrutiny and self-correction.

Condition 5: Baseball: An early history. The control condition recounted the origins of baseball.

To sum, our study employs a between-subject design, where each participant is randomly assigned to one of five conditions, and then again randomly assigned to one of three stories representing this condition. In other words, our study randomly assigned each participant to one of 13 stories (three for each of the four science narrative types and one control). In our analysis, we first examined whether exposure to different stories that were supposed to represent the same category (e.g. exposure to different versions of the crisis narrative) yielded similar effects. If there were no differences between stories representing each category, we aggregate participants into the five conditions (i.e. instead of the 13 conditions represented by each story).

Measures

Trust in scientists

Three items ranging between 1 (low) to 5 (high) capturing three dimensions of credibility (Fiske et al., 2007): “thinking about scientists in general, to what extent would you say that scientists in general [‘share your values’/‘are competent at what they do’/‘are honest and trustworthy’]” (α = .81).

Belief that science is beneficial (three items)

“Think about the scientific findings produced by U.S scientists in the past decade just your best guess, how much of that science [has benefited the country as a whole / has benefited people like you / has produced important knowledge], with the following possible responses: ‘Not much at all’, ‘Not too much’, ‘Some’ or ‘A lot’” (α = .86).

Belief that science is self-correcting

Three items ranging from 1 (rarely) to 5 (often): “When a study is flawed, the scientists involved in it catch and correct the mistake prior to its publication,” “When fraud occurs in scientific research, how often do you think it is caught,” and “When scientists make mistakes in their research, how often do you think other scientists catch it”? (α = .73).

Support for funding of science

Four items ranging from 1 (funding should be decreased significantly) to 5 (funding should be increased significantly): “Please indicate which of the following statements comes closest to your view: Regarding the funding for [cancer research / research about the planets, stars, and universe / research by social scientists / research in medicine]” (α = .73).

Representativeness

“Just your best guess, how often do the kinds of scientific activities presented in the article actually occur in SCIENCE,” ranging from 1 (very rarely) to 5 (very often), with 3 defined as “some of the time.”

6. Results

Overall, across conditions, trust in scientists was moderately high (M = 3.68, SD = .75). Beliefs that science is self-correcting (M = 3.30 SD = .75) and beneficial (M = 3.07, SD = .73) were moderate to high. Support for increasing funding was moderate (M = 3.64, SD = .75).

H1 predicted that trust in scientists will be strongest among readers of the honorable quest or discovery stories, followed by the control, problem-explored, counterfeit, and crisis conditions. A one way analysis of variance showed that the effect of exposure on trust was significant F(4, 4492) = 2.851, p = .022. The strongest trust was expressed among those who have read the discovery stories (M = 3.74, SD = .74), followed by counterfeit (M = 3.70, SD = .73), problem explored (M = 3.67, SD = .74), control (M = 3.67, SD = .75), and crisis (M = 3.62, SD = .77). The planned post hoc contrasts analysis revealed that only the difference between discovery and crisis was significant (p = .010). As expected, the discovery condition yielded the strongest trust and the crisis condition the weakest, though other comparisons were not statistically significant. H1 was partially supported.

H2 predicted that agreement with the beliefs that science is beneficial and self-correcting will be more positive among readers of the honorable quest or discovery stories, followed by the control, problem-explored, counterfeit, and crisis conditions. An one way analysis of variance showed that the effect on the belief that science is beneficial was significant F(4,4492) = 9.201, p < .001). Belief that science is beneficial was strongest among those who were exposed to the control condition (M = 3.18, SD = .75), followed by discovery (M = 3.10, SD = .70), counterfeit (M = 3.04, SD = .75), crisis (M = 3.01, SD = .73), and problem explored (M = 3.00, SD = .73). The planned post hoc contrasts analysis revealed that the difference between the control and the counterfeit, crisis, and problem explored was significant (p < .001) and the problem explored was significantly lower than the discovery condition (p < .001). All other comparisons were not significant.

As for the belief that science is self-correcting, the one way analysis of variance showed that the effect on the belief that science is self-correcting was significant F(4,4492) = 8.711, p < .001). The belief that science is self-correcting was strongest among those reading stories from the discovery condition (M = 3.37, SD = .76), followed by the control (M = 3.34, SD = .76), counterfeit (M = 3.33, SD = .74), problem explored (M = 3.26, SD = .75), and crisis (M = 3.18, SD = .75). The planned post hoc contrasts analysis revealed that the difference between the crisis condition and the control, discovery, and counterfeit conditions was statistically significant (p < .001). The difference between problem explored and the discovery was also significant (p = .034). H2 was partially supported.

H3 predicted that support for funding science will be strongest among readers of the honorable quest or discovery stories, followed by the control, problem-explored, counterfeit, and crisis conditions. An one way analysis of variance showed that the effect on support for funding was not significant F(4, 4492) = .748, p = .559. No significant differences were found between the control (M = 3.62, SD = .83), counterfeit (M = 3.58, SD = .83), discovery (M = 3.57, SD = .82), problem explored (M = 3.57, SD = .82), and crisis conditions (M = 3.55, SD = .84). H3 was not supported.

The means per condition for H1–H3 can be seen in Table 1.

Table 1.

Means and standard deviations for trust, beliefs, support for science as a function of exposure to condition.

Dependent variable	Honorable quest	Counterfeit quest	Science is broken	Problem explored	Control
Trust	3.74 (.74)	3.699 (.73)	3.623 (.77)	3.669 (.74)	3.674 (.75)
Science is beneficial	3.097 (.70)	3.039 (.75)	3.01 (.73)	3.00 (.73)	3.178 (.75)
Science is self-correcting	3.366 (.75)	3.33 (.74)	3.179 (.75)	3.263 (.75)	3.343 (.76)
Support for science	3.565 (.82)	3.582 (.83)	3.555 (.84)	3.567 (.82)	3.615 (.83)

Standard deviations appear in parentheses next to each mean.

H4 predicted a three-way interaction for the effect on the belief that science is self-correcting between exposure to conditions, trust, and perceived-representativeness. Specifically, we hypothesized that (a) those who perceive the counterfeit quest and problem explored (but not crisis) stories to be representative would be more likely to believe that science is self-correcting compared to those who believe the opposite and (b) higher trust in scientists would strengthen and lower trust in scientists reverse the effect (i.e. those who see the story as more representative but have lower levels of trust in scientists would be less likely to conclude that scientists catch errors and fraud for participants exposed to the two narratives portraying systemic problems in science but not for those exposed to the counterfeit story).

As can be seen in Table 2, a multiple regression found a significant three-way interaction between condition (problem explored vs counterfeit), representativeness, and trust (p = .004). On average, trust increased the perception that science is self-correcting (p < .001) but perceived-representativeness did not (p = .796). However, when looking at the three-way interaction, participants in the counterfeit condition who perceived the activities to be representative of science expressed more belief that science is self-correcting, independent of the level of trust (see Figure 1, where the slope for counterfeit across levels of trust remains the same). However, the effect of perceived representativeness on the belief that science is self-correcting for participants in the problem explored condition was moderated by trust. Specifically, among readers of the problem explored condition, for those with lower trust in scientists (the left panels in Figure 1), the more they believed the stories are representative the less likely they were to believe science is self-correcting. However, the effect was canceled for those with moderate trust (trust = 3), and reversed for those who had high trust in scientists (right panels of Figure 1). For those with high levels of trust in scientists who read stories from the problem explored condition, the more representative the stories were, the more they believed that science is self-correcting. Since the three-way interaction was significant for the problem explored but not the crisis condition, H4 was supported.

Table 2.

Regression analysis for interaction between condition, trust, and perceived-representativeness on the belief that science is self-correcting.

Effect	Estimate	SE	p
Intercept	1.325	.295	<.001***
Condition (crisis vs counterfeit)	−.803	.464	0.083*
Condition (problem explored vs counterfeit)	.862	.421	.041*
Representativeness	.022	.087	.796
Trust	.495	.076	<.001***
Crisis × representativeness	−.272	.128	.034*
Problem explored × representativeness	−.329	.125	.008**
Crisis × trust	−.174	.122	.153
Problem explored × trust	−.258	.111	.020*
Representativeness × trust	.007	.022	.721
Crisis × representativeness × trust	.052	.033	.120
Problem explored × representativeness × trust	.091	.032	.004**

SE: standard error.

p < .05; **p < .01; ***p < .001.

Figure 1.

The interaction between condition, perceived representatives, and trust on the belief that science is self-correcting.

7. Discussion

The media, which remain a primary source of scientific information, often fail to recognize the role retractions and failed replications play in scientific progress (Hilgard and Jamieson, 2017). Indeed, a media narrative emphasizing how the identification of scientific mistakes could advance science through the healthy process of self-correction is largely absent in real world journalistic coverage (Jamieson, 2018). Instead of an indicator that science is unreliable, for example, retractions can be thought of as an indication that science is self-correcting (Marcus and Oransky, 2017).

Importantly, we do not claim that scientific failures should not be covered in the media. Scientists make mistakes, some intentional and some not, and we believe that the public benefits from journalistic coverage of such failures. What we argue is that such mistakes are a healthy part of the scientific process (Oreskes, 2019) and that media coverage could better “reflect the realization that because critique and self-correction are hallmarks of the scientific enterprise, instances in which scientists detect and address flaws constitute evidence of success, not failure, and exemplify the underlying protective mechanisms of science at work” (Jamieson, 2018: 6). The coverage we propose in the problem explored narrative also better communicates the limitations of scientific findings. Specifically, it can help readers see scientific discoveries and findings as open for further ongoing scrutiny, conveying the message that “nothing in science is set in stone and only through experience and observation can we successfully advance our understanding” (Denholme, 2020: 121).

The absence of such stories could be the result of multiple factors, including perceived newsworthiness (Stryker, 2002) or the minimal incentives scientists have to share replications with journalists (as opposed to more novel studies and discoveries, see Harris, 2017). An alternative explanation could be that since journalists expect science to yield reliable results (as evident in the popularity of the honorable quest narrative), scientific failures could be seen as more novel and surprising, which in turn could make them more appealing to journalists (Stryker, 2002). That explanation, however, does not account for the fact that when failures are covered, replications and explanations of the nature of science remain absent. In both cases, due to the expected negative effects of problem-focused stories, and the relative absence of stories that put scientific failures in context by explaining attempts to address them, in this study, we assessed the potential effects of exposure to media narratives about scientific success and failure, and examined whether the additional discussion of efforts to address them in the problem explored condition blunted the negative effects of the exposure to scientific failures in the first part of that narrative.

This study demonstrates the adverse, if small, effects of problem-focused media narratives on trust in, beliefs about, and support for scientists and points to the importance of perceived representativeness and audience trust in scientists in the audience’s response to them. Perceived representativeness and trust in scientists affect the likelihood that one will infer from the two systematic problem stories (i.e. crisis/broken and problem explored) that science is or is not self-correcting. When participants were exposed to systemic problem stories but not to the scientist-focused ones, trust in scientists moderated the effect of perceived representativeness on the belief that science is self-correcting. We suppose that those exposed to a counterfeit quest (retraction) story which they saw as representative attributed the described failure to the individual scientist, while those exposed to stories about systematic problems used their level of trust in scientists to determine whether errors and fraud were likely to be caught. Future studies should measure attribution of responsibility directly.

The findings in regard to the problem explored condition could be seen as encouraging from a science communication perspective, though the interaction effect suggests that the potential could be limited by mistrust. On one hand, the range of negative effects of the crisis narrative compared to the more limited ones of the problem explored stories argues that scientists should focus press attention not on problems and failures in science but, assuming that they are in fact being addressed, on the ways in which they can and are being tackled. Such emphasis could be of benefit, at least for those who trust scientists to begin with. On the other hand, the interaction effect suggests that those who distrust scientists may not be persuaded by the addition of information about corrective attempts. Thus, in order for such messaging to be effective, an ongoing effort by the scientific community to signal and bolster the trustworthiness of science is needed (Jamieson et al., 2019).

Meanwhile, the representativeness effect underscores the value of demonstrating that efforts to address identified problems are ongoing and potentially productive. However, we once again acknowledge that news coverage of science is the product of a negotiation between scientists and journalists, both of which may be incentivized to prioritize more sensational, novel stories (Harris, 2017; Stryker, 2002) on the expense of the somewhat pedestrian, yet crucial, topic of self-correction. Better science communication will therefore require a cooperation from both scientists and the media.

This study is limited by its reliance on self-reports and by its inability to assess the effects of recurrent exposure to one of the narratives or to them in combination. Although the impact of single instances of communication is usually short-lived (Hilgard and Jamieson, 2017), recurrent exposure can produce sustained effects (Gerbner, 1998). In addition, our study examined the effects of a one-time exposure to science narratives in a controlled experimental environment. In the real world, people are likely to learn about science through multiple sources, and their exposure could interact with prior knowledge and exposures. It could also be influenced by socioeconomic and other variables. However, to better estimate the effect of the experimental manipulation, we opted not to use demographics as covariates in our models, as differences between groups on multiple observed and unobserved variables are expected to cancel each other out (Mutz et al., 2019). Future studies could shed more light on the role of individual characteristics as well as the cumulative effect of multiple exposures to multiple narratives across media. Meanwhile, scientists and those who communicate about their findings need to develop narratives that reflect the nature of scientific inquiry and its norms and practices as well as the practices it uses to detect and correct error as well as fraud. Another limitation is the use of relatively short media narratives, edited from longer real news articles. The use of short version allowed us to better control the manipulation and thus increase internal validity, but future studies may examine the effect of longer, more complex news articles. Finally, our content analysis, as well as prior research (Jamieson, 2018) have indicated that the most common perspective through which science is being discussed in the news remains the honorable quest story, with its focus on successful discovery efforts. Due to increasing discussions among academics and the press about the so-called “replicability crisis,” and in light of growing mistrust in science among segments of the population, we have focused our analysis on the potential detrimental effects of problem-stories and the need to design alternative stories that better contextualize scientific failures. Future studies would be able to follow trends in media coverage to better understand the real-world effects of problem stories.

In sum, the aim of this study was to provide empirical evidence for the effects of exposure to media narratives about the reliability of science on people’s perceptions about science and scientists (Kaplan and Dahlstrom, 2017). We exposed participants to media narratives with high ecological validity (i.e. similar to real world news stories about science), representing common story types identified in prior research (Jamieson, 2018). While the honorable quest story remains the most common in the media, when failures are discussed, they tend to ignore scientific attempts to address the problems. We argue that such narratives about individual or systemic scientific failures fail to communicate scientific norms of continuing exploration, scrutiny, and skepticism and could, particularly if being presented regularly and consistently, harm public trust and confidence in scientific work. We also show how our suggested problem-explored narrative could ameliorate those detrimental effects and yield more positive beliefs and attitudes about science and scientists, by better communicating scientific norms of continuing exploration, scrutiny, and skepticism. As science communication in news media is the result of a negotiation between scientists and journalists, these results could guide future science communication efforts by both journalists and members of the scientific community. Like others before us (e.g. Valentine et al., 2011), we believe that such a change will require scientific institutions to reconsider the current incentive structure, that prioritizes the promotion of novel, statistically significant discoveries over a rigor self-correction efforts (Harris, 2017).

Supplemental Material

sj-pdf-1-pus-10.1177_09636625211012630 – Supplemental material for The effects of media narratives about failures and discoveries in science on beliefs about and support for science

Supplemental material, sj-pdf-1-pus-10.1177_09636625211012630 for The effects of media narratives about failures and discoveries in science on beliefs about and support for science by Yotam Ophir and Kathleen Hall Jamieson in Public Understanding of Science

Footnotes

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was funded by the Science of Science Communication endowment of the Annenberg Public Policy Center of the University of Pennsylvania.

ORCID iDs

Yotam Ophir

Kathleen Hall Jamieson

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Yotam Ophir, PhD, is an Assistant Professor of Communication at the University at Buffalo, State University of New York. Dr Ophir’s work is focusing on media effects, persuasion, and misinformation in the area of political and science communication. His work employs experiments, surveys, and computational methods for text analysis.

Kathleen Hall Jamieson is the Elizabeth Ware Packard Professor of Communication at the University of Pennsylvania’s Annenberg School for Communication, the Walter and Leonore Director of the university’s Annenberg Public Policy Center. Dr Jamieson employs rhetorical analysis, surveys, and experiments to understand campaign communication, the science of science communication, and ways to blunt misinformation and conspiracy theories.

References

Alberts

Cicerone

Fienberg

Kamb

McNutt

Nerem

, et al (2015) Self-correction in science at work. Science 348(6242): 1420–1422.

Anderson

Brossard

Scheufele

Xenos

Ladwig

(2014) The “nasty effect”: Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication 19: 373–387.

Avery

(2009) Videomalaise or virtuous circle? The influence of the news media on political trust. The International Journal of Press/Politics 14: 410–433.

Binder

Hillback

Brossard

(2016) Conflict or caveats? Effects of media portrayals of scientific uncertainty on audience perceptions of new technologies. Risk Analysis 36: 831–846.

Boykoff

(2008) Lost in translation? United States television news coverage of anthropogenic climate change, 1995–2004. Climatic Change 86: 1–11.

Braddock

Dillard

(2016) Meta-analytic evidence for the persuasive effect of narratives on beliefs, attitudes, intentions, and behaviors. Communication Monographs 83(4): 446–467.

Brosius

Bathelt

(1994) The Utility of exemplars in persuasive communications. Communication Research 21(1): 48–78.

Burns

(2010) Council bars doctor who claimed link between vaccines and autism. The New York Times. Available at: https://www.nytimes.com/2010/05/25/health/policy/25autism.html

Cappella

Jamieson

(1996) News frames, political cynicism, and media cynicism. Annals of the American Academy of Political and Social Science 546: 71–84.

10.

Carter

Bélanger

(2005) The utilization of e-government services: Citizen trust, innovation and acceptance factors. Information Systems Journal 15: 5–25.

11.

CNN Wire Staff (2011) Retracted autism study an “elaborate fraud”. British Journal Finds (WWW document). Available at: http://www.cnn.com/2011/HEALTH/01/05/autism.vaccines/index.html (accessed 23 July 2018).

12.

Denholme

(2020) Some studies are more equal than others. Nature Physics 16: 121.

13.

Errington

Iorns

Gunn

Tan

Lomax

Nosek

(2014) An open investigation of the reproducibility of cancer biology research. Elife 3: 1–9.

14.

Fang

Steen

Casadevall

(2012) Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences of the United States of America 109: 17028–17033.

15.

Federal Register (2015) Findings of research misconduct (WWW document). Federal Register. Available at: https://www.federalregister.gov/documents/2015/11/09/2015-28437/findings-of-research-misconduct (accessed 10 April 2019).

16.

Ferguson

Marcus

Oransky

(2014) Publishing: The peer-review scam. Nature News 515: 480.

17.

Fishbein

Ajzen

(1972) Attitudes and opinions. Annual Review of Psychology 14: 487–544.

18.

Fiske

Cuddy

AJC

Glick

(2007) Universal dimensions of social cognition: Warmth and competence. Trends in Cognitive Sciences 11: 77–83.

19.

Gauchat

(2012) Politicization of science in the public sphere: A study of public trust in the United States, 1974 to 2010. American Sociological Review 77: 167–187.

20.

Gelman

Loken

(2014) The statistical crisis in science. American Scientist 102: 460–465.

21.

Gerbner

(1998) Cultivation analysis: An overview. Mass Communication and Society 1: 175–194.

22.

Haerlin

Parr

(1999) How to restore public trust in science. Nature 400: 499.

23.

Harris

(2017) Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions. New York, NY: Basic Books.

24.

Henderiks

Kienhues

Bromme

(2020) Replication crisis = trust crisis? The effect of successful vs failed replication on laypeople’s trust in researchers and research. Public Understanding of Science 29: 270–288.

25.

Hilgard

Jamieson

(2017) Does a scientific breakthrough increase confidence in science? News of a Zika vaccine and trust in science. Science Communication 39: 548–560.

26.

Hiltzik

(2013) Science has lost its way, at a big cost to humanity. Los Angeles Times, 27 October. Available at: https://www.latimes.com/business/la-xpm-2013-oct-27-la-fi-hiltzik-20131027-story.html

27.

Hmielowski

Feldman

Myers

Leiserowitz

Maibach

(2014) An attack on science? Media use, trust in scientists, and perceptions of global warming. Public Understanding of Science 23: 866–883.

28.

Hwang

Southwell

(2009) Science TV news exposure predicts science beliefs: Real world effects among a national sample. Communication Research 36: 724–742.

29.

Iyengar

(1994) Is Anyone Responsible? How Television Frames Political Issues. Chicago, IL: University of Chicago Press.

30.

Iyengar

(1996) Framing responsibility for political issues. The ANNALS of the American Academy of Political and Social Science 546: 59–70.

31.

Jackson

(1992) Message Effects Research. New York, NY: Guilford Press.

32.

Jamieson

(2017) The need for a science of science communication: Communicating science’s values and norms. In: Jamieson

K. H.

Kahan

Scheufele

D. A.

(eds) The Oxford Handbook of the Science of Science Communication. Oxford: Oxford University Press, pp. 15–24.

33.

Jamieson

(2018) Crisis or self-correction: Rethinking media narratives about the well-being of science. Proceedings of the National Academy of Sciences of the United States of America 115: 2620–2627.

34.

Jamieson

McNutt

Kiermer

Sever

(2019) Signaling the trustworthiness of science. Proceedings of the National Academy of Sciences of the United States of America 116: 19231–19236.

35.

Jasanoff

(1997) Civilization and madness: The great BSE scare of 1996. Public Understanding of Science 6: 221–232.

36.

Kaplan

Dahlstrom

(2017) How narrative functions in entertainment to communicate science. In: Jamieson

Kahan

Scheufele

(eds) The Oxford Handbook of the Science of Science Communication. Oxford University Press, pp. 311–320.

37.

Krause

Brossard

Scheufele

Xenos

Frankie

(2019) The polls - TrendsAmericans’ trust in science and scientists. Public Opinion Quarterly 84(1): 181.

38.

Kreuter

Green

Cappella

Slater

Wise

Storey

. . .

Wooley

(2007). Narrative communication in cancer prevention and control: A framework to guide research and application. Annals of Behavioral Medicine 33(3): 221–35.

39.

Laurance

(1998) Doctors link autism to MMR vaccination (WWW document). The Independent, 22 October. Available at: https://www.independent.co.uk/news/doctors-link-autism-to-mmr-vaccination-1147081.html (accessed 23 July 2018).

40.

McCombs

Shaw

(1972) The agenda-setting function of mass media. The Public Opinion Quarterly 36: 176–187.

41.

Marcus

Oransky

(2017) Is there a retraction problem? And, if so, what can we do about it? In: Jamieson

Kahan

Scheufele

(eds) The Oxford Handbook of the Science of Science Communication. Oxford: Oxford University Press, pp. 119–126.

42.

Morris

(2007) Slanted objectivity? Perceived media bias, cable news exposure, and political attitudes. Social Science Quarterly 88: 707–728.

43.

Mutz

Pemantle

Pham

(2019) The perils of balance testing in experimental design: Messy analyses of clean data. The American Statistician 73(1): 32–42.

44.

National Science Foundation (2018) Chapter 7: Science and technology: Public attitudes and understanding (WWW document). National Science Foundation. Available at: https://www.nsf.gov/statistics/2018/nsb20181/assets/404/science-and-technology-public-attitudes-and-understanding.pdf (accessed 24 July 2018).

45.

Nisbet

(2010) Knowledge into action: Framing the debates over climate change and poverty. In: D’Angelo

Kuypers

(eds) Doing News Framing Analysis: Empirical and Theoretical Perspectives. New York, NY: Routledge, pp. 43–83.

46.

Nisbet

Scheufele

Shanahan

Moy

Brossard

Lewenstein

(2002) Knowledge, reservations, or promise? A media effects model for public perceptions of science and technology. Communication Research 29: 584–608.

47.

Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349: aac4716-1.

48.

Oreskes

(2019) Why Trust Science. Princeton, NJ: Princeton University Press.

49.

Rogers

Amlôt

Rubin

Wessely

Krieger

(2007) Mediating the social and psychological impacts of terrorist attacks: The role of risk perception and risk communication. International Review of Psychiatry 19: 279–288.

50.

Shiffrin

Börner

Stigler

(2018) Scientific progress despite irreproducibility: A seeming paradox. Proceedings of the National Academy of Sciences of the United States of America 115: 2632–2639.

51.

Stryker

(2002) Reporting medical information: Effects of press releases and newsworthiness on medical journal articles’ visibility in the news media. Preventive Medicine 35(5): 519–30.

52.

The Economist (2013) Trouble at the lab. The Economist, 18 October. Available at: https://www.economist.com/briefing/2013/10/18/trouble-at-the-lab

53.

Valentine

Biglan

Boruch

Castro

Collins

Flay

, et al (2011) Replication in prevention science. Prevention Science 12: 103–117.

54.

Wilholt

(2013) Epistemic trust in science. The British Journal for the Philosophy of Science 64: 233–253.

55.

Zillmann

(2006) Exemplification effects in the promotion of safety and health. Journal of Communication 56: S221–S237.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB