Holding Science to Account: A Qualitative Study of Practices and Challenges of Watchdog Science Journalism

Monitoring Science

Our participants found potential stories by monitoring for retractions, fraud, problematic data, paper mills, predatory journals, ethics breaches, problematic use of GenAI, and related integrity issues. A few participants also considered more “human” problems in science, such as equity and diversity-related issues or harassment, or monitored problems adjacent to the scientific system (e.g., international science relations, misinformation, government muzzling of scientists, inappropriate public spending).

Participants used multiple strategies to monitor science, including practices found in previous science journalism research (e.g., newsletters, journal alerts, conferences/events, preprint servers, other media outlets’ reporting). Sometimes participants were commissioned to write watchdog stories by editors, as elaborated below.

Beyond these well-established practices, personal and often anonymous tips (e.g., from suspicious colleagues or co-authors, advocacy groups, lawyers) were key. Often, whistleblowers were familiar with the journalist’s work or had spoken with them during an interview for another story. As J12 explained, “[T]here are scientists I know, trust, and chat to, who will come to me with stuff and say, ‘You know, this is a bit odd.’”

Participants also searched for story ideas using specialized information sources, such as retraction notices, Retraction Watch, and other websites “writing about retractions . . . without emotion” [J2]. These sources were valued because they acted as warning signals that there may be a newsworthy issue warranting further investigation, but also for their perceived objectivity. Whereas sources who tipped off journalists were sometimes “a little promotional” [J9], a retraction was perceived as “a fact” [J17] and an easier hook for a story.

In addition, participants emphasized the value of “spaces where scientists will speak somewhat informally” [J3] for finding potential stories. These included PubPeer, an online post-publication peer review platform where researchers and “science sleuths” can discuss study flaws and integrity issues (Wapner, 2024). Participants also monitored scientific message boards, researchers’ blogs, and scientists and science sleuths on social media. Many noted they had found the former Twitter to be particularly useful, as it allowed them to “eavesdrop on conversations” [J1] and “hear what the community is saying about a particular paper” [J6]. However, this practice appeared to be changing after Elon Musk’s takeover, with BlueSky lamented as “not quite the story generator that Twitter was” [J3]. Several participants were unsure how to replace what had once been one of their most important monitoring tools.

Selecting Stories

Established science journalism practices played a role in journalists’ decisions around which stories to pitch and pursue. Participants considered whether the issue excited them personally, interested their editor, fit the target media outlet’s remit, and had already been covered by competing outlets. Some also used their “own gut and bullshit detector” [J9] to assess whether potential stories were worth exploring.

Added to these are classic news values such as timeliness, novelty, surprise, elites, relevance, conflict/controversy, drama, and, most importantly, impact (for science and society). Participants used assessments of prestige and reputation (i.e., of journals, researchers, and/or institutions) to establish scientific impact, as in science journalism more broadly. However, such newsworthiness evaluations were not universal, as some journalists felt problematic research was “automatically of quite wide interest” [J6] to audiences and thus easy to pitch to editors.

Alongside these well-established selection criteria, journalists described feasibility as a key consideration for whether to pursue stories. They explained that watchdog stories are time and resource-intensive but sometimes lead to dead ends, making it essential to ask, “‘Is this something that we could actually get?’ So that we don’t waste weeks or months on digging around” [J9]. Relatedly, the journalistic “return” on the effort involved was perceived as lower than for other stories. As J12 explained, political watchdog stories were likely to “be on the front page of the paper,” but even the most impactful science watchdog stories could only hope for “a page lead on page seven that exists for one day.” These assessments were thus considered alongside the newsworthiness of the story, leading journalists to balance “How easy is it to get the story versus what’s the public interest?” [J8]. In some cases, feasibility concerns meant important, newsworthy story ideas were abandoned.

Investigating Research Integrity Issues

Participants described investigating watchdog stories as slow and labor-intensive, requiring multiple rounds of collecting, vetting, and triangulating information via source interviews, document gathering, and analysis. Implicit was an unstated—but universally understood—threshold of evidence that had to be met before the journalist could “pull the trigger” [J8] on a story. Participants used colorful metaphors to describe this process: “You usually start from the outside like an onion, and you peel away those layers” [J4], gathering “as much information as you can get that will paint that granular picture” [J13]. The granularity of this picture seemed to depend on the level of risk for journalists and targets, as some stories could ruin scientists’ careers or lead to expensive legal battles. In these higher-risk cases, participants were sure to “cross check or counter check” [J14] more often, “build[ing] my case like a prosecutor in a criminal trial” [J4].

Investigations ideally involved doing “interviews with as many people as possible” [J10], including whistleblowers, science sleuths, co-authors, research integrity experts, statisticians, and experts in the field. Hearing from many individuals was important for contextualizing issues, capturing multiple perspectives, and verification, because some sources were unwilling to go on the record and others were of unknown credibility. By triangulating and corroborating claims from diverse sources, participants felt more confident that their reporting was accurate and balanced.

When risks were high or interviews hard to come by, journalists relied on “as much documentation as possible” [J17], including academic research, reports, retraction notices, legal evidence, other media coverage, datasets, open research code, government websites, and social media. Internal documents (i.e., memos, reports, emails) were highly valued but often hard to access, as sources had to leak them or journalists had to submit freedom of information (FOI) requests—an expensive and slow practice. A small number of journalists used open peer review reports, including those from journals and post-publication review platforms. J21 explained that review reports were “great for accountability” but also “for highlighting good science” because they provide expert perspectives on new studies without the need for interviews.

When relevant documents were scarce, some participants made creative use of whatever they could access, such as public salary information or targets’ social media accounts. As J10 summarized, “You have to do a really good job of thinking, ‘What information is out there and what you can get from it?’”

Whatever the sources used, participants approached verification skeptically and carefully, fact-checking their work (sometimes line by line), keeping rigorous records, and investigating potential conflicts of interest of sources. Some described their watchdog work as being “much like the practice of science” [J4], involving developing hypotheses, gathering and analyzing data, and deriving conclusions.

Once journalists felt the threshold of evidence had been met, they typically took one last step: a “right of reply email” [J17] to the target and/or their institution that provided an opportunity to comment. This practice rarely, if ever, led to useful responses, nor affected whether journalists published their story. Yet it allowed participants to show they had done their due diligence and, especially for those working in the U.K., stave off potential lawsuits. This final step was one of the few parts of the investigative process where press officers played a role, either by providing official statements on behalf of their institutions or, more rarely, connecting journalists to the person accused of wrongdoing.

Reporting Watchdog Science Stories

When it came to writing stories, participants adopted classic journalistic practices, such as using succinct, accessible language and clear explanations, especially when working for outlets targeting broad, public audiences. As with verification, they strove to be accurate and objective, presenting “neutral” [J5] accounts of “just the facts” that would allow audiences to “decide for themselves” [J2] whether disciplinary action was needed. Participants stressed that accuracy was especially important in headlines, as these are often the only things audiences read, especially when covering issues still under debate within the scientific community.

Some journalists tried to entertain, using engaging hooks, telling the “human story” [J5] behind the misconduct, and spotlighting “really interesting characters” [J6] to “sneak integrity stuff in” [J2] without losing the audience’s interest. Importantly, these strategies were not universally applied because some journalists considered watchdog stories to be inherently newsworthy, as mentioned above.

By far, the most important practice was contextualization, as participants felt that providing sufficient background about the larger scientific system was necessary for putting transgressions into perspective. Aligning with Jamieson’s (2018) recommendations, they strove to communicate that science is self-correcting and that a single researcher’s actions do not indicate a broken system. As J1 explained when reflecting on a story they had written:

One thing I like about this piece is that . . . it goes beyond the story itself and looks at what journals are doing to screen figures. I think that’s important to do when it comes to pieces like this—to not just focus on the paper, but to look at the broader structural systems . . . that enable these kinds of problems.

For some, this desire to contextualize was also driven by anticipation of (public) skepticism and distrust in science.

Individual-Level Influences

Experience and reputation appeared as key facilitators of participants’ work across every stage of news production. Well-established watchdog journalists often received tips, so that “you’re not necessarily looking for these stories, but they come towards you” [J3]. Editors were more trusting of these reporters and thus more receptive to their pitches. Experience also made story selection and investigation easier, due to the “gut feeling” [J13] and subject matter expertise developed through years of watchdog work. Having a reputation as a watchdog meant sources were sometimes more willing to go on the record, as “the more you report, the more people see you as someone who they can talk to” [J17].

Journalists’ status within the newsroom was important, with freelancers experiencing additional barriers compared to on-staff journalists and editors. Without the security of a stable job, they didn’t “have the bandwidth at all times to drop things and take a story” [J3]. Editors sometimes trusted them less in relation to editorial standards and were more reluctant to accept their pitches. Some journalists said freelancers’ interview requests were more frequently “knocked down” than those of on-staff journalists, which they saw as “a really terrifying development because lots of freelancers are doing that bread-and-butter work of reporting” [J13].

Finally, participants noted that passion helped them persevere through the weeks or months needed to pursue these stories. Having a background in science was helpful for understanding and vetting research, and strong record-keeping skills were crucial for managing extensive documentation.

Routine-Level Influences

The cycle-driven nature of newswork (Downs, 1972) and the importance of metrics in editorial decision-making (Lamot, 2022) posed routine-level barriers for participants. This is because watchdog stories did not always grab the most attention, and their impact was best understood on a big-picture, long-term level that was difficult to capture via audience metrics. Reporting these stories was described as “antithetical to how journalism is done” [J3] because of the slow nature of investigations.

To address this barrier, journalists used ad hoc strategies, such as “ignor[ing] your editors . . . and just dig[ging] at it yourself” [J17] or ensuring their watchdog story was “the first you write out of the two stories that you have gathered during that day” [J14] to decrease the odds it would get cut due to space constraints. Still, many important stories were never told because “there were other things to do” [J15] or due to pressure “to get the traffic numbers . . . [and] keep subscriptions up” [J17].

Organizational-Level Influences

While the impact of time and resource constraints on science journalism is well-documented (Allan, 2011; Saari et al., 1998), these constraints appear even more problematic for watchdog journalism, which “is expensive, is time consuming, [and] requires a lot of resources” [J18]. These stories require developing in-depth knowledge, meaning participants had to spend considerable time “understanding the science” [J21]. To save time, participants like J20 used a “triage” approach: first assessing the story’s potential, then deciding whether to wade through the “floods” of information ahead.

Dwindling time and resources intertwined with and amplified barriers at almost every other level of influence. Budget cuts resulted in organizations lacking the in-house expertise needed to take on watchdog stories and thus increased their reliance on freelancers. Limited resources made outlets more selective, prioritizing content that was more likely to draw attention over more important but less clickable stories. Financial constraints meant outlets could afford less legal and fact-checking support, increasing the risks for journalists covering these issues. This, paradoxically, raised the threshold of evidence needed to run a story, increasing the time and resources needed to gather and vet information.

Alongside resource-related influences, editorial approaches impacted opportunities for watchdog work. As J10 synthesized: “Editors have different philosophies on how they do things, different thresholds of when you publish or what’s good enough, different priorities.” Editorial priorities were often linked to the target audience, with journalists pitching watchdog science stories more successfully to specialist publications serving science-interested and -educated audiences. Combined with resource-related barriers, this meant watchdog work remained “a minor part” [J21] of science reporting, both within individual journalists’ dossiers and within the wider media ecosystem.

Social-Institutional-Level Influences

Social institutional-level influences primarily centered on access-related factors, such as lack of available sources and documents and PR professionals who acted as “gatekeeper[s]” [J1] trying to “control the narrative” [J4]. When seeking out interviews, participants experienced challenges finding good communicators with relevant expertise, weeding out untrustworthy sources, mitigating sources’ liability concerns, dealing with frustrated scientists, and grappling with sources who were unavailable or slow to respond.

Most importantly, legal and reputational risks meant potential sources were often “not sure that they wanted to put their neck out” [J5]. This was also true of individuals or institutions under investigation. As J20 summarized: “You ask some very reasonable questions about stuff in the public interest and their instinct is to flop off with not answering the questions, not providing data, providing vague answers and spin.” This comment reflects the intertwined issue of accessing (useful) data and document sources, as “crucial data that might reveal things that aren’t necessarily in a government’s or university’s best interest . . . is now being severely restricted” [J4]. This left journalists to “go on what I can access reasonably easily or what I can ask for” [J20].

Press officers and other institutional spokespersons added to the access challenges. Whether dealing with journals, government institutions, hospitals, or universities, the story was the same: “You always have to start with the PR. Sometimes they answer you, sometimes they’ll send you some really anodyne e-mail, but rarely they’ll set up an actual interview with someone” [J7]. Concerningly, this barrier appeared to be getting worse, reflected in observations such as “one trend I noticed . . . is that now you cannot get anywhere without going through a PR department” [J4]. In addition, watchdog stories sometimes led to access issues for future stories, cutting off ties with potential sources. As J14 explained, “I know that the universities and researchers I exposed at that time will never deal with me again . . . So, it comes at a price.”

Social System-Level Influences

Finally, social system-level influences sometimes impacted participants’ work. As J15 expressed, “Canada is not a very open society as far as media goes, and in any kind of institutional situation, the tendency is, ‘We don’t talk to journalists.’” Relatedly, some participants noted that there is a “lack of good Canadian outlets for these stories” [J7], leading freelancers to pitch larger outlets based in other countries, such as the U.S. or U.K. This is reflected in our sample. While we were able to identify journalists who had done watchdog work for Canadian media, these journalists often also freelanced for U.S. or U.K. media.

Reporting for U.K. outlets also came with unique risks, due, in part, to a lack of academic regulation: “In Britain, there is no body with powers like that of the Office of Research Integrity in the US . . . There are just universities regulating themselves” [J17]. Several interviewees reporting for U.K. outlets described highly restrictive libel laws as an additional barrier, as did freelancers based in other countries but reporting for U.K. or Canadian media. These journalists were not always sure whether they were beholden to libel laws where they were physically based, where the outlet was located, or both. Moreover, freelancers experienced different levels of protection at different outlets.

Finally, FOI legislation varied across contexts, with one journalist noting that in Canada, “information that is research is excluded from freedom of information legislation” [J8]. These laws further restricted the access barriers described above, raising the threshold of evidence needed to publish watchdog stories, and compounding the time and resource barriers so central to journalists’ challenges.