The new watchdogs’ vision of science: A roundtable with Ivan Oransky ( Retraction Watch ) and Brandon Stell ( PubPeer )

There are more and more retractions of papers in scientific journals. Are there more instances of misconduct in science, or are the old institutional gatekeepers – the peer reviewers for journals and the ethics bodies supposed to deter misconduct through sanctions – being bypassed? At the same time, we have seen a new pack of watchdogs coming to the fore. Most of them are internet-based, are fed by grass-root researchers, and employ a variety of different shaming techniques. They are connected with one another: Scientists who publish in one watchdog venue read and post in another too, websites cite each other, etc. (Guaspare and Didier, in press).

This pack is here to stay, and it foreshadows one future of scientific evaluation, taking time to dive deeply into data and arguments, radically opposed to the quick statistical metrics and impact factors, perhaps helping to establish some aspects of the 2010 ‘slow science manifesto’ (Slow Science Academy, 2010). Obviously, the STS community should note what they have to say, both as actors and as witnesses, about the scientific field’s contemporary transformation.

Retraction Watch is a website run by science journalists who post daily about corrections and retractions of published papers and about other events related to science publishing, ethics and misconduct. An important part of the content resides in the comments added by readers, who contribute their own points of view or additional information, anonymously or not. The website highlights ethical and procedural breaches but also urges greater transparency in the correction of science. Retraction Watch is now a reference in the field of science blogs, with 150,000 unique visitors and half a million page views each month.

PubPeer is also a website, initially founded as a journal club, where scientists discuss and scrutinize published papers flagged as problematic, a form of post-publication peer review. The simple innovation that transformed the website into a core actor within the new ecology of watchdogs was to allow anonymous posting, starting in 2013. This triggered a host of aggressive interventions denouncing fakes and low-quality papers, including papers by powerful authors. PubPeer is implemented so that when a paper is discussed its corresponding author receives an automatic invitation to respond, which creates animated exchanges.

Discussions on PubPeer go deep into the topic of the papers, and thus are fairly technical. Therefore, they have a small reach outside the scientific community. But the team of Retraction Watch attends to the whys and wherefores of these discussions, and relays interesting controversies to a much wider audience. Since journalists read mainly journalists, Retraction Watch is on the radar of the national press: The New York Times in the US and Le Monde in France, for example, have published papers on recent scandals (such as those associated with Olivier Voinnet and Susana Gonzalez) based on Retraction Watch stories. It appears that many contributors to one site contribute to the other one, though this is impossible to prove since most are anonymous.

We discussed three themes. First, we wanted to know Oransky and Stell’s views on the differences between the new and previous watchdogs. One of the claims that emerged is that the new watchdogs are fed by grass-root scientists who can see, in their labs, what they think is misconduct; they can then voice their criticisms through the ‘technology’ (Oransky), protected by the anonymity provided. Why did these scientists become obsessed with misconduct revelation, since they are not institutionally rewarded for denunciation? ‘I think there is a whole spectrum’ of reasons, says Oransky. Different profiles include ‘the post-doc working in a very prominent lab’, the person ‘frustrated’ or ‘obsessed with one single case’, the person coming from a ‘honor-based culture’, etc. And since Oransky and Stell are themselves whistleblowers, we discussed their own motivations for spending so much time managing their websites.

Second, we discussed whether the appearance of the new watchdogs is associated with new kinds of science misconduct. Our idea was that, very generally, when a surge of misconduct happens in any social sphere, it means not only that it has changed quantitatively, but also qualitatively. This is consistent with Mario Biagioli and Alexandra Lippman’s idea that science is now beyond ‘publish or perish’, having moved into an era of ‘impact or perish’. Misconduct is not just epistemic (when scientists fake evidence for a grand new theory), but often occurs around and outside claims (when scientists fraudulently maximize the work’s impact). We discussed whether the question of reproducibility is still at the core of misconduct. Stell tells a striking insider (but anonymous) story, which was one of the motivations for creating PubPeer, of how fame and power can replace reproducibility of the results.

Finally, we raised the role of journals as gatekeepers of quality papers through peer review. Again, our premise was that when the morals of a social sphere are shaken up by massive amounts of misconduct and the appearance of new watchdogs, actors who had earlier been powerful must adapt and find new roles, or must disappear. We wondered if the appearance of the new watchdogs is transforming the role of peer review. And we discussed the future importance that post-publication peer review might have. More generally, we wonder in a short conclusion what ‘robust’ science now means.

The two guests do not make the same general arguments. For Oransky, as a journalist, open access and transparency in science are the goals for which one should aim. For Stell, the whole concept of journals is outdated. ‘I think that we should completely abandon journals’, he declares. But despite differences, both are core facilitators of a new practical ethics in science, whereby not anything goes. Against the multiplications of new shortcuts in scientific demonstrations, the development of a star system bearing on impact factors, and of repeated evidences of ill treatment of science beginners (grad students and post-docs) in the organization of laboratories, they give the opportunity to scientists, often young and not established, to fight for the implementation of rules that they think are more democratic and more epistemologically sound. They aim for a system of robust demonstration, equality among peers, and mutual respect in the lab.

We invite you to read an edited transcript of the conversation, and/or to listen to an excerpt of it as a podcast, both published on Transmissions (ssstransmissions.com).