Psychology’s Replication Crisis and the Grant Culture: Righting the Ship

The grant culture

About a decade ago, I was a regular attendee at Grand Rounds presentations in a prestigious psychiatry department. Before introducing speakers, the chairperson routinely kicked off sessions by announcing the names of professors who had received large federal grants along with their precise dollar amounts. I was struck that he never announced faculty members’ important publications or scientific discoveries. I have since come to realize that this reinforcement pattern is common in psychology departments, too: Faculty members routinely receive plaudits for receiving grants but frequently find that their scholarly accomplishments go largely unnoticed.

These reinforcement contingencies should strike us as odd for several reasons. First, we do not laud novelists or film producers for securing large contracts for their planned projects, nor should we. Instead, we rightly acclaim them if and when they have generated high-quality artistic work. Similarly, in science, grants should be regarded as means to an end rather than ends in and of themselves (Chambers, 2017; Gallup & Svare, 2016). Second, for a field that prides itself on empirical rigor, psychology’s encouragement of this practice is surprisingly nonempirical. The correlation between grant funding and citation impact in psychology is low and perhaps essentially zero once one controls for potential confounds, such as journal and first-author prestige (Haslam et al., 2008). Third, grants are not needed for many forms of impactful research. For example, most researchers who have authored articles cited 1,000 or more times had no current National Institute of Health (NIH) funding (Nicholson & Ioannidis, 2012); many had produced landmark methodological advances. Furthermore, numerous Nobel laureates in physics, chemistry, and medicine received no federal funding for the work that culminated in their prizes (Tatsioni, Vavva, & Ioannidis, 2010)

To be clear, I am not opposed to grants. For some scientific questions, grant funding is essential for high-quality research or even any research at all. For many of my colleagues in neuroscience-related fields, money is a virtual prerequisite for research. We should encourage these scholars to apply for grants and make allowances in their workloads for grant-related work. Furthermore, we should reward colleagues who obtain training grants to support graduate student education. In addition, the grant culture has its upsides, including provision of funding for postdoctoral scholars and its propensity to spur competition in the marketplace of ideas.

What I am opposed to is the implication that researchers’ scholarly merit should be gauged in large measure by grant success. This fundamental law of academic life has spawned several corollary ordinances. Most notably, in a growing number of psychology departments, faculty members cannot be hired, tenured, or promoted without a solid grant track record regardless of the quality or impact of their work. Hence, even if researchers are generating significant discoveries and influencing the field’s thinking, they may be at risk for termination if they do not obtain grants.

In fairness, many institutions surely push faculty to apply for grants because they are in desperate need of financial resources. In such cases, however, fundamental maxims of intellectual honesty should require them to acknowledge that they are hiring and promoting professors as much on their fundraising success as on their scholarship.

The grant culture has contributed to a number of other troublesome consequences, each of which I describe briefly.

A bevy of negative consequences

In these respects, the grant culture would appear to be a virtual recipe for confirmation bias. Because confirmation bias can be fueled by motivated reasoning (Kunda, 1990), the lure of grant dollars and the fear of losing them induce powerful incentives to detect positive results by means of p hacking, outcome reporting bias, and other QRPs (Nosek, Spies, & Motyl, 2012). Although training in research ethics may be a partial bulwark against QRPs, such training is unlikely to be sufficient because confirmation bias operates largely outside of conscious awareness (Nickerson, 1998).

Furthermore, as Firestein (2015) noted, failure is a crucial element of the scientific enterprise. When studies are well-designed, we learn at least as much from disconfirmation as from corroboration of hypotheses. Nevertheless, much of the grant culture implicitly discourages failure, especially when negative results raise the specter of the investigator’s theory being in error.

Fortunately, the preregistration of hypotheses and analytic plans is a critical safeguard against QRPs (Lindsay, Simons, & Lilienfeld, 2016), as it diminishes the odds that researchers will erroneously present exploratory research as confirmatory. Preregistration will not, however, do much to diminish the foregoing problems emanating from the grant culture.

Still, there are largely unappreciated disadvantages of programmatic research. Such research can foster confirmation bias, especially when it is designed to test the investigator’s favored theory (Greenwald, Pratkanis, Leippe, & Baumgardner, 1986). Research on sunk costs (Arkes & Blumer, 1985) and effort justification (Axsom & Cooper, 1985) further suggests that once individuals have invested a great deal of time and effort in a project, they will feel the need to persist in it even when doing so is no longer fruitful. In addition, programmatic research often runs its course and may yield diminishing knowledge returns following a large number of studies (Peirce, 1967).

In today’s academic environment, big-picture thinkers may at risk for extinction (Wolfe, 2016). Paul Meehl, the most influential clinical psychologist of the latter half of the 20th century, received a grand total of one federal grant in his career. I am hardly the first to observe that psychology’s great generalist thinkers of the past, such as Meehl, Lee J. Cronbach, Donald Campbell, Lloyd Humphreys, Jane Loevinger, and Robyn Dawes, are now few and far between. One has to wonder what would have come of these scholars had they experienced incessant career pressure to apply for funding.