Overpublication as a symptom of audit culture: A comment on Phaf (2020)

Phaf (2020) has asserted that psychologists should read more and publish less. We think that most psychologists (ourselves not excepted) would like to take that advice, but most probably won’t. Here we reflect on that contradiction, arguing that overpublication is a species of problem common to audit cultures (Shore, 2008) and is thus unlikely to resolve as long as academic writing is arbitrated within a marketized calculative regime.

Speaking very broadly, in any system where resources are arbitrated through calculative audits (McKernan, 2012), the hermeneutic—in this case, psychological theory—is subsumed under the algorithmic—in this case, marketized accounting regimes—to the detriment of meaning and interpretation. For most psychologists, publications are organized by metrics (e.g., impact factors) and incentives (e.g., publish-or-perish hiring and promotion) unrelated to scientific theorizing and these “gatekeepers” control production at least as much as our ideas about good scientific theory do. Until the limitations of this larger system are addressed, we suspect that any attempts to change behavior, or even to adjust the metrics or incentives of the system, will ultimately accomplish little in the face of larger calculative forces.

We can help clarify this point by dwelling for a moment on the evolutionary metaphor that Phaf (2020) invokes to explain progress in science. This metaphor, inspired by a broadly Lakatoshian read of science, ¹ accounts for scientific progress in terms of selective forces slowly pruning the least productive theories. This reading is, of course, self-consciously metaphorical and no one suggests that progress in science is a direct analogue of natural selection (what, for example, would reproduction be under this account?), but we think there is some explanatory value in considering the implications of this analogy in the context of scientific culture and practice.

Specifically, we want to make two related points about the evolutionary metaphor that we think help to make sense of overpublication as a predictable response to an audit environment. First, an evolutionary metaphor does not necessarily imply progress, but only adaptation—that is, though selective forces may shape scientific theories in a particular direction, that direction is not necessarily determined by some system-independent standard of “good” or “true” theory. Selective forces are ultimately always adaptations to local conditions and there exists a rich body of sociological, historical, and anthropological research showing the many ways that specific scientific practices are adaptations to political, material, and social conditions (as opposed to purely theoretical ones). The topics of research, for example, may be constrained by available resources (Knorr-Cetina, 1981); the methods by disciplinary norms (Peterson, 2016); and the locations of (and opportunities for) research by disciplinary and interpersonal politics (Dewsbury, 2003). In other words, the forces shaping scientific theories and practices are never purely theoretical or empirical, but also political and institutional. Accordingly, scientific research is never only about producing sound theory or empirical tests—it is also always about the academic survival and potential thriving of researchers, research programs, labs, and the other social units comprising science.

In point of fact, this argument has been made quite effectively in the context of overpublication, under-replication, and Questionable Research Practices (QRPs) generally. Nosek et al. (2012) and many others have pointed to the ways that perverse incentives, in forms like high volume publication requirements in tenure decisions or a general disinclination to publish null results, lead to overpublication and other questionable practices. Thus prolific publication habits, to return to the evolutionary metaphor, could actually be seen as an adaptive response; adaptive in the sense that the survival of a particular body of scientific work (not to mention the scientists who do it) depends upon adapting to a set of local conditions that reward high rates of publication. No doubt, Phaf (2020) is correct, at least in some respects, that overpublication is corrosive to theory building in psychology. However, as long as this practice is an adaptive response to certain structural conditions, saying that it should stop (or attempting to enforce its removal) may have little effect.

But what are those conditions to which hyper-production might be seen as an adaptive response? Speaking generally, the conditions to which we refer are an audit environment where generalized metrics (e.g., impact factors, number of grant dollars, number of publications, etc.) arbitrate resources, encouraging hyper-production in at least two ways: first, by contributing to professional precarity, and second, by linking the survival of scientific work (and theories) to the production of system-defined metrics.

Regarding the first of these difficulties, a number of authors (e.g., Ball, 2003; Shore, 2008) have described how audit environments tend to engineer existential and financial precarity. In brief, audits require surveillance, engendering fear and uncertainty (what Ball, 2003, called “soul terrors”), as well as centralized control of resources, placing employment and other rewards outside the reliable prediction or control of “producers.” Hyper-production can be seen as a quite logical response to this sort of precarity, something the evolutionary metaphor of scientific progress actually suggests. And this is the second point we wish to make about that metaphor. In evolutionary accounts of population growth, hyper-production (e.g., the frenetic reproductive florescence of the mayfly) is generally understood as a response to unstable environments. Such environments (e.g., rapidly changing ecologies like flood plains) are unpredictable and so long-term investments in small numbers of slow-growing, late-maturing, well-cared for, and long-lived offspring are often wiped out by unpredictable storms, floods, predations, etc. Organisms in such environments thus often adapt by producing large numbers of rapidly produced, short-lived offspring, compensating by force of numbers for an inability to predict future conditions.

The hyper-publication we see among academics is, we think, rather reminiscent of this account. The publication environment is certainly an unstable one, not only because of the publish-or-perish pressure most academics experience, but also because both the employment and publication landscapes are in a constant state of transformation. University faculty are being increasingly pushed into more precarious and contingent forms of employment as neo-liberal forms of management take hold (see Ostenson et al., 2017). Furthermore, the publication ecosystem upon which their employment often depends is being mutated by predatory journals, funded publication (often in the name of open access), and other questionable publishing practices. For most psychologists, it would be frankly irrational to treat these publication and employment environments as stable backgrounds for scientific theorizing. To cultivate a slowly and carefully crafted theory, nourished by broad reading habits and tested in carefully designed and replicated research, is an incredibly risky proposition in an environment where neither employment nor publication can be guaranteed and where rapid, prolific production is rewarded.

If the only thing relevant to the production of science were the strength of theory, the sorts of proposals Phaf (2020) makes might be sufficient. However, the precarious conditions under which most researchers operate highlight the ways that science also depends on researchers’ resources, reputations, and employment. Thus, for many academics, a slow and careful program of research is nearly impossible to contemplate; they are struggling to find a job in an oversaturated market or are being paid at (and sometimes below) minimum wage rates; their class-sizes, teaching loads, and administrative responsibilities are increasing while their salaries are not (Clawson, 2009). In other words, for many academics, the local audit environment to which they must adapt often makes it very difficult to read more and publish less.

Audit environments also encourage hyper-production by tying scientific work to production metrics, but it is not simply a particular set of incentives, procedures, or rules that militate against slow and careful theorizing; it is the audit environment itself. Developing theory is a fundamentally hermeneutic endeavor, but calculative audits (like impact factors) are precisely the opposite—they are algorithmic and their purpose is to obviate interpretation in the name of efficiency and scale. It is precisely this evacuation of interpretation, and its replacement with standardized criteria and generalized metrics, that produces the perverse incentives of publish-or-perish academia, or the bad faith “gaming” of impact factors. These questionable practices are symptoms of a neo-liberal form of management—one where the intrinsic meanings and purposes of scientific publication are subverted by algorithmic markers of efficiency and productivity and where these meanings are replaced by a kind of bad-faith compliance in the face of professional precarity. Thus, as long as the process of theorizing is subject to such neo-liberal forms of organization, any incentive will ultimately become perverse and any procedure algorithmic (and eventually meaningless).

We thus find ourselves of two minds about systemic approaches to reducing overpublication, like those proposed by Phaf (2020) and others (e.g., Nosek et al., 2012). On the one hand, these acknowledge the systemic scope of the problem. They acknowledge that this is not merely a matter of individual researchers engaging in questionable practices, but of a whole system rewarding the wrong sorts of behaviors. Yet, such systemic proposals also seem to be always bound up within the neo-liberal logics of the systems they seek to reform. Morawski (2019) has noted this trend, pointing to the ways that many approaches to addressing problems of replication, QRPs, etc. seem to call for more algorithmic requirements (e.g., preregistration) and new forms of surveillance and control, rather than addressing the core political systems that gave rise to these problems. It seems as though even those committed to reforming the professional structures that produce problems like overpublication are often caught within the logics of those systems.

Consider, for example, Phaf’s (2020) suggestion that we use machine learning to identify important sources. On its face, this recommendation makes sense; it is a way of using technology to address the pressures of hyper-productivity. But, on closer examination, it becomes clear that this is not a solution to the problem of reading or thinking too little; in fact, it is a way of reading and thinking less. It is an outsourcing of the discovery and winnowing process. The shadow of the audit is so long and so overdetermines what we consider possible that even the problem of not being able to read enough can seem like its only solution is algorithmic.

Ultimately, however, the only way we will read more is if we have more time to read. And we don’t have that time because the institutions and systems we inhabit monetize, surveil, and control our time and there is little left for thinking or reading. The poverty of theory in psychology is thus a fundamentally political problem; it is a problem that flows from the ways that academics have ceded so many arguments to the corporatized terms of neo-liberal culture, so much so, that we have trouble imagining any solutions that lie outside of those terms.

In this short commentary, there is no space to elaborate on any such solutions; we do, however, want to point briefly to other resources for approaching the audits we inhabit. Elsewhere (Clegg et al., 2018; Ostenson et al., 2017), we have referred to these alternative approaches as “relational accounting practices” and we would argue that such practices confront the audit culture in ways that free up time and opportunity to read more and to better engage theory. These practices come from many sources, though primarily from the critical accounting tradition, and focus on strategically adapting to, disrupting, and re-inscribing the local audit cultures we inhabit. Very briefly, relational accounting practices help us to find spaces within a given system (as well as points of resistance to that system) to do work that is driven by relational values (e.g., strengthening relationships, building local communities, articulating shared meanings, etc.), rather than by the calculative values of efficiency and productivity. These strategies move us from systems to relations—that is, from contracts to covenants; from economic value to shared values; from enterprising self-management to commitment, community, and solidarity; from information structures to social structures.

In the context of overpublication, such practices would be oriented both toward adapting our responses to neo-liberal demands in ways that are sensitive to the human (and hermeneutic) scale of theory building, and toward finding ways to resist and disrupt the audits that inscribe such demands. For example, one particularly valuable relational adaptation to the various publish-or-perish audits is to work collaboratively. No doubt, many authors collaborate to increase production as a response to neo-liberal demands. However, a relational approach to collaboration might convert such efficiency into time to collaborate on work that is intrinsically meaningful to collaborators and through relationships important to them. In this way, we mobilize strategies already available (i.e., collaboration) for relational purposes, even as these satisfy systemic demands. Of course, collaboration merely in the name of efficiency and without regard for these relational purposes would simply perpetuate the problems we hope to address.

Relational accounting practices also include more disruptive strategies. Kamuf (2007), for example, has advocated for strategic substitutions, where particular standard calculative indicators (e.g., an impact factor) are replaced with a more nuanced narrative version (e.g., a description of one’s scholarship and its functions within a scholarly community). Similarly, Dillard and Roslender (2011) have suggested the production of “heteroglossic” accounts, or accounts that include a discussion of purposes and interpretive frameworks, alternative possible forms of accounting and alternative interpretations, as well as minority or dissenting opinions. A tenure case on this model, for example, might initially require more time to conduct, but would do much to disrupt calculative accounts of scholarship.

Perhaps the clearest route to disrupting an audit culture is through administrative service. Faculty who engage in such service, particularly at most smaller institutions, have the power to write rules that ignore things like impact factors and grant demands, and this can alleviate some of the pressures that cause overpublication. Such strategies, no doubt, take courage and push boundaries, but knowing our disciplinary (and physical) geographies can help with this, as these help us discern our limitations and our freedoms to alter the audits we inhabit.

We see such relational accounting strategies as powerful tools in adapting to and resisting the calculative audits at the heart of overpublication, but in recommending these strategies, we in no way wish to discount the importance of system-level reforms. Perverse publication incentives and mechanized metrics should be reformed, but we also need to acknowledge the deeper problematic that underlies such metrics—namely, a neo-liberal audit culture. It is not so much a particular incentive structure that drives overpublication, but a larger algorithmic orientation, driven by the values of scale and efficiency. Strategies that can effectively resist this orientation thus require a move away from the algorithmic and toward a commitment to human scale and relational practices, practices sensitive to the hermeneutic requirements of good theory.