Demand-Side Obstacles to Publishing Economics Research: A View from the South

I. Introduction

The epistemological critique of coloniality in knowledge production has a long tradition and is well developed, but empirical research on the subject is more recent and less advanced (Connell et al., 2017). At the same time, a growing literature provides quantitative evidence that few researchers in the Global South participate in economics conferences or policy debates, publish in economics journals or are cited in the work of others. Variation certainly exists across arenas and methodologies, but, even if we focus on the subfield of development-economics—whose main interest is the economics of low- and middle-income countries (hereafter, LMICs))—estimates generally indicate that only 10%–20% of participants in development-economics fora are researchers working in LMICs (Amarante et al., 2021). This result suggests the marginalization of southern researchers. ¹ A strong defender of the status quo could certainly argue that differences in outcomes reflect differences in productivity and nothing else. According to this view, researchers in developing countries either do not have the skills to conduct top-quality research or that they migrate to northern institutions if they do. We argue that such a view is not only incomplete but that it implicitly undervalues the academic knowledge produced through first-hand experience.

One fundamental question is what constitutes a ‘contribution’ to the field of economics. Though this is a complex question, most economists, in our view, would agree that the criteria for determining whether a manuscript contributes to the state of knowledge include (i) the relevance of the topic and countries covered and (ii) the soundness of the methodology. Do the articles published in top economics journals—which are overwhelmingly written by authors in the Global North—effectively cover relevant topics and countries? Do they follow a rigorous methodology?

In our opinion, the answer is affirmative for both questions. Individually, each article meets the above criteria. The problem occurs at the aggregate level. In the population of published papers in economics journals, we have noted that relevant topics and countries are absent or seldom included and that some countries and topics receive an excess of attention (the United States and microeconomic evaluations of social programmes, for example). The usual explanation for this imbalance in analyses is that investigations of other countries or topics lack data or a solid identification strategy and, therefore, do not meet the accepted benchmarks of scientific rigour.

A similar problem occurs in the subfield of development economics. In a 2021 interview for the World Bank Blogs site, Andrew Foster, Editor in Chief of the Journal of Development Economics, was asked, ‘As far as we are aware, you haven’t worked in Africa at all. Have you ever been tempted to work on an African country?’ Foster responded:

The scarcity of work on Africa by me is not for lack of interest. But a lot of what motivates me … is the study of mechanisms associated with larger economic and demographic change. This sort of work is best suited to household data collected over longer periods of time and perhaps space. Data systems of this sort—whether administrative- or survey-based—were simply not available in Africa when I was starting my career. Of course, data availability in Africa is changing and I do hope to find the right question, data, and partnership in Africa before too long. (McKenzie, 2021)

Foster’s comment illustrates what we believe to be wrong reasoning. Economists should instead engage more actively with difficult and important questions for which only partial or tentative answers exist. Methods should be selected as a function of the problem to be studied rather than the reverse.

In our experience, researchers in developing countries know which topics are relevant because they have first-hand experience in under-researched countries. They and their families work, consume, vote, invest and study in those places, and they are also in a privileged position to determine which research methods are feasible and cost-effective.

Their papers are more likely to be rejected, however, as a 2021 survey by Amarante et al. (2021) suggested. The Amarante group analysed publication pipelines for four development- economics journals—Economic Development and Cultural Change, the Journal of African Economies, the Journal of Human Development and Capabilities and the Review of Development Economics—and found that the rate of acceptance was between two and six times higher for northern than for southern researchers.

When researchers in developing countries seek to publish scientific articles in top economics journals, they face distinct obstacles, which we would group into two main categories. First are supply-side obstacles that make it difficult for researchers in the Global South to write papers ‘like researchers in the North’. Examples of those obstacles include: insufficient networking and visibility (Brogaard et al., 2014), lack of access to technology and funding (Smart et al., 2004; Tijssen & Kraemer-Mbula, 2018), lack of research capacity (Fox & Milbourne, 1999), language barriers (McGrail et al., 2006; Van Weijen, 2013), brain drain and research environment (Ondari-Okemwa, 2004). A second category of obstacle involves demand-side issues—that is, the practices of journal editors and referees and, more specifically, the lack of priority they appear to give to first-hand knowledge. Our focus is on the second category because we think it has become an important impediment to publishing in economics journals for southern researchers.

Figure 1 illustrates an analysis of current publication practices in empirical economics research—that is, our view of what journals should cover versus what they actually do. In fact, we have coined the description ‘the productivity criteria of the status quo’ to describe what we see as one of the main challenges economics researchers in developing countries face in their attempts to publish scientific articles in top journals. According to our observations, journals tend to overpublish papers with high-quality execution (which usually means a strong identification strategy, such as in a randomized controlled trial (RCT)) and under-publish creative papers that confront questions that can be answered only imperfectly, even with the best methodologies available.

OPEN IN VIEWER

Figure 1.

Source: The authors.

We reason that a major cause of this problem is an imbalance in the composition of the editorial boards of economics journals. As of this writing, none of the members of the top five economics journals—the American Economic Review, Econometrica, the Journal of Political Economy, the Quarterly Journal of Economics and the Review of Economic Studies—works in a LMIC. That is zero out of 307 people.

Participation by southern researchers is slightly better on the editorial board of the Journal of Development Economics (5/84), but is again zero at Economic Development and Cultural Change and at the World Bank Research Observer. Only by moving to the field of development studies does it become possible to identify a high-ranking journal at which almost a third of board members work in a LMIC (World Development).

II. Obstacles That Result from Lack of First-Hand Knowledge

Here we provide examples from conversations with colleagues from the Global South and from our own experiences. It is not systematic evidence, but it does illustrate the costs of neglecting academic knowledge produced by developing country authors and can suggest avenues for future research and analyses that can provide higher-quality evidence.

III. Problems Determining Which Assumptions Are Correct

When the government of Argentina, with support from the World Bank, increased the number of beneficiaries of a temporary public-works programme to 15% of the labour force in 2002, a local researcher raised concerns that benefits had been allocated on the basis of political patronage rather than of need, adding that, instead of improving productivity, the programme had fostered dependency. She only had anecdotal evidence from years of social work in poor neighbourhoods on the periphery of Buenos Aires. When she raised her concerns to a fellow researcher from a northern university, her colleague replied:

There is a rigorous evaluation of Argentina Trabaja published in a high-quality economics journal that shows that the program is very well-targeted toward the poor. The good targeting is because beneficiaries must work to receive a small cash benefit, and only the poor are willing to do that. (Jalan & Ravallion, 2003)

In the researcher’s personal experience in Argentina, however, the implementation of the work requirement, though it was in the letter of the program, was actually quite different.

This anecdote has important lessons to teach. First, the authors of the published paper used a sample of programme participants provided to them by the Argentine Ministry of Labour. One possibility is that the authors did not know the sample was biased because they did not have direct contact with programme participants and non-participants. Second, because the legislation stated that participants who received the cash benefit were required to work, another possibility is that the authors assumed the requirement was enforced. They were apparently unaware that, in developing countries like Argentina, the distance between the letter of the law and on-the-ground practice may be large. Our position is that the inclusion of a local researcher on the team would have increased the authors’ ability to judge the validity of the participant sample and reduced the chances of unfounded assumptions.

Similarly, in 2004, the Quarterly Journal of Economics, a top economics journal, published a paper entitled ‘The Regulation of Labor’ (Botero et al., 2004), in which the authors computed controlled correlations between the letter of the labour code and labour- market outcomes across countries all over the world. They interpreted the results as causal effects. Using the letter of the law as a proxy for effective regulation may be a reasonable assumption in the analysis of developed countries because compliance with the law arguably tends to be high. In developing countries, however, a large distance exists between labour law and effective, real-world regulation. Furthermore, as Kanbur and Ronconi (2018) have noted, countries with the most stringent labour laws are often those that devote fewer resources to enforcement. We speculate that an associate editor from the developing world would probably not have found it acceptable for in-form regulations to be used as a reasonable proxy for effective regulations.

IV. Concluding Remarks

We take the position that many editors of economics journals prefer manuscripts with a high-quality execution that tend to focus on already well-researched countries and topics, to creative papers that deal with under-researched countries or topics that can be analysed only imperfectly, even using the best methodologies available. This preference for execution over relevance implicitly undervalues first-hand knowledge and discourages southern researchers from publishing in mainstream economics and development economics journals. More importantly, we view this constraint as a clear obstacle to cumulative scientific progress.

We know that the complaints against the marginalization of southern researchers are partially triggered by the frustration produced by being rejected by journals. But that is not the whole story. In our opinion, economics is a social science whose aim is the overall improvement of human welfare. Being more sceptical about RCTs and their validity, valuing more highly urgent issues in LMICs, valuing more greatly first-hand knowledge in LMICs and valuing alternative methods of investigation will strengthen the overall science of development economics. Achieving these points needs a proactive push for decolonization by journal editors, as well as increased representation of more southern researchers on editorial boards and as reviewers.