History and quantitative conflict research: A case for limiting the historical scope of our theoretical arguments

History and social science

Our aversion to placing these bounds on our research stems in part from how we usually talk about the differences between history and the kind of social science most members of the Peace Science Society do. The most common account of this difference is that historians seek to explain specific events, while social scientists seek to develop generalizations about classes of events. What they do is “idiographic,” while what we do is “nomothetic.” ¹ This account is superficially appealing, but it does not hold up to serious consideration. I doubt very many social scientists or historians really believe it when they think about it, but they often act as if they do.

First, truly “idiographic” explanations of specific events are not possible. This idea is a caricature of what historians are actually doing. You can seek to explain a single historical event, but you cannot do so without a prior theory about the process that produced it. The historical evidence itself will not provide this theory. Without it, there is no way even to know what evidence to consider in the first place. Historical processes leave behind only fragmentary traces—documents, memories, pieces of physical evidence, and other such things. Even describing what happened, let alone explaining it, requires some guide for connecting these essentially isolated pieces of evidence and drawing inferences about the process that produced them. To make matters worse, there may be infinitely many such historical artifacts. A theory about how the process worked is necessary to know which are important and which can be set aside. This theory might be rough and implicit. It might be little more than an amalgam of assumptions, prejudices, and hunches. The researcher might even be in denial about its existence. Nevertheless, the theory has to be present or the researcher will perceive nothing but chaos. And since a theory has to exist, it makes sense to state it explicitly and subject it to critical scrutiny.

In reality, good history has a great deal of theoretical content. For example, histories of American foreign policy often advance strong theoretical claims about the forces shaping it. Some focus on the impact of economic concerns (e.g. LaFeber, 1963; Palen, 2016; Williams, 1972 [1959]). Others have much to tell us about the linkages between domestic political upheaval and aggressive behavior by powerful states (e.g. Dallek, 1982; Hofstadter, 1966 [1951]). Still others emphasize the role of ideas (e.g. Hoganson, 1998; Hunt, 1987; Rosenberg, 1982). The theoretical arguments in these works are not always presented as explicitly as many social scientists would prefer, but they are clear enough from a careful reading. Indeed, some works of history, such as George Kennan’s American Diplomacy (1984), are frequently cited as examples of particular theoretical perspectives on foreign policy and international politics.

So much for the illusion of idiographic explanation. Unfortunately, thinking of the goal of our profession as the development of nomothetic laws about transhistorical categories of events, like “war,” is just as problematic. The trouble is that the object of our study resists these kinds of generalizations in ways that other objects of scientific study do not. Most fundamentally, the changing nature of social actors across time and space can alter the determinants of their behavior. It is reasonable to assume that hydrogen atoms are the same today as they were in 1500 or 1800 in all respects that matter. It is not safe to make this assumption about states or other social actors. Just because you have figured out what determines state behavior today does not mean you understand how the entities we call “states” behaved in the past. We may use the same term for all of them, but they are not really the same animal over time. They are likely to respond to similar events and conditions in quite different ways. In addition to changes in the internal character of the actors, the context in which they exist also changes. Technology, the conventional wisdom about the likely effects of specific policies, and other features of the international system may also alter the determinants of state behavior. We might be able to incorporate some of these differences into our models, but the extent of changes across time and space is more sweeping than we like to believe.

You can observe substantial changes in the nature of the state without going back to the distant past. I have recently been working on American foreign policy during the 1890–1914 period, when the country first emerged as a world power. I began the project by looking for some statement of national strategy comparable to NSC 68 or other policy documents I was familiar with from my past work on the early Cold War era. I knew there would be no exact counterpart, but I thought American policymakers must have written some statement of their goals and of what it would take to achieve them. It soon became clear that I would find no such thing. The reasons for this lacuna revealed a lot of my erroneous assumptions about what the American state—and other states—had to look like. In 1900, the State Department had just 91 domestic employees. This tiny organization contrasts sharply with the post-World War II State Department, which has frequently been more 100 times larger. In 2010, it had 16,565 domestic employees (US Department of State, Office of the Historian, 2018). There were of course no computers or copy machines in 1900, so clerical tasks consumed a great deal more time than they do today, magnifying the difference in the capabilities of the organization. Before World War I, the American state simply lacked the capacity to develop (let alone implement) policy in the way it did after World War II. Even if it had set up a policy-planning office, it would have had to function without inputs from anything like our current intelligence community, since this did not exist either.

There are other differences that bear on the information environment and the range of interests considered in the policymaking process. For one thing, while foreign policy is currently developed in secret within heavily fortified compounds, there were few rules about secrecy before World War I. In 1914, reporters and other interested parties could still walk freely through the State Department checking out more or less whatever they liked (West, 1978: 78). Much of the organization’s most important correspondence would be published at the end of the year anyway, when the annual volume of Foreign Relations of the United States appeared as part of the President’s annual message to Congress. This meant that much of the debate about American foreign policy took place not in secret memos like NSC 68 but in the pages of journals like the North American Review and the Forum, which were read by the sort of people who were interested in the topic. Significantly, nearly all these interested people were upper-class white men from the Northeast. There are many other surprising differences, but these are enough to make the point. It is not safe to assume that such a “policymaking process,” if the term itself isn’t an anachronism, will respond to events and conditions in the same way that our current national security state does.

Of course, the same heterogeneity that occurs over historical time also arises across geographic space. Those of us in political science departments work alongside many specialists in comparative politics who remind us that there are large and important differences in institutions and politics across states. Our contact with historians is typically more limited, so historical differences can be easier to overlook. The point here is that “the past is a foreign country; they do things differently there” (Hartley, 1958: 1). Across both time and space, there are large differences in the character of states and other social actors that might lead them to respond differently to the same events and conditions. These heterogenous responses will limit our ability to generalize about behavior. Theoretical arguments that work well at some times and in some places will be useless in others. We can certainly model some of these differences, perhaps using interaction terms to capture different responses to the same conditions. Unfortunately, the number of these interaction terms will grow rapidly with the temporal and geographic scope of the behavior we want to explain. It will soon become intractable. We may strive for theoretical arguments that have no historical bounds, but our subject matter will resist this effort.

The bottom line is that quantitative social scientists and historians have the same job. Both are in the business of using theory to explain social processes that occur within some historical bounds. Both the theory and the historical bounds are unavoidable. There are certainly disciplinary differences, but these come from the separate development of our research communities, not some fundamental difference in our intellectual objectives or our subject matter.

Handling heterogeneity

How should we handle the changing historical nature of the social actors we study in our research? Apart from simply ignoring the problem and hoping the differences in the character of these actors will be absorbed harmlessly into the error term, there are at least three possible courses of action.

One option is to develop theoretical arguments that are broad and abstract enough to be unaffected by changes in the nature of the social actors we care about across a large swath of space and time, even if they become false or irrelevant at the limit. These arguments will have to identify causal processes that are unaffected by historical changes in the character of the actors. A great deal of research in our profession is conducted in this way. It is a defensible approach, but it comes with a price: you will have to abstract away from causal processes that really do affect behavior in some historical settings. How high this price is will depend on what you are studying. If the very abstract and general causal processes you are modeling are consistently more important than the historically bounded ones that you are forced to set aside, then this approach is justified. The danger is that your attention may be drawn to theoretical processes that are widely applicable but not substantively very important in shaping outcomes you care about. Taking this risk is sometimes justified—I have done it often enough myself—but not always.

A second option is to model the changing nature of the actors we study. This would allow us to explain why a causal process that was important in one time or place is far less so in another. This is an attractive option because it holds out the possibility of subsuming historical change into our theoretical arguments. Research that explains why things work differently in one era than they do in another, or why one state or region features different patterns than one finds elsewhere, is a valuable enterprise and can produce extremely interesting results (e.g. Fazal, 2014; McDonald, 2015). Because we have little choice but to think about the reasons for the historical bounds on our arguments, it is tempting to conclude that we should always theorize the historical bounds on our research. Unfortunately, this conclusion underrates the difficulty of the task. There are many sources of heterogeneity, and the number increases with the historical scope of an argument. Sorting them out is a valuable enterprise, but it is not always the goal that researchers have in mind. While explaining historical change is intimately related to explaining what goes on within a set of historical bounds, it is a separate explanatory task.

The third option is to acknowledge historical bounds and work within them. The existence of these bounds does not mean that no generalizations are possible. Conditions are often stable across enough of time and space for us to identify important patterns rooted in persistent causal mechanisms. It’s just that these patterns have historical limits. I think we should adopt this approach more often then we currently do.

How should we determine the historical bounds on our research? Given the difficulty of fully theorizing these limits, it pays to be cautious about how widely applicable our theoretical claims really are. We might begin by focusing on a relatively narrow range of social actors for which we can make a good prima facie case for relative homogeneity. This might mean a study of a small number of similar states—perhaps even one state—during a limited period of time. For example, postwar American presidents are arguably similar enough in terms of the policy options open to them and the political environment in which they operated to permit generalizations about their conflict behavior. These generalizations might not apply to American presidents before World War II. They are also unlikely to be useful in explaining the behavior of other national leaders operating in very different circumstances, such as those with no military forces at their disposal.

It is useful to be explicit about the historical bounds. Setting them out does not imply that the process in question could never occur in other times and places, but readers need to know what you are seeking to explain. It makes sense to explain why the periods immediately before and after the focus of the research are likely to be different. Fully theorizing and testing the reasons for these differences remains a separate and demanding enterprise, but there is no way around providing at least a minimal explanation for the historical scope of the argument. Refusing to be explicit about historical bounds is akin to some historians’ coyness about their theoretical arguments. Both tendencies arise from the tastes and reward structures of our research communities. Because social scientists strongly prefer broadly generalizable arguments, it can seem professionally unwise to call attention to the limits of your research, even when it is intellectually honest.

In spite of the careerist reasons to claim that you have discovered universal laws about politics, it is better to be cautious in setting historical bounds on your research. Mistakes from overstating the historical scope of your argument have more serious consequences than those that come from understating it. Empirical tests on samples that include cases where your argument does not apply are prone to type II errors, incorrectly suggesting that there is no evidentiary support when the process in question might be quite important within properly specified historical bounds. Overstating the generality of your argument—or just ignoring the scope within which it applies—also invites subsequent researchers to make this same mistake. They may apply your argument in inappropriate settings and erroneously conclude that it is less useful than it actually is. In contrast, unnecessarily restrictive historical bounds set no such trap for future researchers. If they find that the proposed process also took place at other times and in other places, so much the better. If not, the null results invite them to figure out why the argument applies in one historical setting but not another. This is a much more useful exercise than simply setting their null findings alongside yours, shrugging their shoulders, and concluding that there is conflicting evidence about whether the theoretical argument is useful at all. ²

Researchers are especially likely to overstate the historical generality of their theoretical arguments when considering the recent past. (All our data come from the past, of course. Empirical evidence from the present and future is notoriously hard to get.) Research on the truly remote and distant past—say, before 1990—routinely raises questions about whether the patterns it reveals apply in other historical settings, especially the present. These questions are entirely legitimate, but they can and should be asked about research on more recent periods as well. The fact that they usually are not exposes the potentially misleading assumption that current conditions will persist into the future. This assumption makes historical research appear less interesting because it implies that it can tell us little about what will happen next. In reality, the recent past may be quite unusual when viewed from a longer perspective, and the expectation that current conditions will continue is often wrong. If you doubt this, read some of the optimistic takes on the durability and universality of democracy and liberal international institutions written during the 1990s. ³

Present conditions are intrinsically interesting to those of us living in them but we should not let our desire to understand current events entirely dominate our research. We should also be writing for readers living years or even decades in the future. The simple fact that the last decade is recent history in the present moment provides no assurance that it will be especially interesting or relevant when this recency is no longer a factor. In 20 years, an article about the 1900–1914 period might be just as interesting and relevant to readers as an article focusing on 2000–2014. Even if you were certain that some set of past historical conditions would never recur again, the implications of those conditions for the phenomena we care about would still be worth understanding. Such unusual conditions might reveal the limits of what we think we know and would suggest theoretical possibilities that might not occur to you if you remained immersed in current events.

The advantages of historically bounded research

I hope by now that you think placing historical and geographic bounds on our research makes sense, at least sometimes. From the standpoint of generalizable social science, this conclusion may seem like a disappointment. This view is wrongheaded. In fact, historically bounded research has at least four concrete advantages for both theory development and empirical research.

First, accepting limits on the historical scope of what you are explaining permits more precise theoretical arguments. Historical scope limitations can eliminate the need to generalize across institutional differences or the special features of actors’ preferences and abilities that are important but impractical to consider across a wider swath of space and time. Put another way, you can justifiably make more assumptions and leverage them in your theoretical argument. For example, consider the question of whether an approaching election might prompt a national leader to use international conflict to divert attention from his or her poor job performance in other areas. Whether this scenario is plausible depends on many things, including whether the leader can control the timing of the election, whether the electorate tends to support belligerent behavior, whether the leader has adequate military forces for such an action, and whether appropriate targets are available. One might be able to make the necessary assumptions to support such a conjecture about diversionary behavior by postwar American presidents, but probably not for national leaders in many—perhaps most—other historical settings.

Similarly, many important causal mechanisms may only be discerned within specific historical bounds where there are similarities in the nature of the actors and the institutional frameworks in which they operate. For instance, the United States had no foreign aid programs, as we currently understand them, before World War II. With rare exceptions, the American state did not directly transfer resources to foreign governments. However, American policymakers often worked with private bankers, encouraging them to lend in ways that advanced American foreign policy goals (e.g. Moore, 2011; Munro, 1964; Rosenberg, 2003). This process resembled later foreign aid programs in some respects but clearly worked a bit differently because of the private financial interests of the banks. The need to induce their participation limited what the United States government could do. Because the causal process behind this type of foreign aid differed from more recent foreign assistance programs in this important respect, it would be a mistake to theorize about them in precisely the same way.

A second advantage concerns data quality. The acceptance of historical bounds has at least two benefits in this respect. One is conceptual. The specific manifestations of broad concepts like “war” or “military capabilities” vary in different historical settings. As the historical scope of the data expands, the assumptions about the relationship of the specific events and conditions that are actually coded to the underlying concepts become increasingly strained. For example, the significance of iron and steel production for national military capabilities has clearly changed over time, but the widely used composite index of national capabilities requires researchers to assume that these changes do not invalidate the measure. Similarly, the usefulness of the 1000 battle death threshold for identifying wars is open to question in light of recent developments in military medical technology. More recent conflicts with lower casualties may be equally intense (Fazal, 2014). Nevertheless, identifying “wars” over a long period of time requires some consistent marker and others might turn out to be just as problematic. Nearly all data collection efforts require some conceptual compromises, but they are generally less difficult over shorter spans of time and space.

Another data quality advantage concerns the range of useful information available to researchers. To some extent, data availability is a function of historical accidents regarding the things social actors needed to record and which of these records survived to be examined by later scholars. For this reason, information about many phenomena of interest may be unique to a particular historical setting. Even high-quality data that are available for only a small number of states or a short span of time are simply not useful for studies that aim for very broad geographic and historical scope. For instance, an effort to develop a broad cross-national dataset on military aid would run likely aground on the fact that many states simply do not report this information, even for very recent years. Scholars working within specific historical bounds have been able to leverage the available evidence in ways that could not be repeated in many other places and times. Recent micro-level studies of civil war and state repression offer many excellent examples of this kind of research. Working in a diverse range of times and places, scholars have located records on state repression, forced resettlement, refugee flows, and many other phenomena sufficient to permit quantitative analysis as well as qualitative study. ⁴ The same approach can be applied to the study of interstate conflict, foreign policy, and international political economy in specific historical cases. Indeed, data on these topics may generally be easier to locate than those concerning civil war and repression because they more often involve legal and public behavior. ⁵ The point here is not that the sources of data are necessarily obscure but rather that they may be unique to particular times and places.

A third advantage of historically bounded research concerns model specification: it is easier to know what control variables belong in empirical tests. Theoretical arguments may not have a historical context, but observational data always will. This context is what creates the necessity for control variables in empirical analysis. There are always causal processes going on in addition to those of primary interest to researchers. Statistical evidence for a proposed theoretical process will be biased if these other considerations affect the value of the dependent and independent variables. The number of these confounding processes will increase with the historical and geographic scope of the analysis. It is difficult to detect them, let alone include them in the empirical analysis, across many different historical contexts. For this reason, it may make sense just to focus on one historical setting even if you have not fully theorized the reasons it differs from others. The fact that you can consider a more complete and appropriate set of confounders will give people more confidence in the causal relationships you uncover. Most of us intuitively trust claims about causation that focus on quite specific mechanisms in a particular historical context. Research on causality bears out this intuition, suggesting that even some causal effects we think of as very general may actually be quite local (Samii, 2016).

By way of illustration, I have recently finished an article on support for the construction of a battleship fleet in the United States in 1890, an important step in the emergence of the country as a world power (Fordham, 2019). I wanted to test an argument about the effect of trade interests on congressional support for building the fleet. The economic interests of the heavily protected manufacturing sector called for spheres of influence in the less-developed world. Non-industrialized trading partners in these areas would not export manufactured products to the United States, and so would not demand reduction in US tariffs on these products. Because other major powers were also trying to set up similar exclusive empires or spheres of influence, establishing these arrangements was a competitive process. Battleships were an important power projection tool in this contest. On the other hand, building these warships was far less attractive to export-oriented agricultural interests and their political representatives. Less-developed markets held little benefit for them, and the process of securing access to these poor markets would bring the United States into conflict with the agricultural sector’s most important trading partners.

My main hypotheses were that members of congress from states with more import-competing interests should support the program, while those from export-oriented states should oppose it. An important difficulty in testing the argument is that the regional differences in economic structure responsible for different trade interests also had other effects that could have influenced support for the battleship program. These included the local importance of the steel and shipbuilding industries, which had a different economic stake in the construction of battleships (Baack and Ray, 1985; Trubowitz, 1998: 43). Less obviously, some historians have argued that labor unrest and immigration helped motivate support for the battleship program (e.g. Dallek, 1982; Hofstadter, 1966 [1951]). Both phenomena were more intense in regions with more manufacturing industries. Still less obviously, the regional economic interests I cared about were correlated with the causes of the Civil War. The Southern economy was mainly agrarian while the North had a larger manufacturing sector in both 1860 and 1890. In 1860, these differences shaped what side states took in the war. In turn, the war strongly influenced subsequent party divisions in the country. The battleship program was a hobbyhorse of the Republican Party, so this linkage was also potentially important. Omitting these other effects of economic structure raised the possibility that my trade variables would proxy their effects and might produce spurious evidence in support of my argument.

The point of this example is that figuring out what control variables made sense required context-specific knowledge about American politics and the American economy in the late nineteenth century. For once my interest in this period, which extends beyond anything remotely necessary for my research, actually paid off instead of just being a time-consuming diversion. If I had been trying to explain support for power projection across a much wider expanse of space and time, I might not even have recognized these issues. Even if I had, it might have been difficult to get the data needed to address them in my analysis. There is no reason to think that support for battleship-building in the United States in the 1890s is a more complex phenomenon than most of the others that concern our research community. Historically specific confounders are likely to beset most research designs that rely on observational data.

A fourth advantage of working within explicit historical bounds is the opportunity to contribute to the understanding of events we care about. This point is perhaps obvious, but it is nevertheless worth noting that quantitative and formal approaches have underappreciated advantages for answering some widely discussed historical questions. The use of quantitative analysis or formal theory is not especially common in constructing historical explanations. However, uncommon approaches to argument and evidence may illuminate aspects of historical events and processes that might otherwise be neglected.

I have already argued that explaining historical events is impossible without a theoretical apparatus of some kind—at least an implicit one—for deciding which artifacts are relevant for reconstructing a historical process, and for drawing connections between these artifacts. Explicit or formal theory can potentially perform these tasks better, highlighting considerations that might otherwise be overlooked. For instance, outcomes that are rarely observed because they are off the equilibrium path might still affect behavior as actors seek to avoid them. State leaders’ reluctance to back down after issuing an explicit threat, owing to concern about the reaction of domestic or international audiences, is one example (Fearon, 1994). The importance of this phenomenon is hotly disputed, but the debate about it would not have happened if models suggesting the role of audience costs had not been developed. These theories tell historical researchers, including those who doubt that audience costs matter, what they should look for, and when they should expect to see it. ⁶

Quantitative evidence can get at some aspects of historical phenomena better than the documentary sources scholars most often use to construct historical narratives. It is especially useful for examining structural considerations that escape the conscious awareness of social actors, or that they prefer not to discuss openly. The impact of selection processes or constituent interests may be clearer from patterns in political actors’ behavior than in what they say about their own motives. To illustrate this point, let me return to the research I summarized earlier about support for the US battleship program in 1890. Many accounts of the battleship debate, including some written by political scientists as well as historians, stress the role of specific individual advocates of the program, as well as the ideas they propounded to support it. A reading of the documentary record might well lead you to emphasize these things, because that is what comprises most of it. However, these ideas were more persuasive to people from regions of the country that had particular structural characteristics. These regions had a strong tendency to select political leaders who accepted these ideas, thus indirectly shaping the debate by determining who participated in it. Participants in the struggle over the battleship program in Congress did not spend much time discussing the societal forces that brought them to power. Indeed, they may not even have been entirely aware of them. This lack of discussion does not mean that these structural considerations were unimportant, though. There is very strong evidence that they were. The documentary record does not provide much of this evidence, but it is quite clear in data about their behavior.

Of course, quantitative evidence has its own blind spots, and we still need to know what is in the documentary record. Applications of quantitative social science cannot and should not displace historical accounts based on these sources. Indeed, conducting quantitative or formal research with historical bounds will make the relevance of archival evidence and research based on it even more obvious. The work of historians who have cultivated the skills and knowledge to use these qualitative sources is complementary because we are studying the same thing. Just as our work can contribute to understanding specific historical events, so their work often contains useful theoretical arguments, although they may present them somewhat differently than a political scientist would. The bottom line is that we are engaged in the same intellectual project, however different the folkways of our two disciplines may be. Recognizing this common project could be a first step in building an explicitly historical side to our research community similar to the field of economic history within the discipline of economics. This would facilitate a more focused dialogue with international historians.

I have written this essay mainly with an audience of quantitatively oriented scholars in mind. Qualitatively oriented scholars have long embraced historical work of the sort I have advocated here. Just as an explicit historical focus could bring the quantitative conflict research community into greater dialogue with historians, so it could also be a bridge to qualitatively oriented social scientists interested in the same topics. As members of the same discipline and frequently of the same academic department, the opportunities for fruitful collaboration are even greater than they are with historians. The use of both quantitative or formal analysis and primary historical sources in the same work remains uncommon, but it has produced fascinating and impressive results when it has been undertaken (e.g. Goemans, 2000; McDonald, 2009; Simmons, 1994). Work of this sort requires both a substantial amount of space—it is much more likely to be found in books than in journal articles—and several different skill sets. For the same reason that this kind of research makes enormous demands on an individual researcher, it represents an opportunity for collaboration across the methodological divide that still afflicts our discipline.

I conclude with a suggestion for editors and referees, the gatekeepers of our profession. As social scientists, we often focus on the generality of a theoretical argument when assessing the value of a piece of research. The trouble with this criterion is that it necessarily devalues work that has explicit geographic or historical bounds. It discourages researchers from acknowledging these bounds even when they are actually quite important. A successful theoretical argument that applied at all times and in all places would be great, but we need to remember how unlikely we are to find such a unicorn. An unbridled demand for generality mandates scholarly hubris and ignores the nature of our subject matter. Social actors differ enormously across space and time in ways that limit the generality of our arguments. Not all important causal mechanisms are historically or geographically general, and we should not expect them to be. This fact is not rooted in the inadequacy of our theories or our data but in the nature of what we are studying. We need to temper our pursuit of generality with an appreciation of the limits on how general our conclusions can be and of the advantages to working within acknowledged and realistic historical bounds.