Concrete Counterfactual Tests for Process Tracing: Defending an Interventionist Potential Outcomes Framework

Abstract

This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to corroborate process tracing claims. In particular, three steps are prescribed for an interventionist investigation, and each step in turn is shown to make the causal analysis more robust, amongst others by disambiguating causal claims and clarifying or strengthening the existing methodological advice on counterfactual analysis. The article's claims are then illustrated using a concrete example, Haggard and Kaufman's analysis of the Argentinian transition to democracy. It is shown that interventionism could have strengthened the authors’ conclusions. The article concludes with a short Bayesian analysis of its key methodological proposals.

Keywords

causal inference case studies process tracing interventionism counterfactuals potential outcomes

Introduction

Case study researchers are increasingly impressed by process tracing as a method for discerning correlation and causation: process tracing promises to open the ‘black box’ of case study causation by focusing on finding the causal mechanisms behind observed correlations (cf. Beach and Pedersen 2019; Bennett and Checkel 2015; Brady and Collier 2010; George and Bennett 2005; Hall 2013; Hedström and Ylikoski 2010; Mahoney 2001). However, there is a clear lack of consensus about the appropriate underlying philosophical theory of causation for process tracing. This is problematic, given Ingo Rohlfing and Christina Zuber's recent conclusion that philosophical theories entail criteria for when social scientific claims about causal mechanisms are true or false, as well as tell what social scientific evidence can support testing for these criteria (Rohlfing and Zuber 2021). One prominent philosophical disagreement concerns the benefits of a potential outcomes versus a systems approach (cf. Beach 2016; Jacobs 2016; Runhardt 2016). One of the charges levelled against the potential outcomes approach is that it lacks concrete evidential tests at the level of individual case studies.

James Woodward's interventionist theory of causation is a methodological project (Woodward 2003, 2015), amongst others aimed at giving the social scientist concrete evidential tests to establish the truth or falsity of a causal claim. At its foundation, the interventionist theory relies on a counterfactual, potential outcomes view of causation. While the interventionist theory has clear implications for quantitative research (cf. Waldner 2012; Woodward 2000, 2007a, 2007b), its implications for qualitative research methods like process tracing are rarely explored. Given the prominence of similar potential-outcomes approaches to causality in quantitative social research (cf. Rohlfing and Zuber 2021), relating process tracing to this framework is valuable both in its own right and as a potential foundation for mixed-method research combining process tracing with quantitative methods. Moreover, relating process tracing to interventionism can show both methodologists and practitioners that concrete evidential tests are possible at the level of case studies within a potential outcomes framework.

In short, then, this article combines interventionist theory with process tracing methodology, thus bringing the philosophical and methodological literature closer together and filling gaps in both. I will show that interventionism has concrete implications for process tracing: interventionism suggests evidential tests which can corroborate causal claims about the operation of mechanisms within case studies. One of the gains of my interventionist method, I will show, is that its counterfactual analysis is more rigorous than earlier methodologies of counterfactuals in case study research (Goertz and Levy 2007; Harvey 2011, 2015; Levy 2008, 2015; Mahoney and Barrenechea 2019; Nolan 2013; Tetlock and Belkin 1996). Process tracing methodology will thus benefit from a stronger formal embedding in Woodward's leading potential outcomes theory in particular, as opposed to these earlier methodologies.

As I will show below, I recommended that to corroborate singular claims, we must take three steps: circumscribing the causal claim, choosing an appropriate intervention, and finding evidence for this intervention. Each of these new steps will help case study researchers strengthen their causal analysis. The first step, of circumscribing the causal claim, will help researchers avoid false positives and negatives (i.e., the belief that a relation is causal while it is not, or not causal while it is) by making sure the evidence they look for is relevant to the causal claim of interest, and not a similarly worded but different claim. The second step, choosing an appropriate intervention, helps researchers articulate exactly what they may change about the world in a counterfactual scenario without damaging the causal relation of interest. While advice like the ‘minimal rewrite rule’ from the methodological literature also has this aim, my interventionist analysis gives more tangible criteria for a counterfactual so we can ascertain whether the counterfactual is appropriate. The third step, finding evidence for the intervention, will amongst others defeat the criticism that counterfactual scenarios cannot give evidence of what actually happened in a case because of their hypothetical status.

The structure of the article is as follows. After first introducing interventionism, I suggest how one ought to apply the interventionist theory of causation in case study research, based on the theory's technical definition of singular, token causation. In particular, I introduce and discuss the notion of a so-called ‘hypothetical intervention associated with a causal claim’ and detail the three pieces of evidence that one must gather to support interventions. Subsequently, by comparing advice from both my own method and earlier counterfactual methods, I argue that the interventionist approach clarifies the logic behind more imprecise counterfactual approaches in the social sciences. This supports my claim that interventionism suggests concrete evidential tests, despite its counterfactual view of causation. I end this first part of the article by situating the interventionist approach presented within current trends in philosophy of science and methodology.

The second part of the article supports the first by investigating a concrete example, Stephan Haggard and Robert Kaufman's process tracing work in Dictators and Democrats (Haggard and Kaufman 2016). In particular, I examine their process tracing analysis of the Argentinian democratic transition. I argue that the support Haggard and Kaufman need to corroborate their own hypothesis should be described in terms of hypothetical interventions. Evaluating Haggard and Kaufman's work by causal interventionist standards, this article finds that the authors do not meet interventionism's strict demands for corroborating singular causal claims. Although the authors circumscribe the singular causal claim being tested in line with interventionist demands, they do not deliver sufficient evidence for the associated counterfactual. I argue that if Haggard and Kaufman were to use the interventionist technique, they would achieve a more rigorous method. In the last part of the article, I frame this negative conclusion in Bayesian terms.

Interventionism

James Woodward's interventionism is a “set of methodological proposals” (Woodward 2015:3577) which analyses a causal claim $X \to Y$ by asking whether intervening on X would have an effect on Y. If appropriate changes in X are associated with changes in Y, and some specific technical requirements on the intervention and other variables connected to X and Y are met, the relation is causal. Woodward's methodology has many commonalities with the potential outcomes literature: it is a structural counterfactual theory which utilizes causal graphs as identification strategies for corroborating causal claims (see “Why This Philosophical Theory?” below). While it is not often recognized as such (cf. Rohlfing and Zuber 2021:1628, Table 1), Woodward's particular methodology can be applied to both the corroboration of type-level (general) causal claims and the corroboration of token-level (singular) causal claims.

Interventionist evidence, in its simplest form, can include evidence from actual interventions (e.g., in randomized controlled trials) and natural experiments (Reiss 2005; Woodward 2003). Woodward realizes that one cannot always manipulate a putative cause X in practice; interventionism admits that in those cases, we should think of the intervention as purely hypothetical. In other words, in those cases the interventionist theory is a counterfactual approach to causation, in which we search for evidence of what would happen to Y if we intervened on X in a particular way. Woodward's use of counterfactuals is especially apt for case study research, including process tracing analysis. In process tracing analysis, it is rarely possible or desirable to manipulate putative causes and so this area of research is fitting for Woodward's analysis.

While interventionist theory gives us a clear set of technical requirements on what a hypothetical intervention ought to look like to support the claim that $X \to Y$ , Woodward does not consider what evidence for the intervention looks like in qualitative social sciences. What sources of evidence are appropriate? What types of evidence? And how ought we to represent social causal mechanisms or processes using Woodward's framework? Woodward's most detailed examples are from the natural sciences and so of little help. Yet there also exists a body of literature specifically aimed at working out what counterfactual claims are acceptable in the social sciences (cf. Goertz and Levy 2007; Harvey 2011, 2015; Levy 2008, 2015; Mahoney and Barrenechea 2019; Nolan 2013; Schroeder 2004; Tetlock and Belkin 1996). Marrying these approaches and interventionist theory will thus be fruitful in further grounding process tracing; moreover, I will show below that an interventionist approach to counterfactuals is more rigorous than previous methodologies.

Finding evidence for mechanistic claims is at the heart of process tracing. Here, I will follow the analysis in Runhardt (2015), which describes process tracing as such: “In the simplest case (in which there is only one hypothesized mechanism), we may formalize process tracing as follows. Let us call the researcher's own hypothesis $H_{Z}$ . $H_{Z}$ holds that a causal mechanism Z is behind a process linking a putative cause, X, and the observed effect, Y. This mechanism has observable implications, that is, a set of variables $Z_{i}$ such that $X \to Z_{1} \to Z_{2} \to \dots \to Y$ (where $Z_{i} \to Z_{j}$ means that $Z_{i}$ causes $Z_{j}$ ). It is this chain of events that process tracers trace; note that there is a difference between causal mechanisms and the process that they produce.” (Runhardt 2015:1297) Using this representation, we can fruitfully apply interventionism.

In earlier work (Runhardt 2015, 2016), I have shown that in the simplest scenario, interventionism asks us to consider each step $Z_{i} \to Z_{i + 1}$ of the causal chain $X \to Z_{1} \to Z_{2} \to \dots \to Y$ in turn, providing evidence of what would happen to .. under an intervention on $Z_{i}$ . Given this reconstruction, an interventionist approach to corroborating a singular mechanistic causal claim will consist of three main steps: (1) describing each causal ‘step’ under investigation (including the $Z_{i}$ and $Z_{i + 1}$ in the causal relation, but also the wider network of potentially causally related factors) and the associated counterfactual; (2) choosing an appropriate intervention; (3) evaluating evidence of what would happen under this intervention, i.e., guidelines for evaluating evidence for or against the counterfactual.¹ In the remainder of this section, I will detail each of these steps in turn.

Describing the Causal Claim and Associated Counterfactual

To start an interventionist analysis, one must carefully circumscribe the causal claim $X \to Y$ and its associated counterfactual. Doing so makes our causal analysis stronger, because it allows us to “clarify or disambiguate causal claims by explicitly distinguishing among different possible claims (…) that might be intended or by replacing claims involving factors for which the notion of manipulation is not well-defined with claims involving variables for which there is a well-defined notion of manipulation” (Woodward 2003:115).

To see what is intended, consider a simple example, the causal claim “If the Industrial Revolution had not occurred, the British standard of living would have been lower than it was” (Tetlock and Belkin 1996:20). As it stands here, this claim is quite clearly underspecified. Which aspects of the Industrial Revolution, specifically, are hypothesized as the putative cause of the higher standard of living here? Are we only referring to the technological aspects of the Revolution (such as the invention of the spinning jenny or the steam engine)? Or do we refer to some of the large-scale social changes during the Revolution? Specifying the counterfactual can help one clarify the intended causal claim. The associated counterfactual here could be about preventing the invention of the steam engine; alternatively, it could be about preventing some of the large-scale social changes during the Revolution. Considering associated counterfactuals allows the researcher to carefully circumscribe the causal claim at issue; it disambiguates one's analysis. ‘The Industrial Revolution caused the British standard of living to increase’ is too vaguely specified, but once the counterfactual is considered more clearly, X and Y are more easily circumscribed as well. Thus, this first step of interventionist process tracing sharpens our causal analysis. We avoid false positives and negatives (i.e., the belief that a relation is causal while it is not, or not causal while it is) by making sure the evidence we look for is relevant to the causal claim of interest, and not a similarly worded but different claim.

Choosing an Appropriate Intervention

Once we have circumscribed the causal claim, our second step is to choose an appropriate hypothetical intervention (and subsequently, in the third step, provide evidence for it). The search for what is and is not an appropriate intervention is not new to Woodward's work; it comes up, albeit without mention of interventions, in the methodological literature on counterfactuals. Jack Levy, for instance, has argued that to evaluate a counterfactual argument, we must precisely specify both antecedent and consequent, amongst others by clarifying in what way (in our vocabulary: through what intervention) the counterfactual scenario is brought about (Levy 2015:389). How exactly do we propose to prevent the aspects of the Industrial Revolution we have settled on as our putative cause?

Interventionist theory goes beyond the methodological literature on counterfactuals because it provides a more precise logic behind what interventions are acceptable to bring a counterfactual about. As Levy suggests, there are multiple paths through which, e.g., an event like the Industrial Revolution (or aspect thereof) could have been prevented. However, not all these paths will be the result of an appropriate intervention. By linking causal claims with interventions, this second step of interventionist process tracing allows us to clarify which counterfactual scenarios give evidence for the causal claim, and which do not.

In interventionist theory, $X = x$ causes $Y = y$ iff:

“(AC*1) The actual value of $X = x$ and the actual value of $Y = y$ .

(AC*2) For each directed path P from X to Y, fix by interventions all direct causes $Z_{i}$ of Y that do not lie along P at some combination of values within their redundancy range. Then determine whether, for each path from X to Y and for each possible combination of values for the direct causes $Z_{i}$ of Y that are not on this route and that are in the redundancy range of $Z_{i}$ , whether there is an intervention on X that will change the value of Y. (AC* 2) is satisfied if the answer to this question is “yes” for at least one route and possible combination of values within the redundancy range of the $Z_{i}$ .” (Woodward 2003:84)²

The intuition behind the second demand, AC*2, is that not all potential interventions one may use to bring a counterfactual scenario about are appropriate. If the intervention directly or indirectly influences the effect of interest by itself, then it is not an appropriate intervention. Moreover, we are asked to keep fixed at a certain value all variables connected to effect Y not on the path between cause $X$ and effect Y to avoid interference from these variables. This requirement will be familiar to those acquainted with other potential outcome and structural causal modelling approaches as a way of avoiding confounders. The fixing of other variables not on a specific path mirrors mediation analysis of path-specific effects (cf. VanderWeele 2015, 2016), albeit at the level of singular causation.

Evaluating Evidence for the Intervention

The third step of the intervention approach is to evaluate whether there is evidence for or against the specified intervention from step two. According to Woodward, evidence for an intervention can come from “observation or from a combination of observation and experiment” (Woodward 2003:35).³ Moreover, there is no reason to resist the use of a mixed methods approach here, using correlational evidence to support or weaken an intervention claim, as long as this evidence speaks to the described associated counterfactual and follows the demands above.⁴

For process tracing, the pieces of evidence that could corroborate a counterfactual claim of what would have happened under intervention are varied. Jason Lyall's best practices for process tracing (Lyall 2015) show how cross-case process tracing and matching can be used to provide similarity comparisons of the form ‘what would have happened if a certain step of the process had not occurred?’ This information directly provides evidence for the intervention variable, since one finds a sufficiently similar case in which a step in the process $Z_{i}$ was not present, to see what would happen to the subsequent step $Z_{i + 1}$ . Runhardt (2015) works this cross-case approach out for a concrete example, Kirstin Bakke's process tracing analysis of tactical innovation during the Chechen wars (Bakke 2013).

Yet there are other sources of evidence, as illustrated for example by Lebow and Stein in their counterfactual analysis of the Cuban missile crisis (Lebow and Stein 1996). They describe in detail what evidence may be collected to either support or weaken the counterfactual “if Kennedy had had a firmer reputation for resolve, Khrushchev would have sent the missiles to Cuba” (Lebow and Stein 1996:121). As they discuss, evidence that Khrushchev was motivated by his doubt in Kennedy's resolve would support the counterfactual. Such evidence can come from a variety of sources, from archives to witness interviews. This illustrates that what is relevant evidence for a particular intervention claim about a process will depend on context.

Finally, note that none of these pieces of evidence alone are sufficient to confirm the counterfactual claim. We specify the counterfactual claim to clarify for ourselves which pieces of evidence we must collect to corroborate the causal relation. We use these pieces of evidence to bolster our faith in the counterfactual, and thereby the causal relation. The fact that we cannot confirm, only corroborate a hypothesis, should not come as a surprise; this is not a drawback unique to interventionism. I will come back to this in the short Bayesian analysis in “A Bayesian Analysis of the Above” of the article.

Comparisons with the Methodological Literature on Counterfactual Analysis

So far, we have seen that interventionist theory defines singular causation in terms of hypothetical interventions. I have prescribed three steps for an interventionist investigation: to circumscribe the causal claim, to find an appropriate intervention, and finally to establish evidence for the intervention.

Evidence for a hypothetical intervention is by definition evidence for a counterfactual claim. As mentioned in the introduction, there exists a methodological literature on counterfactual analysis in the social sciences which can supplement interventionist theory with concrete evidential guidelines. So, before describing what evidence for interventions would look like in a concrete case study in “Causal Mechanisms in Haggard and Kaufman’s Dictators and Democrats”, I will finish this section by comparing the methodological and interventionist literature. I will analyze three common requirements: the ‘demand for clarity’, the ‘minimal rewrite rule’, and the demand for ‘plausibility of the antecedent’. I show how an interventionist's recommendations compare to these requirements, highlighting their agreements and differences. In doing so, I show that interventionist process tracing delivers higher quality causal inferences than these traditional counterfactual case study research methods.⁵

The Demand for Clarity

The demand for clarity, which we can find in Tetlock and Belkin (Tetlock and Belkin 1996:19–21) is the demand that counterfactuals have “well-specified antecedents and consequents” (Tetlock and Belkin 1996:19): one ought to be specific enough in describing the antecedent that other variables that may influence the outcome are mentioned as well. As we have seen in “Describing the Causal Claim and Associated Counterfactual” above, Tetlock and Belkin discuss the counterfactual “If the Industrial Revolution had not occurred, the British standard of living would have been lower than it was” (Tetlock and Belkin 1996:20). There, we only discussed the careful description of both the putative cause and effect of interest. ‘Clarity’ here means we must also specify other variables which potentially affect the effect of interest. In the example, this means we must specify what the growth rate of the British population is, since this variable is connected both to the occurrence or absence of the Industrial Revolution and to the standard of living.

We may feel this demand is related to interventionist theory's demands on interventions. However, this is not necessarily the case. Tetlock and Belkin's worry is that the absence of the Industrial Revolution would have affected the population growth rate, which in turn would have affected the standard of living. But this, in interventionist theory, would be acceptable; we are asked to keep fixed at a certain value all variables connected to Y not on the path between $X$ and $Y$ . Interventionism specifies that “for each directed path P from X to Y, fix by interventions all direct causes $Z_{i}$ of Y that do not lie along P at some combination of values within their redundancy range” (Woodward 2003:84). In Woodward's framework, we are not required to include in the description of the intervention or antecedent any factors $Z_{i}$ that are on the directed path P from $X$ to Y, unless these factors are independently influenced by the intervention. Yet that is not what is at stake here. When Woodward talks of being specific enough in describing antecedent and consequent, he means something different than Tetlock and Belkin do: he asks only that it is logically possible to change X, and that we can disentangle “the effect on E of changing just C from the effects on E of changes in other potentially confounding variables, including direct effects from the intervention process itself.” (Woodward 2003:131) Figure 1 illustrates this for the Industrial Revolution example, using simple causal graphs. We must distinguish the causal linkage $I \to R \to P \to L$ (with I the intervention, R the industrial revolution, P the population growth, L the standard of living) from any influence of $I \to P \to L$ bypassing R. Thus, our network ought to look like Figure 1's situation (a) and avoid cases like (b) and (c), with X some possible intermediate step connecting I and P not related to R. We do not want the intervention we specify to change the population level except through preventing the industrial revolution.⁶

Figure 1.

Acceptable verus unacceptable interventions for the Industrial Revolution case.

The Minimal Rewrite Rule

The minimal-rewrite rule (cf. Tetlock and Belkin 1996:23) is the methodological literature's equivalent of interventionist theory's demands on appropriate interventions. The minimal-rewrite rule asks that in constructing the counterfactual, we ought to “(a) start with the real world as it was otherwise known before asserting the counterfactual; (b) not (…) unwind the past and rewrite long stretches of history; (c) not unduly disturb what we otherwise know about the original actors and their beliefs and goals” (Tetlock and Belkin 1996:23). One way to do so is to only change “small events and contingent choices that could have easily turned out differently” (Mahoney and Barrenechea 2019:333). In Levy, the minimal rewrite rule comes down to the argument that “counterfactual analysis ideally posits an alternative world that is identical to the real world in all theoretically relevant respects but one” (Levy 2008:635). Interventionist theory shows us the logic behind the minimal rewrite rule; however, interventionist theory also establishes the minimal rewrite rule's limitations. As such, applying interventionist principles will strengthen our causal analysis more than the original minimal rewrite rule alone.

In interventionist theory, an intervention ought to only change the putative cause, X, and not any other factors $V_{i}$ related to Y but not on the path between X and Y. This is necessary for being able to distinguish the effects of $X$ 's change on Y from the effects of the other variables $V_{i}$ 's changes on … The whole ‘event’, if you will, is the combination of all variables $V_{i}$ , X, and Y; to say that the ‘event’ ought to be very similar to the ‘counterfactual scenario’ as the minimal-rewrite rule does, is in essence to ask for the $V_{i}$ 's related to Y but not X to be fixed at some value. The interventionist requirements and those in the counterfactual methodology literature thus far agree.

However, interventionism says nothing about changing variables we know to be unrelated to Y. If the intervention affects such variables, this is unproblematic for the interventionist, but not for someone strictly adhering to the minimal rewrite rule. The minimal rewrite rule is similar to a ceteris paribus condition: all other things must be equal. The interventionist specifies that only those variables related to Y must remain equal. Moreover, while the minimal-rewrite rule can be seen as a way to exclude any effects of intervention I on effect Y that ‘circumvent’ X, we should not see this rule as a description of what types of $X$ 's are usually causes (e.g., as a claim that these $X$ 's are usually small, easily altered events). That would conflate epistemological and ontological considerations. While Mahoney and Barrenechea write that “small and contingent events offer plausible antecedents in counterfactual analysis” (Mahoney and Barrenechea 2019:318), it would be more appropriate to say that interventions ought to be aimed at changing small and contingent aspects of the whole ‘situation’, leaving other aspects fixed.

Plausibility of the Antecedent

We have so far seen how the demand for clarity and the minimal rewrite rule compare to interventionism's technical requirements. The last methodological guideline for counterfactuals that I will discuss is closely related to the minimal rewrite rule: the demand for plausibility of the antecedent. While methodologists like Jack Levy and Paul Schroeder are emphatic that the antecedent of the counterfactual must be ‘plausible’, interventionist theory does not have a straightforwardly equivalent demand (Levy 2008, 2015; Schroeder 2004).

Plausibility refers to the idea that one may not simply introduce a counterfactual into historical analysis without careful consideration of what went on before the counterfactual element's putative place in history. Not any counterfactual is plausible; what is plausible depends on the exact circumstances that led to the event we wish to change. The worry is that counterfactuals are difficult to describe and test because, to avoid historical inconsistency, we must change what went on before the counterfactual, too, even if our counterfactual antecedent is only aimed at one moment in time. This may be summed up with the aphorism that ‘we cannot change history’.

Interventionist theory goes against this view and argues we can disregard, to a certain extent, what went on before the counterfactual element's place in history. To see why, consider Woodward's example of the causal relation between the position of the moon with respect to the Earth and the motion of the tides (Woodward 2003:129–31). This is a singular causal relationship which cannot be tested through an actual intervention or natural experiment (we cannot change the position of the moon, nor is there a similar enough planet with bodies of water and a single moon which is at a different distance from its planet). Thus, the intervention must be hypothetical. Woodward argues that it is not at all clear that such an intervention is even physically possible, i.e., that we may increase the moon's orbit in such a way that we meet the intervention conditions discussed above, without violating physical law.

An intervention on the causal relationship between the moon and the tides, Woodward argues, must be

“sufficiently fine-grained and surgical that it does not have any other effects on the tides besides those that occur through the change that it produces in the position of the moon, and it may well be that the laws of nature guarantee that all real causal processes will have such additional effects. At the very least, it seems wildly optimistic to assume that appropriately surgical intervention processes must be available for all true causal claims.” (Woodward 2003:130)

This example shows us that it is not within interventionist theory to make arguments along the lines that ‘we cannot change history’.

Importantly, interventionist theory is not limited to physically possible interventions. Woodward argues that he has a ‘logical possibility’ approach in mind instead. He finds the following much too strong: “On one notion, an event E is physically possible if and only if it is consistent with the laws of nature and the actually obtaining initial conditions. When conjoined with determinism, this notion of physical possibility implies that interventions on X will not be possible unless they actually occur.” (Woodward 2003:128) Again, nowhere here is Woodward concerned with the plausibility of the antecedent in the ‘concrete’ terms of the methodologists. His interventionist approach allows him to be more ‘fine-grained’ as it were in chooosing what interventions are informative, rather than having to use a more vague notion of ‘plausibility’.

To sum up the above comparison between current counterfactual methodology and interventionism, consider the table below.

Table 1.

Differences Between Traditional Counterfactual Rules and the Interventionist Approach.

Rule for counterfactual reasoning	Interventionist alternative
Demand for clarity - one ought to describe both antecedent and consequent so specifically that all other factors that may influence the consequent are mentioned as well. Changing factors that influence the consequence is unacceptable; we must avoid changing anything but the antecedent in the counterfactual scenario.	One ought to describe all the factors in the causal network, including those not on the path between cause (antecedent) and effect (consequent). Changing factors that influence the consequence (using an intervention) is acceptable, as long as these factors are part of the process between the cause and effect and not independent factors that stand outside this process.
Cotenability (see footnote xi) – Specify everything that would have to be true to sustain the counterfactual.	Specify all the factors in the causal network that contains cause and effect.
Minimal rewrite – Let the counterfactual world be identical to the real world in all but one aspect, preferably some small event or contingency that could have easily turned out differently	It is fine for the counterfactual world to be different from the real world in more than one aspect, as long as any independent influences on the effect that stand outside the process between cause and effect remain the same.
Plausibility of the antecedent – Make sure the counterfactual scenario is plausible given what went on before it.	The intervention does not need to be plausible or even physically possible, just as long as it is informative (i.e., meets demands AC1 and AC2).

Situating the Interventionist Approach

Now that I have presented the interventionist approach to process tracing, including the concrete steps involved, in this section I pull back to situate this approach within the further literature on process tracing. I relate the method to other potential outcomes frameworks and contrast it to its main rival, the systems approach to process tracing. Readers interested in the direct application of the interventionist framework to a concrete case study, rather than a more theoretical discussion of trends in philosophy of science, are invited to skip ahead to “Causal Mechanisms in Haggard and Kaufman’s Dictators and Democrats”.

In the methodological work by George and Bennett which instigated much of the interest in process tracing (George and Bennett 2005; cf. Kittel and Kuehn 2013:1–2), process tracing was defined as the identification of “the intervening causal process – the causal chain and causal mechanism – between an independent variable (or variables) and the outcome of the dependent variable” (George and Bennett 2005:206). Bennett and Checkel later refined this definition to “the analysis of evidence on processes, sequences, and conjectures of events within a case for the purposes of either developing or testing hypotheses about causal mechanisms that might causally explain the case” (Bennett and Checkel 2015:7).

There exists a range of definitions of what a causal mechanism actually is (cf. Mahoney 2001), and, in accordance, a range of further interpretations of what process tracing involves and how it ought to proceed. Some authors writing on process tracing have a minimal interpretation of the method which does not go far beyond the George and Bennett definition. In this minimal interpretation of process tracing, the theoretical causal mechanism linking a putative cause and effect of interest is reduced to an observable chain of events. In this view, the ultimate aim of process tracing is “the identification / description of the intermediate links in a causal chain” (Clarke forthcoming:3) without further unpacking of the mechanism behind these links.⁷

A larger literature relies on a less minimalist definition of causal mechanisms. One can distinguish two main categories of current interpretations of process tracing, each of which arguably relies on a different fundamental theory of causation: (1) systems approaches, which define mechanisms in terms of entities engaging in activities and thereby producing the outcome of interest; and (2) potential outcomes or causal network approaches, which seek support for mechanisms in part by referring to counterfactuals.⁸

Systems approaches. In this leading view (cf. Beach and Pedersen 2013, 2019), the causal mechanisms which process tracing is aimed at tracing are defined using a mechanistic philosophy of causation that first arose in the life sciences (cf. Bunge 1997; Glennan 1996; Machamer, Darden, and Craver 2000). Systems approaches unpack causal mechanisms into parts, each of which can be described as entities engaging in activities (cf. Machamer et al. 2000), “studied empirically in the form of the traces that the activities associated with parts of the process leave within cases” (Beach and Pedersen 2019:38). In this view, each piece of evidence must be linked to each actual step in the hypothesized causal processes (cf. Bennett, Fairfield, and Soifer 2019); we want to know how the parts of the system have actually produced an effect of interest without reliance on difference-making or counterfactual evidence (Beach 2016). “Taking causal mechanisms seriously by adopting a ‘mechanistic’ understanding forces the analyst to conceptualize a theorized mechanism in a manner that explicitly attempts to ‘capture’ the dynamic, causal forces in each part of the mechanism that produces the outcome.” (Beach 2013:14)

Potential outcomes approaches. Other scholars have argued that we ought to define the causal mechanisms in process tracing with a counterfactual theory of causation. The most prominent such philosophical theories are structural theories that represent causal networks using graphical models (Halpern 2016; Pearl 2000; Spirtes, Glymour, and Scheines 1993; Weinberger 2019; Woodward 2002, 2003). Potential outcomes approaches rely on the assumption that, without counterfactual reasoning, we cannot be sure that it was the hypothesized causal process that led to the effect. The claim is thus that structural counterfactual theories are most useful to anchor the causal mechanism claims in process tracing (Rohlfing and Zuber 2021:1645–48), and that causal graphs are useful as identification strategies for corroborating these causal claims. David Waldner, for example, argues that process tracing ought to consist of “careful articulation and defense” of causal graphs (Waldner 2015:143) which are then matched to actual developments in a case setting, “showing that the events in a particular case constitute the theorized value of a random variable as expressed in the causal graph” (Waldner 2015:150). Other potential outcomes defenders, like Jason Lyall and I, have shown that potential outcomes evidence for counterfactuals can come amongst others from cross-case analysis (Lyall 2015; Runhardt 2015).

One might be tempted to consider Bayesian approaches to process tracing to be a third category of process tracing approaches, arguing that it “give[s] rise to different prescriptions for connecting evidence to inference” (Bennett, Fairfield, and Soifer 2019:3) compared to exemplars from the systems and potential outcomes approaches. However, this can be contested; here, I will consider Bayesian process tracing to be a way of judging the weight of evidence for either mechanisms described as systems or mechanisms described with potential outcomes and causal graphs. In fact, authors from both the systems and potential outcomes approaches refer to either the simpler four tests by Van Evera (cf. Van Evera 1997) or to Bayesian updating more generally (cf. Fairfield and Charman 2017; Humphreys and Jacobs 2015). Beach and Pedersen, for instance, refer to a Bayesian logic when discussing the probative value of evidence (Beach and Pedersen 2013, chapter 6, 2019, chapter 7). Waldner, too, calls Bayesian analysis “central to the enterprise” (Waldner 2015:130) and harmonious with his own theory. In general, many authors remark on the “evident connection between the use of evidence in process tracing and Bayesian inference” (Humphreys and Jacobs 2015:657).⁹

As we have seen, the systems approaches and potential outcomes approaches above each reflect a commitment to a different underlying theory of causation. The categories each imply a distinct set of truth conditions under which a process tracing causal claim will be true (Rohlfing and Zuber 2021). However, as Rohlfing and Zuber have pointed out, it is as of yet an open question what these truth conditions are. The framework that I presented in the preceding sections will go some way to answering this question for potential outcomes approaches. My contribution shares with these approaches a focus on using counterfactual reasoning to support causal claims about processes. However, taking a purely interventionist approach based on Woodward's theory of causation means that this article has provided its own, unique set of recommendations. For instance, the Waldner text cited above does not discuss in any detail what truth conditions must be met to corroborate that a causal graph is in fact complete and/or matches events of interest. The interventionist framework discussed in the above, on the other hand, generates a number of concrete truth conditions. By placing process tracing analysis in an interventionist context, including an explicit statement of hypothetical interventions, this article's framework thus builds on previous potential outcomes approaches and answers Rohlfing and Zuber's call to ‘check our truth conditions’ for this particular philosophical commitment.

Why This Philosophical Theory?

So far, we have seen how interventionism fits within the different philosophical approaches to process tracing. Before finishing this discussion, let me pre-empt one potential criticism. As was hinted at in “Interventionism”, Woodward's theory is best known for its applications in type-level causation, and there are alternative potential outcomes philosophies available which are aimed primarily at token-level causation. The most prominent of these is David Lewis’ counterfactual theory of causation (Lewis 1973, 1979). As such, a critic may wonder at the choice for Woodward over Lewis.

There are several reasons for choosing Woodward's interventionist theory in particular to ground a structural, counterfactual approach to causation in process tracing. Firstly, and most importantly, while the type-level theory contained in Woodward's interventionism is best known, interventionism contains a full theory and definition for token causation, which is the one we used in “Interventionism” and “Comparisons with the Methodological Literature on Counterfactual Analysis”.¹⁰ Secondly, there are methodological reasons to prefer Woodward's theory of token causation over Lewis’ theory of token causation. To assess the truth-value of counterfactual claims, David Lewis's theory utilizes the notion of ‘similarity’ between (a) worlds in which the antecedent of the counterfactual holds (so-called ‘possible worlds’), and (b) the actual world. This notion of similarity has commonalities with Woodward's central notion of intervention, but there are also clear differences (cf. Briggs 2012; Woodward 2003:133–45). For example, Lewis relies on laws of nature to define how similar to the actual world a possible world is: simplifying slightly, the fewer violations of the laws of nature this world contains, the closer it is. Relying on laws of nature, however, seems inappropriate in most social scientific contexts. As we have seen above, by design Woodward's interventionist theory does not require the existence of laws. Moreover, while Woodward's definition of token causation in terms of interventions is meant as a methodological theory (cf. Woodward 2015), Lewis's definition of causation in terms of possible worlds is meant primarily as a philosophical tool to reduce the notion of causation to non-causal elements. Therefore, taken altogether I agree with Rohlfing and Zuber that its particulars make Lewis’ theory “unattractive for the social sciences” (Rohlfing and Zuber 2021:1644).

Causal Mechanisms in Haggard and Kaufman's Dictators and Democrats

Thus far, we have seen that the interventionist theory of causation defines singular case causation in terms of hypothetical interventions. Accordingly, I recommended that to corroborate singular claims in process tracing, we must take three steps: circumscribing the causal claim, choosing an appropriate intervention, and finding evidence for this intervention. We have so far seen a brief toy scenario (the example of class size and attainment) to highlight what this last step consists of, and I have discussed how the interventionist theory's demands compare to three common demands in the methodological literature on counterfactuals. I will now turn to a more detailed analysis on how to find evidence for counterfactual interventions, using a recent example of process tracing in political science, Stephan Haggard and Robert Kaufman's analysis of democratic transitions and reversals (Haggard and Kaufman 2012, 2016).

Context and Method

In Dictators and Democrats (Haggard and Kaufman 2016), Haggard and Kaufman study which causal mechanisms played a role in transitions to and from democracy during the Third Wave (1980–2000). The authors analyze and test for several alternative causal processes to democratic transitions and reversions using a combination of statistical work and within-case process tracing.

In their statistical work, the authors show that contrary to work by amongst others Daron Acemoglu and James Robinson (cf. Acemoglu and Robinson 2001, 2006), there does not appear to be a statistical relationship between the level of inequality and democratic transitions. Instead, Haggard and Kaufman hypothesize that capacity for collective action is “a significant and robust predictor of distributive conflict transitions” (Haggard and Kaufman 2016:102). In sum, the authors argue that in some distributive conflict transitions, the density of social organization played a key part, rather than the level of inequality. In those cases, Haggard and Kaufman argue, one mechanism contributing to democratization is mass mobilization, which is more likely when there are organizations (such as unions, middle class organizations, professional associations, or ethnic organizations) that can assume the leadership.¹¹

Haggard and Kaufman use within-case process tracing to establish whether the correlations mentioned above “are supported by causal process observations of the distributive conflict transition cases” (Haggard and Kaufman 2016:102) and to count the number of “cases in which such processes appear to work as expected. In how many distributive conflict transitions were unions and other organized social forces involved in mass mobilization against incumbents?” (Haggard and Kaufman 2016:102). For example, the authors attempt to establish that the density of social organization in Argentina was a key factor in mass mobilization there, which in turn led to democratization efforts in the early 1980s. By establishing this chain of events in Argentina, the authors believe they corroborate that even in a highly unequal society, social mobilization can contribute to a democratic transition. This goes against Acemoglu and Robinson's claim that “highly unequal societies are less likely to consolidate democracy” (Acemoglu and Robinson 2001:938).

While Haggard and Kaufman's analysis is part of a more general project, which attempts to uncover the causal mechanisms behind transitions and reversals during the Third Wave in general, for purposes of brevity I will limit my discussion only to their within-case analysis of Argentina.¹² The question of whether their methodology lends itself to subsequent comparisons across cases, or indeed to wholesale generalization, as well as the merits of Haggard and Kaufman's ‘large-N qualitative testing’ as a mixed method approach (Goertz 2017, chapter 7) are all beyond the scope of this article.¹³

In what follows, I analyze whether Haggard and Kaufman can establish their causal hypothesis in the case study of the Argentinian transition using interventionism's demands from “Interventionism”. I will show that meeting these demands would strengthen several aspects of Haggard and Kaufman's analysis.

Argentina

Haggard and Kaufman focus on the history of the Argentinian transition from the military regime seizing power in 1976 to the transition to a competitively elected government in 1983. After the military regime seized power, it attempted to repress the country's strong union movement, amongst others by “banning parties and strikes and imposing censorship” (Haggard and Kaufman 2016:111), “direct purges of labor-based Peronist adversaries” (Haggard and Kaufman 2016:112), and “neoliberal economic reforms, including reforms of the labor market designed to curtail union power” (Haggard and Kaufman 2016:112). After these reforms, the economy performed poorly, and this, together with structural dislocations, spurred the union movement to respond with a “wave of strikes and general strikes” (Haggard and Kaufman 2016:113). The strikes put pressure on the regime, which made several changes in its leadership and economic policy, each unsuccessful and leading to more strikes. After this continued pressure, the government decided to invade the Falkland Islands, an attempt which ended unsuccessfully. The regime gave in and appointed a caretaker government, which together with amongst others the labor unions organized democratic elections.

To illustrate, Figure 2 contains part of the network described here. Here, D refers to the density of social organization, S to the first strikes, R to autocratic repression, and T to the eventual democratic transition. This network leaves out many other factors in the process, including the government's economic reforms, economic grievances, purges, other strikes, and the invasion of the Falkland Islands. Moreover, some potential causal connections are not displayed, such as between R and S.

Figure 2.

Part of the process in Haggard and Kaufman's analysis of the Argentinian democratic transition.

Let us now consider what an interventionist approach to collecting evidence for this causal hypothesis would look like and why this approach would improve the rigor of Haggard and Kaufman's claims. This will consist of the three phases from “Interventionism”: (1) circumscribing the causal claim; (2) choosing an appropriate intervention; (3) evaluating evidence of what would happen under this intervention.

Circumscribing the Causal Claim

As argued briefly in “Interventionism”, the counterfactual we are concerned with in this process tracing case study is not simply the general ‘if the density of social organization had been lower, the democratic transition in Argentina would not have occurred’. Rather, we must break the process up into its smaller steps. For example, as sketched in Figure 2 one step in the Argentinian process is the causal hypothesis $D \to S$ where D and $S$ refer to the density of social organization and the first strikes, respectively.

To avoid ambiguity and potential false positives and negatives, the description of $D \to S$ needs to meet specific demands, as we have seen in “Interventionism”. For one, we must be able to specify both D and S at the right level of generality. To illustrate what this step involves, consider D, the density of social organization. One may be tempted (as it seems Beach and Pedersen are in their description of the causal mechanism as a ‘union collective-action mechanism’¹⁴) to specify D only in terms of the strength of unions in Argentina, as union membership seems to be an important part of social organization in Argentina. Haggard and Kaufman measure union strength through the proxy of “the per capita membership in unions affiliated with the International Trade Union Confederation (ITUC)” (Haggard and Kaufman 2016:71), because “almost all of the countries with scores of zero are ones in which unions are in effect appendages of the state and/or where unionization rates are in fact low” (Haggard and Kaufman 2016:113). As such, one may think this measure is an appropriate D.

To test whether one should equate D to per capita membership in ITUC unions, interventionists ought to consider the associated counterfactual. In this case, the counterfactual belonging to $D \to S$ would be that the distributive conflict transition in Argentina would have been less likely if membership in ITUC unions had been lower. Yet this contradicts Haggard and Kaufman's hypothesis that different types of organizations can play a role in mobilization. The benefits of an interventionist approach are demonstrated here. We see why considering an associated counterfactual is desirable: without investigating the counterfactual, one may be led astray and specify the putative cause or effect in such detail that the counterfactual and thus the causal claim becomes false. This conclusion would be a false negative. A more general level D is required here. Based on their more general claims in the book, we may conclude that Haggard and Kaufman are interested in the counterfactual that the distributive conflict transition in Argentina would have been less likely in the absence of a strong organization that could mobilize voters. They do not make any claims about the efficacy of one particular form of organization (e.g., unions) over another (e.g., the Church), but rather claim that what matters is the capacity of the organization to mobilize people into the streets in ways which raise the costs of the regime suppressing them.¹⁵

The delineation of S, which I intuitively described as ‘the first strikes’ in the above, is similarly difficult. If we specify S as, say, the first general strike protesting the government's economic policies, organized by the unions in April 1979, we are led into the same trap as above. The effect of interest is more general: Haggard and Kaufman want to know whether levels of mass mobilization raised the costs of repression above what the authoritarian regime was willing to pay. To reflect this, it may turn out to be useful to specify S as some more general measure of the number of mass mobilization events in 1979.

In general, the level of specificity at which we put our putative cause and effect is a tricky balancing act. While delineating cause and effect in a highly specific way may lead to vacuous or false claims, if we make them too general, we may lose track of the motivation for using process tracing (case based) analysis in the first place: we no longer ‘open the black box’ behind a correlation between mobilization and democratic transition. This issue deserves further attention but is beyond the scope of this article.¹⁶

It must suffice here to repeat, in conclusion, that questions of specification are best answered if authors consider two fundamental questions. Firstly, what is the intended counterfactual that goes along with $D \to S$ ? As we have seen above, this helps us clarify that the authors do not suggest that if union membership were lower, no mobilization would have occurred (as mobilization may have been organized by other groups). Secondly, in choosing a specific S it is important to keep track of the next event in the process chain starting with $D \to S$ . S must be specified in enough detail to also be a putative cause of the next step, in this case e.g., the repression by government forces. This step, too, has an associated counterfactual, which the specification of S must be appropriate for.

Describing the Intervention

Setting aside the issues with delineating D and $S$ for the moment, let us turn to the second step of interventionist process tracing. What hypothetical intervention might lower the density of social organization in a way that accords with interventionism's demands? We ought to find an intervention variable I that meets the following (Runhardt 2015; cf. Woodward 2003:98):

Firstly, I should decrease the density of social organization D.

Secondly, I should act as a switch for D, that is, make this density independent of any other variables.

Third, the intervention I cannot itself lead to, or prevent, S in a way that is unrelated to the level of social organization in Argentina. This means, for example, that increasing the repressive capabilities of the government is most likely a poor intervention; in ‘labor repressive’ regimes, the repressiveness of the system itself may be part of the causal process from social organization density to a transition. The repression of unions in Argentina, for example, was as much a part of the unions’ decision to strike as was the economic downturn.¹⁷

Fourth and last, the intervention I must be statistically independent of all variables that increase/decrease S by other means than density of social organization D.

In short, we wish to avoid a scenario where we are imagining an Argentina without a strong organization that could mobilize voters, but in doing so have to change some other set of distal causal factors also influencing the outcome. If the counterfactual scenario is not appropriately thought through, we are not specifying the intervention correctly and thus our causal analysis is weak at best. The intervention allows us to ‘imagine’ the scenario in the appropriate way: it is what brings about the lack of a strong mobilizing organization, and not the lack of such an organization itself. This is illustrated in Figure 3, which adapts the earlier Figure 2 by including a possible intervention I. Given demand 2 for interventions above, the inclusion of I has broken the previous arrow from R to D.

Figure 3.

Part of the process in Haggard and Kaufman's analysis of the Argentinian democratic transition, now under intervention.

Evidence for the Counterfactuals

The key question remaining is: have Haggard and Kaufman provided any evidence, directly or indirectly, for the existence of an intervention variable as described above? Is there any reason to believe that an intervention I, if performed properly on D, would lead to a change in $S$ ? Without such evidence, their causal analysis is weakened, since evidence of an intervention is what ensures their counterfactual scenario is appropriately specified.

In their analysis of the Argentinian transition, Haggard and Kaufman only indirectly corroborate the intervention variable. For example, looking at the demand that by lowering D, we lower the probability of S, one might ask whether there is any overdetermination or pre-emption going on: e.g., if other interested parties would have mobilized had the unions not organized strikes. If the answer to this question is no, then that would help them better understand the causal relations involved. Haggard and Kaufman do not explicitly rule this situation out, but their description of the years leading up to the transition could be interpreted as providing evidence against this specific case of overdetermination and pre-emption. For example, the authors state that while the unions had been organizing strikes since 1977, human rights organizations and political parties did not join in with the strikes until 1982. This arguably makes it more believable that without the unions, human rights organizations and political parties would not have demonstrated instead. Whether this counts as an appropriate counterfactual scenario depends, amongst other things, on their intended variables D and S.¹⁸

In short, my interventionist analysis here is negative. If we accept an interventionist view of causation, we require there to be evidence of intervention claims; such evidence however is never explicitly stated. As such, Haggard and Kaufman's own causal hypothesis is not strongly corroborated.¹⁹

A critic might wonder whether Haggard and Kaufman's rejection of the alternative hypotheses in the literature is similarly unsupported. If we see Haggard and Kaufman's work as aimed, in the first place, at rejecting e.g., the Acemoglu and Robinson theory of distributive conflict transitions, then evaluting their work based solely on whether or not it can corroborate their own hypotheses would be misleading. In brief, my response to this criticism is that any rejection of alternative hypotheses in process tracing must for the interventionist still go hand in hand with specifying and testing an associated counterfactual. Rejecting the alternative hypotheses, for the interventionist, consists of giving strong evidence against the truth of this counterfactual.

For example, consider Acemoglu and Robinson's claim that societies with a high level of inequality are less likely to successfully make a democratic transition. This implies that were we to increase or decrease the equality in a society, then we would thereby increase or decrease, respectively, the probability that democracy is consolidated. In Haggard and Kaufman's work, the finding that a highly unequal society like Argentina nevertheless democratized, is meant to speak against this counterfactual. But we can legitimately ask here under what circumstances, and how strongly, a single case study can provide evidence against a more general claim. Some would argue that one such case study, selected on the dependent variable, may not be compelling evidence against the general claim. The statistical work Haggard and Kaufman undertake will speak more strongly against Acemoglu and Robinson: there does not appear to be a significant correlation between the level of inequality and democratic transitions.

Corroborating General Causal Claims

The relation between singular and general causal claims, and process tracing and statistical work, is not a question solely important in an interventionist framework; as such, I will only explore it briefly here. Given the above concerns, future work should be directed to two questions: firstly, what the associated counterfactual of Acemoglu and Robinson's is for the Argentinian case study alone; secondly, what the interventionist can tell us of the relation between case study and statistical methodologies. In particular, in future work we may ask what the effects are of Haggard and Kaufman's decision to look at all distributive conflict transitions rather than a select number of cases, and how the Argentinian case fits within this much broader context (cf. Runhardt 2022).

Building on the latter point, one may ask more broadly how an interventionist would support Haggard and Kaufman's concerns with not just the individual-level processes within cases, but also some general causal claims about mechanisms. For instance, the authors make claims about distributive conflict transitions in general, and not just about the observable processes within individual case studies. In other words, while the authors trace the processes resulting from their own proposed causal mechanism (density of social organization) in particular case studies like the Argentinian democratic transition in the 1980s, they also hypothesize that this mechanism holds generally for certain types of transitions. Can the interventionist framework help us make sense of the evidence required for such type-level causal mechanism claims?

At first glance, grounding type-level mechanism claims may seem relatively straightforward for the interventionist: Woodward's framework includes both a definition for token-level causal claims like ‘ $X = x$ causes $Y = y$ ’, and for type-level causal claims like ‘(property) X causes (property) $Y$ ’.²⁰ It speaks towards the practicality of an interventionist approach that if we were to apply this type-level framework, we would follow the same three steps as before, viz. describing the claim and counterfactual, defining the appropriate intervention, and evaluating the evidence for that intervention (Woodward 2003, 2007a). However, there are several complicating factors in simply extending Woodward's theory²¹; as such, a further discussion of an interventionist framework for generalization is best left to future research.

A Bayesian Analysis of the Above

In this last section, I wish to present a brief, alternative reading of the methodological advice in the rest of this article. In particular, I wish to follow Macartan Humphreys and Alan Jacobs’ Bayesian analysis of mixed method research and discuss “the varying likelihoods with which potentially probative pieces of evidence may be associated with causal effects” (Humphreys and Jacobs 2015:655). The aim in this section is not to provide an alternative approach to Humphreys and Jacobs, but rather to use their approach to “express the logic” of interventionist process tracing “in Bayesian terms” (Humphreys and Jacobs 2015:657).

Humphreys and Jacobs describe process tracing as: “a search for clues that will be observed with some probability if the case is of a given causal type and that will be observed with some differing probability if the case is of a different causal type” (Humphreys and Jacobs 2015:657). They interpret process tracing with a Bayesian approach in the following way:

“In formalizing Bayesian process tracing, we start with a very simple setup, which we then elaborate. To return to our running example, suppose that we already have $X, Y$ data on one authoritarian regime: we know that it suffered economic crisis ( $X = 1$ ) and collapsed ( $Y = 1$ ). We want to know if X caused Y. We answer the question by collecting one or more clues that we believe are related to the case-level causal effect of X on Y. We use the variable K to register the outcome of the search for a clue (or collection of clues), with $K = 1$ indicating that a specific clue (or collection of clues) is searched for and found, and $K = 0$ indicating that the clue is searched for and not found.” (Humphreys and Jacobs 2015:657).

Let us now express interventionist process tracing using the same language. In the context of interventionism, the relevant ‘search for a clue’ is the search for evidence of the hypothetical intervention (step 3 as described in “Interventionism”). If we search for and find such evidence, $K = 1$ . If we search for and do not find such evidence, $K = 0$ . The relevant probabilities to establish are now the following:

The likelihood of the clue given the hypothesis: $Pr (K = 1 | H)$ . What is the likelihood we find evidence of the hypothetical intervention, if X indeed caused Y ?

The likelihood we find evidence of the hypothetical intervention if X did not cause Y, i.e., $Pr (K = 1 | \neg H)$ . The difference between the two likelihoods $Pr (K = 1 | H)$ and $Pr (K = 1 | \neg H)$ , Humphreys and Jacobs clarify, is the probative value of observing this ‘clue’, i.e., the probative value of the hypothetical intervention data.

The likelihood of the clue in general, $Pr (K = 1)$ , which is a function of the two likelihoods above as well as $Pr (H)$ and $Pr (\neg H)$ .

These likelihoods combine to give us

Pr (H | K = 1)

in the usual way using Bayes’ theorem, i.e.,

Pr (H | K = 1) = \frac{Pr (K = 1 | H) \times Pr (H)}{\Pr (K = 1)}

In the interventionist theory, the causal hypothesis $X \to Y$ is necessary and sufficient for the truth of the counterfactual describing the hypothetical intervention outcome. As such, $Pr (K = 1 | \neg H)$ is 0. However, since it will be difficult to find evidence of the hypothetical intervention outcome, it may well be that $Pr (K = 1 | H)$ is not high either. This means that in the mapping of Humphreys and Jacobs, following Van Evera's terminology (Van Evera 1997), the presence of K is a ‘smoking gun’ for H (the causal relation $X \to Y$ ), in that it would be sufficient to convince us of H. Had Haggard and Kaufman established the counterfactual of what would have happened to mobilization S under lower density of social organization D, this would have strongly supported their causal claim $D \to S$ . On the other hand, the absence of K is a ‘hoop test’ for $\neg H$ ( $X$ does not cause $Y$ ), in that it would be necessary for $\neg H$ to be true but not sufficient evidence to convince us of $\neg H$ . We have seen that Haggard and Kaufman do not provide evidence of K. This does not mean that their causal claim $D \to S$ is necessarily false; however, it also leaves open the possibility that D does not cause S.

I will finish by noting that the above is a simplification of how counterfactual evidence and the causal claim relate. We hardly ever have convincing evidence of the truth of a counterfactual; by this, I mean no more than the old Popperian belief that corroboration of a hypothesis is possible, but confirmation of a hypothesis is not. As such, the variable K which “register[s] the outcome of the search for a clue (or collection of clues), with $K = 1$ indicating that a specific clue (or collection of clues) is searched for and found, and $K = 0$ indicating that the clue is searched for and not found” (Humphreys and Jacobs 2015:657) is an oversimplification of what we have to deal with in searching for counterfactuals. In future work, a more precise analysis of degrees of corroboration of a counterfactual claim is necessary.

Conclusion

In this article, I combined interventionist theory with the process tracing methodology and expressed singular case study causation in terms of so-called hypothetical interventions. I showed that interventionism suggests concrete evidential tests which can strengthen case studies’ mechanistic causal claims. In particular, I prescribed three steps for an interventionist investigation: to circumscribe the causal claim, to find an appropriate intervention, and finally to establish evidence for the intervention. I argued that each step in turn will allow researchers to make their causal analysis in case study research more robust, amongst others by disambiguating causal claims and clarifying or strengthening existing advice on counterfactual analysis.

I spent most of the article analyzing the last of these steps, comparing interventionist theory's demands on such evidence with the already existing methodological demands for counterfactual reasoning in case study research. This led to a technical discussion of which other variables in the causal network can be altered under the intervention (in the counterfactual scenario) and which need to remain fixed.

I then turned to a concrete case study, Haggard and Kaufman's analysis of the Argentinian transition to democracy and argued that this study does not (yet) meet the requirements of interventionist theory. I finished by showing what this means for the causal claims in Haggard and Kaufman using a Bayesian framework. I argued that the absence of tangible evidence for the associated counterfactuals that interventionist theory prescribes means that Haggard and Kaufman have failed to strongly support their own causal hypotheses, but it does not definitively establish the alternative hypotheses they discuss either. By taking on board an interventionist approach to process tracing, they would strengthen their causal analysis.

Footnotes

Acknowledgements

An earlier version of this article was presented as part of the panel ‘Process Tracing. New Directions in Qualitative Research’ at the 2020 American Political Science Association conference. The author thanks the audience at this panel as well as the other presenters for their valuable comments. Finally, thanks to Stephan Haggard for his critiques on an earlier draft.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Availability of Data

Not applicable.

Code Availability

Not applicable.

ORCID iD

Rosa W. Runhardt

Notes

Author Biography

Rosa W. Runhardt is an assistant professor in philosophy of science at Radboud University, Nijmegen in the Netherlands. Her research centers on causation, evidence, and measurement, with a special interest in the philosophy of political science and international relations, qualitative methodologies, and mixed-method research. She has previously published in amongst others Philosophy of Science, Synthese, Philosophy of the Social Sciences and the European Journal for Philosophy of Science.

References

Acemoglu

Daron

Robinson

James A.

. 2001. “A Theory of Political Transitions.” American Economic Review 91(4):938‐63.

Acemoglu

Daron

Robinson

James A.

. 2006. Economic Origins of Dictatorship and Democracy. New York: Cambridge University Press.

Bakke

Kristin M.

2013 “Copying and Learning from Outsiders? Assessing Diffusion from Transnational Insurgents in the Chechen Wars.” Pp. 31‐62 in Transnational Dynamics of Civil War, edited by Checkel

J. T.

. Cambridge, UK: Cambridge University Press.

Beach

Derek

. 2013. “Taking Mechanisms Seriously?” European Political Science 12(1):13‐5.

Beach

Derek

. 2016. “What Are We Actually Tracing? Process Tracing and the Benefits of Conceptualizing Causal Mechanisms as Systems.” Qualitative and Multi-Method Research: Newsletter of the American Political Science Association’s QMMR Section 14(1/2):15‐22.

Beach

Derek

Pedersen

Rasmus Brun

. 2013. Process-Tracing Methods: Foundations and Guidelines. Ann Arbor: University of Michigan Press.

Beach

Derek

Pedersen

Rasmus Brun

. 2019. Process-Tracing Methods: Foundations and Guidelines. 2nd ed. Ann Arbor: University of Michigan Press.

Bennett

Andrew

Checkel

Jeffrey T.

. 2015 “Process Tracing: From Philosophical Roots to Best Practices.” Pp. 3‐38 in Process Tracing: From Metaphor to Analytic Tool, edited by Bennett

Checkel

J. T.

. Cambridge, UK: Cambridge University Press.

Bennett

Andrew

Fairfield

Tasha

Soifer

Hillel David

. 2019. “Comparative Methods and Process Tracing.” Qualitative Transparency Deliberations Working Group III.1 . Report. November 1, 2022. https://ssrn.com/abstract=3333405.

10.

Brady

Henry E.

Collier

David

. 2010. Rethinking Social Inquiry: Diverse Tools, Shared Standards. 2nd ed. Lanham: Rowman & Littlefield.

11.

Briggs

Rachel

. 2012. “Interventionist Counterfactuals.” Philosophical Studies 160(1):139‐66.

12.

Bunge

Mario

. 1997. “Mechanism and Explanation.” Philosophy of the Social Sciences 27(4):410‐65.

13.

Clarke

Christopher

. Forthcoming. “Process Tracing: Defining the Undefinable?” in The Oxford Handbook of Philosophy of Political Science, edited by Kincaid

Van Bouwel

. Oxford: Oxford University Press.

14.

Collier

Ruth Berins

. 1999 “Labor Action in Recent Democratization.” Pp. 110‐65 in Paths Toward Democracy: The Working Class and Elites in Western Europe and South America. Cambridge, UK: Cambridge University Press.

15.

Eells

Ellery

. 1991. Probabilistic Causality. Cambridge, UK: Cambridge University Press.

16.

Fairfield

Tasha

Charman

Andrew E.

. 2017. “Explicit Bayesian Analysis for Process Tracing: Guidelines, Opportunities, and Caveats.” Political Analysis 25(3):363‐80.

17.

George

Alexander L.

Bennett

Andrew

. 2005. Case Studies and Theory Development in the Social Sciences. Cambridge: MIT Press.

18.

Glennan

Stuart S.

1996. “Mechanisms and the Nature of Causation.” Erkenntnis 44:49‐71.

19.

Goertz

Gary

. 2017. Multimethod Research, Causal Mechanisms, and Case Studies: An Integrated Approach. Princeton and Oxford: Princeton University Press.

20.

Goertz

Gary

Haggard

Stephan

. Forthcoming. “Large-N Qualitative Analysis (LNQA): External Validity and Generalization in Case Study and Multi-Method Research.” in The Oxford Handbook of Philosophy of Political Science, edited by Kincaid

Van Bouwel

. Oxford: Oxford University Press.

21.

Goertz

Gary

Levy

Jack S.

. 2007. Explaining War and Peace: Case Studies and Necessary Condition Counterfactuals. London: Routledge.

22.

Haggard

Stephan

Kaufman

Robert R.

. 2012. “Inequality and Regime Change: Democratic Transitions and the Stability of Democratic Rule.” American Political Science Review 106(3):495‐516.

23.

Haggard

Stephan

Kaufman

Robert R.

. 2016. Dictators and Democrats: Masses, Elites, and Regime Change. Princeton: Princeton University Press.

24.

Hall

Peter A.

2013. “Symposium: Tracing the Progress of Process Tracing.” European Political Science 12(1):20‐30.

25.

Halpern

Joseph Y.

2016. Actual Causality. Cambridge: MIT Press.

26.

Harvey

Frank P.

2011. Explaining the Iraq War: Counterfactual Theory, Logic and Evidence. Cambridge, UK: Cambridge University Press.

27.

Harvey

Frank P.

2015. “‘What If’ History Matters? Comparative Counterfactual Analysis and Policy Relevance.” Security Studies 24(3):413‐24.

28.

Hedström

Peter

Ylikoski

Petri

. 2010. “Causal Mechanisms in the Social Sciences.” Annual Review of Sociology 36(1):49‐67.

29.

Humphreys

Macartan

Jacobs

Alan M.

. 2015. “Mixing Methods: A Bayesian Approach.” American Political Science Review 109(4):653‐73.

30.

Jacobs

Alan M.

2016. “Introduction: Mechanisms and Process Tracing.” Qualitative & Multi-Method Research (1/2):13‐5.

31.

King

Gary

Keohane

Robert O.

Verba

Sidney

. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.

32.

Kittel

Bernhard

Kuehn

David

. 2013. “Reassessing the Methodology of Process Tracing.” European Political Science 12(1):1‐9.

33.

Lebow

Richard Ned

Stein

Janice Gross

. 1996 “Back to the Past: Counterfactuals and the Cuban Missile Crisis.” Pp. 119‐48 in Counterfactual Thought Experiments in World Politics. Princeton: Princeton University Press.

34.

Levy

Jack S.

2008 “Counterfactuals and Case Studies.” Pp. 627‐44 in The Oxford Handbook of Political Methodology, edited by Box-Steffensmeier

J. M.

Brady

H. E.

Collier

. Oxford: Oxford University Press.

35.

Levy

Jack S.

2015. “Counterfactuals, Causal Inference, and Historical Analysis.” Security Studies 24(3):378‐402.

36.

Lewis

David K.

1973. “Causation.” The Journal of Philosophy 70(17):556‐67.

37.

Lewis

David

. 1979. “Counterfactual Dependence and Time’s Arrow.” Noûs 13(4):455‐76.

38.

Lyall

Jason

. 2015 “Process Tracing, Causal Inference, and Civil War.” Pp. 186‐207 in Process Tracing: From Metaphor to Analytic Tool, edited by Bennett

Checkel

J. T.

. Cambridge: Cambridge University Press.

39.

Machamer

Peter

Darden

Lindley

Craver

Carl F.

. 2000. “Thinking About Mechanisms.” Philosophy of Science 67(1):1‐25.

40.

Mahoney

James

. 2001. “Beyond Correlational Analysis: Recent Innovations in Theory and Method.” Sociological Forum 16(3):575‐93.

41.

Mahoney

James

Barrenechea

Rodrigo

. 2019. “The Logic of Counterfactual Analysis in Case-Study Explanation.” The British Journal of Sociology 70(1):306‐38.

42.

McGuire

James W.

1995 “Interim Government and Democratic Consolidation: Argentina in Comparative Perspective.” Pp. 179‐210 in Between States: Interim Governments in Democratic Transitions, edited by Shain

Linz

J. J.

Berat

. Cambridge: Cambridge University Press.

43.

Nolan

Daniel

. 2013. “Why Historians (and Everyone Else) Should Care About Counterfactuals.” Philosophical Studies 163(2):317‐35.

44.

Pearl

Judea

. 2000. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press.

45.

Reiss

Julian

. 2005. “Causal Instrumental Variables and Interventions.” Philosophy of Science 72(5):964‐76.

46.

Rohlfing

Ingo

Zuber

Christina Isabel

. 2021. “Check Your Truth Conditions! Clarifying the Relationship Between Theories of Causation and Social Science Methods for Causal Inference.” Sociological Methods & Research 50(4):1623‐59.

47.

Runhardt

Rosa W.

2015. “Evidence for Causal Mechanisms in Social Science: Recommendations from Woodward’s Manipulability Theory of Causation.” Philosophy of Science 82(5):1296‐307.

48.

Runhardt

Rosa W.

2016. “Tracing the Productive Continuity of Social Mechanisms.” Qualitative and Multi-Method Research: Newsletter of the American Political Science Association’s QMMR Section 14(1–2):22‐8.

49.

Runhardt

Rosa W.

2017. “Causal Comparability, Causal Generalizations, and Epistemic Homogeneity.” Philosophy of the Social Sciences 47(3):183‐208.

50.

Runhardt

Rosa W.

2022. “Limits to Evidential Pluralism: Multi-Method Large-N Qualitative Analysis and the Primacy of Mechanistic Studies.” Synthese 200(171).

51.

Schroeder

Paul W.

2004 “Embedded Counterfactuals and World War I as an Unavoidable War.” Pp. 157‐92 in Systems, Stability, and Statecraft: Essays on the International History of Modern Europe, edited by Wetzel

Jervis

Levy

J. S.

. New York: Palgrave Macmillan.

52.

Shpitser

Ilya

Sarkar

Sourjya

. 2017. “Personalizing Path-Specific Effects.” arXiv . August 9, 2022. https://arxiv.org/abs/1709.03862v1.

53.

Spirtes

Peter

Glymour

Clark

Scheines

Richard

. 1993. Causation, Prediction, and Search. New York: Springer-Verlag.

54.

Tetlock

Philip E.

Belkin

Aaron

. 1996. Counterfactual Thought Experiments in World Politics. Princeton: Princeton University Press.

55.

Van Evera

Stephen

. 1997. Guide to Methods for Students of Political Science. Ithaca: Cornell University Press.

56.

VanderWeele

Tyler J.

2015. Explanation in Causal Inference: Methods for Mediation and Interaction. New York: Oxford University Press.

57.

VanderWeele

Tyler J.

2016. “Mediation Analysis: A Practitioner’s Guide.” Annual Review of Public Health 37(1):17‐32.

58.

Waldner

David

. 2012 “Process Tracing and Causal Mechanisms.” Pp. 65‐84 in The Oxford Handbook of Philosophy of Social Science, edited by Kincaid

. Oxford: Oxford University Press.

59.

Waldner

David

. 2015 “What Makes Process Tracing Good? Causal Mechanisms, Causal Inference, and the Completeness Standard in Comparative Politics.” Pp. 126‐52 in Process Tracing: from Metaphor to Analytic Tool, edited by Checkel

J. T.

Bennett

. Cambridge: Cambridge University Press.

60.

Weinberger

Naftali

. 2019. “Path-Specific Effects.” The British Journal for the Philosophy of Science 70(1):53‐76.

61.

Williamson

Jon

. 2011. “Mechanistic Theories of Causality Part I.” Philosophy Compass 6(6):421‐32.

62.

Woodward

James

. 2000. “Explanation and Invariance in the Special Sciences.” The British Journal for the Philosophy of Science 51(2):197‐254.

63.

Woodward

James

. 2002. “What Is a Mechanism? A Counterfactual Account.” Philosophy of Science 69(S3):S366‐77.

64.

Woodward

James

. 2003. Making Things Happen: A Theory of Causal Explanation. Oxford: Oxford University Press.

65.

Woodward

James

. 2007a “Causal Models in the Social Sciences.” Pp. 157‐210 in Philosophy of Anthropology and Sociology, edited by Turner

S. P.

Risjord

M. W.

. Amsterdam: Elsevier.

66.

Woodward

James

. 2007b “Interventionist Theories of Causation in Psychological Perspective.” Pp. 19‐36 in Causal Learning: Psychology, Philosophy, and Computation, edited by Gopnik

Schulz

. Oxford: Oxford University Press.

67.

Woodward

James

. 2015. “Methodology, Ontology, and Interventionism.” Synthese 192(11):3577‐99.