A Unifying Framework for Causal Analysis in Set-Theoretic Multimethod Research

Introduction

Multimethod research (MMR) combining regression analysis with process tracing, also known as nested analysis (Lieberman 2005), has gained increasing attention in the methodological and empirical literature. ¹ Based on this and the spread of Qualitative Comparative Analysis (QCA) as a set-theoretic method usually operating on the cross-case level (Rihoux and Marx 2013), set-theoretic MMR has recently been added to the toolbox (Schneider and Rohlfing 2013). Set-theoretic MMR combines QCA with process tracing to enhance causal inference of claims of necessity and sufficiency. ² Thus far, the set-theoretic MMR literature has primarily focused on issues related to methods, including the role of case study-based insights for QCA (Rihoux and Lobe 2009), the informal and formalized choice of cases on the basis of QCA results (Ragin and Schneider 2011; Rohlfing and Schneider 2013; Schneider and Rohlfing 2013, 2018) and the logic of causal inference in QCA-based case studies (Rohlfing and Schneider 2013; Schneider and Rohlfing 2016).

The development of set-theoretic MMR is laudable and the combination of QCA with process tracing is steadily becoming more popular in empirical research (e.g., Alemán 2010; Samford 2010; Segura-Ubiergo 2007). ³ What is still lacking is a foundational discussion on how the integration of QCA as a cross-case method and process tracing as a within-case method is possible. We believe that tackling this issue on a fundamental level is crucial to understand how we can generate coherent causal arguments in set-theoretic MMR.

The fact that both QCA and process tracing have been based on set theory (Goertz and Mahoney 2012:chapter 1) does not automatically imply that they allow for the formulation of coherent causal inferences. Set theory is only about the relation between two or more sets, which is not about causation per se. Set relations are not causation, to paraphrase a common saying.

With regard to the combination of regression analysis and case studies (Lieberman 2005), which forms the template for set-theoretic MMR (Schneider and Rohlfing 2013), it has been argued that the two methods are difficult, if not impossible, to reconcile because the supposed regularity conception of causation that underlies regression analysis does not fit squarely with the analysis of one case or a small number of cases (Chatterjee 2013). In light of this and more general concerns about the way in which multiple methods fit in MMR designs (Ahmed and Sil 2012; Beach and Pedersen 2013:section 8.4; Blatter and Haverland 2012:section 5.5; Creswell and Plano Clark 2011:chapter 2), we see a need to consider whether set-theoretic MMR allows for valid inferences based on integrated cross-case and within-case analyses. The primary goal of our article is to demonstrate that QCA and process tracing share common ground in set-theoretic MMR and that integrated causal analysis is feasible. The core element of our argument is a single-case, counterfactual notion of causation.

Our article is based on two premises that are widely shared in the social science literature on set theory. First, QCA is employed for causal analysis, as opposed to the search for mere associations (Ragin 2008:9). ⁴ Second, sound causal analysis should establish a causal effect on the cross-case level and involve a complementary causal explanation via the analysis of causal mechanisms (Cartwright 2004; Gerring 2005). In principle, causal inference is feasible with a valid identification strategy and does not need to shed light on the underlying mechanisms (Gerring 2010). However, causal explanations that focus on the link between a cause and an effect yield theoretical and policy-related added value and should be an integral component of causal analysis (Dessler 1991). The importance of opening the black box between cause and effect has long been acknowledged in the qualitative literature (Bennett and Elman 2006) and is now also reflected in the concern of the quantitative literature about mechanisms, where this issue is discussed under the label of “mediation” (VanderWeele 2009).

Based on these two premises, we develop a unifying foundation for set-theoretic MMR that highlights the roles and interplay of QCA and process tracing for causal analyses. We argue that causal analysis on the cross-case level and within-case level is compatible in set-theoretic MMR and does not suffer from the alleged problems of nested analysis and, more generally, MMR. Causal inference via counterfactuals on the level of single cases integrates QCA and process tracing as two methods operating on different levels of analysis and assigns proper roles to each method.

The starting point for the mechanismic foundation for set-theoretic MMR is Machamer, Darden, and Craver’s (henceforth MDC 2000) conception of mechanisms as a chain of entities and activities. This concept has received considerable attention in philosophy of science (McKay Illari and Williamson 2012), but has so far barely resonated in the social sciences (Beach and Pedersen 2013; Rohlfing 2012:chapter 2). In the second section, we introduce MDC’s conception in more detail and elaborate on how it can be anchored in set theory. In the third section, we summarize the key elements of set-theoretic MMR and link it to MDC’s mechanismic account. In the fourth section, we show that causal inference via counterfactuals in single cases is able to unify causal inference in set-theoretic MMR. The fifth section concludes.

Mechanisms as Chains of Actors and Their Behavior

Machamer, Darden and Craver (MDC) define a mechanism as comprising “entities and activities organized such that they are productive of regular changes from start or set-up to finish or termination conditions” (2000:3). ⁵ In the social science literature on set theory, the terms “start-up condition” and “finish or termination condition” are usually referred to as the sufficient term, X (a condition, conjunction, or equifinal conditions or conjunctions) and the outcome, Y. ⁶

The mechanism linking X to Y includes a chain of entities with corresponding activities. We speak of actors and their behavior in the following instead of entities and their activities in order to transfer MDC’s conception to the social science domain and common social science terminology. ⁷ Capturing mechanisms by specifying actors and their behavior can be clarified with the example of Owen’s (1994) explanation of the democratic peace phenomenon and a visualization of the two mechanisms that link the presence of liberal ideas (X) to peace between two democracies (Y). ⁸ In Figure 1, the upper panel reproduces the original figure from Owen (1994:102). For the lower panel, we modify his chart in order to render it in line with MDC’s conception of mechanisms.

OPEN IN VIEWER

Figure 1.

Two conceptualizations of causal mechanisms in Owen (1994).

We elaborate the causal mechanisms in the lower panel of Figure 1, since it exemplifies the conception we advocate as being the basis for set-theoretic MMR. A closer look at Owen’s verbal formulation of his theory of democratic peace reveals that the lower panel provides a more precise visualization of his arguments with multiple advantages over his original figure that we discuss below.

The common starting point of the two mechanisms is “the elite of a country holding liberal political ideas” (Owen 1994:93-012). According to the first mechanism, these liberal political ideas lead to the pursuit of a liberal foreign policy by the elite. An important consequence of a liberal foreign policy is that members of the elite develop the idea that foreign liberal democracies are trustworthy and peaceful and should not be attacked. Since this idea is held by many, although not necessarily all, members of the elite, the government of a liberal democracy is facing tight constraints with regard to foreign policies toward other liberal democracies. Even in times of crisis between liberal democracies, these constraints account for the democratic peace phenomenon because then members of the elite lobby for peace and mobilize the public to do the same.

The second mechanism is depicted by the lower path of the bottom panel in Figure 1. Liberal ideas that are held by the elite lead to the creation of institutions that give the public political rights such as voting and freedom of speech. Freedom of speech is used to evaluate different foreign policies in relation to other liberal democracies. It is assumed that the public, as the actor incurring the costs of war, is not prepared to carry these costs in a conflict with a liberal democracy. The opportunity to discuss and evaluate foreign policies and the right to vote constitute a second constraint on the government. This constraint limits the available foreign policy options toward liberal democracies to the pursuit of peaceful relations.

The conception of causal mechanisms as a chain of actors displaying a certain behavior has three important features. First, the lower panel is more detailed and conveys much more information that is more precise to the reader. It highlights the difference between common path diagrams in the social sciences, such as the upper panel in Figure 1, and diagrams of mechanisms for the purpose of causal explanation. Second, the decomposition of a mechanism into actors and their behavior establishes productive continuity (Machamer et al. 2000:3). The goal of causal explanations is to elucidate whether and why one event is causal for another. In this conception of mechanisms, why-questions are answered by pointing to an actor behaving in a certain way, which prompts another actor to perform a certain behavior, and so on. ⁹ For every step in the sequence, we are able to explain how we got there and where we will be led in the next step and those that follow. In the Owen study, he theorizes that elites share the idea that two democracies should not fight each other because it is a consequence of the more fundamental commitment to liberal political ideas. Similarly, Owen conjectures as to how the process develops further by arguing that a peace-oriented foreign policy toward other democracies is a cause of government constraints.

Third and related, in confirmatory process tracing, the view on mechanisms as a causal sequence of actors and their behavior has advantages over the derivation of multiple, but isolated observable implications that are tested. Being a long-standing practice in the social sciences (Campbell 1975; King, Keohane, and Verba 1994:chapter 1; Mahoney 2010), this is now referred to as the congruence method (see Blatter and Blume 2010). ¹⁰ In theorizing the democratic peace phenomenon, we could, for example, hypothesize that the government faces tight constraints in launching a war with another democracy and that the public is generally war-averse in crises involving another liberal democracy (Owen 1994:102-4). These are constitutive elements of the mechanisms in the bottom panel of Figure 1. Yet, this form of theorizing leaves open questions as to why the public is war-averse in the first place and why it enjoys the right to prevent the government from initiating a war. In contrast, a fully specified mechanism is able to answer these why-questions by pointing to preceding actors and their behavior (Beach and Pedersen 2013:chapter 3).

Theorizing multiple observable implications is a valuable endeavor, yet it only captures parts of the mechanism that is able to fully answer the question of how we get from the sufficient term to the outcome. In addition, the theoretical specification of a causal mechanism in terms of actors and their behavior yields the maximum number of causally related observable implications. This, in turn, makes it easiest to find disconfirming evidence and offers the highest level of theoretical leverage in a confirmatory analysis.

Fourth, the entire mechanism and each constitutive step can be conceptualized in set-theoretic terms (Beach and Pedersen 2013:chapter 3; George and Bennett 2005:chapter 10; Rohlfing 2012:chapter 2). ¹¹ Each actor and its behavior, including the triggering condition, can be framed as sufficient for the subsequent actor carrying out its behavior and, combined, are jointly sufficient for the outcome. For one causal chain, this implies that each actor and its behavior are a necessary element of the mechanism because their absence causes the discontinuity of the process (see below on causal inference). Whether an actor is also necessary for the outcome depends on the number of mechanisms that are expected to be in place. When there is more than one mechanism operative, as in the Owen example, each actor and its behavior constitute an insufficient, but necessary part of a mechanism that is unnecessary, but sufficient (INUS) (Mackie 1974).

In closing this section, one might wonder why we specifically use this definition of mechanism, given that conceptions of mechanisms are abundant (Hedström and Ylikoski 2010). MDC’s account has achieved huge resonance in philosophy of science and biology because it is simple but powerful, and we are of the belief that it can be transferred to the social sciences. ¹² In relation to this point, we emphasize that we rely on MDC’s conception as a template for causal explanation and do not necessarily subscribe to MDC’s view that it is ontologically accurate. ¹³ The claim of ontological accuracy implies that we can describe every causal process in terms of actors and their behavior. In biology, it has already been questioned whether this is possible (Nicholson 2012), and we leave it as an open question as to whether all social and political processes can be subsumed under MDC’s model of mechanisms. Regardless of this, our point of departure that MDC’s conception does not need to be ontologically accurate still allows for using it as an explanatory template when possible.

Set-Theoretic MMR in Short

Set-theoretic MMR combines a cross-case method for the analysis of cases with process tracing (Ragin and Schneider 2011; Rohlfing and Schneider 2013; Schneider and Rohlfing 2013, 2016). ¹⁴ It is possible to invoke typological theory on the cross-case level (see Møller and Skaaning 2015) but we focus on QCA as the standard set-relational cross-case method. For our purposes, it suffices to broadly refer to “QCA” because the mechanismic foundation that we develop in the following applies to all of its variants. ¹⁵ Here, we conceive of QCA as a technique combining set-theoretic reasoning with the analysis of a truth table using an algorithm, the truth table analysis. Furthermore, the mechanismic framework extends to different varieties of set relations (necessity, sufficiency, INUS, and sufficient, but unnecessary attribute of a term that is insufficient, but necessary ¹⁶ ). For ease of presentation and without a loss of generality, we only develop our arguments for sufficiency and provide the basis for the following two sections by giving a brief exposition of set-theoretic MMR. ¹⁷

QCA and process tracing follow the division of labor that is known from nested analysis (Lieberman 2005; Schneider and Rohlfing 2013). The role that each method plays in set-theoretic MMR varies conditionally on the sequence in which they are implemented (see Beach and Rohlfing 2018). ¹⁸ In a process tracing-first design, the analysis is confined to typical cases with the aim of identifying a sufficient term for the outcome and the mechanism located in-between. ¹⁹ The purpose of a QCA that is run on the basis of an initial round of process tracing is to test whether the sufficient terms identified in one or a small number of cases can be generalized to more cases (see below on causal inference), and whether the truth table yields sufficient terms in addition to those discovered in process tracing. We are also interested in the consistency and coverage of each sufficient term and the entire QCA solution. ²⁰

The alternative is a QCA-first design in which process tracing is done on the basis of the QCA results. We derive the truth table from existing, possibly case-based empirical research and theory rather than from process-tracing evidence. ²¹ Process tracing after the truth table analysis fulfills multiple roles. With regard to the causal analysis, the most obvious is the study of causal mechanisms that are triggered by the sufficient term. For this purpose, we select typical cases that allow us to discern the causal mechanism via exploratory or confirmatory process tracing. ²²

When a case is a member of a sufficient term and the membership in the term exceeds membership in the outcome, we are confronted with a deviant case consistency. Exploratory process tracing in such cases serves to solve this puzzle by adding an omitted conjunct to the term. ²³ The missing conjunct is identified by finding within-case evidence on why the mechanism is not operative in the deviant case consistency. A deviant case coverage is a member of the outcome, but not of any sufficient term derived by the truth table analysis. This constitutes a dilemma in Y-centered research because its aim consists of comprehensively explaining the outcome. In an analysis of a deviant case coverage, the goal of exploratory process tracing is to determine a formerly ignored sufficient term with which the deviant case coverage can be explained. ²⁴

A Proposal for Unified Causal Analysis in Set-Theoretic MMR

MMR is now a widely embraced idea because two methods operating on the cross-case level and within-case level yield complementary inferences on causal mechanisms and causal effects (e.g., Collier, Brady, and Seawright 2010). However, linking two methods is only possible if the corresponding inferences follow a template of causal inference that is compatible with the cross-case and within-case method. The combination of regression analysis and case studies has been criticized on precisely this point (e.g., Chatterjee 2013). ²⁵ It is said to rely on the regularity conception of causation, that is, causation is inferred from observing a systematic pattern on the cross-case or within-case level. ²⁶ Case studies can be employed based on a regularity premise, but the uncertainty of the causal inferences derived from one or a small number of cases tends to be considerable (Baumgartner 2008; Kühn and Rohlfing 2010). ²⁷ This has led to the criticism that process tracing is pointless in nested analysis when the cross-case analysis is grounded in a regularity conception of causation.

Some might argue that the same criticism extends to set-theoretic MMR. QCA is run on a population of cases (Ragin 2000:chapters 2 and 7), or at least a medium to large number of cases with the goal to find patterns. Here, ‘pattern’ means that members of the same term are also members of the outcome and that non-members of any term are non-members of the outcome. Such cross-case evidence entails the expectation that members of the same term also display the same mechanism (Falleti and Lynch 2009; Machamer et al. 2000:3).

Search for patterns in QCA seems to be in conflict with the fact that we do process tracing in single cases and make inferences about mechanisms without incorporating information from all the other cases in the analysis. Likewise, in the truth table analysis we usually make arguments about cross-case relationships without having detailed knowledge about the mechanisms in all cases under scrutiny. The discrepancy between our reliance on multiple cases in the truth table analysis and a single case in process tracing gives rise to multiple questions: Should we make inferences based on the QCA finding that the sufficient term triggering the mechanism is consistently linked to the outcome? Or should we make our causal inferences based on the deep knowledge of the process that we reconstruct via a within-case analysis? Or is it possible that both methods contribute their share to the causal analysis?

Many philosophers of science who embrace the conception of mechanisms, introduced above, as an ontological or epistemological template, including Machamer et al. (2000), also endorse difference-making as the basic template for causal inference. ²⁸ The specific variant of difference making that is widely endorsed is Woodward’s (2003) interventionist account, which rests on type-level counterfactuals (e.g., Kuorikoski 2012). In contrast, we propose counterfactuals based on single cases as the basis for causal inference on terms and mechanisms and the actors and their behavior constituting it. ²⁹ We show that single-case counterfactuals assign proper roles to process tracing and QCA in set-theoretic MMR and invalidate concerns that the cross-case method and within-case method are fundamentally incompatible. ³⁰

The details of single-case counterfactuals have been extensively dealt with before in philosophy of science and the social sciences. ³¹ Given our focus on set-theoretic MMR, we limit ourselves to discussing the elements that are salient for our purposes. A generally appealing property of single-case counterfactuals is that causal inferences are located where the actual mechanism and causal relation rest, namely, in the cases. This nicely emphasizes the notion of QCA as a case-based method. ³²

Single-case counterfactuals assess causal claims by asking what the outcome in a given case would be if one feature of the actual case was different (Lebow 2000; Lewis 1973a, 1973b; Paul and Hall 2013). Counterfactuals are based on the premise that a factor (to use a general term here) only qualifies as causal when it makes a difference to the outcome. A counterfactual confirms the causal relevance of a given factor if we can argue that the presence and absence of this factor make a difference to the presence and absence of the outcome. For instance, if we want to assess the cross-case hypothesis that liberal ideas are a sufficient cause of peace, we can analyze the case of Franco-American relations from 1796 to 1798 for which Owen argues that liberal ideas maintained by U.S. elites were a cause of peace between the two countries. In counterfactual-based causal inference, we would make this inference on the basis of within-case evidence and the conclusion that France and the United States would not have maintained peaceful relations if liberal ideas had not been entertained by the U.S. elite. ³³

The generation of inferences on an entire mechanism requires that we make an inference on every actor and its associated behavior that constitute the mechanism. We concur with MDC that an explanation is only meaningful if we consider an actor together with its behavior (2000:3); an activity must be carried out by an actor in order to contribute to productive continuity, and an actor must engage in some behavior. For purposes of causal inference, however, we must detach actors from their behavior and ask two counterfactuals. Would the process break down and the outcome be absent if a given actor did not behave as she did in the actual case? And, would the process break down and the outcome be absent if a given behavior were not performed by the actor in the actual case, but by another actor? ³⁴ If both questions are answered in the affirmative, we can conclude that the outcome depends on this component of the mechanism and is causally relevant.

We believe that tracing the actual process is in itself not sufficient for establishing explanatory relevance (Hitchcock 1995), although it is of course integral for counterfactual reasoning (Lebow 2000). ³⁵ Tracing a process is not sufficient for causal inference because a traceable process can be noncausal in fact (Woodward 2011). ³⁶ Neither is a process necessary for attributing causal influence because of cases of negative causation in which the absence of an event is a cause (Paul and Hall 2013). This is worth emphasizing because one might think that explanatory relevance is achieved by “peering deeply into the box of causation” (Gerring 2008:160). In practice, one might make a plausible argument by tracing an actual process. However, natural scientists who work with mechanisms on a much lower level of analysis than social scientists do not accept this as a standard for explanatory relevance (Woodward 2004), so it is not clear to us why social scientists should settle for this in principle.

Regarding the democratic peace phenomenon, take for example the claim that “elite holding liberal ideas” is sufficient for the elite’s adherence to a peaceful foreign policy toward other democracies. We can assign explanatory relevance to this element of the process if we are able to substantiate two claims. First, the elite would not have subscribed to a peaceful foreign policy if it had not entertained liberal political ideas, that is, we counterfactually consider the consequences of the absence of the observed behavior. Second, the elite would not have subscribed to a peaceful foreign policy if another actor or group of actors had subscribed to liberal political ideas, that is, we counterfactually evaluate the consequence of the absence of the actor. If both counterfactuals lead us to conclude that a nonliberal elite would not have adhered to a peaceful foreign policy, we have established the explanatory relevance of this part of the mechanism.

From a set-relational point of view, a twofold counterfactual on an actor and its behavior is necessary for the assessment of causal relevance. In empirical research, however, we consider it legitimate to focus on either the actor or the behavior, depending on the theoretical interest of the study at hand. In the example just given, we would deem it more interesting to look at the causal relevance of the elites’ behavior because, at least implicitly, Owen’s research interest seems to be that liberal ideas, as opposed to other ideas, trigger the mechanism that results in democratic peace.

If causal inferences are generated in process tracing, one might wonder what role is left for the simplification of conjunctions in QCA. In an ideal process-tracing scenario, we identify all sufficient terms and corresponding mechanisms via process tracing which is, however, highly improbable to ever happen in research practice. Most, if not all, social phenomena can come about by multiple sufficient terms and causal mechanisms, and we consider it unlikely that we would be able to select a case for process tracing in which all the terms and mechanisms are present. Even if we leave this issue aside, the availability and quality of sources create epistemic uncertainty and constrain our causal inferences in process tracing (George and Bennett 2005:chapters 5 and 6).

The first reason why it is beneficial to run a QCA stems from these challenges of causal inference in single-case studies. A relation of counterfactual dependence is taken as a sufficient criterion for separating causal from noncausal relationships, but not a necessary one (Hall 2009). Two prominent problems that render counterfactuals nonnecessary are overdetermination and preemption which come in various types that are not relevant here (see Paul and Hall 2013). In a set-relational setting, overdetermination describes a constellation in which two terms seem to be causally related to one outcome. Preemption means that one term brought about the outcome, but that another term would have brought about the outcome just as well if it had not been for the first term. A classic example from philosophy of science is two bullets hitting a vase simultaneously (overdetermination) and one bullet hitting the vase earlier than a second bullet (preemption; Northcott 2010). ³⁷ Both constellations are a problem for counterfactual inference because it is evident that in both scenarios both bullets are causes of the vase shattering. In a naive counterfactual analysis, however, we would infer that neither bullet is a cause because if one bullet had not been shot, the vase still would have been shattered by the other bullet.

Philosophers of science committed to single-case counterfactuals submit various proposals for enhancing causal inference in the face of overdetermination or preemption (Paul and Hall 2013). In single-case settings, the fundamental problem is one of a low-information environment because we only have one case at hand. In set-theoretic MMR, in contrast, the truth table analysis has the potential to ameliorate causal inference because it introduces additional cases that might increase the amount of information. With regard to the two-bullets example, more cases would solve the problem if they included a case in which only one of the two bullets was shot and we observe that the vase shattered, and cases in which no bullet at all was fired and the vase remained intact. It still holds that the causal inferences are made in process tracing, but, under overdetermination and preemption, we are left uncertain about what the proper inference is and QCA, by looking at more cases, can help us sort out which of the possible inferences are accurate.

The second added value of QCA relates to its exploratory nature and the fact that it can point our attention toward sufficient terms that, based on process tracing, we did not infer to be present (see Beach and Rohlfing 2018). This might be because we overlooked these terms in process tracing or they simply were not operative in the selected cases.

The third important task QCA performs is that it allows us to assess the degree to which a causal inference on a sufficient term travels from the examined case to similar cases under analysis. Although this step is not about causal inference, it is equally valuable because MMR is concerned with populations or samples of cases and rests on the assumption of causal homogeneity. We address this assumption by calculating the consistency value of a term. We cannot directly evaluate the degree to which the mechanism is operative in other cases because a QCA only involves data on the term and the outcome. However, when the QCA results indicate that the sufficient relationship between X and Y we derived from process tracing generalizes to more members of X and Y, we can invoke the causal homogeneity assumption and be more confident than in stand-alone process tracing that the same mechanism is present in all cases sharing the same X and Y. Just because in set-theoretic MMR causal inferences are made in process tracing, QCA should not be considered to be of second-order importance. It performs important complementary functions in causal analysis.

Advantages of the Unified Framework for Set-Theoretic MMR

There are multiple templates for causal inference (Brady 2008), which gives rise to the legitimate question of why we advocate single-case counterfactuals. We do not claim that case-level counterfactuals are foolproof. We are aware that they involve challenges, such as establishing the truth value of the counterfactual and preemption and overdetermination (see above). However, all templates for causal inference have their share of problems and the search for criteria that are necessary and sufficient for valid causal inference has so far proven to be fruitless (Paul and Hall 2013:chapter 2). Against this background, we argue that causal inference via single-case counterfactuals has four advantages in the context of set-theoretic MMR.

First, counterfactual inferences based on single cases are fully in line with the case-centered nature of QCA (Ragin 1987:chapter 1). They reinforce QCA’s case-based and qualitative component by assigning the task of causal inference to process tracing. Case-based counterfactuals dispel the notion that QCA hampers qualitative research by distracting qualitative scholars from their focus on cases (Collier 2014a, 2014b). Second, single-case counterfactuals address long-standing criticisms of, and questions about, the basis for causal inference in QCA (e.g., Lieberson 2001). The truth table analysis can invoke a regularity-centered criterion based on observing actual patterns even when a term only has one case as a member (Baumgartner 2013). “However, the type-level based inferences of regularity theories can hardly be reconciled with process tracing in single cases where token-level features such as time and space matter.” We therefore argue that the truth table analysis is important for causal analysis in set-theoretic MMR, but it does not directly contribute to causal inference, our issue of interest here. Causal inference via single-case counterfactuals can more easily accommodate sufficient terms having only one case as a member.

Third and related, when we use the Quine–McCluskey algorithm, we simplify conjunctions by determining redundant conjuncts the presence and absence of which do not make a difference to the outcome (Ragin 1987:chapter 6). ³⁸ Single-case counterfactuals invoke the difference-making criterion with the primary goal of confirmation. Causal inference via case-based counterfactuals establishes difference-making as a unifying principle for set-theoretic MMR and additionally takes into account that the intermediate and parsimonious QCA solution also rely on counterfactuals (Schneider and Rohlfing 2014, 2016). This would be different if causal inferences on mechanisms were based on any criterion that does not entail the idea of difference making, such as, for instance, the process tracing of spatiotemporally ordered actual events (Waskan 2011). ³⁹ Besides the problem of establishing causal relevance without difference-making (Hitchcock 1995), this would entail that set-theoretic MMR required a twofold standard of causal inference, that is, separate criteria would apply to QCA and to process tracing. Twofold criteria are not unknown to theories of causation (Russo and Williamson 2011). However, one sacrifices parsimony by relying on two criteria, which, so far, have been only discussed for probabilistic theories of causation that face many challenges. Parsimony is not an end in itself, but if one favors a dual criterion, one has to carefully consider whether the loss in parsimony is worth the inferential gain.

Fourth, our proposal aligns the idea of difference-making and counterfactual reasoning, on the one hand, with set relations, on the other. This is not an end in itself, but worth emphasizing because it hints at counterfactuals as a common basis for set-theoretic and quantitative research. The development of the potential outcomes framework has the individual treatment effect (ITE) as its basis but then moves on to the average treatment effect (ATE) and its variants because the ITE cannot be estimated (Morgan and Winship 2014). Based on our proposal, the differences between set-relational and quantitative analyses boil down to the focus on different types of “treatment effect”. This we consider an important difference because it sheds light on why it is difficult to conceptualize set relations in terms of the ATE.

Gerring (2012:chapter 12) proposes to unify social science methodology by transferring the potential outcomes framework, which has a solid counterfactual foundation (Morgan and Winship 2014:chapter 2), to the realm of set relations. He argues that patterns of necessity and sufficiency can be inferred by calculating the ATE or 2 × 2 tables, as they are known from crisp-set QCA. As has been shown before, this transfer of the potential outcomes framework is problematic because the ATE can produce significant results when a visual inspection of the 2 × 2 table shows that no set relation whatsoever is involved, and, vice versa, a set relation can be in place when the ATE is not significant (Schneider and Rohlfing 2012).

Table 1, reproduced from Schneider and Rohlfing (2012), exemplifies for a sufficiency relation the perils of relying on the ATE. The left panel in Table 1 represents an ATE of 0 because the presence and absence of the condition (elite holding liberal ideas) do not make a difference to the outcome on the cross-case level. In the right panel, in contrast, we get an ATE of 0.5 that is significant at .05. Yet, the presence of elites holding liberal ideas is definitely not sufficient for democratic peace, as a look at the right-hand column with a consistency score of 0.5 shows.

OPEN IN VIEWER

Table 1.

Average Treatment Effects and Sufficiency.

Democratic Peace	Yes	100	100	Democratic Peace	Yes	0	50
	No	0	0		No	100	50
		No	Yes			No	Yes
		Elite holding liberal ideas				Elite holding liberal ideas

Note: Reproduced from Schneider and Rohlfing (2012).

Instead of comparing the right and left column of a 2 × 2 table, as is required for calculating the ATE, in process tracing at least one case in the upper-right cell is subject to counterfactual causal inference. The criterion of single-case counterfactuals implies that a symmetric relation between X and Y on the within-case level, which follows from the requirement that a term must make a difference to the outcome, can be reconciled with an asymmetric X-Y relation on the cross-case level.

When the two tables represent results of a truth table analysis that was run before process tracing, we of course would only do process tracing for one or few of the 100 cases in the upper-right cell of the left table because the condition is consistently linked to the outcome. A follow-up cross-case analysis producing the result in the right-hand table would show us that the causal inference derived from process tracing does not generalize because 50 cases that are similar to the examined case do not display the outcome. This does not automatically invalidate our process-tracing inference because it is about a single case, but we would learn that it is not generalizable to all purportedly similar cases, that is, all cases in the right column of the table.

Conclusion

MMR is becoming increasingly popular among empirical researchers. At the same time, some methodologists remain skeptical as to whether MMR can work on a principled level. In this article, we build on Machamer et al.’s conception of mechanisms as entities and activities for the formulation of a unifying framework for causal analysis in set-theoretic MMR that takes single-case counterfactuals as the template for causal inference. Causal inference via single-case counterfactuals is certainly not without problems, just as no theory of causation is infallible (Paul and Hall 2013:chapter 2). We have shown, though, that inferences based on single-case counterfactuals have attractive characteristics that render them highly compatible with QCA using the Quine-McCluskey algorithm and the way in which it aims to substantiate causal claims.

We want to stress that our framework for set-theoretic MMR is also relevant for single-method research invoking either process tracing or QCA. Even if, for whatever reason, we are not combining two methods in one design, it might still be possible that the process-tracing insights form the basis for a QCA study at some point in the future and that the QCA results are the starting point for future process tracing. Under the assumption that the combination of both methods is generally appropriate for the research interest at hand (Ahmed and Sil 2012), our framework clarifies what and how exactly process tracing and the truth table analysis contribute to set-theoretic research even when carried out by multiple researchers in a sequential order.

In the division of labor we propose, single-case counterfactuals might seem to put process tracing in the driver’s seat because it is the method that is used for the generation of inferences. This impression is misleading because set-theoretic MMR involves a population or sample of cases. It is important to determine how far the single-case inferences travel to other cases that are members of the same sufficient term, and how many cases are members of the outcome but do not fall under the causal explanation found via process tracing. Taken together, we are not only convinced that process tracing and QCA can be combined without friction, but also that their combination represents a fruitful combination for causal analysis in MMR.