Ontology,Epistemology,and Multimethod Research in Political Science

The Epistemological Consequences of Humean Ontology

That small N is not merely an epistemological problem becomes evident when we ask under what definition of “cause” should small N be a problem for establishing causal relations. The answer has to do with statistical theory and the conception of causation that sufficiently—though not necessarily—justifies it. The origins of this metaphysical view can be traced to David Hume ([1748] 1999, 136)—hence often referred to as “Humean.” It assumes that any particular causal event necessarily implies the existence of causal laws that the former is an instantiation of. For example, if we see that in a particular case A is followed by B or increases in A are followed by increases in B, we come to the conclusion that A caused B if and only if there exists a law stating that all instances of A (or A-like properties and events) are followed by instances of B (or B-like properties and events). Thus a single causal event—or what is often called token causation—necessarily assumes causal generalizations, or type causation. Notice that this automatically precludes all singularist views and descriptions of causation. This has an important epistemological and methodological consequence. Under no circumstances can one infer a causal relationship from a single instantiation unless of course one already has knowledge of all the relevant laws. The laws in question state the exact form in which events or properties of type A cause events or properties of type B. Now very briefly, Humean definitions come in both deterministic and stochastic versions. Causes precede their effects, and are either necessary, sufficient, or both necessary and sufficient conditions (in the deterministic versions), or increase the conditional probability of their effects (in probabilistic versions). In both cases, every singular causal statement must be an instance of one of more general causal laws. The singular phenomenon itself need not be repeated as long as the unique phenomenon can be shown to be the result of a combination of laws that recur in other singular phenomena. Epistemologically therefore, the singular phenomenon cannot play a role in the establishment of a causal relationship since it is itself dependent on preexisting regularities that have already been established. Both the deductive nomological (D-N) scheme of explanation, proposed most clearly by Hempel and Oppenheim (1948), and Hempel’s (1942) inductive-statistical (I-S) scheme follow directly from such conceptions of causation.

So given a Humean view, how would one go about discovering causal relations? The epistemological problem is that of discovering regularities when many laws are instantiated simultaneously. Under ideal conditions experimentation would be the first best method (this obviously is not unique to Humean views; experimentation as a method is consistent with almost all ontologies of causation, but interpretations of experiments would differ depending on the definition of causality). One way to overcome the problem of simultaneous instantiation would be to isolate individual causes and observe their effects repeatedly to establish law-like regularities. When we move from the experimental sciences to the social or nonexperimental sciences, the goal remains the same, that is, the discovery of regularities, but this time they have to be detected from purely observational data. This is where statistical models come in. Such models try to approximate the experimental situations described above. These models assume that the data being generated are akin to the result of a series of independent experiments or observations generated from mutually independent processes where nature manipulates the independent or explanatory variable under different background conditions or controls (again, it is also possible to give other interpretations to inferential statistics). Inferential statistics is also consistent with the definition of causes as generalizations; that is, the “regularity” part of the definition, or alternatively the definition of causes as “types.” The latter is obviously because insofar as it informs one of average effects, generalization (over a particular population) is built into the interpretation of inferential statistics. Let us interrogate in slightly greater detail why an attempt to fit case studies into this framework would fail by reviewing the various defenses of case studies that at least implicitly accepted a reductionist and regularist view of causality.

Some of the earliest defenses of case studies took for granted that case studies had a small N problem, and judging from the arguments, underlying their acceptance of the weaknesses of case studies was a Humean view of causation. These arguments recognized that not only does statistical control approximate controlled experiments, but they also approximate experiments that have been repeated multiple times. The latter part is very important since given this ontology, one cannot rely on a small number of experiments to establish a regularity as such: the experiments have to be varied and repeated under different conditions to attest to the general validity of the regularity so discovered. For example, Sartori defended comparative case studies as a third-best method behind experiments and statistical studies (1991, 16). He admitted that

[C]omparative control is but one method of control. It is not even a strong one. Surely experimental control and, presumably, statistical control are more powerful “controllers.” But the experimental method has limited applicability in the social sciences, and the statistical one requires many cases. We are often faced, instead, with the “many variables small N problem.” . . . and when this is the case our best option is to have recourse to the comparative method of control.

The defense—without interpreting the passage above in a question begging way—is basically that though experiments and statistical studies are superior to comparative case studies in drawing causal inferences, the phenomena that most interest certain political scientists do not occur enough times to lend themselves to statistical studies. The problem with this defense is that the acceptance of the logic of statistical inference entails that a few cases cannot or should not lead us to believe that a cause exists. This is the crux of Lieberson’s (1991) argument against drawing causal conclusions from a few comparative cases (also see Sekhon [2004]). Using the example of automobile accidents, Lieberson shows how fragile our conclusions can be as to the causes of accidents if we rely on only a few cases, assuming that knowledge of causes entails knowledge of regularities. The most logical conclusion in this instance would be to state that given the paucity of cases one cannot say anything about the presence or absence of causes.

Lieberson’s critique applies equally to solutions to the problem that urge us to somehow increase the number of cases by, among other things, performing “within case analyses” by looking at multiple implications or consequences of a particular theory or causal statement within the same case (Campbell 1975, 184-89). But as Collier and Mahoney (1996) point out, this does not alleviate the problem of control since case selection is still likely to be biased. More fundamentally, if we assume that regularities are most basic and knowledge of causes entails knowledge of regularities, it is difficult to count multiple implications as an augmentation of the number of cases. For at a given level of analysis, each implication of any causal statement must be considered separately. For example, if causal statement A has implications b and c respectively, we would still require more than one case to establish this causality. If statement A is in the nature of a general statement of regularity, the fact that is has multiple implications for a single case should not cause us to attach much significance to it since the regularity can only be established by the demonstration of multiple instantiations of the same implication in many different cases. This is true even if the two implications are considered in conjunction, for then we would look for the two implications in conjunction in multiple cases. It is for this reason that statistical models (both classical and Bayesian) require each observation to be independent. And multiple implications of the same causal statement or theory can never be considered independent from each other. The assumption of homoskedasticity of error terms in linear models is one methodological manifestation of this fact. There is a rebuttal to Lieberson’s argument, but as we shall shortly see, the rebuttal makes sense only within a decidedly non-Humean ontology of causation. Within the Humean ontology, Lieberson’s position is very convincing indeed.

Some early discussions on the merits of case studies seemed to have realized the above, and as a result eschewed any discussion of the role of case studies in causal explanations. An alternative use for case studies is evident in Lijphart’s recommendation that though “if at all possible one should generally use the statistical (or perhaps even the experimental) method instead of the weaker comparative method” (1971, 685), given that time and money are in short supply,

the intensive analysis of a few cases may be more promising than a superficial statistical analysis of many cases. In such a situation, the most fruitful approach would be to regard the comparative analysis as a first stage of research, in which hypotheses are carefully formulated, and the statistical analysis. . . . in which these hypotheses are tested in as large a sample as possible. (Lijphart 1971, 685)

Considering his assertion that statistical analysis is normally better, this recommendation by Lijphart does not clearly define “superficial statistical analysis” compared to which “intensive analysis of a few cases” is more fruitful. But his suggestion that case studies can be used to formulate hypotheses can be seen as a partial justification for such studies. Eckstein’s (1992) “plausibility probes” are similar to this inasmuch as they are “attempts to determine whether potential validity [of a theory or causal statement] may be reasonably considered great enough to warrant the pains and costs of testing” (Eckstein 1992, 147). More recently, Gerring (2004, 347) has contended that though such studies can hardly illuminate causal relations, they can nonetheless be used for other purposes such as providing descriptive inference by furnishing answers to “what” and “how” questions, thus presuming a certain definition of causality that apparently excludes “what” and “how” questions.

Another justification that applies more to single rather than comparative case studies is that certain kinds of case studies that Lijphart (1971, 691-93) terms “atheoretical” case studies generate data and new information, which are raw materials for theory building. Finally, the assertion that single case studies are basically explications of singular causation in light of existing theories that have been established by statistical studies forms a final category of defense under this heading. Lijphart’s (1971, 691) respective theory confirming, theory infirming, deviant and interpretive case studies, and Verba’s (1967, 114-15) “disciplined configurative approach” are all examples of this defense. The basic claim of this position is that case studies are explications of particular events that especially interest the researcher. The researcher then seeks to explain the event with the help of established regularities and general causal statements. This is essentially an example of respectively Hempel and Oppenheim’s D-N model or Hempel’s I-S model of explanations at work, both of which rely on a conception of cause that is wedded to regularities.

Verba’s (1967, 114) pronouncement that “[t]he uniqueness of the explanation of any particular case arises from the fact that the combination of relevant factors that accounts for a nation’s pattern of politics will be different from the combination in other cases” implies that general causal laws operate in different combinations and under different initial conditions and this is the reason why every explanation appears unique. This is reminiscent of Hempel’s position that historical explanations can be shown to consist of information about general laws and initial conditions. Thus Verba (1967, 114) points out that a configurative explanation, in addition to “plausibility of the account of a political event,” also rests on “the acceptability of a variety of unexplicated general laws that underlie the argument.” It is important to note here that though disciplined configurative explanations rest on general laws, the explanation itself does very little to strengthen or weaken the validity of the said laws.

Similarly in what Lijphart (1971, 692) calls interpretive case studies, “a generalization is applied to a specific case with the aim of throwing light on the case rather than of improving the generalization in any way.” His other three kinds of case studies, viz theory confirming, theory infirming, and deviant case studies, are not much different despite his claims that they help in building theory. Similar to an “interpretive” study, a “theory confirming” or “theory infirming” study takes place “within the framework of established generalizations” (1971, 692). Are these generalizations in the nature of regularities? It would seem that they are, given Lijphart’s statement that, “assuming that the proposition is solidly based on a large number of cases, the demonstration that one or more case fits does not strengthen it a great deal. Likewise theory infirming case studies merely weaken the generalization marginally” (ibid). The same can be essentially said of an interpretive study though in this case the purpose of the researcher is different. It can be said that though an interpretive researcher does not care about theory construction per se, her success or failure at interpreting a particular case in light of existing generalizations does marginally strengthen or infinitesimally infirm those generalizations. The purpose of the researcher is irrelevant here.

These concessions—precluding any role for case studies in causal analysis—are sometimes accompanied with arguments that cannot easily be reconciled with the former. Thus, for example, Lijphart’s subsequent assertion that such studies can be considered “crucial experiments” if values on the variables are extreme is difficult to reconcile with his statement quoted above. Why should extreme values on variables in one case cause us to reexamine our prior theory, especially since the latter could be based on a large number of cases? The same applies for “deviant case” analyses. As McKeown (1999) has also observed in a slightly different context, a single additional case can never, by this logic, lead us to weaken an original proposition that, in Lijphart’s own words, “solidly based on a large number of cases” (1971, 692). The problem is that some of Lijphart’s epistemological points about the contributions of case studies make sense only when decoupled from his ontological orientation which seems to underlie the bulk of his other points.

Another popular defense of case studies—that such studies are better at handling determinism (Gerring 2004, 347; Munck 1998, 33)—is based on conflation of ontology with epistemology. It is perfectly consistent to have a deterministic and Humean view of causality—indeed, the original Humean view was in fact deterministic and some philosophers have argued that “Hume and indeterminism don’t mix” (Dupre and Cartwright 1988)—and still claim at the epistemological level that statistical inference is the best way to establish this causality. As Laplace observed a long time ago, an (ontologically) deterministic relation can appear to be (epistemologically) stochastic because of ignorance of all relevant laws and initial conditions. It does not matter whether the view of causality is deterministic or stochastic because the problem is not stochasticism per se but the reduction of causes to regularities. As a result ontological determinism does not even sufficiently justify case studies. The argument from determinism becomes important only once we abandon the reduction of causes to regularities. The problem is that what we discover from what are sometimes called Mill’s methods of difference, agreement, or concomitant variation cannot even be considered “cause” in this sense.

The reference above was to deterministic sufficient conditions. But can deterministic necessary conditions justify case studies, as Dion (1998) has argued? Dion’s argument protects case studies against the small N criticism only under extremely restrictive conditions. The argument has more to do with the problems that classical inferential statistics faces in tackling necessary conditions than the inherent strengths of case studies. In fact it could be seen primarily as an advocacy of Bayesian statistics over classical statistics when it comes to necessary conditions.

Since Bayes’s rule depends crucially on known probabilities to determine posterior probabilities, its applicability is limited to only certain kinds of systems. ¹ To be precise, it is crucial that the mechanism that generates prior probabilities is well known, and alternative hypotheses have well-known probability outcomes or likelihoods. The prior probability is a source of great debate in both philosophy and statistics (see Sober [2002], for example). It is uncontroversial in cases of systems where there is a clear way of assigning prior probabilities. But it is slightly more controversial in cases where we can’t. Then the question is: what should the prior probabilities be based on? Should they be based on statistical regularities, “common sense,” case studies, subjective opinions, and/or collective opinions of “experts”? As soon as we ask these questions, we realize that we are back to the square one, that is, the same basic question that we asked at the outset. Additionally, and more pertinent to the use of such statistics to defend small N’s is the fact that we would have to consider multiple hypotheses with determinate likelihoods for effective statistical control; at which point the difference in terms of sample size between classical inferential statistics and Bayesianism begins to disappear.

Similar difficulties apply, with some modifications to Dion’s argument about necessary conditions, which he clearly recognizes. For instance, in reformulating Theda Skocpol’s argument, he assumes that the prior probability for the necessary condition is 0.5; in other words, he assumes that the prior probability of state crisis is about half, or given the observation of a social revolution there is a 50% prior chance of state crisis. Moreover, he considers only one alternative hypothesis. Now if there were more than one alternative hypothesis, the prior probability for the necessary condition would likewise decline (because some probability would have to be assigned to each alternative, and the total probability obviously cannot exceed 1). In addition as Dion (1998, 135) points out, it is very difficult reach a respectable degree of posterior probability if the likelihoods of the alternative hypotheses are similar to the likelihood of the hypothesis in question. A fundamental question would however still remain even for necessary condition Bayesian analyses: where do prior probabilities come from and on what basis do we assign them numerical scores? Even this argument, as a result, cannot provide sufficient justification for case studies.

This brings us to the final and most popular justification for case studies and their incorporation in mixed-methods research, namely, that case studies are uniquely suited to discover or enunciate what are called “causal mechanisms,” which statistical studies are less able to do. We saw, however, that something akin to statistical generalization is integral to the regularist and reductionist view of causality. That is precisely why case studies are seen to have, among others, a small N problem. Thus to advocate incorporation of case studies along with inferential statistics in the same research design would be to fall into a logical contradiction by advocating contrary views of causality in the same breath, unless the very concept of “mechanism” is defined statistically. But if “mechanism” is defined statistically, there cannot possibly be any place for case studies in research designs. Let us consider then, first the statistical view of “mechanisms,” consistent with a Humean view, and then other, nonregularist definitions of “mechanism” that are contrary to this view by virtue of not being either regularist or reductionist. Though the latter definitions support both large N inferential statistics and case studies independently, depending on whether they are either nonreductionist, or nonregularist, they imply very different interpretations and practice of mixed or multimethod research.

If mechanisms are defined as, “in effect, variables that operate in sequence” (Sambanis 2004, 288), or any variation thereof, some of the same criticisms that we started with apply. The difficulty of defending case studies while holding this particular understanding of mechanisms stems from the fact that it implies just another version of the Humean definition extended to intervening variables. It is possible to multiply the number of steps between cause and effect while remaining steadfastly Humean. Indeed many philosophers have demonstrated how to build chains of causal relevance (Salmon 1993).Each link or mechanism in a longer chain can be represented by equations that can be construed as statements of regularity, and as such the same epistemological concerns that were raised earlier about the confirmation of causal claims with case studies apply here too. Various statistical models such as path models would seem to be the natural recourse. If this is a fair representation of some definitions of causal mechanisms, then again the sufficiency of case studies cannot be defended.

More avenues open up once we abandon either reductionist or regularist (or both together) understandings of the concept of “mechanism.” But these latter conceptions, though equally supportive of inferential statistics independently, cannot easily accommodate the usual manner of performing multimethod research without running into logical contradictions.

The Epistemological Consequences of Singularist Accounts and Some Difficulties in Combining Methods

“Singularist” definitions of causality hold that singular events and not regularities are more basic. The definition decouples generalizations from the definition of causality (Ducasse 1926; Salmon 1980, 1997a). Epistemologically therefore one need not look for generalities and the explanation of a single event or case can count as a causal explanation. But the definition presents us with a problem that is pervasive in singular or quasi-singular accounts of causation. An example, due to Ducasse (1926), is that if a brick thrown with a particular velocity strikes a glass window and breaks it, and at the same time airwaves from a canary singing strike the window, we would not know which event was relevant to the breaking of the window by this definition (i.e., at the level of ontology). On the other hand if we were to try to make the brick rather than the canary’s sound relevant to the breaking of the window, one of the options would be to rely on statistical relations, in which case the latter would seem to be bearing most of the causal burden. Thus such reductionist (inasmuch as they seek to reduce causes to something more basic) but singularist (insofar as they do not rely on regularities) definitions of causality face a basic dilemma, which is that of distinguishing spurious causes from “real” causes at the definitional level.

The epistemological manifestation of this problem is that it is difficult know that the mechanism connecting a cause with an effect in a particular case is the same mechanism connecting causes to effects in all the other cases. If it is, then we do not need to trace the process—and the processes in turn are just like general regularity statements—and if it is not then the statistical part of any study plays no role in causal explanation.

Nonetheless, a partial way out of this problem could be to attach singular counterfactuals to singularist or process conceptions of causation. Combining process conceptions of causation with counterfactual reductions of causation such as the one offered by the philosopher David Lewis (1979, 1973) could be helpful. Without going into too much detail, one standard of judging the truth conditions of counterfactuals would be to claim that the truth condition of counterfactuals depends on the contrast space of any explanation and therefore causality is also context and contrast space dependent. For instance, sustained economic crisis was the cause of the Indonesian dictator Suharto’s resignation relative to a contrast space where there is no resignation, but it is not necessarily a cause if the contrast space is resignation in May 1998 rather than in December 1998, or if the contrast space is exile rather than resignation. It is likewise possible to build contrasts into the explanatory process too. Thus the statement above can be rewritten as, “sustained economic crisis (as opposed to no crisis), was the cause of Suharto’s (rather than some other Indonesian political figure’s) resignation (rather than continuation in power).” Therefore depending on the contrast space, one would devise different criteria for similarity.

How does one go about demonstrating a causal relation given this ontology? The first task is to carefully ascertain and trace causal processes and pathways. As Salmon (1997b, 476) observes this would automatically eliminate “irrelevant factors that are not present at the right place and time.” Among political scientists writing about these issues, this view is also implicit in some defenses of case study research (see Waldner [1999, 230-41]). At the same time, these arguments, though sufficient in justifying case studies, cannot equally accommodate mixed methods. This is because notwithstanding its decoupling of generalizations from the very definition of causality, the problem of reconciliation between the singularist view and large-scale generalizations reappears.

Since in principle such counterfactuals are metaphysical pronouncements and hence cannot be literally “tested,” one can at the most assert the plausibility of the counterfactual under consideration. One way to do this would be to attend carefully to the nature of the counterfactual and its contrast space. For instance, the counterfactual stated above could be taken to assert what would have happened in Indonesia to Suharto rather than some other country to some other head of state if there were no economic crisis. Since this is a singularist conception of causation, what happened in other countries is irrelevant to the credibility of the causal account; it is irrelevant if economic crisis was accompanied by resignations of heads of state in other quasi-totalitarian regimes. One can certainly form hypotheses connecting economic crises to resignations based on other cases, but unlike in regularist conceptions of causation, the fact that such crises may or may not have led to resignations in other cases does not either strengthen or weaken the singularist account.

The counterfactual would be open to doubt if it were demonstrated that previous cases of economic crises very similar to the one asserted in Indonesia did not result in Suharto’s resignation under background conditions similar to those mentioned in the counterfactual. We would, however, not cease calling economic crisis the cause of resignation. This would depend on whether the counterfactual was interpreted as asserting a singular necessity or a singular sufficiency. If the counterfactual was interpreted as a necessary condition, nothing more could be said since Suharto’s resignation as an event could happen only once, that is, we cannot look at multiple Suharto resignations to ensure that economic crises were present in all of them. On the other hand we could observe more than one economic crisis in Indonesia during Suharto’s reign and note what happened. Again, Waldner (1999, 230-41) comes close to implicitly exemplifying this ontological position.

Singular counterfactuals therefore are very sensitive to contrast spaces. We would thus get different causal explanations if we phrased our questions differently, for instance, by asking for the causes of resignation of any president of Indonesia or the president of Indonesia rather than the Prime Minister of Malaysia. Being a singularist view of causation, it is amenable to more general questions only in a piecemeal manner. Generalizations, if any, must be constructed from below by comparing two cases at a time where cases can be conceived synchronically or diachronically. Furthermore, since singular events are more basic, there is no expectation that generalizations will necessarily emerge. The objection to this ontological view and its epistemological consequences also follows from this fact. Say we want to define a contrast space such that it contains all cases of nonresignations of every head of state by asking, “what accounts for the resignation (as opposed to the nonresignation) of every head of state (as opposed to other political figures)? Nothing in the singularist definition prevents this question from being asked. Indeed this would be equivalent to asking, “what accounts for the resignation of the president of Indonesia and the Prime Minister of Thailand and the Prime Minister of India and . . . and so on rather than the Prime Minister of Malaysia, the Prime Minister of Great Britain . . . and so on.” Now even if we call both a legitimate answer to this question and a legitimate answer to the more limited question asked earlier, “cause,” clearly the former is to be preferred if we prefer more generality over less. And if we prefer the more general statement to the less, the same problems that we talked about in reconciling case studies with large N inferential statistics recur at the epistemological level.

For instance, Evan Lieberman’s (2005) latest attempt to suggest a framework for multimethod research faces this particular problem. Though he generally steers clear of any theoretical justifications for confidence in putatively causal relations discovered in the course of research, Lieberman’s implicit ontological views can at times gauged from his methodological pronouncements. He writes, for instance, that “a nested research design implies that scholars will pose questions in forms such as ‘What causes social revolutions?’ while simultaneously asking questions such as ‘What was the cause of social revolution in France?’” (2005, 436). For an answer to both questions to qualify as “causes” almost necessarily implies a singularist view of causation. Under a regularist view an answer to the second question cannot differ from an answer to the first, and the former has to be at least a subset of the latter. Lieberman’s two questions can also therefore be restated as respectively, “What differentiates the set of social revolutions (i.e., every single case of successful social revolution) from the set of unsuccessful ones?” and “Why was social revolution in France (as opposed to Germany?) successful rather than almost successful or unsuccessful?” But though a singularist view justifies both questions, it does not presume that an answer to one—what causes social revolutions—will necessarily have any bearing on the other. Lieberman does not seem to sufficiently realize this, and as a result some of his methodological strictures are difficult to make sense of. His advice is to start with a large N analysis and then—in case of robust and satisfactory results—“test” the model with small N analysis by choosing cases that fall within the average prediction of the large N model (2005, 437). Why should we expect the small N cases to be consistent with the large N predictions? Even if they are, why should we have any confidence that the average prediction of the large N analysis and case study research point to the same causal relationship? In the absence of robustness Lieberman advises model building and analysis of predictions that fall in the average, and also are outliers (2005, 439-40). The criteria for “robustness” and “satisfaction” must be statistical; it is therefore difficult to see why lack of robustness should motivate case studies. Indeed there are well-known remedies within inferential statistics for such problems as lack of statistical significance or any bias in a model and none of these involve looking at case studies. Note that all the questions raised here do not imply that Lieberman is wrong, but that the argument contains large gaps, owing to insufficient appreciation of the metaphysical implications of methods. Additional arguments have to be supplied to reconcile mixing of the two methods.

Nonreductionist Causes and the Prospects for Multimethod Research

Our discussion thus far has indicated the difficulty of reconciling inferential statistics and case studies in the same research design (insofar as both claim to illuminate causal relations) at the methodological level. This in turn can be traced to the difficulty of clearly separating methodology from metaphysics, and the fact that methodologies at times imply (and are implied by) ontologies that are not easily mutually reconciled. The abandonment of a reductionist view of causality can, however, open up promising avenues of reconciliation. The following example will help motivate the discussion.

Consider Lieberson’s (1991, 313-14) example about traffic accidents again. He shows how by relying on what are called “Mill’s methods” we might be mistaken about the cause of accidents. In the first example he uses Mill’s method of difference to show that reliance on this method would lead us to the conclusion that neither speeding, nor drunk driving or running a red light was the cause of the accident since both drivers (we will call them j and k, respectively) were doing all three. In fact the cause was j’s entering from the right-hand direction. Then he uses the method of agreement to demonstrate that if we change the outcome variable to accident and alter one variable from no to yes, we come to completely different conclusions about what causes traffic accidents (both are reproduced below).

OPEN IN VIEWER

Table 1.

Mill’s Methods of Difference and Agreement: Lieberson’s Example.

Accident	Drunk Driving	Car Entering from Right-Hand Side	Driver Speeding	Runs a Red Light
	Difference
Yes (driver j)	Yes	Yes	No	Yes
No (driver k)	Yes	No	No	Yes
	Agreement
Yes (driver l)	Yes	Yes	No	Yes
Yes (driver m)	Yes	No	Yes	Yes

First notice than if we look at the table from the singularist and process ontology, there are absolutely no problems with the table or with Mill’s methods. Looking at the first set of cases, assuming that the causal mechanisms between each of the factors and the outcome were legitimate and that all the relevant background conditions are accounted for, it is not at all a strange conclusion that what causes driver j’s accident (as opposed to driver k’s accident) is that he was entering from the right-hand direction (rather than from the left-hand direction). Now it is a different matter altogether that it may not be generally true that most accidents are caused by cars entering from the right-hand direction, but it definitely explains why j, rather than k, had an accident. It may turn out that when we further compare j with others the explanation might change since at that point our contrast space would also have changed. If my twin and I have been lifelong smokers and we share all relevant physical attributes except for the fact that he is an avid runner and I am not, there is nothing wrong in claiming that this difference explains why I have heart disease and he does not. There may be others who are avid runners and smokers but have heart disease, but this does not invalidate my prior judgment. Nonetheless if we care about generalizations or view generalizations as a part of the very definition of “cause,” we might object to this line of reasoning for the reasons sketched out earlier.

There is a deeper and more fundamental response to Lieberson’s argument. This is better illustrated with Lieberson’s second example (though we will come back to his first). He writes,

Ten people apply for a job; five are black and five are white. One of the five whites and all of the five blacks are hired. Applying the method of difference, one would conclude that race did not affect employment. Rather, it would have to be some variable that separates all the employed persons from the four who did not get a job. (1991, 313)

This is correct insofar as it goes in demonstrating that Mill’s methods cannot account for degrees of effects, but Lieberson seems to assume here that what convinces—or in any case, should convince—the observer of a causal relationship should be the data alone. Race could have been a factor, but it is also possible that some other factor was correlated with both race and the eventual outcome, in which case race would no longer remain causally relevant. This is an example of Simpson’s paradox, or what is sometimes referred to as the Cohen-Nagel-Simpson paradox, that “any association . . . between two variables which holds in a given population can be reversed in the subpopulations by finding a third variable which is correlated with both” (Cartwright 1979, 422; Hesslow 1981). For example, if we think that smoking causes cancer, it is not difficult to find “subpopulations in which smoking prevents cancer” (Dupre and Cartwright 1998, 524). This has led some philosophers to relativize causal claims to particular populations, which is not so different from what we were doing earlier with contrast cases (Dupre and Cartwright 1998, 528). The problem, again, is not small N but the reduction of causes to statistical relations.

Let us take Lieberson’s accident example again. One will notice that it seems absurd to Lieberson that drunk driving, speeding, or running a red light should not cause accidents. But if one were to exclusively rely on data sets about traffic accidents one would not even begin to understand what causes them. Almost anything under the “right” background situations can both cause and hinder accidents. All we have to do is to find a suitable variable to condition on that can reverse any relationship. Lieberson assumes that drunken driving and so on are legitimate causes of accidents, and it is this assumption that helps him make sense of the apparent absurd outcomes of Mill’s methods. Yes Mill’s methods are seriously flawed, but they are not flawed for the reasons Lieberson thinks they are. They are flawed for the same reasons that large N analyses would be flawed if we were to solely rely on them for our causal judgments. The essential principle therefore applies to large N as well as small N analyses. Prior causal knowledge, in other words, restricts and informs the kinds of inferences one is able to make both from statistical relations and case studies. It bears reiteration that Liberson’s basic critique is sound from a perspective that sees regularities as basic. The discussion then, brings us to an important point. What options are we left with if we do not consider regularities to be basic? And if they are not basic, where does our causal knowledge come from?

Elizabeth Anscombe (1975) has argued that in fact our primary knowledge of causality comes from our ability to speak:

In learning to speak we learned the linguistic representation and application of a host of causal concepts. Very many of them were represented by transitive and other verbs of action used in reporting what is observed. Others—a good example is “infect”—form, not observation statements, but rather expressions of causal hypotheses. (1993, 92-93)

Furthermore, the word cause itself is very abstract and having such a word in the vocabulary presupposes the knowledge of language.

The word “cause” can be added to a language in which are already represented many causal concepts . . . [b]ut if we care to imagine languages in which no special causal concepts are represented, then no description of the use of a word in such languages will be able to present it as meaning cause. (1993, 93)

The causal terms themselves are very abstract and must find their manifestation in the more concrete circumstances in which they are used. Nancy Cartwright has made related claims that help us make sense of why even single case studies are sometimes so influential in illuminating causal relations.

She contends that it is the arrangement of capacities in certain ways that produce regularities; “nomological machines,” or “socio-economic machines” as she calls them, are particular arrangements of capacities that “in the right sort of stable (enough) environment will, with repeated operation, give rise to the kind of regular behavior that we represent in our scientific laws” (1999, 50). Capacities, furthermore, cannot be identified by any particular manifestation. They can be compared to qualities such as kindness or tenacity that are carried by human beings. Such qualities are not identified with any one particular behavior; instead they are instantiated in multiple circumstances as different behaviors all of which have in common the fact that they are displays of kindness or tenacity (1999, 51). Socioeconomic machines are essentially fables that illuminate important aspects of how the world works, while capacities can be equated with morals of such fables. The relationship between the fable and the moral is not that of similarity but “that of the general to the more specific . . . [e]ach particular is a case of the general under which it falls” (1999, 39). This means inter alia that “satisfying the associated concrete description that applies on a particular occasion is what satisfying the abstract description consists of in that occasion” (ibid). Thus any particular arrangement of capacities is also general, and in turn, every general capacity finds its manifestation only in particular arrangements. Once we understand capacities well enough (as is the case in certain natural sciences) we can further manipulate these capacities and arrange them in different ways to produce different laws. As Cartwright observes, “anything can cause anything else. In fact, it seems . . . not implausible to think that, with the right kind of nomological machine, almost anything can necessitate anything else” (1999, 72).

The epistemological consequences of this view urge us to treat both (most) large N statistical studies and case studies as essentially alike in that both can be interpreted as attempting to “guess” the arrangement of hypothesized capacities in the world. Sometimes when we know about enough capacities and other background conditions “[w]e accept laws on apparently slim experimental bases . . . [and] the data plus the description of the experimental set-up deductively imply the law to be established” (1999, 93). Case studies, both single and comparative, can therefore be considered similar to fables that substantiate morals. The fables, however, have to be very carefully constructed with great attention to capacities and their arrangements. They are necessarily concrete, but they are at the same time general. This is precisely why studies like John Gaventa’s (1980) of one particular locality in one country are also general. Notice that domain restriction—an issue that is of some concern to defenders of case studies since it is always seen as a weakness of such studies—finds its best justification under this ontology. In fact if we follow this logic, restrictions of domains is imperative, since what we are describing are particular nomological machines, the very definitions of which carry the connotation of restriction. This is because as we observed earlier, it is the arrangement of particular capacities in certain orders and under certain conditions that could generate laws. But domain restriction does not mean restricted generalization. The fact that some physical laws are literally true only within the confines of the laboratory does not prevent them from being also general. This answers certain criticisms of case studies based on their domain restriction. Thus to say that domain restriction in case studies necessarily implies limited causal force is to implicitly accept an ontology that cannot justify case studies in the first place.

Even in this case, however, the usual manner of combining case studies with a large N (inferential) statistical analysis cannot be logically supported because of the reasons pointed out earlier. On the other hand, one way of avoiding the usual contradictions in mixing methods would be to truly “triangulate” within the general framework of a case study. In other words, instead of using the usual procedures of picking one case out of any sample, one could try to empirically describe the arrangement of capacities (of course, in the context of prior background knowledge of capacities) of any one case, and then examine the implications of such an arrangement using quantitative evidence. This would work because as Cartwright pointed out, it is the particular arrangement of capacities that produces regularities. But it must be a necessary preliminary to first explain why and how the arrangement of capacities came to be. This kind of suggestion is most relevant to the literature on institutions in political science and sociology, especially the ones based on single cases. Thus in explaining the theoretical conditions underlying the formation of particular socioeconomic machines (i.e., institutions, organizations), the researcher, in addition to tracing processes or verbally explaining cases, could also statistically test the implications of such processes.

To take one example, in his work Socializing Capital, the sociologist William Roy (1997) seeks to explain the emergence of the corporate form in the United States. His explanation is power-based: briefly, and to simplify a bit, he argues that corporations and corporate forms are the manifestation of collective power of capital, and their emergence was an inherently “political” act. Roy is also questioning the “efficiency” explanation of the rise of corporations advanced among others by Alfred Chandler, namely, that the corporate form emerged because it was a far more efficient, purely “economic” competitor in the market. The bulk of the work is a verbal explanation of the evidence that businessmen and factory owners acted collectively, self-consciously, and strategically—quite at variance with what would be expected of purely “economic” actors—and overcame opposition in the political arena in successfully getting the state to recognize and legitimize this form of organization with all its privileges and immunities. At the same time, Roy recognizes that the plausibility of his larger account would be enhanced with the demonstration of other ramifications of his argument, quite separate from his primary case study. If his argument was correct—and the efficiency thesis incorrect—we would also expect to see little correlation between growth and productivity of units and the extent of incorporation. Indeed, this is what his statistical model seems to demonstrate (1997, 32-34). Growth and productivity had little to negative effects on the extent of corporate capital, demonstrating that for most firms, efficiency was hardly sufficient for incorporation. He subsequently also tests other implications such as the effects of sheer size on profitability, net of factors normally associated with efficiency to come to similar conclusions (1997, 36-38). Thus two independent sources of evidence—qualitative description of how capitalists mobilized to win political battles, and the predictable or predicted quantitative implications of the latter—point to the same basic answer. Roy’s work can also be seen at the same time as a general explanation for the emergence of corporations that illuminates certain capacities (or general tendencies) and their arrangements. The knowledge capacities from this single case can guide further research as investigation is carried out on how similar motivations and interests play out in other social contexts.

Conclusion

The problem this article sought to identify was that many researchers writing about mixed methods take (perhaps not a very well-articulated) metaphysics of causality as given and then try to justify multimethod research designs on those grounds. Yet metaphysics does constrain epistemology and vice versa. It is very difficult to justify case studies while holding a causal ontology that asks one to look for regularities (rather than say, capacities) in nature. Similarly, some seemingly sound epistemological arguments when paired with others accepting the weaknesses of case studies risk running into contradiction due to ontological incommensurability. For example, in commenting on selection bias in qualitative research, Collier and Mahoney (1996, 63) say,

What issues arise if researchers are working only with the smaller set of cases and do not care about generalizing to the larger set that has greater variance on the dependent variable. The answer is that, if these researchers seek to make causal inferences, they should, in principle, be concerned about the larger comparison.

They have at this point already accepted the ontology that underlies this statement since it does not occur to them that such issues could be irrelevant to causal inferences provided that others held a different conception of cause. Therefore despite appearances, this is not necessarily a methodological point. It would be a methodological point under the assumption that the researchers who do not wish to generalize shared the same conception of causality as Collier and Mahoney. But the conception of cause that underlies their methodological point cannot logically support case studies. As was demonstrated, case studies cannot identify regularities, while statistical inference does sufficiently allow us to do so. Though in principle the task of finding regularities is strictly speaking, independent of the task of inferring population characteristics from a sample, the latter provides us with the best conceivable vehicle for the former. In fact, a few pages later Collier and Mahoney go on to say that “[s]tudies that achieve greater generality could be seen as doing so at the cost of parsimony, accuracy, and causality” (1996, 69). How can this statement be reconciled with their earlier statement since that statement implied that one could not make causal inferences if one were restricted to only a few cases? Is a study that is very particular closer to “causality” than a study that is very general since causal heterogeneity is a problem? Moreover since “with a complex regression model, it may be possible to deal with heterogeneous causal patterns,” (1996, 68) why not use the model instead? Collier and Mahoney then observe that this is partly because of a difference in opinion between scholars who believe that it is “possible to develop valid concepts at a high level of generality” and those who are “fundamentally ambivalent about generalization, are committed to careful contextualization of their findings, and in some cases explicitly seek to impose domain restrictions on their studies” (1996, 69). What if the latter contended that their study and findings only apply to the case they studied? Will their argument be considered causal then? How much contextualization is too much? And how does this reconcile with Collier and Mahoney’s previous statement about causality? It is hard to see how they can endorse both kinds of research at the same time given their previous statement about causality. These and other differences cannot be explained except with reference to ontological differences about the nature of causality.

Thus far we have argued that ontological assumptions (whether explicit or implicit) affect epistemology and, as a consequence, methodology. There is, however, a very reasonable argument to the effect that empirical social “scientists” should avoid ontological discussions. For, may it not be argued that it is practice that should inform philosophical discussions, not the other way round? Surely, much conversation in the philosophy of science is in fact informed by actual scientific practice: witness Imre Lakatos’s (1970) theory of scientific methodology or Kuhn’s (1970) or Feyeraband’s (1975) work on scientific methods and discovery. Similarly, why shouldn’t the ontology of cause be informed by practice?

The argument, though cogent, still does not count against the consideration of ontological assumptions of causation. This is because the relationship between ontology and methodology is not one-way. They actually reinforce each other. For example, if we begin with a strictly regularity (or regularist) conception of cause, we would be predisposed to look for regularities in nature and our methods would reflect this. Of course, our methods might fail to reveal such regularities and we might change our conception in that light, but the initial conception itself cannot be defended except on analytic and logical grounds. On the other hand, we might even continue to choose to ignore alternative conceptions in the face of empirical ambiguities, insofar as results from practice do not conclusively support one view over the other. In this situation, it would be incumbent on us to clearly defend our choice, on ontological grounds. Second—and this argument holds especially for the social sciences—it is never very clear when a particular conception has “worked” on purely pragmatic grounds. At least with regard to the applied sciences, one could argue that they are able to function quite well while being indifferent to causal concerns. In other words, whatever conceptions they have “work,” and we are clearly able to see how: a television remote control “works,” for example. Even if we granted this argument, one would be hard pressed to see how and what conceptions of causality (or lack thereof) in “applied” sociology or political science have “worked.” Therefore in the absence of clear performance criteria, it is important to explicitly defend one’s causal ontology. Thus if a principal purpose of the social sciences is explanation, and to the extent that explanations are causal (and certainly there can be very good noncausal explanations), they also implicitly assume causal ontologies. Insofar as these assumptions are being made anyway, it is worthwhile to render them explicit in the course of embarking upon a research program that may not sit comfortably with the ontological underpinnings of the theory and, as a result, produce findings that are at best marginally useful for establishing its validity.

Finally, it is worth pointing out that the discussion so far does not nearly exhaust the universe of ontological questions with epistemological and methodological implications. It would therefore be worthwhile to conclude with a list of issues that this article neglected, sidestepped, or otherwise left unexplored. First, causal explanations do not exhaust the universe of possible explanations (explanations, for instance could also make use of noncausal laws) and as such the article only addressed one possible kind of explanation. Related to this is the argument that since social sciences concern human action, social explanation should concern itself with reasons for action, and furthermore, it can be argued that reasons are not causes and should instead be understood in terms of meanings for social actors (Taylor 1971; Mcintyre 1973). This in turn would involve interpretation of intentional actions and the meanings ascribed to them. A further step could be an examination of the reconcilability of causation with intentional explanations. This article, however, sets these questions aside for the moment.

Second, it was not the objective of this article to pronounce on the relative superiority of any particular methodology. The implication of the argument made in this article therefore is not that case studies are somehow inferior to inferential statistical studies but that the arguments advanced in favor of case studies in the methodological literature in political science cannot sufficiently justify such studies because the ontological presuppositions of such arguments preclude any role for such studies in establishing causal relations. That is also the reason that multiple research methods within the same research design (where multimethod consists in combining inferential statistics with case studies within the same design) faces logical problems. Conversely, that the same ontological presuppositions sufficiently (though not necessarily) support inferential statistics should not imply that the latter is a “better” methodological approach to causal analysis.

Third, it was also not the intention of this article to comment on the relative strengths, weaknesses, or superiority of any particular ontology or theory of causation. It was merely to underline what kinds of epistemologies and methods would be consistent or inconsistent with particular conceptions of causation. This was also the reason that the article did not discuss at any length the larger literature on causation in analytical philosophy. It discussed the literature only insofar as certain conceptions are implicated in the arguments actually made in the methodological literature in political science.

Finally, this article also circumvented some larger debates in the philosophy of science such as those between empiricists and philosophical realists. This is because the latter, in principle, are independent from debates specifically about the nature of causality. Thus, for example, the issue of causal realism versus reductionism is quite distinct from that of scientific realism as against logical empiricism. Thus the debate between causal realism and reductionism is also a debate among various stripes of logical empiricists. As our discussion of the logical empiricist Nancy Cartwright’s conception of cause demonstrated, it is logically quite consistent for those subscribing to a larger empiricist philosophy of science to be realist (or nonreductionist) about causes. One of the implications of the discussion in this article is therefore that one need not stray from an empiricist philosophy of science to seek ontological justifications for case studies and certain kinds of mixed method research designs. Thus though it does not in any way slight defenses of case studies on nonempiricist grounds—it does not preclude nonempiricist interpretations of the exemplary studies cited above, for instance—it implicitly questions the necessary association often made between empiricist ontology and statistical methods.