Causal Comparability,Causal Generalizations,and Epistemic Homogeneity

Abstract

The issue of causal comparability in the social sciences underlies matters of both generalization and extrapolation (or external validity). After critiquing two existing interpretations of comparability, due to Hitchcock and Hausman, I propose a distinction between ontological and epistemic comparability. While the former refers to whether two cases are actually comparable, the latter respects that in cases of incomplete information, we need to rely on whatever evidence we have of comparability. I argue, using a political science case study, that in those cases of imperfect information, an epistemic homogeneity criterion can be an adequate justification for generalization.

Keywords

causation comparability generalization heterogeneity social science methodology

1. Introduction

A key question in causal inquiry in the social sciences is causal comparability, that is, when we can fruitfully compare the causal relationship between a putative cause $X$ and effect $Y$ across different case studies. This question underlies matters of both internal validity and external validity. If we wish to evaluate a claim about the relation between $X$ and $Y$ as event types, that is, a general claim, then the extent to which the individual cases generalized over are causally comparable is key. Similarly, if we wish to extrapolate from the relation between $X$ and $Y$ in some test population to the relation between $X$ and $Y$ in some target population, then the causal comparability between individual cases in the population studied on the one hand and individual cases in the target population on the other is at stake.

Demanding causal comparability in the social sciences faces two main problems. First, arguably all instances of social phenomena have at least some causally relevant differences (cf. Kuorikoski 2012; Little 1991), and thus a population of cases under study may not contain similar enough individual cases to draw general conclusions. One may therefore argue that general conclusions in the social sciences are often unwarranted. The second issue with demanding causal comparability is that one almost never has a complete causal picture of the social phenomena under study (cf. Steel 2008), especially given the issues of multiple causation and INUS factors (cf. Cartwright and Hardie 2012). Without knowing what kinds of factors may interfere with $Y$ other than $X,$ it is difficult to see whether some value $X$ will have the same effect on the value of $Y$ for two cases.

Although we may take these two problems, the lack of true comparability and lack of evidence for comparability, as a reason to stick with single-case research and avoid generalization altogether, this solution is unsatisfactory for some purposes; causal comparability is required for any kind of generalization, and generalization in turn is important for both theorizing and policy purposes. A push to forego causal explanations at the population level altogether is incompatible with social scientific practice.

How, then, should we analyze causal comparability? The main focus in the debates on causal comparability centers on criteria for comparability themselves. Broadly speaking, there are two interpretations of what causal comparability amounts to. One derives from Christopher Hitchcock’s highly theoretical characterization of the relationship between singular and general causation, the “causal homogeneity condition,” and the other from a more pragmatic characterization of this relationship by Daniel Hausman, the “average effect condition.” However, as I will show in this paper, both interpretations are problematic in the social sciences. The former is too strong, because Hitchcock requires a strict similarity between different individual cases that is almost never met. The latter is too weak, because Hausman cannot adequately describe causal explanation in cases in which a population contains many causally dissimilar subpopulations.

This paper moves the causal comparability debate on by starting from our limited causal knowledge of social phenomena. I argue that Hitchcock and Hausman’s interpretations are more pertinent to different epistemic situations: in the former, we are aware of all the relevant probabilities for individual cases in our population of interest and so we have evidence for the stronger generalization, and in the latter, we are unaware of the specific probabilities and so we merely have evidence for a weaker generalization. I then take this analysis further. Following an older distinction by Wesley Salmon, I distinguish between ontological and epistemic comparability. The former has been the focus point of debates in causal comparability, internal validity, and external validity; ontological comparability refers to whether two cases are actually comparable. Epistemic comparability, on the other hand, has been relatively ignored.

While ontological comparability refers to whether two cases are actually comparable, epistemic comparability respects that in cases of incomplete information, we need to rely on whatever evidence we have of comparability. I argue, using a political science case study, that in those cases of imperfect information an epistemic homogeneity criterion can be an adequate justification for generalization. I carefully distinguish different types of epistemic comparability: first, depending on whether one is concerned with comparability relative to one causal factor or several (“relative” vs. “total” epistemic homogeneity), and second, depending on whether the assumption of homogeneity is due to practical constrains on time and effort, or due to a true lack of knowledge (“pragmatic” vs. “true” epistemic homogeneity). This paper thus provides a framework for describing how generalization and extrapolation in the social sciences actually happen. I illustrate my claims by critically analyzing Nicholas Sambanis’ theory on ethnic civil wars using my “epistemic homogeneity” account. A direct consequence of my new taxonomy is that one can fruitfully discuss the comparability of cases even in the scenario that individual cases in a sample are not entirely similar. Using the notion of epistemic homogeneity, I argue that we can generalize even in such scenarios.

2. Two Interpretations of Comparability

To introduce the first interpretation of comparability, the Hitchcock causal homogeneity condition, consider the general causal claim that “smoking causes cancer” for a set of individuals including both John and Mary (Hitchcock 1995). Hitchcock interprets this general causal claim as saying that the relation between smoking and cancer is sufficiently similar for John and Mary. In brief, he argues that when we know that every individual will respond in the same way to the causal variable being at a certain value, all other things being equal, then we can make a general causal claim. John and Mary, all other things being equal, are equally likely to develop lung cancer when they smoke a certain number of packs per day.

More formally speaking, for Hitchcock’s causal homogeneity condition, we need the following ingredients. Hitchcock constructs a “little probability space” {Ω _i , $ℱ_{i}, P_{i}}$ for every individual $i$ in the set we wish to generalize over, consisting of a sample space $Ω_{i}$ which contains all possible instances $X_{i} = x$ of the process being modeled (e.g., as we have seen, smoking $x$ packs per day), a set $ℱ_{i}$ which contains “events,” that is, combinations of outcomes from $Ω_{i}$ (e.g., as we have seen, smoking more than $x$ packs per day), and a probability function $P_{i} : ℱ_{i} \to [0, 1]$ to model the conditional probability of the effect $E_{i}$ of interest (e.g., the probability of contracting lung cancer) given an event from $ℱ_{i}$ . Moreover, $Ω_{i}$ can be partitioned into background contexts $G_{i}^{1}, G_{i}^{2}, . . .$ , which include other factors that may be relevant to $E_{i}$ besides $X_{i}$ . The “little probability space,” then, is a distribution function $f_{i α} (x) = P_{i} (E_{i} | X_{i} = x, G_{i}^{α})$ that tells us (for an individual $i \in I$ ) the probabilities of the effect $E_{i}$ given that the causal variable $X_{i}$ has value $x$ and given background context $G_{i}^{α}$ . For example, it tells us the probability that John contracts lung cancer given that he smokes $x$ packs per day and given his particular medical history (regular exercise), living arrangements (no asbestos), work hazards (working in a power plant), and other potential influential background factors. A general causal claim, under Hitchcock’s interpretation, can be made over the set of those individuals for which the little probability space is the same, that is, for those sets $I$ of which all elements $i, j$ have the same distribution, that is, $f_{i α} = f_{j α}$ . For instance, we can make a general causal claim about the set containing John and Mary if the probability each contracts lung cancer is the same, all other things being equal, if they both smoke $x$ packs per day.

The Hitchcock causal homogeneity condition is a very strong demand for social science generalization. First, it is difficult to find evidence that its requirements are met in any actual cases of interest; it will be difficult to calculate the individual probability spaces {Ω_i, $ℱ_{i}, P_{i}}$ in any real case, among others because we may not be able to ascertain what (the values of) the factors included in $G_{i}^{α}$ are. Second, even if we did manage to calculate the individual probability spaces for all the cases we wish to generalize over, there are often at least some known causally relevant differences between these cases. Moreover (and relatedly), we are hardly ever faced with two base units of analysis that have the same background conditions, that is, we do not actually observe both $f_{i α}$ and $f_{j α}$ but are more likely to observe $f_{i α}$ and $f_{j β}$ (cf. Cartwright and Efstathiou 2011, 232). Thus, this interpretation of comparability is unhelpful in social scientific practice. This is the argument made among others by Daniel Hausman (2010). Hausman argues that the causal homogeneity condition (which he dubs the “contextual unanimity condition”), is often not what is intended when we make general causal statements, and thus that we need a different characterization of the relationship between singular and general causal claims.

Hausman shows that a general claim such as “smoking causes lung cancer” will be technically false if we interpret it, as Hitchcock does, as saying that the probability of lung cancer increases for every individual who smokes, all other things being equal. As Hausman points out, there are individuals whose smoking leads to a fatal heart attack before they can even develop lung cancer; these individuals do not fit with the general causal claim as defined by Hitchcock. After all, their background context will be different from the background context of individuals who do not develop a fatal heart attack. Call the set of all individuals whose smoking leads to a fatal heart attack before they develop lung cancer $I$ , and call the set of all other individuals $J .$ Now, define for each $i \in I$ the distribution function $f_{i α} (x) = P_{i} (E_{i} | X_{i} = x, G_{i}^{α})$ as before, that is, $f_{i α} (x)$ is the probability distribution for individual $i$ developing lung cancer given that $i$ smokes $x$ packs per day under background conditions $G_{i}^{α}$ . Define for each $j \in J$ the distribution function $f_{j α}$ in the same way. We will conclude, since all individuals in $I$ die before they develop lung cancer, that $f_{i α} \neq f_{j α}$ . Thus, we cannot generalize over the set $I \cup J$ along the lines that Hitchcock presents.

This seems counterintuitive if, despite some individuals dying of heart attacks first, we still want to say that smoking causes lung cancer. For that reason, Hausman argues for his alternative characterization of the relation between singular and general causal claims, the average causal effect. Using the same notation as before, under this characterization, in population $I$ , $X = a$ as compared to $X = b$ causes $E$ is true (e.g., smoking one pack per day as compared to smoking no packs per day causes lung cancer is true) if and only if two demands are met. First, $P (E | X = a) > P (E | X = b)$ . Second, the probability difference in the first demand (the fact that for this population, $P (E | X = a) > P (E | X = b)$ ) must be due to the causal influence of $X = a$ as compared to $X = b$ in some causally homogeneous circumstance occupied by members of the population $I$ . This second demand is meant to exclude cases where the relationship between $E$ and $X = a$ is spurious.

Note that this interpretation of generalization does not ask us about any individual conditional probabilities for the $i \in I$ ; it simply asks us to compare the average probabilities (which can be calculated using, for instance, observed correlations for population $I$ , collecting the frequencies with which $E$ occurs in those members of the population for which $X = a$ , and for those for which $X = b$ ). This fits with one of the motivations for formulating this particular characterization of generalization in the first place: it is useful for those cases where we simply cannot sum over the effects under causally homogeneous background conditions. Secondly, the average effect condition will be of use when we are unaware of the different causally homogeneous background conditions, or unaware of which background condition any one individual belongs to.¹

Although the average effect condition can be useful in those practical contexts, there are problems with this characterization of comparability. If there are many different subpopulations of $I$ , for each of which the relationship between values of $X$ and some effect $E$ differs significantly, then the average effect in $I$ can be misleading. Consider, for instance, a test population in which one type of individual shows strong negative side effects from some medication, and another type of individual does not show any side effects. If we are unaware that there exist two types of person, we may be misled by the average effect to think that the medication only has some moderate negative side effects. Hausman (2010, 59) therefore admits that “if the consequences of $X$ for $E$ differ appreciably over different subpopulations, then it can be harmful to generalize over the whole population. It is usually better to generalize concerning the narrowest populations for which the information is available.”

3. Epistemic Homogeneity Defined

There exist, then, two different interpretations of comparability, each of which has its issues. One way of construing this difference is to say that the two interpretations are more pertinent to different epistemic situations: in the one, we are aware of all the relevant probabilities for individual cases in our population of interest and so we have evidence for the stronger generalization, and in the other, we are unaware of the specific probabilities and so we merely have evidence for a weaker generalization.²

Synthesizing the two interpretations, in the rest of this paper, I will argue for distinguishing epistemic homogeneity and ontological homogeneity. The latter has been the focus point of debates in causal comparability, internal validity, and external validity; the former, on the other hand, has been relatively ignored. In the second half of this paper, I show how epistemic comparability respects that in some situations we may not know the full story and thus need to rely on whatever evidence we have of comparability, and I carefully distinguish different types of epistemic comparability. I illustrate my claims by critically analyzing Nicholas Sambanis’ theory on ethnic civil wars using my “epistemic homogeneity” account. This paper thus provides a framework for describing how generalization and extrapolation in the social sciences actually happen.

3.1. Wesley Salmon’s Reference Class Rule

The notion of epistemic homogeneity that I will defend here is based on an earlier argument by Wesley Salmon. Salmon first proposed the distinction between objectively and epistemically homogeneous reference classes in an attempt to solve the reference class problem for probabilistic explanation (Salmon 1971). After introducing Salmon’s argument, I will use it to give an account of causal inquiry.

The reference class problem is the issue that calculating a probability of an event requires us to specify a reference class for that event, while there is no straightforward “correct” reference class but rather a variety, each of which gives a different probability. For example, to calculate the probability that John will die of a heart attack, we must know what reference class to use: the class of all Caucasian men? All middle-class people? All professional tennis players?

Salmon solved the reference class problem by arguing for the use of the “broadest homogeneous reference class.” Let me first introduce the concept of a homogeneous reference class with an example. Assume we are investigating a class of events or phenomena, $A$ , and are interested in the frequency with which some property $B$ occurs in that class. For instance, assume we are investigating the class of all English sixth-form pupils, and are interested in the frequency of A* grades among them. Class $A$ is called a “homogeneous reference class” for property $B$ if there is not a property $C$ according to which we can partition the class further such that the probability of having property $B$ is different in the resulting subclasses (i.e., if it is not the case that $P (B | C, A) \neq P (B | A)$ )). (Any partition that refers to property $B$ will always be a partition that results in a different frequency of $B$ ; that is why we exclude such partitions.) So, for instance, if there is no way of dividing up the class of all English sixth-form pupils, so that in one subclass the frequency of A* grades is significantly different than in the other subclass, then we call the class of all English sixth-form pupils homogeneous with respect to getting A* grades. (In reality, of course, the class of all English sixth-form pupils is not homogeneous with respect to getting A* grades; for instance, we know students from underprivileged backgrounds typically do worse than students from other backgrounds.)

Consider a few more examples. The class of all pregnancies is not homogeneous for the property of babies born with physical, developmental, and functional problems, because we can further divide the class according to the place selection that picks out mothers who drink more than two glasses of wine per day. The class of university graduates is not homogeneous for the property of income, because we can further divide the class according to the place selection of gender.

A first thing to note is that, at least in the university graduates example, we could also partition the class with other place selections, for example, in this case according to social class. It is not always clear which of these classes we should work with, which is why Salmon argues for investigating “the broadest homogeneous reference class to which the single event belongs” (Salmon 1971, 43), that is, the homogeneous class with the most members. For Noa, a female graduate from a working-class background, this homogeneous class will be the class of all female working-class graduates. For Clarence, a male graduate from an upper-class background, this homogeneous class will be the class of all male, upper-class graduates. Note that if it were to later turn out that the highest level of education attained by Noa and Clarence’s parents also matters to Noa and Clarence’s income, this property would have to be included in the delineation of their homogeneous class, and thus we would have to narrow the homogeneous classes further.

A second thing I wish to stress is that in Salmon’s framework, a class $A$ is never simply homogeneous—it is homogeneous with reference to a particular property under investigation. This means that a class that is homogeneous with reference to one particular property $B$ may not be homogeneous with reference to another property $C$ . Thus, whereas a theory concerned with property $B$ may have scope conditions $a_{1}$ (may only apply to entities with property $a_{1}$ ), a theory concerned with property $C$ may have scope conditions $a_{2}$ (may only apply to entities with property $a_{2}$ ). For instance, if true, the democratic peace theory (the theory that democracies are hesitant to go to war with each other) has all (pairs of) democracies as its scope, since it claims this class of democracies is homogeneous with respect to the probability of war onset. For other theories, however, we will be forced to partition the class of democracies and thus give the theory a more narrow scope; for instance, a political scientist interested in the causes of party polarization may wish to distinguish multi-party democracies from two-party democracies.

Problematically, in the social sciences, researchers are often unaware of the full causal picture, that is, which properties are relevant for the cause and effect under discussion, and which are not. Salmon’s suggestion, in those cases of “incomplete information,” is as follows: “[w]hen we know or suspect that a reference class is not homogeneous, but we do not know how to make any statistically relevant partition, we may say that the reference class is epistemically homogeneous” (Salmon 1971, 44).

How does this solution to a problem for probabilistic explanation relate to the interpretations of generalization I have outlined in Section 2? As argued there, the two interpretations are meant to apply to different epistemic situations. In Hitchcock’s interpretation, we are aware of all the relevant probabilities for individual cases in our population of interest, and in Hausman’s interpretation, we are either unaware of, or choose to ignore,³ the specific probabilities. If we take Hausman’s interpretation as a suggestion to look for epistemic homogeneity rather than ontological homogeneity, that is, to assume homogeneity until proven otherwise, we will be able to make epistemic progress regarding a set of cases.

3.2. Epistemic Causal Homogeneity in a Formal Framework

Let me return to the framework from Section 2, in order to clarify what epistemic homogeneity would amount to there. A general causal claim, under Hitchcock’s interpretation, can be made over the set of those individuals for which the little probability space is the same, that is, for those sets $I$ of which all elements $i, j$ have the same distribution, that is, $f_{i α} = f_{j α}$ . In cases when not all elements $i, j$ of the set $I$ we are interested in have the same distribution function, the scope of the population $I$ we generalize over must be restricted. In the simplest case, there is one particular property $C$ which has led to the problem. Now, following Salmon’s terminology, it is this property $C$ that we could use to partition $I$ into two homogeneous classes. The result will be two subsets of $I$ , $I_{1} \subset I$ , and $I_{2} \subset I$ (with, seeing as this is a partition, $I_{2} = I \ I_{1}$ , that is, $I_{2} = {x \in I | x \notin I_{1}}$ ), which each in turn are causally homogeneous but for which it is the case that for every $i_{1} \in I_{1}$ and for every $i_{2} \in I_{2},$ $f_{i_{1}} \neq f_{i_{2}}$ .

So far this framework cannot deal with cases of incomplete information. This is a problem because researchers are hardly ever in possession of all relevant information about the distribution functions $f_{i α}$ . Researchers do not know which properties of an individual $i \in I$ might disrupt the causal relation between $X_{i}$ and $E_{i}$ , nor which properties might support it. We may, in those cases, not wish to bet that the strict generalization is true; we may prefer to instead adopt a weaker notion of generalization of which we are relatively sure it is true in cases of interest.

So, in general, if researchers have incomplete information about the causal structure of the area they are investigating, they might not be able to come up with a partitioning $C$ . It is exactly this “incompleteness” that Salmon captures with his notion of epistemic homogeneity. As long as they do not have a partitioning $C$ that they know will show that the class $I$ is heterogeneous with respect to the causal relation of interest, researchers ought to call that $I$ epistemically homogeneous. For that set $I$ we can use the Hausman average effect condition to make general claims, since it does not require further information about the individuals in that set; once we find more information, we can then (following Hitchcock) narrow down the set further. This synthesis between the two frameworks for generalization, I believe, is key to making epistemic progress in causal inquiry in the social sciences.

4. Does Causal Modeling Trivialize the Epistemic–Ontological Distinction?

Before I illustrate the distinction between ontological and epistemic homogeneity with a case study, I wish to clarify one potential misunderstanding. So far, I have argued that we are hardly ever aware of all the relevant probabilities for individual cases in our population of interest, but may not wish to settle for a weaker generalization that merely averages out over the probabilities in these individual cases. One response might be that causal modeling approaches can help us find all relevant variables for a particular causal structure (cf. Pearl 2000; Spirtes, Glymour, and Scheines 1993; Woodward 2003), and that as such the distinction between ontological and epistemic homogeneity is relatively trivial. In this section, I will show why this is an oversimplification by discussing a limitation of causal modeling that is closely related to the issue of causal homogeneity, focusing on Spirtes, Glymour, and Scheines’ account (cf. Spirtes, Glymour, and Scheines 1993). I will also briefly rebut a closely related objection, namely, that econometric methods that uncover heterogeneity in populations trivialize the distinction between ontological and epistemic homogeneity.

Key within Spirtes, Glymour and Scheines’ causal modeling technique is the notion of a causal search. A causal search is, in simple terms, a computer algorithm that takes statistical data (including a list of all measured variables) as its input and produces an equivalence class of all the causal structures that fit these data as its output. Such structures can then, in turn, be used to answer predictive questions; for instance, we may ask what would happen to a particular variable in the causal structure under an intervention. Moreover, and worth noting given the discussions in this paper, a causal search can show whether a data set is compatible with the existence of one or more unspecified, latent common causes. As such it can, in some cases, tell us whether there is or is not enough information in the data to answer causal questions.

This, I believe, is where a potential misunderstanding could arise. A critic may argue that Spirtes, Glymour, and Scheines’ techniques of causal modeling trivialize the epistemic–ontological distinction, because they allow us to detect hidden variables. Thus, the techniques allow us to break down the epistemic boundary between the variables we know might have an effect, and the ones we do not know about, but actually do have an effect. However, this response is an oversimplification of the issues at hand. Without going too far beyond the scope of this paper, I wish to evidence my response briefly by describing particular data sets that a causal search has difficulty with, and which arguably illustrate why causal modeling does not (yet) trivialize the problems of comparability in this paper.

In their 2004 paper “Causal Inference of Ambiguous Manipulations” (Spirtes and Scheines 2004), Spirtes and Scheines draw attention to an issue that could arise when one uses “defined variables” in a causal search. In some of those cases, one’s causal search includes measurements that define a variable as homogeneous while in fact (without our knowledge) it consists of two subvariables which have heterogeneous effects under manipulation. Manipulating one of the subvariables will give a different result than manipulating the other will; therefore, we call such manipulations “ambiguous” (Spirtes and Scheines 2004, 834).

Spirtes and Scheines give the example of the variable “total cholesterol” (call this $X$ ) and the variable “risk of heart disease” (call this $Y$ ). $X$ is an ambiguous variable, since it consists of the two subvariables “level of high-density lipoprotein” (HDL) and “level of low-density lipoprotein” (LDL). This case is illustrative because the difference in causal relevance for heart disease between HDL and LDL went unrecognized for a number of years. It turns out that while HDL decreases the risk of heart disease, LDL increases the risk. For that reason, the manipulation of $X$ is ambiguous; lowering total cholesterol by lowering HDL will increase the risk of heart disease, while lowering LDL will decrease the risk. One can show that if one is unaware of the existence of these two subvariables, and treats total cholesterol as a homogeneous variable, the equivalence class of causal structures that the causal search produces will be agnostic about the relation between heart disease and total cholesterol (Spirtes and Scheines 2004; see also Spirtes and Scheines 2003). For some individuals in the data set, decreasing total cholesterol will have a positive effect on the risk of heart disease, while for other individuals, it will have a negative effect. This means that two causal structures are compatible with the data: one in which there is a positive causal relation and one in which there is an inhibitive causal relation. As such, the causal search’s answer to a question “What is the effect of increasing total cholesterol on the risk of heart disease?” will be “can’t tell” (i.e., the relation between total cholesterol and the risk of heart disease in its equivalence class of causal structures will be undecided).

Given these difficulties with defined variables, I believe it is an oversimplification to say that causal modeling techniques trivialize the question of comparability and homogeneity in this paper. Causal modeling will struggle with ambiguous defined variables, which abound in social science. Think, for one, of the concept “democratization.” There are many different kinds of democratization and so if, like in the LDL/HDL example, states undergoing certain types of democratization behave differently than states undergoing other types of democratization, it is highly likely that a manipulation of democratization as a total category will lead to many “can’t tells.” As such, we need the distinction between ontological and epistemic homogeneity to accurately describe the issues faced when building general theories in the social sciences.⁴

In a similar vein, a critic may argue that recent developments in econometric methods trivialize the distinction between ontological and epistemic homogeneity. In econometric terms, this paper describes a taxonomy of how one may describe (our knowledge of) populations in light of unknown independent variables, and how one can taxonomize populations in the social sciences even in light of the inherent variability between subpopulations. A critic may argue that some econometric methods (such as regression analysis) do just that; specifically, some methods are specifically attuned to find out the potential effects of any unknown variables by systematizing these variables as error terms. These methods tell us the ceteris paribus effect of the systematized independent variables of interest on the dependent variables, exactly as Hitchcock requires for his causal homogeneity condition.

However, arguably such econometric methods do not trivialize the distinction between ontological and epistemic homogeneity but rather underline it. Information about a sample and its statistics, including the ceteris paribus effect of independent variables on dependent variables, should be described in terms of epistemic homogeneity. This information should not be confused with a full causal picture of the population and its parameters, which we can describe in terms of ontological homogeneity.

Cases of selection bias provide a relevant illustration of when a sample and its statistics on the one hand differ from a population and its parameters on the other. In these cases, we are concerned with the possibility that the differences within a sample at hand damage a study’s internal validity, that is, the degree to which a causal conclusion is warranted for the entire sample studied. Although one can limit the influence of selection bias using methods such as instrumental variables and Heckman corrections (cf. Antonakis et al. 2010), these methods rely on specific assumptions about the data under scrutiny and thus we cannot always implement them. For some econometric models suffering from selection bias, “it is difficult to anticipate whether the biased regression estimates overstate or understate the true causal effects” (Berk 1983, 390; see also Xie 2011). Here, we can fruitfully employ the distinction between ontological and epistemic homogeneity. If we can control for selection bias, we make epistemic progress by further narrowing down the reference classes under study. If we cannot, we are in a situation of epistemic, and not ontological homogeneity.

To rearticulate the rebuttals in this section, this paper does not aim to comment on econometric solutions, nor on causal modeling approaches, but rather aims to provide a more foundational analysis, that is, of how one can describe inherently variable populations using the concepts of ontological versus epistemic homogeneity. We can then use this analysis to describe any epistemic progress made by econometric methods or causal models.

5. Example: Civil War Studies’ Search for Epistemic Homogeneity

In this section, I wish to discuss an example of the search for causal homogeneity, namely, the move from a general theory on civil war onset to a more specific theory on ethnic civil war onset by Nicholas Sambanis. I will show that the class of states at civil war was an epistemically homogeneous class with respect to several properties, including economic and political factors, until Nicholas Sambanis figured out a way to make a statistically relevant partition in the class, namely, between ethnic and non-ethnic civil wars.⁵ I will make clear what the results of the partitioning of this class were, and thereby illustrate the notions of epistemic homogeneity and epistemic progress discussed above.

5.1. Civil Wars as an Epistemically Homogeneous Class

In 2001, Nicholas Sambanis asked whether ethnic and non-ethnic civil wars start for the same reasons. The ordinary theories in the civil war literature at that time assumed so: the then prominent “economic” theory of Collier and Hoeffler (2004) aggregated civil wars since 1960 into one class⁶ and suggested that such wars start mainly because of economic factors (such as financial incentives for the rebels) rather than political factors (such as the level of democracy of the country and of its neighboring countries).

In the terminology introduced in the previous section, one may say that Collier and Hoeffler treated the class of all civil war onsets as an epistemically homogeneous reference class with respect to the properties under investigation (i.e., with respect to the potential causes). Sambanis’ contribution to the civil war literature was suggesting that there might exist a relevant partition, “ethnicity,” which divides the class of civil war onsets into causally dissimilar subgroups. In this section, I will highlight the differences between ethnic and non-ethnic civil wars in Sambanis’ framework. I briefly outline the aggregate theory of civil wars as presented by Collier and Hoeffler, and then discuss the properties relative to which Sambanis believes there is a partitioning which shows the ethnic heterogeneity of the class of all civil wars.

Collier and Hoeffler and Sambanis define “civil war” in a similar manner, based on the definition used by the Correlates of War database, one of the most commonly used sources for data in quantitative studies of war. Collier and Hoeffler do not distinguish between different types of civil war, but instead consider the class of civil wars as causally homogeneous. They discuss this decision in later writing, noting that although some data sets define conflicts in terms of the underlying issues (as is the case when scholars make a distinction between ethnic and non-ethnic civil wars) they have decided not to do so because “the classification of conflicts according to their causes does not seem helpful . . . if we want to analyze the causes of civil war” (Collier and Hoeffler 2001, 5).

In terms of the interpretation outlined in the previous section, Collier and Hoeffler consider the class of all civil wars, $A$ , and search for the frequency with which property $B$ , “ethnic fragmentation” (a potential causal variable), occurs in that class (which is then compared with the frequency with which property $B$ occurs in a similar class where civil war did not break out). According to Collier and Hoeffler, ethnic fragmentation is a measurable proxy for the “coordination costs” of a rebellion. Higher coordination costs, they argue, are an inhibiting cause of civil war onset. Ethnic fragmentation is also related to civil war onset via a different route: according to Collier and Hoeffler, ethnic dominance leads to minority oppression, which in turn increases the risk of civil war onset.

5.2. Partitioning Civil Wars into Ethnic and Non-Ethnic Wars Relative to Ethnic Heterogeneity

In contrast, Nicholas Sambanis (2001) argues for partitioning $A$ , the class of civil wars, in relation to property $B$ , “ethnic fragmentation,” according to the place selection $C$ of “ethnicity.” Having thus partitioned the class of civil wars into ethnic and non-ethnic civil wars (the definition of which I will outline below), Sambanis uses statistical methods to test the hypothesis that the probability of ethnic war breaking out is an increasing function of the degree of ethnic fragmentation.⁷ He finds that ethnic fragmentation is not significant in any of the regressions which treat civil war as a homogeneous class, including the regressions in Collier and Hoeffler’s study. After partitioning, however, the numbers look significantly different. Ethnic fragmentation “is among the most robustly significant variables and is positively correlated with the onset of ethnic war: as a country becomes more heterogeneous, the probability of occurrence of an ethnic war increases” (Sambanis 2001, 273). However, for non-ethnic war onset, ethnic fragmentation is no longer statistically significant.

Sambanis defines ethnicity following earlier work by Donald L. Horowitz (1985), in which an ethnic group can be defined in terms of anything from “color, appearance, language, religion, some other indicator of common origin, or some combination thereof” (Donald L. Horowitz 1985, 17-18) and covers other terms like “tribes, races, nationalities, and castes” (Donald L. Horowitz 1985, 53). A civil war is an ethnic civil war, Sambanis argues, if the core issues in the conflict are “integral to the concept of ethnicity” (Sambanis 2001, 261-62). Or, alternatively, an ethnic war is a “war among communities (ethnicities) that are in conflict over the power relationship that exists between those communities and the state” (Sambanis 2001, 261). He codes a conflict as an ethnic civil war if it is a civil war (for exact requirements, see Sambanis 2001, 262) and if it counts as an episode “of violent conflict between governments and national, ethnic, religious, or other communal minorities (ethnic challengers) in which the challengers seek major changes in their status” (Sambanis 2001, 262).

In light of the discussion of Salmon’s reference class rule, whether one can call “ethnicity” a proper partitioning for the class of civil wars in relation to the property of ethnic fragmentation is dubious. At first glance, one may suspect that taking “ethnicity” as a place selection breaks the rule that a partition of class $A$ is only a place selection if it does not refer to the property $B$ under investigation; or, to repeat Collier and Hoeffler’s words, we cannot classify conflicts “according to their causes . . . if we want to analyze the causes of civil war” (Collier and Hoeffler 2001, 5).

On the other hand, ethnic fragmentation refers to the number of different ethnic groups within a state. Thus, it is not the case that the partition into “ethnic” and “non-ethnic” civil wars refers to the property of “ethnic fragmentation.” As Sambanis puts it, “not all wars that involve ethnic groups as combatants should be classified as ethnic wars. The issues at the core of the conflict must be integral to the concept of ethnicity” (Sambanis 2001, 261-62). A country can in theory have a low degree of ethnic fragmentation and still be at ethnic civil war; two ethnicities is all it takes. And, vice versa, a country can have a high degree of ethnic fragmentation without an ethnic civil war being fought there—and if there is a civil war being fought in the country, it may have started for different reasons.⁸ For those reasons, one might argue that “ethnicity” is a proper partitioning, despite first appearances to the contrary.

However, the correlation between ethnic heterogeneity and ethnic civil war is arguably not unexpected given that both are defined in terms of ethnicity; they might not be interchangeable but they are closely related. Unless we can show the conceptual independence between the two, it is not useful to make a distinction between ethnic and non-ethnic civil wars merely on the basis that they have a different causal connection to the property ethnic fragmentation. What Sambanis needs to show, I would argue, is that ethnic wars are different from non-ethnic wars in a way that goes beyond their causal history of ethnic fragmentation.

Though it is not immediately obvious that Sambanis’ distinction between ethnic and non-ethnic wars is a proper partitioning for the property of “ethnic fragmentation,” there are other properties that Sambanis investigates. He shows that ethnic and non-ethnic civil wars also differ in relation to those other properties, that is, the class of “civil wars” is heterogeneous relative to other properties besides the (dubious) ethnic heterogeneity. Sambanis considers several such properties, including political variables such as the polity score of the country, the polity score of its neighboring countries, and economic variables such as real per capita income. He shows that while the polity score of a country is statistically significant for ethnic war onset,⁹ its polity is non-significant to non-ethnic war onset (Sambanis 2001, 276). Moreover, real per capita income is more significant to non-ethnic war onset than it is to ethnic war onset, leading Sambanis to speculate that economic variables are a more important causal factor in non-ethnic war onset than in ethnic war onset. And indeed, Sambanis concludes that “[ethnic] wars are predominantly caused by political grievance, and they are unlikely to occur in politically free (i.e. democratic) societies” (Sambanis 2001, 280).

5.3. Lessons Taken Forward

So, I have shown in this section that civil war onset was treated as a homogeneous class, despite indications to the contrary from other areas of the civil war literature. Sambanis showed that for the property of ethnic fragmentation, the class could be fruitfully partitioned into two causally dissimilar subgroups, that is, ethnic and non-ethnic civil wars. I have analyzed to what extent this partitioning is a proper partitioning. I have also shown that Sambanis’ partitioning into ethnic and non-ethnic civil wars was relevant for other properties which are more straightforwardly independent of ethnicity, that is, political and economic factors. If we accept Sambanis’ statistical results, we must also accept that general theories of the causes of civil war cannot have all civil wars as their scope; the scope conditions had to be limited to ethnic civil wars or non-ethnic civil wars (not both).¹⁰

This case study of Sambanis can be taken to reiterate that we must check for the independence of the place selection and property, a requirement I introduced in my analysis of Salmon’s reference class rule in Section 3.1. But the case study also highlights a relevant distinction in the notion of epistemic homogeneity: whether we consider epistemic homogeneity relative to one particular property, or total epistemic homogeneity. In the former case, researchers are merely interested in a particular cause, and in the latter, researchers aim to find a complete causal picture of a particular social phenomenon. I will discuss this distinction in more detail below.

6. Further Refinements of the Notion of Epistemic Homogeneity

6.1. Classes That Are Epistemically Homogeneous Relative to More Than One Variable

So far, I have argued that if we do not know what the right partitions in a class are to show causally relevant differences, then we may call this class epistemically homogeneous. We might say that things are equal until proven different.¹¹ Without such a ruling, any policy that requires some assumption about the probabilities of all cases under its scope will be incomplete. As already indicated when I discussed Hausman’s average effect condition, assuming a population is homogeneous will have more serious consequences if there turn out to be large causally relevant differences in subpopulations; if, for instance, territorial ethnic civil wars respond quite differently to international interventions than non-territorial ethnic civil wars do, then the average effect of international interventions for the class of all ethnic civil wars will be a misleading source of information for anyone deciding whether or not to intervene in an individual conflict.

In the case study above, I found that there are cases when a researcher does not simply investigate the homogeneity of a population relative to one particular variable. There are instances when researchers are not simply interested in the relationship between one property and the class under consideration (as when we try to investigate whether the polity score of a country is a [contributing] cause of civil war). Instead, researchers may wish to link a whole list of properties to the class under consideration (as when we are interested in a complete causal picture of civil war). Sambanis, in his case study, looks at more than just ethnic fragmentation; he concludes that there are “statistically significant differences between the means of core variables (e.g. political rights, ethnic heterogeneity, and war duration) sorted by war type” (Sambanis 2001, 272). This leads him to argue for researching the differences between ethnic and non-ethnic wars in more detail. We may call the sort of homogeneity we are looking for when it comes to a whole list of properties “total causal homogeneity,” as opposed to the “relative causal homogeneity” researchers are looking for when they only consider one (potentially causal) property.

As anticipated in the introduction, a critic might respond that there are no “social kinds,” that is, there are reasons to believe that all social science cases have causally relevant differences (cf. Little 1991). This means that total ontological causal homogeneity is far too strong a requirement for social science. Nevertheless, I would argue, one can fruitfully employ a notion of total epistemic causal homogeneity. In other words, though after comparative research it will always turn out that all homogeneous classes have size one, that is, contain only one element, researchers may work with what they believe to be a completely causally homogeneous class until they find a relevant partition.¹²

Consider the following formalization of this search for total causal homogeneity. Looking for total ontological causal homogeneity would involve the following. Call $X_{i} = {X_{i}^{1}, X_{i}^{2}, \dots, X_{i}^{n}}$ a set of variables about which we wonder whether they are causally related to an effect of interest $E_{i}$ for each individual $i$ in a population $I$ , the further properties of which we describe by a background context $G_{i}^{α}$ (defined before as a subset of $Ω_{i}$ ). Now index the little probability space for any $i \in I$ not only to the individual $i$ , but also to the causal variable $X_{i}^{k}$ of interest, as such: $f_{i}^{k} : {x}_{k} \to P_{i}$ , with $f_{i}^{k} = P_{i} (E_{i} | X_{i}^{k} = x, G_{i}^{α})$ . A set $I$ of individuals will then have total ontological causal homogeneity if for any $i, j \in I$ , for all $X_{i}^{k}, X_{j}^{k}$ , it is the case that $f_{i}^{k} = f_{j}^{k}$ . If this total ontological causal homogeneity condition is not met, in the simplest case, we will be able to find a partition $C$ of the population $I$ into two subpopulations, $I_{1}$ and $I_{2}$ , such that for all $i_{1}, j_{1} \in I_{1}$ , for all $X_{i_{1}}^{k}, X_{j_{1}}^{k}$ , it is the case that $f_{i_{1}}^{k} = f_{j_{1}}^{k}$ , but this is not the case for any $i_{2} \in I_{2},$ that is, $f_{i_{2}} \neq f_{i_{1}}$ .

Total epistemic causal homogeneity, then, is the weaker notion of generalization that we adopt when we do not have sufficient information to formulate such partitions $C$ . In doing so, however, we must respect that any stronger notion of generalization will be inapplicable in the social sciences, since as stated above, all totally homogeneous classes in the social sciences have size one.

Table 1 summarizes the distinction between total and relative homogeneity on the one hand, and epistemic and ontological homogeneity on the other.

Table 1.

Distinguishing total and relative, and epistemic and ontological homogeneity.

	A partitioning . . .
	. . . relative to one variable . . .		. . . relative to a list of variables . . .
	. . . exists	. . . does not exist	. . . exists	. . .does not exist
We know	Relative ontological heterogeneity	Relative ontological homogeneity	Total ontological heterogeneity	Total ontological homogeneity
We do not know	Relative epistemic homogeneity—e.g., the democratic peace theory states that the class of pairs of democracies is homogeneous with respect to the probability of war onset		Total epistemic homogeneity—e.g., as far as we know, the class of ethnic civil wars is homogeneous with respect to a range of causes, like the country’s polity score and its real per capita income

6.2. Pragmatic Epistemic Homogeneity

The criticism that there always exist causally relevant differences between social phenomena now brings me to another way in which we can taxonomize epistemic causal homogeneity. Although one may agree that there are always going to be causally relevant differences between cases, this has so far not stopped social scientists from attempting to generalize, whether relative to one particular property (in the case of epistemic causal homogeneity relative to a property) or relative to all properties (in the case of complete epistemic causal homogeneity). In this section, I will highlight what reasons researchers can have to generalize despite these relevant differences, and distinguish between “true epistemic causal homogeneity” (when researchers truly cannot think of a relevant partitioning of a class) and “pragmatic epistemic causal homogeneity.” Researchers are working with the latter when they know particular partitionings of the class exist, but treat the class as homogeneous anyway for pragmatic reasons (including, for instance, not having the time, energy, or resources to adjust policy to any subclasses).¹³ In the latter case, the causal claims one makes about a class are not tracking the true causal structure of the world, or even what we believe may be the true causal structure given epistemic limitations, but such claims are nonetheless useful simplifications.

To highlight these two constraints on homogeneity considerations, consider the following example. If we wish to know whether there is a type-level causal relation between students’ ethnicity and their educational attainment, we must measure the relation between the two (properly systematized) variables for a particular population: for example, we may calculate the correlation between ethnicity and educational attainment for all Key Stage 4 pupils in England.

Now, crucially, assuming that each ethnic group in English Key Stage 4 pupils is causally homogeneous with respect to educational attainment will rely on two considerations. First, as I have argued in the first part of this paper, such an assumption will depend on whether researchers have any evidence to the contrary (whether there exists a place selection that would show there exist causally heterogeneous subgroups within at least one ethnic group of English Key Stage 4 pupils with relation to the property of educational attainment; think of, for example, gender).

Second, which group researchers treat as causally homogeneous will depend on the purpose for which they seek the causally homogeneous group. To name but a few aim-related considerations, which group researchers treat as causally homogeneous will depend on whether they can afford (the time, energy, finances) or risk to generalize over causally heterogeneous subgroups. Do they have the time, energy, or finances, say, to research educational attainment for each ethnic subgroup? What would they risk if they did not, and what might the consequences be? Moreover, even if they have good reasons to believe there exist causally relevant differences between some subgroups, do the researchers have the time, energy, or finances to treat each subgroup differently?

The distinction between true and pragmatic causal homogeneity is also related to the following distinction. Generalization always amounts to describing properties of some population. On some occasions, all one cares about are facts about the population at large; in that case, averaging out over subpopulations we know to be heterogeneous is acceptable, that is, we can work under pragmatic epistemic homogeneity. On other occasions, one cares about painting as accurate a picture as possible for each individual in the population, and not just for the population at large. In that case, averaging out is unacceptable; we will not work under pragmatic epistemic homogeneity.

The distinction between true and pragmatic causal homogeneity completes my taxonomy of the different kinds of homogeneity. We can thus expand Table 1 to include this distinction, into Table 2.

Table 2.

Distinguishing pragmatic and true homogeneity.

	A partitioning . . .
	. . . relative to one variable . . .		. . . relative to a list of variables. . .
	. . . exists	. . . does not exist	. . . exists	. . . does not exist
We know	Relative ontological heterogeneity	Relative ontological homogeneity	Total ontological heterogeneity	Total ontological homogeneity
We know, but choose to ignore it	Pragmatic relative epistemic homogeneity—e.g., the class of white working-class students is heterogeneous with respect to attainment, because we could partition according to gender, but we ignore this in the classroom and treat both genders equally	(empty)	Pragmatic total epistemic homogeneity—e.g., the male U.K. population is heterogeneous with respect to a range of effects, like the probability of developing obesity and the probability of heart attacks (we could partition according to family history), but the National Health Service chooses to give every man the same health advice	(empty)
We do not know	True relative epistemic homogeneity—e.g., the democratic peace theory states that the class of pairs of democracies is homogeneous with respect to the probability of war onset		True total epistemic homogeneity—e.g., as far as we know, the class of ethnic civil wars is homogeneous with respect to a range of causes, like the country’s polity score and its real per capita income

The distinction between true and pragmatic epistemic homogeneity raises a few questions. First, we may ask whether true epistemic causal homogeneity can still be called epistemic (as opposed to ontological). I would argue that it can; it is a description of the state of our knowledge about the causal structure of a particular class of units, and not a description of the state of the world. As Table 2 illustrates, a class can be ontologically heterogeneous yet epistemically homogeneous. Second, we may ask whether there is any reason to use the term true epistemic causal homogeneity given that, as stated above, among others Daniel Little (1991) has convincingly argued there are no social kinds: there are always causally relevant differences between units researchers aim to generalize over. I would argue there are reasons to use the term: it describes the state of our current knowledge. To make a distinction between true epistemic causal homogeneity and pragmatic epistemic causal homogeneity allows us to highlight the difference between not knowing a way to subdivide a group into idiosyncratic individuals, and knowing such a subdivision exists, yet not acting on it.

7. Conclusion

In the above, I have argued that ontological causal homogeneity is too strong a requirement for comparability, and thus for building general theories. It requires one to have knowledge about all the events or phenomena in one’s class of interest, which does not reflect scientific practice. I defended one particular way of weakening the notion of homogeneity so as to better understand the process of generalizations in social science, that is, making a distinction between ontological and epistemic homogeneity. I showed how the notion of epistemic homogeneity can bring us closer to accounting for what constitutes an adequate justification for the scope conditions of a general theory, and illustrated this with the development of Nicholas Sambanis’ theory on ethnic civil wars. I then refined the notion of epistemic homogeneity by showing one can delineate a class of events or phenomena according to one property (is this class of pupils homogeneous with reference to their educational attainment?) or according to all properties (is there class of civil wars homogeneous with reference to all causally relevant variables?), that is, I distinguished between relative epistemic homogeneity and total epistemic homogeneity. Moreover, I discussed and illustrated a further distinction between pragmatic epistemic homogeneity, which takes into account the practical constraints on how much time and effort one can put into distinguishing a group into subgroups, and true epistemic causal homogeneity, which merely refers to the case when a researcher does not know a relevant partitioning.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biography

Rosa W. Runhardt is a research associate at the Centre for Philosophy of Natural and Social Science (part of the London School of Economics and Political Science), as well as Honorary Research Associate at King’s College London. Her research interests include philosophy of the social sciences, philosophy of causation, and qualitative research methods, with a particular focus on the use of causal mechanisms in political science (process tracing). She has previously published in Philosophy of Science.

References

Antonakis

John

Bendahan

Samuel

Jacquart

Philippe

Lalive

Rafael

. 2010. “On Making Causal Claims: A Review and Recommendations.” The Leadership Quarterly 21: 1086-1120.

Berk

Richard A.

1983. “An Introduction to Sample Selection Bias in Sociological Data.” American Sociological Review 48 (3): 386-98.

Cartwright

Nancy

Efstathiou

Sophia

. 2011. “Hunting Causes and Using Them: Is There No Bridge From Here to There?” International Studies in the Philosophy of Science 25 (3): 223-41.

Cartwright

Nancy

Hardie

Jeremy

. 2012. Evidence-Based Policy: A Practical Guide to Doing It Better. New York: Oxford University Press.

Collier

Paul

Hoeffler

Anke

. 2001. “Data Issues in the Study of Conflict.” Data Collection on Armed Conflict, Uppsala, 8-9 June.

Collier

Paul

Hoeffler

Anke

. 2004. “Greed and Grievance in Civil War.” Oxford Economic Papers 56 (4): 563-95.

Hausman

Daniel M.

2010. “Probabilistic Causality and Causal Generalizations.” In The Place of Probability in Science, edited by Eells

Ellery

Fetzer

J. H.

, 47-63. Dordrecht: Springer.

Hitchcock

Christopher R.

1995. “The Mishap at Reichenbach Falls: Singular vs. General Causation.” Philosophical Studies 78 (3): 257-91.

Horowitz

Donald L.

1985. Ethnic Groups in Conflict. Berkeley: University of California Press.

10.

Horowitz

Shale

Heo

2008, March 26. “Do Ethnic and Non-Ethnic Civil Wars Have Distinct Causes?” Annual Meeting of the International Studies Association, San Francisco, CA.

11.

Kuorikoski

Jaakko

. 2012. “Contrastive Statistical Explanation and Causal Heterogeneity.” European Journal for Philosophy of Science 2:435-52.

12.

Little

Daniel

. 1991. Varieties of Social Explanation: An Introduction to the Philosophy of Social Science. Boulder: Westview Press.

13.

Pearl

Judea

. 2000. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press.

14.

Salmon

Wesley C.

1971. Statistical Explanation and Statistical Relevance. Pittsburgh: University of Pittsburgh Press.

15.

Sambanis

Nicholas

. 2001. “Do Ethnic and Nonethnic Civil Wars Have the Same Causes? A Theoretical and Empirical Enquiry (Part I).” The Journal of Conflict Resolution 45 (3): 259-82.

16.

Spirtes

Peter

Glymour

Clark

Scheines

Richard

. 1993. Causation, Prediction, and Search. New York: Springer-Verlag.

17.

Spirtes

Peter

Scheines

Richard

. 2003. Causal Inference of Ambiguous Manipulations. Pittsburgh: Department of Philosophy, Carnegie Mellon University. Technical Report No. CMU-PHIL-148. Accessed online October 27, 2015, http://repository.cmu.edu/cgi/viewcontent.cgi?article=1136&context=philosophy

18.

Spirtes

Peter

Scheines

Richard

2004. “Causal Inference of Ambiguous Manipulations.” Philosophy of Science 71 (5): 833-845.

19.

Steel

Daniel

. 2008. Across the Boundaries: Extrapolation in Biology and Social Science. Oxford: Oxford University Press.

20.

Wimmer

Andreas

Cederman

Lars-Erik

Min

Brian

. 2009. “Ethnic Politics and Armed Conflict: A Configurational Analysis of a New Global Data Set.” American Sociological Review 74 (2): 316-37.

21.

Woodward

James

. 2003. Making Things Happen: A Theory of Causal Explanation. Oxford: Oxford University Press.

22.

Xie

. 2011. “Population Heterogeneity and Causal Inference.” University of Michigan Population Studies Center Research Report 11-731. Accessed online October 27, 2015, http://www.psc.isr.umich.edu/pubs/pdf/rr11-731.pdf