The role of social science in action-guiding philosophy: The case of educational equity

Abstract

Education policy decisions are both normatively and empirically challenging. These decisions require the consideration of both relevant values and empirical facts. Values tell us what we have reason to care about, and facts can be used to describe what is possible. Following Hamlin and Stemplowska, we distinguish between a theory of ideals and descriptions of feasibility. We argue that when feasibility constraints are used to rank competing states of affairs, two things must be articulated. First, one must explain why one feasibility constraint is preferred over another. Second, because of empirical uncertainty, one must describe the upper and lower bounds of a specified feasibility constraint. The first case implies that different optima are possible depending on, for example, what one takes to be fixed about the world. The second case implies that different optima are always possible, and the upper and lower bounds of these optima will depend on the empirical uncertainty of an estimated feasibility constraint. We then describe three distinct forms of empirical uncertainty. Careful consideration of these sources of uncertainty can help to mitigate the risks of imprecision. The article closes by considering a case study whereby a meritocratic conception of fair equality of opportunity is considered alongside competing values of priority and parental partiality.

Keywords

Action-guiding philosophy causal inference educational equality empirical uncertainty feasibility constraints nonideal theory public policy

Introduction

This article addresses the role of social science evidence in action-guiding or nonideal theory.¹ It is intended for those engaged in projects relevant to the social world – policy makers, social scientists, and normative theorists. Nonideal theory addresses two concerns: optimal states of affairs and empirical constraints on the optimal arrangements. In an education setting, decisions about how to best maximize educational opportunities for different groups of children is an ongoing challenge. Action-guiding theorists approach this challenge by considering how educational institutions ought to be structured and how resources ought to be distributed.

The work of action-guiding theorists is to identify an objective and to develop a plan of action to realize the objective. The plan must specify the population targeted by the objective, the agents responsible for carrying out the action, and a timeline for the realization of results.

It is clear that both values and facts are relevant for action-guiding theories. Values are relevant because they are required to rank states of affairs, to specify that one objective is superior to another. Facts are relevant because action-guiding theory specifies realizable values. The realization of the values requires the consideration of feasibility constraints, which are empirical in nature. Swift (2008) has described a three-pronged approach for producing action-guiding theory. For a particular problem, normative theorists first articulate a set of values and set the assumptions that frame their analysis. They then weight and rank the relevant values, and consider the acceptability of various trade-offs among these values, assuming that trade-offs are necessary and can be made. Second, empiricists describe possible states of affairs based on the implementation of policies under a variety of assumptions. They might, for example, describe a scenario in which one policy produces X equality and Y efficiency, while another produces X′ equality and Y′ efficiency. Two bundles of efficiency and equality are described so that different states of affairs can be compared. Finally, normative theorists and empiricists together consider an array of states of affairs and rank them in accordance with the values that were initially presented. An action-guiding principle is then developed from the highest ranked, all-things-considered state of affairs.

This is an outline of the project in broad strokes. As is evident, the project has semicompartmentalized roles for philosophers and social scientists. And while a fair amount of argumentation has taken place among philosophers about the first step in this process, much less has been said about the second. Regarding the first, Sen (2006) has questioned the usefulness of ideal theory in the face of existing grave injustices. Farrelly (2007) has argued that ideal theorists, by not acknowledging feasibility constraints, are incapable of considering values-based trade-offs. Moreover, there has been disagreement about what constitutes ideal and nonideal theory in the first place (see, for example, Farrelly, 2007; Simmons, 2010; Stemplowska, 2008; Stemplowska and Swift, 2012). Even more fundamentally, there is disagreement about whether values must be ‘fact free’ or should be ‘fact sensitive’ (see Cohen (2003), for why values are fact free, and Miller (2008) for why values should be fact sensitive). Even in the absence of agreement among all parties, the richness of this literature has helped to develop and refine the way in which one might go about developing a ranking of values.

This rich discussion, however, has not translated into an equivalently rich discussion about how empirical evidence ought to be incorporated into this multidisciplinary project. Swift (2008), for example, has written, ‘[i]t is for the empirical, descriptive/explanatory, social-scientific disciplines to (try to) tell us what states of the world can indeed be realized by what means – with what probabilities, over what time scales – given where we are now’ (p. 369). He goes on to say that social scientifically informed action-guiding principles presuppose sufficient understanding of social process, causal mechanisms, and explicit time horizons, factors that may themselves be only weakly defined or understood. Even imperfect descriptions of possible states of affairs, he notes, are extremely difficult because of the limitations of empirical evidence about the social world. But Swift’s account of the challenges inherent in such a project is only a call to action. It is not a full explanation of how social scientists and philosophers ought to tackle this project and cope with these challenges.

In this essay, we address two overarching questions. First, how should empirical evidence be used in action-guiding theory? And second, what are the consequences of empirical imprecision for action-guiding theory? The article proceeds as follows. In section ‘The relationship between facts and values for action-guiding theory’, we provide a stylized example of the relationship between values and empirical knowledge for action-guiding theory. We show that the set of factors considered to be fixed – the ‘facts on the ground’ – will affect the values that are realizable in practice. In section ‘Feasibility and empirical uncertainty’, we formalize the example by defining feasibility constraints and showing that different conceptions of feasibility are possible. We identify two reasons why multiple feasibility sets arise. First, different actors may regard different facts about the world as fixed, and second, empirical uncertainty creates imprecision in knowledge. The consequence of the first case is that different feasibility-constrained states of affairs will be identified; the consequence of the second case is that the actual ranking of states of affairs is confounded. In section ‘Sources of empirical uncertainty’, we provide a careful account of three sources of empirical uncertainty. Informed consideration of these sources of empirical imprecision can minimize the risk that uncertainty poses. And finally, in section ‘Analyzing complex educational problems’, we build upon Brighouse and Swift (2008) to demonstrate how the consideration of values, facts, and uncertainty – the three components of an action-guiding theory – can help unpack complex problems faced by educational theorists and policy makers.

The relationship between facts and values for action-guiding theory

We do not argue that values depend on facts (see Cohen, 2003; Miller, 2008 for divergent points on this matter); however, facts about the world lead to the consideration of different values. To clarify this claim – that facts about the world lead to the consideration of relevant values – consider a hypothetical case involving charter schools. Suppose we know that the allowance of charter schools in low-income neighborhoods has two effects. First, students who attend charter schools in these areas perform better on academic tests, and their higher achievement can be attributed to their schooling. Second, the students who remain in traditional public schools are worse off as a result of ‘cream skimming’ – that is, the students who remain in traditional public schools have lower-achieving (though demographically similar) peers, on average, because higher-achieving students have sorted into charters. For a variety of reasons, the presence of lower average achievement in a school may negatively affect individual students’ achievement.²

Suppose the best evidence we have supports this stylized example. What values are we led to consider? First, there is a question about fair equality of opportunity. When charter schools that improve achievement are made available to children in low-income neighborhoods, children who enroll in these schools are provided with an opportunity that they were previously lacking. Whether this constitutes an equal opportunity will depend on our conception of equality. Formal equality might only require these charters to be as good as public schools in high-income neighborhoods. However, if schools are to compensate for social background conditions, such as poverty, then equality might require that charters exceed the quality of public schools in high-income neighborhoods. We do not wish to sort out the various conceptions of equality of opportunity here. We simply seek to show that the existence of these charters leads to, at a minimum, the consideration of the value of equality of opportunity.

Other values are brought into consideration as well. In this example, we suppose that geographically proximate students who do not attend charters are made worse off, and that the students who are made worse off are also the least advantaged in the school system. This leads to considerations of priority principles. A priority principle demands that attention be paid to improving the situation of the least advantaged (Parfit, 1997). The existence of charter schools, at least as we have described it, leads to the consideration of how equality of opportunity ought to be traded off against principles of priority.

If the facts on the ground were different, then one would consider different values. Suppose that charter schools improve academic outcomes among attendees, do not harm the achievement of traditional public school students, as was indicated in the previous example, and also increase racial segregation between schools.³ In this case, charter schools are academically superior, are not causally related to any academic harm to a worse-off group, but are racially more homogeneous than public schools. This would shift concern away from priority principles and toward, perhaps, the value of equal citizenship.⁴ If racially segregated schools undermined equal citizenship, one would be led to consider how equality of opportunity ought to be traded off against equal citizenship. In Swift’s tripartite schema, this is the role set aside for normative theorists. Normative theorists articulate the relevant values and provide tools for ranking bundles of values.

We now turn to the facts on the ground where we focus on three distinct but related points: what is considered to be fixed in the world will depend on both empirical description and empirical assumptions; the facts that are taken as fixed will affect the realization of values, as well as the quantity of those values; the less precisely the facts relate to the values at hand, the more difficult it will be to identify a values-optimizing plan.

Through the charter school example, we showed that an empirical description of the world is necessary to understand the ways in which a policy affects values. The more subtle point we wish to make is that an empirical description alone is not sufficient to support the statement, ‘the facts on the ground lead to the consideration of value’. What one takes to be the relevant state of affairs also requires assumptions about the aspects of the world that are fixed.

To see this, imagine two public decision makers, a Governor and a Supreme Court Justice. A Governor is bound by election cycles and is under pressure to implement policies that will have immediate impacts. She takes the current state of affairs to be firmly fixed – the existence of income-segregated neighborhoods, race-based residential preferences, and the status-quo effectiveness of the public schools are the conditions under which she makes policy. A Supreme Court Justice, however, operates under different conditions. For her, the current state of affairs is less certain, and her ability to take a longer view lets her relax certain empirical constraints. Perhaps she supposes that race-based preferences are rapidly dissipating, and in a reasonable amount of time, charter schools will not be as racially homogeneous as the facts currently suggest. The Justice makes assumptions that end up mitigating the trade-off between opportunity and civic equality. Presented with the same facts, the governor and the judge see different states of affairs, and neither is wrong. Their conceptions of what is feasible are determined by different constraints and therefore a different set of values can be considered.

Thus far we have shown that the facts on the ground lead to consideration of different values; what one takes to be the relevant facts will vary depending on what one takes to be fixed; and the final set of relevant facts can entail a different set of values considerations. We now discuss the last piece of the empirical puzzle – uncertainty. What effect might empirical uncertainty have on the values considered and on the best way of realizing those values?

Returning to our first example, first suppose that there is certainty in the available facts and that the current state of affairs is fixed. Under these conditions, charter schools are discouraged on the grounds that increasing opportunity for low-income children is not justified if it harms the less advantaged among them. Priority trumps opportunity. In this case, once the values are arranged, a decision about what ought to be done can be made based on the effects of the policy. Now suppose that it is known that charter schools improve academic outcomes but there is uncertainty about whether those left behind in traditional public schools – the least advantaged – are made worse for this. In this case, one’s capacity to rank competing states of affairs is stymied. Proponents of charter schools will argue that opportunity is assured, and that harm is only a possibility. Opponents will argue that the harm is a real possibility and, because priority principles trump opportunity principles, it is a risk not worth taking. Evidence about the degree of uncertainty or the magnitude of risk might be levied to adjudicate the disagreement. If there is only a small chance that the least advantaged will be harmed, opponents might be convinced to change. If the magnitude of potential harm to the least advantaged is very high, even if uncertain, proponents might be convinced to change. Additional information about the uncertainty itself cannot negate the fundamental challenge, but it can help us make better-reasoned decisions.

In this section, we used concrete examples to illustrate the relationship between values and facts in action-guiding theory. In section ‘Feasibility and empirical uncertainty’, we generalize the concepts of feasibility and uncertainty by providing a model to understand the interplay between the three parameters of action-guiding theory: values, facts, and uncertainty.⁵ This model shows, more generally, that the relevant feasibility constraints will vary depending on what one takes to be fixed, and that uncertainty clouds the identification of actions that could produce the optimal bundle of values.

Feasibility and empirical uncertainty

In this section, we build on a framework described by Hamlin and Stemplowska (2012) that distinguishes between a theory of ideals (or a theory of values) and nonideal theory. Theories of ideals are necessary for defining, articulating, and comparing the values that are at stake in different circumstances (Hamlin and Stemplowska, 2012: 53).⁶ Descriptions about the realization of different states of affairs are necessary for providing an account of what is possible. Hamlin and Stemplowska describe two offsetting values – equality and welfare. This can be seen in Figure 1, which is based on a figure from their article. Along the indifference curve I1, units of equality are exchanged for units of welfare (and vice versa). Any point along I1 is superior to any point along I2, but all points along I1 are equivalent.⁷ These indifference curves elucidate the values in play and the trade-offs between them. What has traditionally been considered ideal theory is, under this heuristic, a theory of ideals. Theories of ideals allow us to consider arrangements of relevant values without invoking empirical considerations.

Figure 1.

Two offsetting values equality and welfare from Hamlin and Stemplowska (2012).

Hamlin and Stemplowska (2012) argue that what is often described as nonideal theory is better understood as a theory about the realization of possible states of affairs. The realization of equality and welfare will be constrained in different ways, leading to the term feasibility constraints. Quite generally, we define a feasibility constraint to be a set of obstacles in the world that affect the capacity to realize values. At the most basic level, one might consider the availability of natural resources. Scarcity dictates difficult and unavoidable decisions about the distribution and production of goods and services. Perhaps less rigid are the obstacles of human moral sentiments and rational faculties. Society’s capacity to realize the value of equality may be constrained by the need to provide incentives to those who would otherwise not perform useful tasks (Cohen, 1992). And finally, there are obstacles that are malleable and continually reassessed, such as the tax base in a local township. The ability of that township to provide basic services to its least advantaged members will be constrained by the amount of resources at its disposal. The complete set of obstacles affecting our capacity to realize values defines a feasibility constraint.

In the simplest case, consensus about the relevant obstacles produces just one feasibility constraint. However, multiple conceptions of what is feasible are possible, and in this case, there will be variation in the set of obstacles considered to be relevant. We highlight two reasons for this: variation in time horizons and assumptions made when facts are unknowable. In the first case, those with longer time horizons might have less restrictive feasibility constraints than those with shorter time horizons. This was the case described above regarding the Governor and Supreme Court Justice. In the second case, we suppose that some things are unknowable, such as the moral sentiments of Americans 20 years in the future. When multiple feasibility constraints exist, the specification of assumptions will dictate the set of obstacles that constitute the feasibility constraint.

Figure 1 helps to portray this. In addition to indifference curves I1 and I2, we now emphasize two feasibility frontiers F1 and F2.⁸ The feasibility frontiers represent different assumptions about institutional and structural constraints. Consider the case of ability-based course-tracking. This is a widely studied feature of the American education system that often has negative implications for racial and socioeconomic segregation within schools, especially in schools with diverse student populations (Lucas and Good, 2001; Oakes, 1985). This feature of schooling, what Mickelson (2003) has called ‘second generation segregation’, creates intrainstitutional segregation, and likely has consequences for the aspirations, expectations, and future opportunities of minority and low-income students. But tracking may also encourage white and middle-class families to remain in diverse schools. In Figure 1, we might now have the value of classroom integration on the y-axis, and the value of within-school diversity on the x-axis. Indifference curves I1 and I2 represent trade-offs between a highly diverse school and highly diverse classrooms. F1 might be a description of what is expected in the short-term given empirical evidence about parental behavior, and F2 an appraisal about what is possible in the future. Whereas point B is the best combination of values in the short term, point A represents an outward shift in what is possible in the long term. In the course-tracking example, point A represents a combination of both greater within-school diversity and greater racial and socioeconomic integration within classrooms. One can express different conceptions of what is feasible by describing the conditioning assumptions that undergird those conceptions.

Building upon the ideas of Hamlin and Stemplowska (2012), we have thus far presented a conception of feasibility constraints and argued that it is possible to have more than one. Next we return to the simplest case – consensus about underlying assumptions and the identification of a single feasibility frontier. But we will assume a feasibility-constrained optimal state of affairs remains unknown. This scenario occurs when there are different estimates of what is true about the world. In this case, multiple optimal states of affairs are possible even if a single indifference curve I and a single feasibility frontier F are specified. We refer to this problem as empirical uncertainty.

We discuss the role of empirical uncertainty in social science knowledge in section ‘Sources of empirical uncertainty’, but for expository purposes, we can use a simple and plausible case in which one policy study suggests an effect size of F−, while another suggests an effect size of F+. This is represented in Figure 2 by an upper and lower bound around an agreed-upon feasibility frontier. F corresponds to F1 from before. F is a description of what is possible under certain assumptions about what one takes to be fixed. F+ and F− represent alternative feasibility frontiers, the upper and lower bounds of the policy’s expected effect. Note that the uncertainty about the policy’s effect can potentially result in a downward or upward shift between indifference curves, represented by points A+ and A−. In the absence of additional information, it is impossible to adjudicate between the status quo and adoption of the policy. Figure 1 showed that different optima were possible under different feasibility frontiers, represented by the points A and B. However, it was possible to adjudicate between points A and B by explaining why one feasibility frontier is specified over another. In the case of empirical uncertainty, differentiating between the upper and lower bounds is difficult. Points A+, A, and A− are all possible maxima under a specified feasibility frontier, the upshot of which is the confounding of different feasibility frontiers. This hinders decision-making and complicates the assessment of values trade-offs in response to the facts on the ground.

Figure 2.

A simple and plausible case in which one policy study suggests an effect size of F−, while another suggests an effect size of F+.

It is important to note that greater certainty reduces the difference between F and F+ or F−. This implies that greater uncertainty for a given feasibility constraint raises the stakes − the policy is riskier due to the wide variation in the possible bundles of values. The mitigation of risk is one of the principle justifications for further social scientific inquiry.

This section has outlined two considerations regarding the use of facts in action-guiding theory. First, different feasibility constraints are possible, and second, empirical uncertainty complicates the construction and ranking of different states of affairs. In the next section, we explain our conception of uncertainty in social scientific knowledge in more detail.

Sources of empirical uncertainty

We proceed by distilling three distinct features of social scientific evidence that create empirical uncertainty: the identification of causal pathways, the coarseness of knowledge, and the contextual nature of all social facts. The vast majority of criticism about social scientific evidence focuses on the first limitation, the determination of causality. Although issues of causal inference do raise serious concerns, the determination of causality is just one of our three categories of uncertainty. We argue here for a broader, more theoretical conception of uncertainty. As we describe below, we conceive of uncertainty as not just a technical problem, but also as a problem of answering the most relevant question, and of producing information that is generalizable over a range of time and place. These challenges should not preclude the use of social science evidence. To the contrary, social science evidence is essential for understanding economic, social, and political processes that are crucial to people’s welfare, and to inevitable, values-laden policy decisions. There is uncertainty in social knowledge, but when uncertainty is minimized, decisions carry less risk. Thus, on account of empirical uncertainty, action-guiding theorists should be judicious in their use and interpretation of facts, and social scientific researchers should be motivated to minimize the uncertainty of relevant facts.

Before proceeding, we would like to clarify two points. First is our use of the term facts. In our view, there are at least two brands of social facts – descriptive and causal. Descriptive facts simply characterize the world (e.g. Hispanic students are more likely to drop out of high school than students of any other race/ethnicity (US Department of Education, National Center for Education Statistics, 2013)), while causal facts connect cause and effect (e.g. the release of school districts from court-ordered desegregation plans leads to the resegregation of schools (Reardon et al., 2012)). Although descriptive facts have desirable properties, and are easier to acquire, they are of limited use for policy makers who seek to understand how inequality, for example, is generated. Conversely, causal facts help to identify how social patterns emerge, but they are much more difficult to acquire. It is causal facts, or mechanistic explanations, that we focus on in this section. For the reasons we delineate below, facts of this nature often carry with them a great deal of uncertainty, which creates difficulty for action-guiding theory. We use the term facts as shorthand for this more causal, process-oriented knowledge about the social world.

The second point is related to the most obvious source of empirical uncertainty, statistical uncertainty.⁹ Although statistical uncertainty can operate similarly to our three broad sources of uncertainty, we set it aside in this article as a point of discussion. We do this because statistical uncertainty is fundamental to the practice of empirical analyses in the social sciences and is therefore embedded in all quantitative social science evidence. We do not intend for this article to rehash statistical theory; instead, we are interested in clarifying less obvious sources of uncertainty. Thus, in the remainder of the article, our use of the term uncertainty refers to the three broad technical and theoretical sources of uncertainty we introduced above.

Causal pathways

Causal inference is itself a difficult enterprise, and this is compounded when the goal is to isolate causal pathways, as opposed to correlational associations. Causal pathways identify the mechanisms that create the patterns we observe, and they help to determine if and how public policy can affect social outcomes. Why is it so difficult to identify a causal link between two phenomena? The first difficulty is that our statistical models do not identify the causes of effects, but rather the effects of causes. For example, we are not able to determine why children of less educated mothers complete college at a lower rate than children of better educated mothers. But we are able to answer a modification of this question: do children of less educated mothers who receive good quality early childhood education have higher rates of college completion relative to children of less educated mothers who do not receive early childhood education? This is an important distinction because it dramatically narrows the field of questions for which causal explanations are possible (Holland, 1986).

Yet, even identifying the effects of causes is not easy. Consider how individual characteristics often come bundled. One might anecdotally notice that children with less educated mothers tend to have lower educational attainment. In addition, these children are more likely to live in lower income households, be raised by a single parent, attend more crowded schools, and have less qualified teachers. The fact that these social features hang together in the real world makes it complicated to determine if any one of these factors affects any given outcome. It would be naive to conclude, for example, that because children with less educated mothers are less likely to complete college than children with better educated mothers that the education of one’s mother has a causal effect on college completion (though it is plausible that there is a causal effect).

Even using standard statistical techniques to account for other possible observable characteristics is not sufficiently causal because unobserved factors can lead to a selection effect or, more generally, omitted variable bias. It is difficult to fully account for all possible causes of an effect in the absence of observing the same person under two competing conditions.¹⁰ Because it is impossible for the same person to experience two conditions simultaneously – what Holland refers to as the ‘fundamental problem of causal inference’ – the next best option is to estimate an average effect of some factor (average treatment effect) by observing groups of individuals who are alike in expectation in two different conditions.¹¹

Once a causal pathway is determined, there are still limitations with which to contend. For example, some experimental evidence suggests that smaller class sizes have a causal effect on students’ test scores (see, for example, Chetty et al., 2011; Krueger, 1999), but that is not to say that class size is the only way to change test scores (because it is only one possible cause of test scores), or that a reduced class size will change test scores for all students. Average treatment effects are not necessarily indicative of the effect on any particular observation, such as a school, classroom, or an individual. Effect sizes often differ by race/ethnicity, gender, or age, for example. The possibility of heterogeneous effects is especially salient when causal evidence is derived from a narrow population, where grasping the generalizability of the finding is difficult. As we grapple with these complexities, we move toward a fuller conception of the ways in which achievement, for example, is determined.¹²

Coarseness of knowledge

A second complicating feature of empirical evidence is its level of specificity – known facts may identify broad stroke social processes. Causal pathways are often blunt, especially in relation to identifying the effects of complex social institutions, such as schools, neighborhoods, or the family. Understanding social processes is like peeling away layers of an onion. Each layer represents a discrete fact or causal pathway that brings us closer to the core. The school effects literature, which aims broadly to determine whether schools affect student outcomes, presents a useful example of this phenomenon. The outer layer, the broadest test of this question, simply attempts to attribute some change in student outcomes to enrollment in a certain school. Even if the researcher is able to determine a causal pathway between attending certain schools and test scores, the conclusion can only be that the school as an entire institution affects achievement. This is a coarse finding and it represents the ‘black box’ of causality. As stated by Morgan and Winship (2007), ‘causal explanations require theory and concepts that organize knowledge about (typically) unobserved processes or mechanisms that bring about the effect’ (p. 197). Experimental conditions isolate effects, but they often contribute little to the understanding of social processes.¹³

Much of the school effects literature attempts to address slightly more narrow questions, such as measuring the effects of charter or Catholic schools. But attributing an effect to a type of school still leaves much room for explanation. What causes the effect? Freedom from regulation? Nonunionized teachers? Longer school days? Or is it simply that students feel special because they won a lottery to attend a nontraditional school? Each one of these causal pathways is like peeling away another layer of the onion, revealing mechanisms that might explain student achievement in certain contexts. The current interest in the effects of teachers is an example of a subinstitutional mechanism. Researchers have successfully identified teachers as one of the primary mechanisms through which schools make a difference (see, for example, Clotfelter et al., 2010; Rivkin et al., 2005; Rockoff, 2004),¹⁴ but there is currently little knowledge about the characteristics of teachers that make them most effective. This information would be useful in hiring prospective teachers, and also in distributing teachers across schools to reduce inequalities. What makes a teacher effective will likely be the focus of the next round of causal studies, representing the next layer of facts to discover. In short, social facts are layered, and the discovery of mechanistic processes is iterative.

Contextual nature of social processes

And finally, social science evidence is necessarily contextual in nature because social processes are spatially and temporally contingent. In addition, human processes are dynamic – an effective equalizing intervention at one point in time may produce inequality at a later point in time. Even the most rigorously determined causal social relationships are context specific.

School integration serves as one example of the contextual nature of social processes. If the effect of a school integration policy was dependent on public opinion about the need for racial equality in schools, the state of racial attitudes more generally, or the extent of state-sponsored racial segregation, then one might guess that the effect of such a policy would differ over time and place. The impetus for, and the implementation of, such a policy in the early 1960s was quite different than it would be in 2010, even though the racial segregation between schools was unacceptably high at both times. Empirical evidence on topics such as school integration, which encompass a broad swath of policies and varied contexts of reception based on time and place, generates mixed results, in part because of the context-specific nature of social facts.¹⁵

Indeed, the generalizability of research findings is one of the primary concerns of most social scientists. Great efforts are made in large studies to ensure that the external validity is high, meaning that the results are representative of the entire population. Case studies, on the other hand, generally are lacking in external validity, but can trump larger studies in internal validity, meaning they more closely adhere to the principles of causal inference. In addition, case studies allow for in-depth exploration of social processes in a way that is difficult to do in larger studies. Ideally, knowledge would be both internally and externally valid, but that is rarely possible to achieve perfectly in social research.

Although uncertainty is endemic to the kinds of facts most useful for action-guiding theory, it is not the case that all facts are equivalently uncertain. Knowledge of the causes and magnitude of imprecision makes it possible to use evidence strategically. In the case of causal pathways, evidence that merits strong causal warrant, models heterogeneity of effects, or uses repeated experiments reduces uncertainty, and therefore reduces the distance between the upper and lower bounds in Figure 2. Evidence with more certainty should be weighted more heavily than evidence with less certainty. In this way, one can incorporate the bounds of potential outcomes into the decision-making process. Naturally, the process of weighting evidence can be extended to our other two categories of uncertainty as well.¹⁶

Analyzing complex educational problems

We have argued that defining social problems requires attention to several factors: relevant values, the priority of those values over others, conceptions of feasibility, and the certainty of those conceptions of feasibility. In this section, we demonstrate that attention to these factors can usefully diagnose complex educational problems. We choose to model a series of policy choices, described by Brighouse and Swift (2008), whereby a meritocratic conception of fair equality of opportunity (FEO) is considered alongside competing values of priority and parental partiality.

Brighouse and Swift (2008) define FEO such that a person’s ‘prospects for educational advancement may be a function of that individual’s talent and effort, but they should not be influenced by his or her social class background’ (p. 447). They consider two objections to the meritocratic conception of FEO. Their first criticism is that the FEO principle is overly tolerant of inequalities between those with different talents. They consider the case of a state in which opportunities are equalized between social class backgrounds. If this state comes into a windfall of money, the meritocratic version of FEO provides no reason to allocate funds to the less talented. Their second criticism is that the FEO principle may, under certain conditions, place restrictions on privileged familial relations.

The authors respond to each of these criticisms by subordinating FEO to a higher principle. In the case of talent inequalities, the authors subordinate FEO to a principle of priority. If the pursuit of a policy that increases equality of opportunity harms the least advantaged, then we should not pursue that policy. This can be restated positively by saying that departures from FEO are tolerated if the benefits are largely realized by the least advantaged, which the authors note is a variant of Rawls’ difference principle.

In relation to their second criticism, the authors subsume FEO to a principle that allows for the production of familial relationship goods. In short, they argue that there is a set of unique relational goods produced by the family, and that these goods should not be sacrificed for FEO (for a full account, see Brighouse and Swift, 2009). To define terms, these goods are unique because the factors of production for these goods are nonsubstitutable (i.e., they cannot be produced outside the family); these are goods because they have value; and they are familial because they are produced by the family. The relationship goods that fall within this set have lexical priority over processes conforming to FEO. Thus, to the extent that the factors of production for these relationship goods are nonsubstitutable, and to the extent that these factors of production conflict with processes conforming to FEO, then those processes that produce familial goods are not to be blocked.

The authors have presented two values with lexical priority to FEO. Figure 3 represents the relationship between prioritarian principles and the production of familial relationship goods on the vertical axis, and meritocratic FEO on the horizontal axis. The horizontal indifference curve I indicates that no amount of FEO can be exchanged for either priority or familial relationship goods.

Figure 3.

The relationship between prioritarian principles and the production of familial relationship goods on the vertical axis, and meritocratic FEO on the horizontal axis.

Having described the values, we now consider how feasibility affects policy decision-making. For sake of parsimony, our examples concentrate on the trade-off between FEO and priority principles, but they can be readily extended.

Brighouse and Swift consider a policy to eliminate elite private schools, under the assumption that the closure of elite private schools will increase meritocratic FEO. This is represented by a move along the feasibility frontier from A to B (or A′ to B′). In order for this policy to conflict with priority principles, it must be the case empirically that the closure or prohibition of elite private schooling harms the least advantaged. Two mechanisms are proposed. One possibility is that affluent families vigorously battle the policy, which costs the state a great deal of resources that would otherwise have gone to the poor. A second possibility is that elite private schools inculcate a sense of noblesse oblige, and in the absence of these institutions the rich become cynical and tax avoidant, thus diminishing the redistribution of funds to the poor.

So, does the policy cause a shift from A to B or from A′ to B′? In the example presented in Figure 3, the shift from A to B is not optimizing, whereas the shift from A′ to B′ is optimizing. What matters, then, is whether the feasibility set is best described by F or F+. Brighouse and Swift note that whether F or F+ is specified will depend on a number of conditioning assumptions,¹⁷ which will in turn determine whether F or F+ represents the shift. Real-world constraints and the conditioning assumptions that are made determine whether it is possible to improve FEO while maintaining prioritarian commitments, or whether another policy (or the status quo) must be pursued.

Because there are different reasons for assuming one feasibility constraint over another, it is possible that F and F+ describe two possible worlds. As we argued earlier, uncertainty creates a band of error around each feasibility frontier, as can also be seen in Figure 3 If we assume that there is agreement about the conditioning assumptions, and that these conditioning assumptions describe the feasibility frontier represented by F, then uncertainty bounds this estimate between F− and F+.

If uncertainty truly masks the effect of a policy on a desired bundle of values, then there is no obvious way to make a policy decision. Brighouse and Swift seem to suggest a solution to this problem by placing the burden of proof on those opposed to FEO. They say, ‘[t]he challenge to the defender of particular mechanisms that violate the meritocratic conception of educational equality is to show precisely why it is that they are necessary for improving prospects for flourishing of the less advantaged’ (Brighouse and Swift, 2009: 453). This implies a chain of reasoning akin to the following:

It is given that policy X (closing elite private schools) will improve FEO.

Policy X is not to be pursued if it conflicts with priority principles.

Defenders of priority principles must demonstrate empirically that the least advantaged will be harmed if policy X is pursued.

In the absence of this demonstration, pursue policy X.

Brighouse and Swift express concern that those who introduce uncertainty into the decision-making process do so in order to stymie FEO-enhancing policies. We are sympathetic with this worry, but we do not see why any particular party – defender of FEO or priority principles – must carry the burden of proof. The goal is to describe the world as accurately as possible: the real trade-offs, the assumptions that undergird those trade-offs, consideration of alternative feasibility sets that minimize the trade-offs, and an account of the certainty of the estimations.

In this section, we have used an example from the normative literature on educational equity to show how explicit attention to values, facts, and uncertainty is necessary to properly define a problem. Incorporating careful knowledge of all three features of action-guiding theory will rarely identify a specific local maximum, but it may help to identify superior regions of the values space and to therefore move a decision-making process forward. Similar to many of our other examples, this example is clearly oversimplified. Few policy decisions, especially in education, involve just two competing values. More common are policy decisions that involve multiple values trade-offs to be considered simultaneously and a pile of empirical studies that each only addresses aspects of these trade-offs. We set this much more complicated, but realistic, scenario aside in the hopes of distilling our main argument. However, in the spirit of contributing to action-guiding theory, we acknowledge the limitations of a stylized example.

Conclusion

In this article, we have argued that there are three necessary components of action-guiding theory – values, facts, and uncertainty. At the outset, we said that this article was intended for policy makers, normative theorists, and social scientists. We conclude by returning to these target groups and clarifying our intentions.

First, for policy makers, these categories can help to clarify and maybe resolve disagreements about policy proposals. A disagreement about whether to increase taxes to raise educational revenues might be disputed on values-based grounds. There is variation in commitment to efficiency, priority, and equality principles. The disagreement may hinge on the established facts. Parties might be equally committed to egalitarian principles but disagree about the facts. Those opposed to the tax might expect the tax to cause wealthier state residents to move, thereby shrinking the tax base and shifting the tax burden onto the less well-off; parties in favor of the tax might expect minimal out-migration. Finally, parties may converge on principles and facts, but differ on the degree of confidence they have in those facts. There could be general agreement that the rich will not flee the state, but those still opposed suspect this description of the facts is very imprecise, and, mindful of the worst outcomes, opt to keep the status quo.

Second, in order to maximize clarity of purpose and the accurate use of social science evidence, normative theorists involved in socially relevant projects must be mindful of all three components. At a minimum, scholars should articulate whether they are presenting a theory of ideals or an action-guiding theory. If the purpose is action guiding, then feasibility constraints must be considered. One should articulate the relevant feasibility constraints and the conditioning assumptions that undergird those constraints. In other words, explain why F1 is relevant and not F2. Finally, empirical uncertainty should be addressed based on the available facts. Ideally, scholars would describe the appropriate upper and lower bound consequences of their proposed principle.

Finally, social scientists can benefit from a careful accounting of these three components as well. First, as we have argued, certain values considerations rely on the facts on the ground, and in many cases, the relevant values are unidentified because of empirical uncertainty. Social scientists, by being mindful of the relevant values affected by educational policies, can more consciously work to reduce the confidence intervals around feasibility estimates. Second, social scientists could better articulate the assumptions underlying the feasibility constraints they describe. Even in more complex analyses, social scientists often work from cross-sectional perspectives in which moral sentiments are considered fixed and time horizons perceived as narrow. Finally, social scientists spend a great deal of energy debating causal warrant and statistical uncertainty and very little energy on the broader conceptions of uncertainty that we have emphasized – the generalizability of facts through time and space, and the coarseness of empirical evidence. Attention to these dimensions of inquiry enriches our understanding of empirical uncertainty and can help direct us to more relevant values.

It may seem that we have advocated for an overly ambitious process of incorporating social knowledge into normative theory, given the challenges posed by uncertainty in empirical facts. Indeed, the project is difficult. However, this article was motivated by the fact that scholars and citizens are already engaged in this activity. In applied ethical debates, we see a potentially powerful iterative relationship between philosophy and social science. Our goal in this article is to open a dialogue about the ways in which philosophers and social scientists can engage in an interdisciplinary project that will facilitate the design of normatively grounded, empirically sound social policy decisions.

Footnotes

Acknowledgements

We are grateful to two anonymous reviewers, participants in the Ethics of Education Equity Workshop at Stanford University, and Sean Reardon for providing us with thoughtful feedback and detailed comments.

Funding

Research for this article was supported by the Spencer Foundation’s initiative on Philosophy in Educational Policy and Practice (postdoctoral support to Bischoff through the Ethics in Society Center at Stanford University), Cornell University’s National Science Foundation ADVANCE program (faculty development grant to Bischoff), as well as the Institute for Education Sciences Interdisciplinary Doctoral Training Program (fellowship to Shores).

Notes

Author biographies

Kendra Bischoff is an assistant professor in the Sociology Department at Cornell University, and previously completed a postdoctoral fellowship in the ethics of education equity at Stanford’s Center for Ethics in Society. Her current research focuses on the causes and consequences of racial and economic segregation in neighborhoods and schools, as well as the effect of school context on youth civic engagement.

Kenneth Shores is a PhD candidate at the Stanford Graduate School of Education. His interests are in the various educational inputs that can ameliorate the effects of social background and improve equality of opportunity. He is currently studying school finance reform and its effects on narrowing the achievement distribution.

References

Abdulkadiroğlu

Angrist

Dynarski

. (2011) Accountability and flexibility in public schools: Evidence from Boston’s charters and pilots. Quarterly Journal of Economics 126(2): 699–748.

Anderson

(2007) Fair opportunity in education: A democratic equality perspective. Ethics 117(4): 595–622.

Bettinger

(2005) The effect of charter schools on charter students and public schools. Economics of Education Review 24(2): 133–147.

Bifulco

Bulkley

(2007) Charter schools. In: Ladd

Fiske

(eds) Handbook of Research in Education Finance and Policy, pp. 425–446. New York: Routledge.

Bifulco

Ladd

(2007) School choice, racial segregation, and test score gaps: Evidence from North Carolina’s charter school program. Journal of Policy Analysis and Management 26(1): 31–56.

Brighouse

Swift

(2008) Putting educational equality in its place. Education Finance and Policy 3(4): 444–466.

Brighouse

Swift

(2009) Legitimate parental partiality. Philosophy & Public Affairs 37(1): 43–80.

Chetty

Friedman

Hilger

. (2011) How does your kindergarten classroom affect your earnings? Evidence from project STAR. Quarterly Journal of Economics 126: 1593–1660.

Clotfelter

Ladd

Vigdor

(2010) Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources 45(3): 655–681.

10.

Cohen

(1992) Incentives, inequality, and community. The Tanner Lectures on Human Values 13: 263–329.

11.

Cohen

(2003) Facts and principles. Philosophy & Public Affairs 31(3): 211–245.

12.

Farrelly

(2007) Justice in ideal theory: A refutation. Political Studies 55(4): 844–864.

13.

Frankenberg

Siegel-Hawley

Wang

(2011) Choice without equity: Charter school segregation and the need for civil rights standards. Educational Policy Analysis Archives 19(1): 1–96.

14.

Greene

Forster

Winters

(2003) Apples to Apples: An Evaluation of Charter Schools Serving General Student Populations. New York: Manhattan Institute for Policy Research.

15.

Hamlin

Stemplowska

(2012) Theory, ideal theory and the theory of ideals. Political Studies Review 10(1): 48–62.

16.

Holland

(1986) Statistics and causal inference. Journal of the American Statistical Association 81(396): 945–960.

17.

Hoxby

(2002) How school choice affects the achievement of public school students. In: Hill

(ed.) Choice with Equity, pp. 141–178. Stanford, CA: Hoover Institution Press.

18.

Hoxby

Rockoff

(2004) The Impact of Charter Schools on Student Achievement. Cambridge, MA: Department of Economics, Harvard University; Manhattan, NY: Graduate School of Business, Columbia University.

19.

Krueger

(1999) Experimental estimates of education production functions. Quarterly Journal of Economics 114(2): 497–532.

20.

Ladd

(2003) Comment on Caroline M. Hoxby: School choice and school competition: Evidence from the United States. Swedish Economic Policy Review 10(1): 67–76.

21.

Lucas

Good

(2001) Race, class, and tournament track mobility. Sociology of Education 74(2): 139–156.

22.

Mead

Green

(2012) Chartering Equity: Using Charter School Legislation and Policy to Advance Equal Educational Opportunity. Boulder, CO: National Education Policy Center.

23.

Mickelson

(2003) The academic consequences of desegregation and segregation: Evidence from the Charlotte-Mecklenburg schools. North Carolina Law Review 81: 1513–1562.

24.

Miller

(2008) Political philosophy for earthlings. In: Leopold

Stears

(eds) Political Theory: Methods and Approaches, pp. 29–48. Oxford: Oxford University Press.

25.

Morgan

Winship

(2007) Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge: Cambridge University Press.

26.

Oakes

(1985) Keeping Track: How Schools Structure Inequality. New Haven, CT: Yale University Press.

27.

Orfield

Luce

(2013) Charter Schools in the Twin Cities: 2013 Update. Minneapolis, MN: Institute on Metropolitan Opportunity, University of Minnesota Law School.

28.

Parfit

(1997) Equality and priority. Ratio 10(3), 202–221.

29.

Reardon

Grewal

Kalogrides

. (2012) Brown fades: The end of court ordered school desegregation and the resegregation of American public schools. Journal of Policy Analysis and Management 31(4): 876–904.

30.

Rivkin

Hanushek

Kain

(2005) Teachers, schools, and academic achievement. Econometrica 73(2): 417–458.

31.

Rockoff

(2004) The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review 94(2): 247–252.

32.

Sampson

(2008) Moving to inequality: Neighborhood effects and experiments meet social structure. American Journal of Sociology 114: 189–231.

33.

Satz

(2007) Equality, adequacy, and education for citizenship. Ethics 117(4): 623–648.

34.

Sen

(2006) What do we want from a theory of justice? Journal of Philosophy 103(5): 215–238.

35.

Simmons

(2010) Ideal and nonideal theory. Philosophy & Public Affairs 38(1): 5–36.

36.

Stemplowska

(2008) What’s ideal about ideal theory? Social Theory and Practice 34(3): 319–340.

37.

Stemplowska

Swift

(2012) Ideal and non-ideal theory. In: Estlund

(ed.) The Oxford Handbook of Political Philosophy, pp. 373–392. New York: Oxford University Press.

38.

Swift

(2008) The value of philosophy in nonideal circumstances. Social Theory and Practice 34(3): 363–388.

39.

US Department of Education, National Center for Education Statistics (2013) The condition of education 2013. NCES 2013-037, May. Washington, DC: US Department of Education, National Center for Education Statistics.