Bringing in Theory Building and Social-Political Context to Understand and Reduce Inequality: A Commentary

A Split Between the Certainty Desired by Policy Makers and the Fundamental Uncertainty of Scientific Knowledge

Policy makers (and hopefully politicians too) are interested in the question, Does policy D work? Of course, questions are often more nuanced, such as, How much can policy D be expected to reduce or increase Y? The desire for a solid quantifiable answer is clear. With the emergence of causal designs for observational data, rewarded with the 2021 Nobel Prize in Economics to Joshua Angrist, David Card, and Guido Imbens, scientists can provide such answers. We now know more about the causal effect of minimum wages on employment, of class size on student performance, of ability grouping in schools on student outcomes, and of activating labor market policies on moving out of unemployment. We even know for which specific subgroup the effects are identified. Notwithstanding how valuable such studies are, it may emerge in the exchange between scientists and policy makers that “now we know for sure” whether policy D has the anticipated effect on Y. And if there is no evidence for a causal effect of D, we may think that D has no effect. The veil of certainty is ingrained in the exchange between evidence-inclined policy makers and social scientists. We should inform policy makers of the conditionality of our findings and of the uncertainty that is a fundamental property of scientific knowledge. Such uncertainties could include, for instance, problems of scaling up effective small-scale interventions (Kizilcec et al. 2020) and the uncertainty of statistical effects (Conaway and Goldhaber 2020). In a way, social scientists exchanging knowledge with policy makers are pulled between the certainty requested by policy makers and the fundamental uncertainty of scientific knowledge.

Let me give a few examples of where I think our interactions with policy makers can go wrong in this regard. A first example comes from educational policy analysis in the Netherlands. Economists from the influential Netherlands Bureau of Economic Policy Analysis (Centraal Planbureau) published a review of research on many different education policies, including class-size reduction, ability grouping, teacher performance pay, preschool quality improvement, parental involvement programs, inspectorate visits, and much more (CPB 2016). The motivation for this review was to inform political parties so they could take their positions in the upcoming parliamentary election campaign. Centraal Planbureau had included only studies that followed the golden standard of experimental (or quasi-experimental) research from any possible country, mostly if not only published in economics journals. One of the discussed policies was ability grouping in schools, to enhance learning. Ability grouping is a highly salient issue in Dutch education debates, possibly because differentiation in the classroom is hard and because the Netherlands has an early tracking system whereby secondary school children are in separate tracks and often in separate schools depending on their supposed learning potential. It was this particular issue that was on the national news on public television, where the authors made the case for ability grouping. It appeared that the conclusion in favor of ability grouping in primary schools was based on one experimental study done in Kenya (Duflo, Dupas, and Kremer 2011), leaving aside a whole lot of longitudinal education research on countries more similar to the Netherlands and from other fields. Was that the scientific knowledge on which basis we would want to inform political parties to determine their position?

Another example is the debate on wearing face masks in the coronavirus disease 2019 pandemic. The Dutch government has been reluctant to make face masks mandatory in public buildings. The Outbreak Management Team (OMT), a scientific task force to support policy responses on the outbreak of the coronavirus, stated that there was no scientific evidence for the use of masks. Only around mid-October 2020 did the OMT change its position, and the Dutch government moved accordingly (although still not, I must say, wholeheartedly). The OMT, which bases its policy recommendations on input from the National Institute for Public Health and the Environment, was hesitant because there was no randomized controlled trial on the use of face masks showing that they have the anticipated effect among citizens outside the health professions. New reviews, which included observational studies on the use and effectiveness of face masks, were not taken into consideration (e.g., Peeples 2020), plausibly because such studies do not follow the golden standard of RCTs to examine causal effects. Again, what is the kind of scientific knowledge relevant for policy?

When it comes to ability grouping, or wearing face masks to curb the spread of the virus, there may not be a final answer about their effectiveness. But that is inherent to scientific knowledge. The false sense of certainty that we may sometimes distill from policy-relevant causal research may be countered by more theory-driven research. We need social science theories on why ability grouping may have an effect, and we can pile up evidence on its effectiveness and the underlying mechanisms, using various research designs. Together, they may help us define, to paraphrase Watts, “what is the social science perspective on ability grouping,” because we can stipulate what ability grouping does or does not do to students of different ability groups, or socioeconomic backgrounds, and why.

It may be good to recall Karl Popper’s work here. Popper ([1945] 2013) developed one approach that promotes both the achievement of scientific progress and the organization of an open society with “piecemeal social engineering.” According to Popper, all humans are biased in their views, and all knowledge is provisional, so the only way to get to the truth is to stipulate propositions, testable hypotheses. Along similar lines, Goldthorpe (2001) argued that we can develop theories about the micro-level foundations that constitute a causal explanation for a phenomenon, and each of the building blocks of such an argument can be put to empirical scrutiny. If we integrate this hypothesis-testing approach with a focus on programs, policies, and practices to reduce inequalities, it becomes clear that we need to understand not only whether effects are found but also what the mechanisms are for such effects, the conditions under which effects are found, and the agency that can then be promoted among agents in the process that we study. Voilà—one sees the similarities to DFW’s feasibility studies, perhaps from a more theory-driven perspective.

With such theoretical-empirical approach, a useful knowledge base for policy enables us to look at social phenomena from different angles. We can accumulate knowledge on well-grounded specific research questions that are part of overarching research problems of a field, aiming to contribute to theory development. I would argue that, with good intentions, the two ambitions of “understanding” and “reducing” inequality may be less separated than thought. If sociologists are willing to engage with the field(s) in specifying concrete research questions, in ways that make specific field questions to be part of larger, overarching social scientific problems, it may be possible to help the field as well as to contribute to fundamental understanding of inequalities.

Using a major skill of researchers—formulating research questions and testable hypotheses—to help the field in turning their practical problems into questions of broader scientific interest can foster research that leads both to scientific progress and provide tools for schools (van de Werfhorst 2017). This requires close collaboration among researchers, schools, and policy makers. As an example, such collaborations are institutionalized in the field of education in the Netherlands’ Initiative for Education Research (Nationaal Regieorgaan Onderwijsonderzoek [NRO]), a body that is part of the Netherlands Organisation for Scientific Research, the main national research funding agency. In the NRO, three parties work together—research, policy, and the education field—to fund research, make research accessible and applicable in the field, and help design education policies and practices. ¹ Interactions are facilitated in “knowledge roundabouts,” and stimulated by research funding programs developed by academics, education professionals, and policy makers. Collaborations among these three parties have led to the creation of register databases on school careers (the Netherlands Cohort Study on Education) that are interesting for academics and deliver information that is fed into the schools on a regular basis (Haelermans et al. 2020).

Social and Political Context as a Complement to the Rationalistic Perspective to Public Policy

Research that demonstrates the causal effects of interventions (“policies, programs, and practices”) on inequality has a technocratic, rationalistic character. It suggests that policy makers have tools to solve problems, by adopting successful interventions. Policies, programs, and practices may, under such a rationalistic model, be seen as “buttons to press” for the policy maker. But besides taking account of the fundamental uncertainty described in the previous section, a rational-functional perspective to interventions may need to be complemented with other forms of knowledge relevant for effective public policies.

In part, the social-political context perspective is, like the rationalistic model, aimed at understanding policy effectiveness to reach certain goals. However, it emphasizes the relevance of the social and political context in understanding under which conditions interventions are indeed being evaluated and under which conditions interventions can be more or less effective. There are at least four ways in which the social and political context can be included in ambitions to reduce inequality.

First of all, the process of policy evaluation itself is a political process. Social problems are constructed within policy arenas in which interest groups compete for attention for the social problem they want to address (Hilgartner and Bosk 1988). Also, knowledge about policies requires political leverage and interactions between policy makers and researchers to make a case for a particular evaluation, defining the salience of policy issues, or interpretation of the research results (Bovens, ’t Hart, and Kuipers 2008; Cairney and Oliver 2017). Bovens et al. (2008) described the argumentative approach to policy evaluation as “a contribution to an informed debate among competing interests” (p. 324). It abandons the rationalistic separation of facts and values; designing policy is a value-laden, normative activity, and policy evaluations are part of the political debate. A relevant question is, for instance, which policies will be evaluated. From a very different perspective, Cairney and Oliver (2017) also emphasized the relevance of interactions between researchers and policy makers. They asked how far scientists should (and can) go to persuade policy makers to act in line with their findings. Cairney and Oliver discussed three approaches to defining a hierarchy of types of evidence: (1) give prominence to hardcore evidence-based methods (randomized controlled trials), (2) instead focus on “storytelling” based on testimonies of relevant actors, or (3) the effective mixture of these two, called “improvement science.”

A second way in which the social and political context matters for the effectiveness of inequality-reducing interventions concerns the role of professionals who carry out the intended policies. Most famously, of course, is the idea that professionals are “street-level bureaucrats” who may not always act in line with the intended policy goals (Lipsky 2010). More recently, support for policies among professionals in schools has been argued to be conducive to the effectivity of education policies (Cohen and Mehta 2017; Cohen, Spillane, and Peurach 2018). By and large, this literature concludes that education policies can only be effective if teachers support the underlying values. Effective policy making thus requires interactions between policy makers and professionals in the field(s) to discuss the underlying values of policies.

This brings us to the third way in which the social-political context is relevant in understanding effective policies: be explicit about the values underlying the policies. It may not be specific enough to say that a field looks for tools to reduce inequality. Policy makers can be more specific about the underlying distributive values of their equality-enhancing education policies (Brighouse et al. 2016). Is the principle of adequacy leading, assuming that all students should reach a certain minimum set of skills to be able to participate in society? Or should policies promote equality to make children better able to develop equal opportunities to lead their lives? NYC follow a similar logic in making the case for studying equality of opportunity. Or should policies always benefit the least advantaged most? Although such values vary among individuals and social groups, wider societal cultural tendencies can also be observed relevant for addressing inequality. Mijs (2021) showed, for instance, that meritocratic values in a society are positively associated with the level of income inequality. Inequalities are more easily legitimized if they are seen as the product of meritocratic allocation (Sandel 2020).

A fourth way to bring context into sociological approaches to policy evaluation is to address more fundamentally the institutions that are formed through the application of policy. Under a rationalistic model of policy evaluation, one may think that policies can be rationally devised to address emergent inequalities. However, such rational choice institutionalism is poorly able to explain how institutions take shape (Mahoney and Thelen 2010). Instead, to understand institutional formation, one needs to understand, historically, the role of power dynamics and competing interests.

As an example, we can look at selection processes in schools, as these are well-known practices that generate and perpetuate inequalities. More specifically, educational inequalities are produced by processes of both sorting (who goes to which school?) and learning (who learns what within a school?) (van de Werfhorst 2021). Both processes can theoretically follow a rational-functional logic: sorting in a way that aims to optimize outcomes by bringing each student to the school that best serves their abilities (or at least without harming others), and in schools students learn equally from the offerings of the school. Under such a model, when a causal effect of an intervention on inequality reduction has been found, one may rationally conclude that the intervention can be rolled out. However, if sorting and learning follow the logic of boundary making rather than rational allocation, sorting into educational tracks may happen to “construct and reinforce highly salient social categories” (Domina et al. 2017:312). If, once students are in school, children from disadvantaged backgrounds suffer in elite school environments because of a lack of cultural capital, the learning process may then also become unequal as a consequence of boundary-making efforts. If categories in education, including the “policies, programs, and practices” that define them, follow cultural boundary-making efforts, as would follow from a relational-inequality theory as propagated by NYC, it may be much harder to rationally address inequalities by demonstrably effective interventions. Boundary-making efforts may be so fundamental that an effective intervention will be countered by new forms of middle-class family behavior to maintain children’s advantaged position. Public knowledge about boundary making (as opposed to rational allocation of students) may be essential to reduce inequalities, as policy makers and school boards are all too often, somewhat naively, building their policies upon rationalistic views on optimal sorting and learning. A public concern with such fundamental theories of stratification may help, as NYC (p. 8) propose, to “delineate and counter the mechanisms that enhance the power of elite institutions.”

Importantly, a more serious engagement with the social and political context of reform may also lead us to conclude that step-by-step “piecemeal social engineering” is not radical enough to reduce inequality (Jackson 2021). With small steps we lose sight of the cumulative process of how disadvantage accumulates over domains, such as health, education, housing, and work.

Conclusion: Which Knowledge Is Relevant for Reducing Inequality?

In line with the articles by Gamoran (2021), DiPrete and Fox-Williams (2021), and Nalani et al. (2021), I have highlighted that sociology should both be concerned with identifying causal effects of interventions and understand the social and political context of the evaluation of policies, programs, and practices to understand and reduce inequalities. When the context is more seriously included in an intervention-oriented social science, in ways that incorporates theory building in collaborations between social scientists, policy makers, and the field, the connection between understanding and reducing inequality becomes evident. Such an approach would mean that social scientists can take a role in influencing policy debates, help schools, housing agencies, employment activation organizations, or health institutions, and at the same time follow theory-building efforts in bridging research and practice. It would require social scientists to be more concerned with working with partners outside the academy, in ways that are more common in other fields such as health and engineering. It will remove the distinction between applied and fundamental research, as DFW have advocated. It produces feasibility research as DFW describe it. And it will produce the data infrastructure and knowledge base that NYC propose.

Following a theory-driven approach that more explicitly tests hypotheses on mechanisms underlying causal effects, it will also be evident that a wider array of knowledge becomes useful for policy makers. Rather than strictly defining evidence-based knowledge on the basis of the golden standard of experimental research, we can use a wider array of research designs to test hypotheses on (mechanisms behind) policy effects, including ethnographic fieldwork, internationally comparative research, and (quasi-)experimental research. Such knowledge may help us understand the relational processes behind existing inequalities, heterogeneous effects of policies for different subgroups, and the context within which interventions can be made effective.