Abstract
Many recent studies of the judiciary and public opinion adopt a model that views court decisions as aggravating division within the public. The authors question the image of Court as polarizer, arguing that the persuading influence of the U.S. Supreme Court is broader than contemporary authors acknowledge. Using a potential outcomes framework, the authors analyze public attitudes in response to the decision in Roe v. Wade, the original test case in Franklin and Kosaki’s seminal article. The authors’ evidence suggests that members of diverse groups who were aware of the Roe decision were more supportive of abortion than their decision-unaware counterparts.
Do citizens follow the guidance of the Supreme Court on controversial issues? To a greater degree than for other American political institutions, the question of public assent is important for the Court. In Federalist 78, Alexander Hamilton declared that “[t]he judiciary . . . has no influence over either the sword or the purse; no direction either of the strength or of the wealth of the society; and can take no active resolution whatever. It may truly be said to have neither force nor will, but merely judgment.” This statement articulates the Court’s fundamental weakness: a body with no formal enforcement mechanism naturally depends on its ability to convince the other branches and their constituents of the legitimacy of its judgments. And unlike the legislative or executive branches, the Supreme Court cannot be forced to quickly respond to public disapproval, as the norm of stare decisis constrains the Court from reversing itself in the face of an unreceptive public even if it were so inclined. During the past half century—a time of significant activity by the Court—a debate has emerged over the Court’s ability to convert its stores of legitimacy in the public’s mind into support for specific political stances. While early scholarship posits the Court as “republican schoolmaster” (Lerner 1967), efficacious in bringing public opinion to its side, the past two decades have seen the rise of a view emphasizing the divisive fallout from decisions.
Franklin and Kosaki (1989) contend that the republican schoolmaster theory has suffered from an absence of empirical evidence supporting a legitimacy-conferring effect of the Court, particularly in the prominent ruling in Roe v. Wade (decided January 21, 1973), which entered into the divisive political issue of abortion. Addressing the apparent inability of data analysis to demonstrate a causal link between Roe and aggregate public opinion change, Franklin and Kosaki contend that the Court instead acted as a polarizing force in American politics, producing a restructuring of opinion along socioeconomic and sectarian lines. Nevertheless, a number of studies that have emerged since Franklin and Kosaki’s article—particularly those utilizing an experimental research design to examine cases other than Roe—have found an overall opinion change in the direction of Supreme Court rulings.
Franklin and Kosaki’s finding is not easily reconcilable with these newer works, but their article has nevertheless remained hugely influential in the literature on the Supreme Court and public opinion. This apparent conflict recommends closer examination and specifically as to whether the methodological assumptions present in Franklin and Kosaki’s observational analysis may be responsible for the founding of the structural response hypothesis.
This article revisits the question of how the public reacted to Roe v. Wade using a more explicit causal model based on the potential outcomes framework. Its key finding is that, contrary to Franklin and Kosaki, the public actually responded positively to the Court’s decision in Roe; moreover, the increases in public support are almost uniformly nonnegative across social groups said to be polarized in Franklin and Kosaki’s article.
Legitimation versus Structural Change
Much of the literature on the Supreme Court has discussed to the institution’s ability (or inability) to legitimate various policies (Adamany 1973; Casper 1976; Dahl 1957; Funston 1975; Hoekstra and Segal 1996; Lerner 1967). At the heart of this discourse is the idea that the Supreme Court enjoys a certain degree of public standing and that it is able to transfer that legitimacy to the policy positions its decisions favor. Lerner (1967) argues that early in the Court’s history, the justices were aware of their roles as civic educators and intentionally used their position to influence the citizenry. Lerner’s argument is that the founding generation understood the necessity of public support for “the faithful discharge” of judicial power to occur and organized the practices of the Court—such as the requirement that justices ride circuit—to encourage understanding of the judicial system and to develop civic virtue generally. In more recent times, the Court has taken cognizance of the public when issuing some of its most important rulings, a noted instance being Brown v. Board of Education (1954), in which Chief Justice Earl Warren took pains to compile a unanimous Court and also sought to limit the length of the Court’s opinion to help ensure that newspapers would reprint the decision in its entirety (Abraham 1977, 372). Scholars have also shown on many occasions that the Court maintains a high degree of public confidence as an institution—the Court’s legitimacy has been tied to core understandings of the democratic process and is believed to be fairly unshakeable (Caldeira 1986; Caldeira and Gibson 1992; Gibson, Caldeira, and Spence 2003). It has been suggested that this institutional legitimacy acts as political capital that the Court may choose to spend (Grosskopf and Mondak 1998; Mondak 1992). However deep the roots, the Court’s good standing with the public is thought to be a key factor in gaining a positive public reaction the nature of its substantive findings.
This theory, that the Court is able to transfer its good standing with the public to popular support for its decisions, draws additional support from social psychology. Studies in this field have demonstrated that, when messages come from credible and likeable sources, individuals are more likely to be persuaded (Chaiken 1980; Petty, Cacioppo, and Goldman 1981). While there has been debate as to whether or not perceptions of procedural fairness contribute to the Supreme Court’s high marks in this area (Gibson 1989; Tyler and Rasinski 1991), survey data have repeatedly shown that the public holds the Court in high esteem, particularly when compared to the other two branches of government (Mate and Wright 2008). As such, this high regard for the institution combined with psychological theories of persuasion are at the foundation of Supreme Court legitimation theory.
In attempting to build on this notion that the Court may act to legitimate certain policies, several studies have struggled with the lack of findings that would readily indicate such a phenomenon. Many of these look, as we do, at Roe v. Wade and the subsequent shifts in public opinion on abortion. Although some have noted positive trends in favorable attitudes toward legal abortion following the Court’s 1973 decision (Arney and Trescher 1976; Ebaugh and Haney 1980), others have found an absence of any causal evidence that connects these shifts on abortion opinions to the Court decision (Blake 1977; Rosenberg 1991; Uslaner and Weber 1979). In addition, studies that have analyzed other cases have had mixed results in their attempts to show a causal link between Supreme Court rulings and aggregate opinion (Adamany 1973; Marshall 1987).
Franklin and Kosaki organize this strain of research into what they call the “positive response” model, which predicts a positive net change in public opinion on a given issue in response to a Court’s ruling. Using Roe v. Wade, Franklin and Kosaki challenge the positive response hypothesis, countering with a “structural response” hypothesis, which does not depend on an aggregate change in public opinion. Instead, the structural response hypothesis predicts a polarization of attitudes between groups that were already on one side or the other of the issue in question. Their findings with respect to Roe indicate that certain groups, such as Catholics and blacks, who were already disposed to oppose abortion, became more disapproving of the practice in “discretionary” situations. Within other groups, attitudes became more favorable toward abortion. Considered together, such opposing changes may preclude a noticeable net effect in the entire population. Franklin and Kosaki argue that this polarization can be substantial and that Court decisions therefore remain vital to understanding public opinion. They conclude that the positive response hypothesis cannot account for the structural change that occurred immediately following Roe and that only the structural response hypothesis provides the machinery necessary to understand public reaction to this case.
This alternative view of the Court’s influence has been prevalent in the literature exploring public reaction to high courts. Franklin and Kosaki’s model has been foundational in works examining other specific aspects of the relationship between the judiciary and the public, such as the Court’s effect on local communities (Hoekstra 2000, 2003; Hoekstra and Segal 1996), the role of media coverage and political context in shaping the Court’s effect (Stoutenborough, Haider-Markel, and Allen 2006), and the effect of courts on the public in other nations (Baird and Javeline 2007; Gibson and Caldeira 2003). Johnson and Martin (1998) build off the structural response hypothesis to formulate their “conditional response” hypothesis, which posits that the Supreme Court may have little influence on public opinion in salient cases where the Court has already issued a landmark decision in the same topic area. Their study looks at Webster v. Reproductive Health Services (1989), a subsequent abortion case, as well as three death penalty cases, Furman v. Georgia (1972), Gregg v. Georgia (1976), and McClesky v. Kemp (1987), to determine whether or not structural attitude change occurs after the Court revisits a salient issue. They find that structural change is conditional on the Court deciding the issue for the first time. However, in spite of their revision to Franklin and Kosaki’s theory, Johnson and Martin do not dispute the original study’s findings but merely extend the Franklin and Kosaki’s methodology to other Court cases.
However, there may still be a glimmer of hope for the positive response hypothesis. Experimental studies by Mondak (1990, 1991, 1992, 1994) and others (Clawson, Kegler, and Waltenburg 2001; Hoekstra 1995) searching for a legitimation effect from the Court have been much more successful in producing results. However, these experimental studies still draw from Franklin and Kosaki’s model for structural change and have even criticized early experimental models (such as Baas and Thomas 1984) for neglecting the structure of public opinion. Still, Franklin and Kosaki’s work, being an observational study, inevitably lacks the strength of the experimental model. It is possible, therefore, that the results of their study depend on the functional form used to estimate the effect of the decision. Mindful of the inconclusiveness of the findings in the literature on this subject, we turn to a thorough examination of the Franklin and Kosaki finding.
Roe v. Wade as a Case Study
The advantages to using Roe v. Wade as our test case are twofold. First, it is the case used by Franklin and Kosaki in their pivotal study. If we are to use a different identification strategy to arrive at a result that supports the positive response hypothesis, it is best that we respond directly to the study that initially challenged it. Second, scholars have placed a great deal of emphasis on how public opinion of legal abortion varies by subgroup. Interestingly, there is little evidence for a gender gap on abortion opinion (Craig and O’Brien 1993; Strickler and Danigelis 2002), and when such a gap is found to exist, it is men who favor the practice more than women (Cook, Jelen, and Wilcox 1992; Luks and Salamone 2008). A number of studies have noted a racial gap on the issue (Combs and Welch 1982; Hall and Ferree 1986; Scott and Schuman 1988) that converged in the late 1980s through the mid-1990s (Strickler and Danigelis 2002; Wilcox 1990). In addition, many have found that religion is associated with abortion opinion; Catholics and evangelicals are often found to be less supportive of abortion than Jews and mainline Protestants (Blake 1977; Cook, Jelen, and Wilcox 1993; Granberg and Granberg 1980; Legge 1983). However, several scholars suggest that it is religiosity, not the religion itself, that is associated with antiabortion sentiments (Craig and O’Brien 1993; Singh and Leahy 1978). Finally, high educational levels are also correlated with support for abortion rights (Luks and Salamone 2008). Thus, as subgroup membership has been shown to be highly connected to one’s opinion on this issue, it is certainly worthwhile to take a closer look at the Court’s effect on the structure of abortion opinion.
If we look then for a case likely to have an effect that creates a polarization and restructuring of opinion on public opinion, Roe is a prime candidate. While abortion was a fairly salient issue in the early 1970s, media coverage of Roe and its companion case, Doe v. Bolton (1973), was relatively light as compared with the attention surrounding follow-up Supreme Court cases decided in the late 1980s and early 1990s (Luks and Salamone 2008, 85). Though many held passionate views on the question of abortion before Roe, public opinion on the issue indicated a level of unsettledness. Greenhouse and Siegel (2010) attempt to reconstitute the political landscape surrounding abortion at the moment of Roe and find “many moving and intersecting parts” to the debate in the 1960s, invoking oft-forgotten background considerations such as the thalidomide scandal and the zero population growth movement. Hence, Roe presents a case where any polarization may be thought to have been a consequence of the Court’s decision and not prolonged public attention to the case in the months or years leading up to the decision.
Before considering the specific effect of the Roe v. Wade decision on abortion attitudes, it is instructive to see, again at the bivariate level, how opinions changed between 1972 and 1973 on two dependent variables measuring attitudes about abortion changed. In doing this, we can get a rough impression of what dynamics were at work during this period. Following prior research on this subject, we divide into two groups a battery of six questions on the General Social Survey (GSS) asking whether abortion should be legal under specific circumstances. Responses to whether abortion should be legal if a pregnancy results from rape, if the pregnant woman’s life is seriously endangered by the pregnancy, or if there is a strong likelihood that the unborn child will have a serious defect are grouped into what is called a health scale. Questions on the discretionary scale ask whether abortion should be legal if a family is poor and cannot afford any more children, if a pregnant woman is single and does not want to marry the man, or if a married woman does not want any more children. 1
As seen in Table 1, the full sample appears to be more permissive (in terms of both the health and discretionary scales) in 1973 than in 1972, a difference that is statistically significant. Furthermore, the subgroup-specific results already cast some doubt on Franklin and Kosaki’s structural response hypothesis; rather than the expected polarization, we observe that almost every category of respondents is more permissive, on average, in 1973 than it had been in 1972, and most of these differences are statistically significant. No single subgroup appears to move in a less permissive direction, considering only statistically significant results. When discretionary and health scores are considered together for the entire GSS sample, the changes 1972–73 are substantively greater than those observed in any other year in the period 1972–78. Table 2 shows mean increases in both the discretionary and health scales from 1972 to 1973 followed by a fairly stable series of observations. Even though the support for discretionary abortions dipped from 1977 to 1978, this is not accompanied by a large change in support on the health scale. Of course, simply showing mean differences between years is a useful first cut but does not allow us to accurately gauge the role of the Supreme Court’s decision on these attitudes. Other research suggests that attitudes about abortion were already changing in the late 1960s and early 1970s prior to the Court’s decision (Barnum 1985; Blake 1977). In addition, the 1972 and 1973 GSS field periods were separated not only by the Roe decision but also by a presidential election and other events. Comparison of opinions pre- and postdecision by itself provides no leverage on assessing the effect of Roe versus the alternative maturation and history explanations. Fortunately, the 1973 GSS gives us a tool that, properly considered, may help us to isolate the effect of the Court’s decision. In addition to being asked their opinions on abortion, respondents were asked whether they had “heard or read of the recent Supreme Court decision concerning abortion.” We now turn our attention to structuring a research design that will exploit variation on this question to permit stronger inferences about the effect of the Supreme Court’s decision on public opinion.
Mean Abortion Support, Unmatched Population Group Comparisons
p < .05. **p < .01. ***p < .001 (Student’s t-test, two-tailed).
Mean Support for Abortions on Discretionary and Health Scales
Identification and Estimation
As noted above, the key question is whether or not the Supreme Court’s decision in Roe v. Wade had any causal effect on public opinion, both on the whole and within subgroups. From an empirical standpoint, the “effect” of a court decision is usually understood in one of two different ways. The first understanding construes broadly the effect of a court’s decision by looking at opinion change within the entire population exposed to a court decision. Such an approach would encompass both individual reactions to the court’s decision itself as well as effects mediated by a host of other actors (politicians, media, interest groups, opinion leaders) who may influence individual opinions regardless of whether an individual is aware of the decision itself or its content. This understanding suggests a simple comparison of aggregate opinion before and after a court decision, but any such analysis suffers from the possibility that some other factor or event is responsible for the observed change. Some commentators have speculated that the Supreme Court—rather than influencing public opinion—follows the public, a hypothesis that is difficult (if not impossible) to rebut given evidence of a change in opinion. 2
A second understanding of the effect of a decision would be restricted to opinion change among individuals who are cognitively aware of a decision and its source. This second aspect of a decision’s effect is convenient for survey experiments that manipulate information presented to respondents or present hypothetical court decisions. But while survey experiments provide freedom in formulating treatments and permit random assignment to treatment conditions, it is more difficult within this framework to provide treated individuals an opportunity to ruminate over the decision or talk with others about it. Such a design would successfully identify a parameter of interest—the causal effect of a court’s reputation on individual attitudes toward a public policy. However, this parameter might represent only a portion of the effect of a decision on public opinion. Alternatively, if discussion and individual contemplation of a given decision (or the passage of time itself) dull a person’s reaction to a judicial decision, a survey experiment could overestimate the effect of a decision on opinions.
Lacking the ability to control experimentally whether individuals are subject to a judicial decision, we rely on a comparison of respondents who told an interviewer that they had heard about the Roe v. Wade decision to those who replied that they had not. This approach attempts to capture the outcomes of real-world treatments while avoiding the danger that other events or trends contemporaneous with the court decision were in fact responsible for a change. The key challenge present in using observational data to determine the effect of the Court’s decision is that the “treatment” (hearing about the Roe v. Wade decision) is not randomly assigned; rather, it is highly probable that treatment depends on a number of factors. For instance, one could easily imagine the likelihood of a given respondent hearing about the decision being affected by his or her religious affiliation, education level, gender, or any number of other such factors. In the absence of random assignment, we cannot make an a priori claim of equivalence between “hearers” and “nonhearers.” 3
Franklin and Kosaki go to some length in considering its implications for their research. Their solution is to look at respondents in the 1973 GSS who had heard of the Roe decision and compare them to all respondents in the 1972 GSS and, separately, to look at respondents who had not heard of the decision in 1973 and compare these to all 1972 respondents. Franklin and Kosaki concede that the covariate distribution in the 1972 sample is likely to be different from that of either of the 1973 groups with which it is fused. For instance, when 1973 decision-awares and 1972 respondents are analyzed, the group of 1973 decision-awares contains a greater proportion of women and whites and the 1973 decision-aware group on average attends church more often, has a greater amount of formal education, and tends toward a more conservative party identification. 4 The imbalance between treatment and control in Franklin and Kosaki’s analysis therefore requires correct specification of their ordered probit model to produce unbiased estimates of causal effects. Incorrect specification introduces the possibility that the changes in slopes postdecision (the β2k coefficients in Franklin and Kosaki) could be the result of any improvement gained in adding a second coefficient to model the covariate–outcome relationship over the more homogeneous set of 1973 cases. Finally, Franklin and Kosaki’s model does not permit the effect of church attendance on opinion to change after Roe is decided, despite the role of religious observance in their theory of polarization. Unquestionably, churches and church communities have an important influence on opinions, but we consider it strongly likely that Roe would affect the attention paid to the abortion issue in church settings. As a result, the relationship between being present in those church settings and abortion opinion should be different than before any effect of the Roe decision on church discourse.
We seek to reexamine the results of the Franklin and Kosaki article using an identification strategy that does not require correct specification of the causal model. We claim that by matching members of the relatively small pool of Roe-unaware respondents to Roe-aware individuals we can produce pairs wherein there exists an equal likelihood of either individual receiving treatment and that therefore treatment assignment can be thought of as randomly assigned within the pair. The selection on observables assumption requires that all variables (both observed and unobserved) that have an effect on both the participants’ outcome and treatment status be equally distributed between treatment and control groups if unbiased estimates are to be obtained.
To obtain pairs in which Roe-aware respondents are twinned with respondents who are unaware of the Court’s decision, we employ GenMatch, a software program developed by Sekhon (forthcoming), which uses a genetic algorithm (Mebane and Sekhon 1998; Sekhon and Mebane 1998) to implement a nonparametric matching technique. GenMatch uses a process similar to a Markov chain to create a vector of weights for the covariates in a given list—these weights then provide the relative importance of each covariate in the procedure that will choose matches for the control cases. 5 If a vector of weights results in matches that have good balance between treatment and control, then the vector “survives” to help create new vectors of weights. If a vector does not improve balance, it is discarded. After repeated iterations of the vector-producing routine, GenMatch arrives at a set of weights (and therefore matches) that all but ceases to improve under subsequent modifications. Because GenMatch can optimize over a number of different balance metrics and does not rely on assumptions about the distribution of covariates or sample size, the matches found lead to higher efficiency and lower bias than other matching techniques (including Mahalanobis or propensity score matching; Diamond and Sekhon 2005). GenMatch has been used in a variety of applications both in political science (Gordon and Huber 2007; Herron and Wand 2007) as well as outside (Grieve et al. 2008).
Compared to the experimental benchmark, our analysis falls short in that we are forced to match on characteristics relevant to treatment assignment and outcome if we are to obtain unbiased estimates, and this matching in turn relies on the stable unit treatment value assumption (SUTVA), the common support assumption, and the selection on observables assumption. SUTVA requires that the outcome of a given treatment allocation for a participant is identical to the outcomes that would occur under any allocation of treatments where he or she receives treatment. The two most common ways in which this assumption can be violated are (a) when there are multiple versions of the treatment varying in effectiveness and (b) when there is interference between participants (Rubin 1990). SUTVA poses a small difficulty for our analysis, as we are unable to discover how individuals learned about the decision. It is likely that at least some individuals receive treatment because of the treatment assignments of other individuals. These treated individuals differ from members of the control group in that they have family members or friends who talk to them about current events and may consequently also differ from control units in a number of important ways.
Moreover, our matching (described below) shows that the lack of common support is not a critical issue. One could still argue that we are not conditioning on a sufficient number of relevant covariates, and thus, our estimates are still being biased. We investigate this possibility by comparing treatment and control groups on a range of controversial social issues. If we have found appropriate matches for each of our nonhearers, membership in the treated and control groups may be thought of as random within each pair, and this should be inconsequential in predicting views on issues other than abortion. (We consider it unlikely that individuals will systematically form or reevaluate opinions about other social questions in response to knowledge of court action on abortion.) If, instead, differences in consumption of current events information, wealth of social networks, and so on persist, we might expect the treated and control groups to have divergent opinions.
Before any matching is performed, it is fairly clear that hearers and nonhearers have different stances on a range of social issues. Persons in the GSS sample who heard about the Roe decision were more liberal in attitudes toward homosexuality, less likely to say that they would oppose a family member inviting a black friend to dinner, and more confident in the scientific community. Nonhearers tended to be less favorable toward the death penalty, with these results narrowly missing a p = .10 standard.
As seen in Table 3, by matching on covariates, we are able to remove much of the difference between hearers and nonhearers on most issues. While Roe-aware respondents remained somewhat more liberal on questions relating to interracial marriage (t-test p value = .15) and considerably more tolerant of openly gay college professors (p = .04), hearers do not appear to be more tolerant than nonhearers when it comes to the prospect of a black dinner guest (t-test p = .11, Kolmogorov–Smirnov [KS] test p = .85) or the subject of homosexuality more broadly (t-test p = .79, KS test p = .45). The failure of matching to provide full balance on death penalty opinion is somewhat troubling, as the Supreme Court in 1972 handed down a decision in the case of Furman v. Georgia striking down state death penalty statutes and invalidating hundreds of death sentences nationwide. Like Roe v. Wade, Furman was decided in the period between the 1972 and 1973 GSS field periods. The fairly substantial (though again, not statistically significant) remaining difference between treated and control suggests one of two possibilities. The first is that the groups are somehow ideologically different, with hearers more likely to support both abortion and capital punishment. The second is that persons who heard about the abortion decision were more likely to hear about the death penalty decision and changed their opinions with respect to both. Ideally, our matching would be so effective that members of the treatment and control would be equally likely to have heard about Furman, the death penalty decision, but we cannot ensure this. Even if hearers in the matched sample also heard about Furman, they do not appear to possess dramatically more liberal views across all social issues. 6 A residual tendency toward some liberal positions among hearers may remain postmatching, but liberalism or access to liberal streams of information does not appear responsible for whether a respondent falls into the treatment or control group.
Group Mean Differences (Treatment – Control) and Balance Tests for Social Policy Questions
Note: Scales for homosexuality, extramarital sex, and pornography have been recoded to a 0–1 interval. A positive mean difference indicates greater acceptance, tolerance, or support by the group of “hearers” than “nonhearers.” t-test and bootstrapped Kolmogorov–Smirnov (KS) columns denote p values. Higher p values indicate a lower likelihood of inequality between groups.
Estimation Strategy
Our estimation strategy is as follows: First, our treatment variables are as specified above. Second, our outcomes make use of a battery of GSS questions that ask respondents whether or not they would permit abortion under a given circumstance. Parallel to Franklin and Kosaki, we report the results for items grouped into two 3-item additive scales, one including “health”-related circumstances and the other including “discretion”-related circumstances. We utilize the same covariates as did Franklin and Kosaki: education level, gender (denoted as female), a white/nonwhite indicator variable, a measure of church attendance, and an indicator for Catholic respondents. We supplement these covariates with a number of additional variables, adding vote 1972 (shown in balance statistics as vote—or intention—for Richard Nixon, Nixon ’72), age, income, single status, place size, and willingness to have a black dinner guest. These additional covariates help to maximize balance across the treatment and control groups. Including explanatory covariates follows Ho et al.’s (2007, p. 216) counsel that “all variables in Xi that would have been included in a parametric model without pre-processing should be included in the matching procedure.” Therefore, covariates that would be used in regression models to predict abortion opinion—gender, religion, and so on—are incorporated when matching hearers to nonhearers. We are limited in our ability to include covariates that would predict hearing about the decision but that might not meaningfully influence abortion opinion itself (such as a variable capturing media exposure). A question about newspaper readership was not asked in 1973 or 1974, and other questions about media consumption were not asked until 1975. There were neither questions about whether individuals were aware of other events nor factual questions, either of which would allow us to test whether hearers were more knowledgeable—objectively or in their own perception. The GSS has only seldom asked questions about interest in politics. Since we cannot demonstrate that persons hearing about the Court’s decision in Roe were not more aware about public affairs than those who did not hear about the decision, our analysis relies more heavily on the finding that hearing about the Supreme Court’s decision correlates with opinion on abortion questions but is unrelated to opinion on a variety of issues on which we might think political awareness might be associated with opinion.
Considering the dependent variable, one source of debate in the use of multi-item scales in survey data concerns the handling of cases where a respondent did not provide a yes or no answer to one or more of the index components. Franklin and Kosaki removed cases from their data set if a respondent declined to answer a question in either the discretionary or health scale. Our examination of respondents answering “don’t know” to at least one question indicates that these cases are not ignorable. Respondents who answered that they did not know whether abortion should be permitted in a discretionary case were likely to say that they supported abortion in each and every one of the health scale scenarios. Consider a respondent who answers “don’t know” to a question pertaining to abortion when a woman’s health is endangered; such a person almost certainly opposes abortion in all so-called discretionary instances. Conversely, respondents who were uncertain about whether abortion should be permitted in a discretionary case—when a mother is married and wants no more children or is too poor to afford to raise a child—overwhelmingly supported abortion rights where rape, birth defects, or the mother’s health was a factor. To take a pair of representative illustrations, the mean discretionary score was .28 (out of 3) among respondents who answered “don’t know” to the mother’s health example; this is one-fifth the average score of for the entire sample. The average health scale score for a respondent who answered “don’t know” to the (discretionary) poverty question was 2.93.
It seems to us abundantly clear that persons who fail to answer yes or no to a question about a particular abortion question have bona fide attitudes toward abortion generally. Failure to respond definitively to a question about abortion in one or two instances thus does not suggest the presence of nonattitudes on the subject of abortion generally; a faithful attempt to assess opinion change in this area should account for the possibility that a salient event may cause individuals to become uncertain on applications for which they previously had a definite opinion. Similarly, individuals who had been uncertain about the permissibility of abortion in a particular scenario may come to a conclusion after being presented with new information.
In the results presented in this article, “don’t know” responses are coded as .5 (between 0 for “no” and 1 for “yes”). Empirically, by expanding our sampling frame slightly by the inclusion of respondents with “don’t know” answers, we are able to reduce imbalance between hearers and nonhearers on marital status, and among our placebos, better balance is achieved when we incorporate the “don’t know” respondents. (As shown in Table 6, when we perform matching using the Franklin and Kosaki method of case deletion, we do not achieve a statistically significant treatment effect on the discretionary scale, but the results hold for the health scale.)
Matching Results
As noted above, our design involves matching individuals in a control group to those in a treatment group based on the five covariates in Franklin and Kosaki’s article and six additional covariates not incorporated in the article. The treatment group (those having heard of the decision) contains 1,288 cases and the control group 216.
We used the GenMatch program to improve balance between treatment and control based on a number of covariates. To examine the Franklin and Kosaki hypothesis of structural change, we require the matching procedure to only produce pairs where gender, Catholicism, and race are identical across the treatment and control. Thus, white Catholic men are matched to other white Catholic men, African American Protestant women to African American Protestant women, and so on. We did not specify exact matching—but nevertheless sought balance—on remaining covariates.
This procedure worked quite well. We can see by looking at Table 4 that, prior to matching, the distributions of four of the five Franklin and Kosaki covariates (the Catholic dummy variable being the exception) were considerably different between treatment and control groups. After the match, however, balance was considerably higher, exceeding the critical values of our test statistics. By exactly matching on race, gender, and Catholicism, we achieve balance on those characteristics by construction.
Group Mean Differences (Treatment – Control) and Balance Tests for Covariates, Pre- and Postmatching
Note: Except for mean difference columns, cells denote p values. The Kolmogorov–Smirnov (KS) test is not applicable for dichotomous variables.
Results
Even before presenting the matching results, we examine how the basic socioeconomic factors in our analysis relate to attitudes as measured on the health and discretionary scales first used by Franklin and Kosaki. Bivariate correlations, presented in Table 5, are broadly consonant with previous work on this subject. Permissiveness increases with years of education and decreases with respondents’ level of church attendance. Catholics and blacks are substantially less permissive than non-Catholics and whites, respectively, and females are slightly less permissive than males (though only in terms of the discretionary scale).
Relationships between Matching Covariates and Abortion Scales
Note: Pearson correlation coefficients (significance tests are two-tailed).
p < .05. **p < .01. ***p < .001.
In Table 6, we report estimates of the effect of hearing about the Supreme Court’s decision on opinions toward abortion. On the health scale, respondents who had heard of the decision increased their permissiveness for abortion by .208 (on the 3-point scale) more than respondents who had not heard of the decision. Using standard errors for matched data as specified by Abadie and Imbens (2006), we find this result to achieve a high degree of statistical significance (p = .012). This effect parallels the finding of Franklin and Kosaki’s polychotomous probit model with respect to health-related abortions—respondents in 1973, as a whole, had higher levels of support than 1972 respondents, and there was not a statistically significant relationship between race, religion, or other factors and the observed effect of the decision.
Treatment Effects for Roe Awareness
On the discretionary scale, respondents who had heard of the decision increased their scores by .25 relative to Roe-unaware respondents. The Abadie–Imbens standard error for this analysis similarly indicates a strong degree of statistical significance (p = .039). The difference, which we take to be the effect of the Court’s decision itself, indicates evidence for a positive response not found by Franklin and Kosaki.
When we employ Franklin and Kosaki’s method for handling “don’t know” responses—namely, to drop from the sample any respondent who answers “don’t know” to any of the six questions, even if he or she has a complete set of yes–no answers to one scale—the results are largely the same as with our approach. On the discretionary scale, the average treatment effect on the untreated group is virtually unchanged: .241 rather than .25. The standard error rises to .134, meaning the .05 level of significance is no longer attained (p = .073). For the health scale, the treatment effect increases from .208 to .247 and the level of statistical significance become stronger, to p = .007.
The positive response hypothesis implies large effects in the direction of the decision and small deviations by group from the decision effect. The structural change hypothesis, by contrast, implies that overall change should be “small” but group effects should be large and increase the group’s deviation from the population median. So if the structural change hypothesis is correct, we would expect non-Catholics to respond more positively than Catholics, highly educated people more positively than poorly educated people, whites more than blacks, and so on. In Table 7 we present estimates of the mean change in a number of demographic groups. The first column of each side of Table 7 gives the mean score for matched respondents who were unaware of the decision in Roe; the second column gives the mean value for the matched individuals who were aware of the decision. The third column of each half represents the difference between unaware and aware respondents.
Effects for Population Groups
If we look solely at the observed effects among those who had heard of the decision, we see that support for abortion increased in nearly every one of the demographic groups studied. Among Catholics, the treatment group does have a lower level of support for health-related abortion than the control group, consistent with a theory of structural response. It bears noting, however, that the magnitude of this effect is quite small: .17, or approximately one fewer proabortion answer in a group of six Catholic respondents. However, among respondents who had heard of the Court’s decision, Catholics and non-Catholics have approximately the same level of permissiveness toward abortion, with the average respondent believing that abortion should be allowed in 2.47 (Catholics) versus 2.55 (Protestants) of the 3 instances provided in the survey instrument. This result is hardly consistent with a story of opinion polarization attributable to group memberships.
For all groups considered except infrequent church attenders, the effect of the decision appears to have been higher levels of permissiveness toward abortion. The negative effect among low church attenders, compared with the positive results for medium and frequent attendance respondents, calls Franklin and Kosaki’s theory into even sharper question, since it is institutions (and the social networks that accompany them) that are posited to be doing the work of counter-Court agitation. Yet our analysis shows that it is at the lowest levels of institutional exposure where assent to the Supreme Court’s decision appears to be the weakest.
Further illustration of consistency toward abortion support can be seen by isolating the matched pairs where respondents expressed different abortion opinions. Creating matches between the treatment and control groups permits us to examine for each pair the amount by which the “treated” individual gave different responses from his or her “untreated” counterpart. In this manner, we can think about each match as falling into one of three separate categories: the treated participant is more permissive of abortion than the control, the treated participant is less permissive, or the treated and control participants have the same level of permissiveness. If the Supreme Court’s decision in Roe had a consistent effect on permissiveness across groups, membership in a group should not be related to the likelihood that the more permissive member of a matched pair will be the individual who was treated, as opposed to the control. If instead the Court’s decision had a polarizing effect, we would expect different results by groups as to whether the more abortion-supportive member of a pair is in the treatment or control group. Following Franklin and Kosaki’s findings that response to Roe differed by gender and Catholicism, we break up respondents into gender and religious group (and also race) to test whether within a certain group (as opposed to others) hearers, on average, are more likely to have a higher abortion permissiveness than nonhearers.
On the health scale, 106 of 216 pairs had a discordant outcome, while on the discretionary scale, 143 out of 216 pairs had a discordant outcome. Examination of Table 8 does not support the standard structural response narrative with respect to differences in discretionary scale responses to the abortion decision. Here, it is among non-Catholic women that nonhearers are more supportive. And as we have previously seen, Catholic and non-Catholic respondents who heard about the decision did not differ appreciably from one another on the health scale. Considering the breakdown of discordant pairs on the health scale by race and gender, the rate at which the more permissive member of the pair found himself or herself in the treatment group ranged only from 50 percent among black men to 72 percent among white men. Unfortunately, the small number of discordant Catholic pairs argues against the ability of these data to rigorously test the hypothesis that structural change occurred in addition to the positive responses already noted.
Summary of Discordant Pairs, by Religion and Gender, and by Race and Gender
Conclusion and Discussion
Our results indicate that a positive public response to the Supreme Court’s decision in Roe v. Wade was more robust and widespread than has been acknowledged in the recent literature. This finding is consistent with a depiction of the prolife movement as unorganized and fractious in the years following the decision (Epstein and Kobylka 1992, 207). In an issue area where changing public attitudes yielded to many years of essentially stable division (Stanley and Niemi 1992), the Court’s action appears nonetheless to have boosted public support for abortion in at least the short term.
The present work does not aim to be generalizable in the usual sense; indeed, we cannot say with any degree of certainty that the positive response apparent after the Roe decision should materialize in other instances as well. The limitations of extant opinion data make this difficult. Franklin and Kosaki’s original study was made possible by the fact that the GSS was in the field at the time and postdecision respondents were asked whether or not they had heard about it—this fortunate pattern of data collection exists for Webster v. Reproductive Health Services (see Johnson and Martin [1998], who find no structural change in opinion) but does not appear to have been employed surrounding any other Supreme Court decisions. The lack of survey items specifically tapping respondents’ awareness of Court decisions is an impediment that can be overcome only with new data collected as cases are handed down. It remains plausible that, within a more polarized context, high-salience Supreme Court rulings will not change aggregate opinion on a particular issue.
Admittedly, our work is a critical examination of the conclusions researchers have drawn about a single high-leverage Supreme Court case—Roe v. Wade. Franklin and Kosaki used this case to build support for their theory of structural response, and their work has served as a reference point for a generation of scholars studying public response to Supreme Court decisions. That contribution is important, but our work indicates that the conclusions reached are not robust when alternative modes of identification are used. Given the difficulties all social scientists face in mapping causal relationships, this is an important issue.
That the positive response hypothesis is borne out by Roe leaves us one fewer instance where structural response occurs in the manner described by Franklin and Kosaki—a small overall coefficient of change (the population treatment effect), but great variation between different groups in the population. The results of our exercise indicate that the Court is able to sweep opinions to its side even in highly controversial areas it is deciding on for the first time. Whether this short-run effect can be maintained is another question altogether—in the case of abortion, it would appear that Roe did catalyze an organized response among activists that produced sharp conflict over abortion rights in the late 1970s and up to the present day.
The Supreme Court may have ample reason to be concerned with structural change in public opinion in the medium or long term—increased conflict among activists may lead to a more protracted stalemate in the public on virtually the same battle lines as preceded the Court’s decision. But focus on what may be a long-term effect obscures the immediate effect of the Court’s decision: net opinion change as one side in a debate is validated at the expense of another. Whether this effect is enough to drive the losers away or whether the losers are able to subsequently regroup is a matter beyond the Court’s control, but not necessarily beyond its interest.
Our application of matching techniques to test the positive response hypothesis also shows that study of the courts and public opinion, already a rich field of research in political science, provides further opportunities for the use of different methodologies to assess the validity of causal claims. Seeking to extend these findings to other cases and issue domains would be of great value to our understanding of these processes. We conclude, thus, with considerable enthusiasm about the prospects for future research.
Footnotes
Acknowledgements
The authors wish to thank Jack Citrin, Jasjeet Sekhon, Gordon Silverstein, Laura Stoker, Rocío Titiunik, and numerous seminar participants for helpful comments.
The authors declared that they had no conflicts of interests with respect to their authorship or the publication of this article.
The authors received no financial support for the research and/or authorship of this article.
