Abstract
Recent work on the federal judicial nominations process finds relationships between nominees’ characteristics, such as partisanship and gender, and American Bar Association (ABA) ratings. While the findings inform public debate about ABA involvement in the nomination, the studies do not take into account the characteristics of the individuals who investigate the nominees. This study adds investigator partisanship to understand more completely the relationship between nominees and their ABA ratings. The results indicate that the Standing Committee on the Federal Judiciary (SCFJ) investigators’ partisanship contribute systematically to a nominee’s likelihood of receiving a higher or lower ABA rating. The probability that a Republican nominee receives the highest rating does not vary with the investigator’s partisanship. Democratic nominees, however, have the highest chance of the top rating after an SCFJ investigation led by a co-partisan. An analysis of matched data from the whole dataset reproduces the basic pattern of results, while the implementation of matching to partisan subgroups of nominees uncovers that both parties may benefit roughly equally from investigations led by co-partisans.
On 3 August 2017, President Donald Trump transmitted to the Senate the nomination of Leonard Steven Grasz, the former Nebraska deputy attorney general who co-wrote the petitioner’s brief in the controversial Supreme Court abortion rights case, Stenberg v. Carhart, 530 U.S. 914 (2000), for a seat on the Circuit Court of Appeals for the Eighth Circuit. At the end of October, the American Bar Association’s (ABA) Standing Committee on the Federal Judiciary (SCFJ) announced that it unanimously rated Grasz “not qualified” to sit on the circuit court. In the report to the Senate Judiciary Committee (SJC), SCFJ Chair Pamela Bresnahan (2017, 7) explained that the not-qualified rating—the first such rating for a circuit nominee since 2006, and only the fifth since 1958—stemmed from Grasz’s “temperament issues, particularly bias and lack of open-mindedness.” Conservative nomination watchers derided the Grasz rating, arguing that the lead investigator into Grasz’s background, University of Arkansas Law School Professor Cynthia Nance, held a “strong ideological bias” against conservative judicial philosophies (Whelan 2017). It was not the first time that partisan commentators suggested that the SCFJ’s investigator influenced its ratings, nor has such criticism come entirely from the political Right (Grassley 1990; Grossman 1965; U.S. Congress 1979).
Is the objection to an ABA rating based on the lead investigator’s identity an empirically grounded one? To what extent does the partisanship of the lead investigator affect the nominee’s rating? Recent studies of the federal lower-court nomination process discovered that certain traits of potential trial and appellate judges lead to systematically lower ratings from the SCFJ (Sen 2014a, 2014b; Smelcer, Steigerwalt, and Vining 2012; 2014). The mechanisms that create the empirical relationships between ratings and nominees’ attributes, though, remain unclear. American Bar Association rating studies treat the SCFJ and its members as a “black box” (e.g., Lindgren 2001, Lott 2013). We do not know how the apparent bias against Republican courts of appeals nominees or female and minority district court nominees comes about.
Using a new dataset comprising all 250 members of the SCFJ and the 643 circuit court nominees whom they investigated and rated from 1958 to 2020, this study examines whether there is any systematic relationship between investigator and nominee partisanship. It considers two theories—political ratings and ABA standards—to explain how the SCFJ arrives at its ratings. Briefly, the political ratings theory posits that the relationship between the SCFJ member and nominee’s partisanship affects the rating. The ABA standards theory imagines that the SCFJ rates on professional qualifications only.
Both theories find at least some support in the data. The partisanship results are complex, but analyses of the full dataset and matched samples reinforce that politics matter in the process. Republican nominees have around a 50 percent chance of receiving the highest rating regardless of their investigator’s partisanship. In contrast, Democratic nominees have the highest chance, around two-thirds, of the top rating after an SCFJ investigation led by a co-partisan. Analysis of the full dataset after matching reinforces the pattern. Yet, an analysis of matched data within partisan subgroups supports the strongest partisan hypotheses: Democratic and Republican nominees both may benefit significantly from investigations led by co-partisans.
Greater professional qualifications—the kind the ABA says it values—also push the rating upwards. More experience as a federal judge, state judge, federal judicial clerk, and in private practice increase the probability of a top ABA rating. The size of the effect varies from around 10 percentage points to over 20 percentage points. Also, the ABA consistently applies its own rules about non-legal experience: Nominees who took time out from practicing law to hold partisan political office have a significantly lower chance of receiving the top rating.
The study makes at least four contributions to the emerging literature on ABA ratings, partisan bias, and the lower court confirmation process. First, the data are the most comprehensive to date. Not only do the data include the major addition of the SCFJ investigators, but they also extend the timeframe back to the first official ABA ratings and forward to the end of the last presidential administration. Second, some of the results reinforce Republican claims of bias from SCFJ investigations. Previous research suggested the bias might come from Democratic SCFJ members; the full data tend to support that argument. Third, the subgroup analysis complicates the picture significantly. It shows that the partisanship of the SCFJ investigators may benefit or hinder both parties’ nominees. Fourth, it is the first to find a negative effect on ABA ratings for minority racial or ethnic identity among circuit court nominees.
History of the ABA’s Involvement in the Judicial Confirmation Process
Unlike the basic standards it outlines for House, Senate, and presidential candidates, the Constitution sets out neither explicit prerequisites nor explicit bars for those who would occupy federal judicial office. Indeed, the only constitutional test of potential judges’ fitness is that the president nominates and the Senate confirms them. For most of its existence, then, the federal judiciary served as a vehicle for party patronage, with senators often playing a crucial role in identifying nominees for district and appellate court seats in their home states (Goldman 1997, 7–9). With the institution of divided government for most of the post-war period, party service became more difficult to justify as a primary qualification for confirmation. A Republican-controlled Senate was much less likely to install mere Democratic lawyers, and vice versa.
The new SJC chair in 1947, Alexander Wiley (R-WI), enlisted bar associations as informal advisors during the confirmation process. At this initial point of involvement, the bar associations’ exclusive identification with the Republican caucus limited their influence. But, the associations occasionally could block a nominee if they raised a particular disqualification (Grossman 1965, 64–65). While not giving its findings as much esteem as his predecessor did, SJC Chair Pat McCarran (D-NV) kept the ABA on as a formal adviser after Democrats retook Congress in 1949.
In the wake of a scandal in the Truman Administration’s Department of Justice (DOJ) that led to the replacement of many top officials in 1952, the ABA found new friends in high places, though too late for a formal liaison between the SCFJ and DOJ. The incoming Eisenhower Administration, however, continued talks with the ABA. After five years of negotiations, the ABA and DOJ entered into a regular working relationship in 1958 (Grossman 1965, 70–76). From 1958 on, with the exceptions of the W. Bush, Trump, and Biden administrations, the SCFJ investigated and rated all potential nominees to the federal judiciary before the president officially announced and transmitted their nominations to the SJC. During the non-Obama 21st century administrations, the SCFJ became aware of the nominee’s name only after the administration made it public.
Standing Committee on the Federal Judiciary Rating Procedure
The SCFJ has followed the same basic rating procedure, outlined in the semi-periodical ABA “backgrounder,” The SCFJ: What It Is and How It Works (American Bar Association 2017, 4–7), since it began its regular relationship with the DOJ. First, when a vacancy occurs, the DOJ informs the SCFJ chair of the name(s) under consideration as a replacement. The chair then assigns the investigatory task to the member of the SCFJ representing the judicial circuit that the vacancy is in, with some exceptions if the circuit member is unavailable or overworked. The SCFJ member investigates the potential nominee(s) and prepares an informal report, including a tentative rating of “well qualified,” “qualified,” or “not qualified.” 1 The investigator sends the report to the SCFJ chair, who reviews it and gives the DOJ a summary. If the DOJ decides to move forward with a nominee, it informs the chair, who in turn instructs the investigator to prepare a final report. The chair then circulates the final report to all members of the committee, who vote on the nominee’s final qualification rating using majority rule. The majority’s rating is the committee’s official rating. Since the late-1970s, the SCFJ has reported minority votes, if any. The chair votes only if there is a tie.
Key Actors
In light of the ratings procedure, three actors could plausibly affect the final ABA rating. The first is the SCFJ chairperson. The chair performs important administrative functions, including liaising with the DOJ and SJC, as well as “assuring consistency in the committee ratings” (Slotnick 1983a, 360). In both functions, a chair can exercise discretion in ways that could inject personal preferences into the rating process. In working with the DOJ at the pre-nomination stage, the chair could negatively frame the results of the preliminary investigation. The chair could also persuade committee members to adopt evaluative standards that reflect the chair’s ideology, thereby achieving consistently favorable—to the chair—results through ostensibly neutral means. Additionally, in reviewing informal and formal reports for completeness, a chair could tell a member to continue the investigation if the chair does not agree with the provisional rating. Or, the chair could approve an otherwise incomplete report if the rating fits with a pre-conceived perception of how the nominee rates.
The second individual of interest is the committee member. The committee members are possibly the most influential, yet least studied, components of the SCFJ. Each circuit, including the Courts of Appeals for the DC and Federal Circuits, has at least one representative on the committee. The Ninth Circuit has two due to its size. From 1958 to 1965, the chair of the SCFJ was also a circuit representative. The SCFJ added an at-large seat in 1964, and since 1965, the chair has occupied the at-large seat.
The circuit members do the actual work of the committee, investigating the qualifications of the nominee for federal judicial office. The investigation process presents many opportunities for a member to introduce personal preferences into the ratings. Whereas the FBI vets a nominee to ensure she was never, for example, a member of the Communist Party or an alcoholic, the SCFJ investigator examines the legal writings of the prospective nominee, … reviews reported and unreported court decisions, briefs, legal memoranda, publications, speeches, hearing and argument transcripts, articles, and other writings, … [and] conducts extensive confidential interviews of a broad cross-section of judges, lawyers and others to obtain their assessments of the prospective nominee's integrity, professional competence and judicial temperament, and the underlying bases for such opinions (American Bar Association 2017, 4).
Former committee member and SCFJ Chair Robert Trescher described the investigatory process in slightly different terms: “If you talk to 10 or 15 people, all of whom say [the nominee] is a bum, you stop. If 10 or 15 all say he’s good, you might also stop, there’s no point in going on. But if you get a division, you usually keep going until you are certain you have him sized up accurately” (qtd. in Goulden 1974, 55).
Nearly every part of both the ABA’s and Trescher’s descriptions of the process creates an avenue for bias to enter the investigation. The investigator may not agree ideologically with the positions that the potential nominee took in legal writings and thus suggest that she is not qualified. Despite the formal requirement of a “broad cross-section” of interviewees, the investigator may interview only like-minded legal professionals in the circuit. Indeed, Grossman (1965, 104) anticipated this possibility: “The type of people on whom [the investigator] relies may significantly determine the ultimate rating given a candidate.” And, with respect to Trescher’s approach, the investigator’s discretion defines “10 to 15,” “bum,” “good,” division,” “certain,” and “accurately.” Therefore, investigations may run as long or as short as they need to in order for the report to reflect the rating the investigator wants. The member also interviews the nominee in person, an interview during which the nominee can respond to any “adverse comments” that might have come up during the investigation (American Bar Association 2017, 5–6). In the interview, the member may amplify, distort, or downplay negative information so that the candidate cannot reply fully and accurately.
Two contrasting nominee interviews illustrate the potential for bias concretely. First, consider the interview of J. Kenneth Porter, a Reagan nominee to a district court seat in Tennessee. According to Porter, the Sixth Circuit SCFJ representative, John Elam, assisted by Fifth Circuit member Gene W. Lafitte, presented him with “absurd … allegations … from a small group of zealots.” On Porter’s own admission, he “did not give [Elam and Lafitte] the perspective essential to full understanding” of the “patently specious” charges, and the SCFJ subsequently reduced Porter’s “qualified” rating to “not qualified” (U.S. Congress 1989, 240–242). Reagan eventually withdrew Porter’s nomination.
Second, consider the interview of Alfred Goodwin, a Nixon nominee to the Court of Appeals for the Ninth Circuit. Goodwin had his interview over lunch with the Ninth Circuit SCFJ member, John Sutro. Early in the meal, the men “learned they shared an interest in calf-roping” and spent much of their time discussing the sport (Wasby 2014, 279). Rated “exceptionally well qualified” for the circuit court, Judge Goodwin heard federal appellate cases for the next 50 years until his death in 2022.
The third actor of interest is the committee as a whole. The fact that a majority vote determines the final rating suggests that researchers should not ignore the SCFJ’s overall composition. Indeed, by not including data on individual committee members, all previous studies on potential bias in ABA ratings implicitly assume that aggregate committee composition produces their findings. Kern (1990) and Slotnick (1983b) both suggest the committee as a whole influences ABA ratings. After their finding of bias against Republican circuit court nominees, Smelcer, Steigerwalt, and Vining (2012, 837) argued that “[i]f the committee is not dominated by Democrats … that would cast doubt on claims that its output is the result of partisan bias.”
One-party domination of the committee is a sufficient, but not necessary, condition for partisan bias to emerge. Observing that much of the work in gathering, preparing, and reporting information on the candidate is already complete by the time the committee as a whole votes, Grossman (1965, 106) argued, “[I]t would appear that the influence of the noninvestigating members in each case is more apparent than real.” In other words, the other committee members can base their votes only on what the chair and investigator allow them to see, so their votes are more likely a rubber stamp than a critical check. Kamenar (1990) listed several cases in which he argued that the investigator and chair, not the entire committee, conspired to produce biased ratings.
The balance of previous studies and commentary on ABA ratings strongly weighs in favor of the investigator as the most probable direct influence on the final rating. This study thus proceeds on the assumption that the investigator’s partisanship will interact with the nominee’s partisanship to influence the rating. On the one hand, this approach leaves out the chair and committee, which both have at least some theoretical or evidentiary support as potential influencers. On the other hand, a narrow focus on the individual investigator not only makes the empirical analysis more straightforward, but it also presents a “best case” test of the hypothesized bias. If most previous studies identify the individual investigator as the primary influence on final ratings, yet the present data do not support the notion that the individual investigator matters, then the null result would suggest that the more contingent chair or committee influences are even less likely to operate.
What Might Affect a Nominee’s ABA Rating?
I examine two competing explanations of nominee ABA ratings: the political ratings theory and the ABA standards theory.
Political Ratings
First, consider the political rating theory. In the strong version, the SCFJ investigator’s conscious effort to prop up co-partisan nominees and knock down opposite-party nominees is the mechanism that produces the observed bias in the ratings. The ABA’s conservative critics often ground their arguments in the strong political rating theory. For example, Kamenar (1990) noted the ways in which former SCFJ members John Elam, Steven Keane, Joan Hall, Jerome Shestack, and Robert Fiske may have unfairly investigated or oversaw the investigations of four Reagan appointees. More recently, the Federalist Society’s ABA Watch published the political biographies of the then-current members of the SCFJ, noting that “six of the seven Standing Committee appointments … have given money to the Democratic Party and Democratic candidates” (ABA Watch 2006, 16). The clear implication is that liberal Democratic members of the SCFJ worked to downgrade and defeat then-President Bush’s nominees.
Smelcer, Steigerwalt, and Vining’s (2012, 837) conjecture regarding the possible partisan composition of the SCFJ producing their results of anti-Republican bias reflects a weaker version of the political ratings theory. In the weak version, the bias is not necessarily conscious. Rather, it arises from Democrats’ commonly held ideas about qualifications that systematically disadvantage Republican nominees. In other words, party identifiers—everyone from SCFJ members to presidents—hold fundamentally different views about what it means for a nominee to be “qualified.” Thus, SCFJ investigators see nominees from the opposing party as less qualified not due to their perceived partisanship, but due to their lack of qualities that all people who share the investigator’s partisanship agree are important. An excellent example of the weak view is the partisan controversy over President Obama’s identification of “empathy” as an important judicial qualification (Hook and Parsons 2009).
While they cannot determine the exact mechanism that may produce observed bias, the data can test the political rating theory’s primary implication: a relationship between the partisanship of the SCFJ investigating member and the partisanship of the appointing president. More specifically, investigations led by Democratic (Republican) members of the committee should produce higher ratings for Democratic (Republican) nominees.
The ABA Standards
Second, consider the ABA standards theory. The bar association itself suggests it rates the nominees on their “integrity, professional competence and judicial temperament[,] … not … philosophy, political affiliation or ideology. The committee’s objective is to provide impartial peer evaluations of the professional qualifications of judicial nominees” (American Bar Association 2017, 1; emphasis added). While social scientists can specify statistical models with appropriate proxies for “professional competence”—for example, judicial experience—the models fall short with respect to “integrity” and “judicial temperament.” Despite researchers’ sophisticated attempts to isolate the effects of non-professional variables like partisanship and identity (Sen 2014a; Smelcer, Steigerwalt, and Vining 2012, 2014), the studies potentially miss much of what actually goes into ABA ratings without measuring the other two “legs” of the “stool” that supports them (Saks and Vidmar 2001, 2003).
Nevertheless, this study’s data allow for a new, more straightforward test of the ABA standards. The association suggests that the empirical bias arises from factors to which only SCFJ investigators have access, and the SCFJ objectively defines these factors. If this proposition is true, then there will be no relationship between the partisanship of the investigator and the partisanship of the appointing president. In other words, if they support the null hypotheses of no difference in ABA ratings between investigations conducted by Democratic (Republican) SCFJ members for Republican (Democratic) nominees, then the data would vindicate the ABA’s defenders.
New Data on SCFJ Members
To test the relationships between nominee and investigator characteristics, I constructed a new dataset of the 250 individuals who occupied a seat on the SCFJ from 1958 to 2020 and coupled them with the 643 ABA-rated nominations to the courts of appeals during the same period. (See Online Appendix I for information on the SCFJ data collection effort.) Some of the nominee data came from the Federal Judicial Center’s website, 2 which gives professional experience information for all confirmed federal judges, and the Lower Federal Court Confirmation Database, 1977–2004 (Martinek 2005). For the most recent nominees, I relied on responses to their SJC background questionnaire. 3 The data begin in 1958 because the SCFJ regularized its rating process to the “exceptionally well qualified,” “well qualified,” “qualified,” “not qualified” scale that year (Grossman 1965, 76).
With few exceptions, the SCFJ assigns the investigatory task to the representative for the circuit in which the vacancy occurs. So, I identify the circuit representative on the committee at the time of nomination as the nominee’s investigator. 4 In the case of the two Ninth Circuit representatives, I attach the California representative to nominees who reside in California and the other representative to all other Ninth Circuit nominees. In total, 194 SCFJ members investigated the 643 rated nominees. The modal number of nominees per investigator is one, but the count goes up to 12 for Mark Martin, the Fifth Circuit member from 1975 to 1981. Because a circuit member undertakes an investigation only if there is a vacancy in her circuit while she is on the SCFJ, I count 47 circuit members who did not investigate a nominee. 5
Partisan Backgrounds of SCFJ Members
The measurement of SCFJ members’ partisan attachments is tricky. The measurement problem is most acute when it comes to how to treat the SCFJ members to whom I could not assign Republican or Democratic Party identification. As Figure 1 shows, the highest proportion of “unknown” partisans are on the SCFJ in the early years under study. Recent committees still have between 10 and 20 percent unknown partisans. Proportion of unknown partisans on the SCFJ over time. The figure presents the fraction of SCFJ members per year for whom I could not assign a Republican or Democratic Party identification.
There are at least three solutions to the measurement problem. First, I could model partisanship with three categories: Republican, Democratic, and unknown. While this approach would retain all of the observations, the unknowns are not necessarily independents or moderates. Thus, there is no warrant to expect their behavior to fall between the identified Republicans and Democrats. In addition to the lack of a hypothesis for the unknowns, the substantive interpretation of the estimated effect of an SCFJ member’s unknown partisanship is not at all clear. If, as a group, they appear more or less favorable to the nominees of a particular party, there is no ex ante reason to explain the result.
Second, I could initially model the unknowns separately, then lump them in with the party, if either, to which they behave more similarly. For example, if I found that they tend to favor Republican nominees, I could re-label the unknowns as Republicans in a two-party, all observations model. 6 This approach again preserves all of the observations, which maximizes the statistical power available to test hypotheses. But, the re-assignment strategy raises two problems. One, there is no reason to believe that it is harder to determine the partisanship of one party’s identifiers than the other’s is. Two, the strategy raises the real specter of data dredging. That is, a fundamental aspect of this study is looking into possible partisan bias from SCFJ members. Reconfiguring the data after discovering a fraction of members’ bias could “load the statistical dice” in favor of finding an effect. The strategy would leave, paradoxically, a study about bias with a potentially significant bias itself.
Third, I could eliminate the unknown partisans from the analysis, which is the strategy I adopt. The two drawbacks of this approach are that it (1) eliminates 76 observations from 18 investigators and (2) makes the inclusion of committee-level effects intractable. Including committee effects is difficult because it requires accurate measurement of the committee’s composition, which is impossible when ignoring the presence of at least one member in most years. The gain from eliminating the unknowns, however, outweighs the loss of observations and restricted analysis. The approach avoids any temptation to develop post hoc rationalizations for statistical results, and it does not unjustifiably alter the data. Figure 2 displays the proportions of Democratic and Republican members of the SCFJ over time. Proportion of known partisans on the SCFJ over time. The figure presents the fractions of the SCFJ for whom I could assign a partisan identification.
Variables and Measurement
There are two independent variables of primary interest. First, the party of the nominating president takes a value of 1 if a Republican made the nomination and 0 if a Democrat did. Using party of the appointing president is consistent with recent research on the ABA in the confirmation process (Sen 2014a; Smelcer, Steigerwalt, and Vining 2012, 2014). 7 Second, the party of the investigating member takes a value of 1 if Republican and 0 if Democrat. The coefficients for each of the variables provide, respectively, estimates for the effect of a Democratic investigator on a Republican nominee and a Republican investigator on a Democratic nominee. Their interaction provides an estimate for the effect of a Republican investigator on a Republican nominee. A Democratic investigator of a Democratic nominee is the reference category.
Independent variables capturing the ABA standards include the nominee’s number of years as a federal judge, number of years as a state judge, number of years in private practice, and number of years as a government attorney, which includes time spent as a Judge Advocate General, elected state attorney general, or prosecutor. Previous research shows that these variables, especially the number of years as a federal judge, are important predictors of a nominee’s final ABA rating (Smelcer, Steigerwalt, and Vining 2012). 8
I include an indicator variable for whether or not the nominee graduated from a Top 14 law school, coded 1 if the nominee did. Because all lawyers in the United States have graduated from law school, there is no variation among the levels of education of judicial nominees. There is variation in the law schools they attended, however (Sen 2014a, 2014b). The clearest line to divide elite law schools from the rest is drawn between the Top 14 schools, which are those schools that have appeared ranked one through 14 in nearly every iteration of the US News & World Report law school rankings since 1990, and all others (Hopkins 2012). Yale, Harvard, Columbia, Stanford, Pennsylvania, Virginia, Michigan, Chicago, Duke, Cornell, Georgetown, Berkeley, New York, and Northwestern comprise the Top 14 law schools.
I also include indicator variables for whether or not the nominee ever taught in a law school, held a federal clerkship, or participated in publicly partisan political activity, as opposed to merely donating money (all coded 1 if “yes”). The ABA prefers “substantial” trial experience for nominees, yet it claims “[d]ue consideration will be given to distinguished accomplishments in field of law … [which] may be considered as a substitute for a prospective nominee’s lack of substantial courtroom experience.” Moreover, there is “somewhat less emphasis on the importance of trial experience as a qualification for the appellate courts” (American Bar Association 2017, 3). Therefore, it is not clear what effect having taught law school would hypothetically have on the ratings.
Judges award clerkships to law schools’ top graduates. The clerkship provides experience in the mechanics of judging (see, e.g., Peppers 2006, Ward and Weiden 2006). Thus, I expect former law clerks to receive higher ABA ratings. I expect partisan political activity, in contrast, to have a negative effect on the final ABA rating. The ABA standards “recognize[] that civic activities and public service are valuable experiences for a nominee, [but] they are not a substitute for significant” legal experience (American Bar Association 2017, 4).
The models have demographic controls for gender (1 if female), minority status (1 if non-white), and age and age squared. Recent research finds that ABA ratings disadvantage racial minorities and female nominees at the district court level. But, there is little evidence that a non-traditional gender or racial-ethnic background leads to a lower ABA rating at the courts of appeals level. Given that the ABA has historically been skeptical of nominees both “too young” and “too old,” the squared term for age is theoretically appropriate (see Grossman 1965, 85; and Sen 2014a, 39).
Finally, I include a control for whether or not the nominee came up during the “No Split Votes” era (1 if “yes”). Before 1978 the SCFJ did not release the minority views of the committee if they existed, for example, it reported “qualified-not qualified” as “qualified.” If a minority of the committee were just as likely to think more highly of the nominee as it were less highly of her, the reporting change would not pose a problem. But, since the SCFJ started reporting split ratings, the minority rating has been higher only about 15 percent of the time. Without the era control, post-1978 nominees would appear lower rated, but the appearance would stem from the SCFJ’s changed reporting procedure. See Online Appendix II for the dependent and independent variables’ summary statistics.
Results
Partisan Interactions in ABA Ratings
Ordered Logit Models of Courts of Appeals ABA Ratings, 1958–2020.
The dependent variable is a six-point ordinal measure of the nominee’s ABA rating.
Standard errors clustered on committee year.
*: p < 0.05 **: p < 0.01 (all two-tailed).
Model I provides some evidence of political bias, though the pattern of results is complex. Republican nominees receive lower ratings after investigations by Democratic SCFJ members. Holding all other variables constant, a Republican nominee investigated by a Democratic SCFJ member has a 52 percent chance of receiving the ABA’s top rating of “well qualified,” while a Democratic nominee has a 67 percent chance of receiving the top rating after an investigation conducted by a co-partisan.
10
The 15 percentage-point difference is statistically significant (
Other political relationships, however, do not find statistical support. I cannot reject the null hypothesis that Republican-led SCFJ investigations lead to equal outcomes whether the nominee is a Republican or Democrat. A Republican investigation will result in a “well-qualified” rating for a Republican nominee about 52 percent of the time and a Democratic nominee about 60 percent of the time (
Matched analysis: Full dataset
Though its parameter estimates derive from the most comprehensive dataset yet of circuit court nominees paired with SCFJ investigators, Model I suffers from the same deficiency that many models based on observational data suffer from: imbalance on the main covariates of interest. Concretely, we want to know what causal effect the SCFJ investigator’s partisanship has on the nominee’s rating, conditional on the nominee’s partisanship. Republican and Democratic nominees, however, are not exactly the same in the aggregate on other variables that influence the rating. So, the imbalance between partisan subsets of the data could bias inference in profound and misleading ways (see, for example, Boyd, Epstein, and Martin 2010, 395).
To ameliorate the problem, researchers use matching methods to pre-process their observational data. Broadly, matching requires first the identification of an indicator variable that splits the data into two groups, a “treatment” group and a “control” group. Then, the matching procedure attempts to reduce the difference in means between the treatment and control groups on all identified confounding variables by discarding or weighting observations. The procedure thus leaves a reduced, yet balanced, dataset. 11
Previous sophisticated studies of ABA ratings adopt one of two matching methods. Smelcer, Steigerwalt, and Vining (2012, 2014) use genetic matching (Diamond and Sekhon 2013). Sen (2014a) uses coarsened exact matching (CEM) (Iacus, King, and Porro 2011). Both methods fit ABA studies well because they are more flexible in dealing with continuous variables, for example, years of legal experience, than exact matching. Whereas exact matching would pair—and thus retain—two observations only if they had equal years of experience, genetic matching and CEM allow for pairs of observations that are similar but not identical. The choice of which specific matching method to use turns on two major factors: which will reduce imbalance more and which will preserve more observations.
With the present data, CEM generally performs better on the former, while genetic matching maximizes the latter. Crucially, though, genetic matching does not appreciably improve the balance on a number of covariates, but CEM improves balance on all of the covariates except experience in private practice. Therefore, I use CEM to pre-process the data, valuing balance over a larger “matched” dataset that retains significant imbalance (Iacus, King, and Porro 2009). 12
Model II in Table 1 reports the parameter estimates for the ordered logit model using the matched data. Although the estimated coefficients for most variables look significantly different from Model I, it is important to remember that matching isolates the effect of the expected “treatment” variable, partisanship. The matched variables act as pure controls. At first glance, it appears that partisanship has no effect, either directly or through the interaction with the SCFJ member’s partisanship. Yet, the predicted probabilities derived from the model estimation reflect a pattern consistent with the one that the full data presented. Democratic nominees have a 66 percent chance of receiving the top rating after an investigation by an SCFJ co-partisan. Republican have only a 44 percent chance of a unanimously well-qualified rating after a Republican-led investigation. The 22 percentage point difference is statistically significant (
Matched analysis: Partisan subgroups
Ho, Imai, King, and Stuart (2007, 205) argue that “projects” involving “more than one causal variable of interest” pose a unique challenge to the estimation causal effects. They suggest matching “separately for each [variable] and [to] work[] hard to avoid post-treatment bias in the process.” While the present study is such a project—the two causal variables are the nominee’s partisanship and the SCFJ member’s partisanship—I take a different tack: I perform CEM within each partisan subgroup of nominees. The subgroup approach better captures the interactive effects of partisanship than successive matching.
Differences in Means Within Matched Partisan Subgroups Between Democratic and Republican SCFJ Investigators.
Cell entries are the estimated value of the dependent variable after a bivariate OLS regression of SCFJ partisanship on nominee rating. Standard errors of the estimates appear in parentheses. Please see Online Appendix IV for more details on the matching procedure.
*: p < 0.05.
The analysis of within-subgroup matched data fits the strongest version of the political theory’s hypotheses. Here, the data show what one would expect if SCFJ investigators were as nakedly partisan as possible: Democrats investigating Democrats and Republicans investigating Republicans lead to much higher ratings than the cross-partisan investigations do. Co-partisanship moves the expected rating up 25 percent of the entire range of the dependent variable, a nearly 50 percent increase for Republican nominees and nearly 40 percent increase for Democratic nominees. The size of the estimated effects are not statistically distinguishable from each other (diff: 0.49, st. err.: 0.69; p > 0.45). The matched subgroup analysis reinforces the pattern of results from the full dataset that showed Democratic nominees’ lower and higher ratings are higher than the Republican nominees’ lower and higher ratings, respectively. The results further complicate the straightforward picture of anti-Republican bias that Smelcer, Steigerwalt, and Vining (2012) found.
ABA Standards
Substantive Effects of Professional and Demographic Variables.
The table displays the change in probability of receiving a unanimously “well-qualified” rating from the SCFJ for different values of non-partisan independent variables. The left column indicates the variable. The middle column shows the specific values the variable takes to calculate the marginal difference. The third column reports the marginal difference, subtracting the first comparison value from the second. All are significant at the p < 0.05 level. I calculated the margins using estimates from Model I in Table 1.
With respect to demographic variables, Model I demonstrated that minority status and age are significant predictors of ABA ratings. Nominees who are members of racial and ethnic minority groups receive significantly lower ratings than white nominees do. The substantive effect of minority status is roughly the same magnitude as having nine years of federal judicial experience. The data additionally support the notion that there is an “optimum” age of appointment (Grossman 1965, 85). The probability of a unanimously “well-qualified” rating crests at 63 percent for nominees between 49 and 52 years old. The probability is statistically significantly lower for nominees 42 years old and younger, as well as nominees 56 years and older.
Discussion and Conclusion
To sum up the primary conclusions of the study: (1) The partisan bias identified in previous studies of courts of appeals nominees’ ABA ratings remains even if (a) researchers include all nominees since the ABA began its official ratings and (b) the model controls for the partisanship of the investigating SCFJ member. However, (2) straightforward pro-Democratic, or anti-Republican, bias is too simple an explanation for the pattern of results. Additionally, (3) this is the first study of courts of appeals nominees to find evidence of racial bias in ABA ratings. (4) The qualifications that the ABA deems important, for example, years of judicial experience, are consequential for the ABA rating, too.
The first and second conclusions are important updates to understanding partisanship’s role in the ABA rating process. The first extends Smelcer, Steigerwalt, and Vining’s (2012) findings of anti-Republican bias. The second stems from the observation in Figures 3 and 4 that Republican- and Democratic-led investigations produce similar ratings for Republican nominees, while both types of investigations lead to higher ratings for Democratic nominees. The Democratic investigator-Democratic nominee pair leads to an even higher probability of a well-qualified rating than the Republican-Democratic pairing. Speculative interpretations of these findings abound, yet these data cannot discern with any precision which interpretations are more or less plausible. The interaction of investigator partisanship and party of the appointing president with full data. Estimates are the predicted probabilities (with 95 percent confidence intervals) of receiving a well-qualified rating after an investigation conducted by a Democratic or Republican SCFJ member, using results from Model I in Table 1. The interaction of investigator partisanship and party of the appointing president with matched data. Estimates are the predicted probabilities (with 95 percent confidence intervals) of receiving a well-qualified rating after an investigation conducted by a Democratic or Republican SCFJ member, using results from Model II in Table 1.
The third conclusion—racial and ethnic minority nominees receive consistently lower ratings—reinforces the importance of continued research on ABA ratings. The finding is new to the understanding how circuit court evaluations work. It builds on findings of anti-minority bias at the district court level, findings derived from data that encompass over 50 years of nominations (Sen 2014a, 2014b). In contrast, the fourth conclusion extends the pattern from all previous studies, regardless of level in the judicial hierarchy, that a nominee’s educational and legal experiences are significant determinants of the final rating. Future studies that incorporate investigator identities should ask whether effects of SCFJ members’ gender, racial, or experiential backgrounds mirror the effects of partisanship. Indeed, the non-effect of gender in this study might mask countervailing significant effects from male and female investigators relative to male and female nominees. 14 Moreover, if the racial identity of the investigator leads predictably to the negative effect of minority status, policymakers would likely scrutinize the SCFJ’s role in the process even more closely.
Without more data, such the exact vote breakdown of all non-unanimous ratings, lists of interviewees, and provisional investigator reports, researchers will have a difficult time pin-pointing the exact mechanism that produces the observed results. Data and other limitations restrict the scope of the conclusions drawn from this study in other important ways. First, data on potential nominees filtered out at the pre-nomination stage where the SCFJ and its chair arguably exercise much of their real power remain elusive (Saks and Vidmar 2001). Second, the difficulty in identifying the partisanship of some SCFJ members leaves an important variable—the composition of the committee as a whole—unexamined. While qualitative impressions cast doubt on the possibility of committee influence (e.g., Grossman 1965, 106), its (non-)existence is ultimately an unexplored empirical question. In light of the ABA’s recent relegation for the first time in a Democratic administration to a “post-selection check,” as opposed to a pre-nomination player, the association could make the committee’s history and process more transparent, if only to quiet critics from both sides of the partisan divide.
The analysis of matched partisan subgroups finds that both Democratic and Republican nominees benefit from investigations led by co-partisans. The finding enriches the narrative surrounding the change in partisan criticism of the SCFJ over time. The SCFJ began its regular involvement in the nomination process under a Republican president. Until the late-1970s, the committee’s critics were mainly Democrats, including former Vermont Senator Patrick Leahy, who memorably compared the SCFJ’s judgment of judicial nominees to “Jack the Ripper determining the qualifications of surgeons in 19th century England” (U.S. Congress 1979, 10). Republican critics emerged in the 1980s after perceived unjustly low ratings of Reagan Supreme Court nominee Robert Bork and circuit nominees like Frank Easterbrook and Richard Posner. The subgroup finding, paired with the relative partisan strength on the committee displayed in Figure 2, suggests that the historical pattern of criticism reflects a rational response to the SCFJ’s behavior. Democrats criticized when Republicans had a consistent majority on the SCFJ, and Republicans criticized when Democrats had a similar enduring majority. 15
So, is the objection to an ABA rating on the basis of the lead investigator’s partisanship an empirically grounded one? This study’s findings point to “yes.” From these data, there is clear evidence that how a nominee rates depends on who investigates. To what extent does the partisanship of the lead investigator affect the nominee’s rating? The answer to this question is more complicated. No single explanation describes the patterns observed in the data. The story might be anti-Republican bias from Democratic investigators, but in the whole dataset analysis, Republican-led investigations also led to marginally higher ratings for Democrats. And, there is evidence that pure partisanship affects the ratings for both Republican and Democratic investigators. Lacking data, researchers and popular commentators have only untestable hypotheses. More openness from the association would lead to a better understanding of what it means when it rates a nominee “qualified.” More openness would in turn help the public and policymakers incorporate better the SCFJ’s evaluation in the “battle over the bench” (Steigerwalt 2010).
Supplemental Material
Supplemental Material - How You Rate Depends on Who Investigates: Partisan Bias in ABA Ratings of US Courts of Appeals Nominees, 1958–2020
Supplemental Material for How You Rate Depends on Who Investigates: Partisan Bias in ABA Ratings of US Courts of Appeals Nominees, 1958–2020 by James A. Sieja in Political Research Quarterly
Footnotes
Acknowledgments
The author thanks Ken Mayer, Alex Tahk, Ryan Owens, Steve Wasby, Ellie Powell, Sida Liu, Justin Wedeking, Rich Vining, Ronnie Olesker, Gwen Higgins, Daniel Lempert and anonymous reviewers for comments, assistance, and encouragement at various stages of the project. A previous version of the paper was presented at the 2014 Midwest Political Science Association Conference.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
