Abstract
This article investigates whether constituents are able to accurately infer their senators’ votes when the senator frequently votes against the party line. We find that when senators repeatedly vote against the party line, constituents’ ability to correctly identify their senators’ votes drops precipitously while levels of misinformation rise. We then show that citizens represented by senators who tend to vote against the party line are also less able to connect their policy positions with their evaluations of those senators. These findings indicate that there is substantial variation across senators in the ability of their constituents to hold them accountable for their votes while in office. Constituents simply know less about the positions taken by moderate senators and have a harder time aligning their levels of policy agreement with a senator with their evaluation of that senator if she frequently votes against her party.
Introduction
Whether or not voters have the information necessary to hold politicians accountable is the source of extensive scholarly attention and debate. For voters to hold elected officials accountable for the positions they take, voters must know, or at least be able to infer, what those positions are in the first place. Scholars have recently begun a renewed effort to understand how much constituents know about their representatives’ and senators’ positions on specific votes (Ansolabehere & Jones, 2010; Barabas, Pollock, & Wachtel, 2012; Dancey & Sheagley, 2013; Fortunato & Stevenson, 2014; Jones, 2011, 2013; Mitchell, 2009; Wilson, 2012). Although this question has received scholarly attention in the past (e.g., Alvarez & Gronke, 1996; Hutchings, 2001; Wilson & Gronke, 2000; Wolpert & Gimpel, 1997), new survey questions in the 2006 Cooperative Congressional Election Study (CCES) have facilitated more in-depth analysis in recent years. Such analysis is important because if we believe elected officials should be held accountable for the positions they take in office, then we should desire to know if citizens possess the requisite information to hold their leaders accountable.
The recent strain of research demonstrates that citizens hold relatively accurate perceptions of their representatives’ and senators’ votes (Ansolabehere & Jones, 2010) and use these perceptions to hold legislators accountable (Ansolabehere & Jones, 2010; Jones, 2011, 2013). Missing from this research, however, is an assessment of whether, and to what extent, this accountability varies across senators. Put differently, is there systematic variation in how much constituents know about their senators’ votes that can be explained by the characteristics of the senators themselves?
One potentially important senator-level characteristic that could affect overall levels of constituent awareness is the frequency with which a given senator votes with his or her party on high-profile votes. For instance, there is evidence that voters have less accurate, and in many cases inaccurate, perceptions of their legislators’ positions when they deviate from the party line (Ansolabehere & Jones, 2010; Dancey & Sheagley, 2013). What we do not know, however, is how a record of consistently deviating from the party line affects overall levels of constituent knowledge. In other words, when a senator frequently votes against her party (e.g., former senator Olympia Snowe [R-ME]), do constituents simply know less about her votes in the aggregate or do such senators develop a reputation for moderation that voters can rely on to make accurate inferences about their votes (on the latter point, see Fortunato & Stevenson, 2014)?
We rely on the 2006 CCES to study how variation in senators’ adherence to the party line shapes the accuracy of inferences constituents make about their senators’ votes. The survey contains six questions about respondents’ knowledge of their senators’ votes on high-profile issues in the 109th Congress (free trade, immigration, stem cell research, capital gains taxes, minimum wage, and withdrawing troops from Iraq). Utilizing variation in how often individual senators vote with their party, we demonstrate that constituents’ ability to identify their senators’ votes drops precipitously when a senator consistently votes against her party. Averaged across issues, there is a roughly 30-percentage point increase in the accuracy of constituents’ inferences as we move from the least loyal to most loyal partisan senators.
After establishing that constituents are less able to accurately identify the positions of their less partisan senators, we examine the consequences for the link between citizens’ policy preferences and their evaluations of their senators. More simply, we ask whether there is a weaker link between constituents’ levels of policy agreement with a senator and their evaluation of that senator if the senator frequently votes against the party line. To answer this question, we rely on the 2006, 2008, 2010, and 2012 CCES surveys. Although the 2006 study is the only one to ask respondents how they think their senator voted on several recent high-profile votes, all four surveys ask respondents how the respondent would have voted on several key votes from the previous Congress. For all 4 years, we are thus able to create a measure of actual policy agreement between CCES respondents and their senators on several major roll call votes (e.g., health care reform, Dodd-Frank, Iraq, stem cell research). In all 4 years, we find that the link between constituents’ agreement with their senators’ positions and approval of that senator is stronger for senators who are more partisan in their voting behavior.
The findings demonstrate significant heterogeneity across senators in the level of awareness of their roll call votes and the ability of their constituents to link their policy agreement to approval. Citizens do indeed possess the ability to hold their senators accountable for their positions in office (Ansolabehere & Jones, 2010; Jones, 2011), but this ability varies across senators. Just as the context of Senate campaigns can change the effect of certain variables on vote choice (Kahn & Kenney, 1999), the characteristics of a senator can shape citizens’ ability to link their policy preferences with their evaluations of the senator. Constituents simply know less about the positions taken by moderate senators and have a harder time aligning their levels of policy agreement with a senator with their evaluation of that senator if she frequently votes against the party line. In the conclusion, we consider the normative implications of these findings.
The Study of Constituent Awareness
The study of constituent awareness of elected officials’ positions has a long history. Miller and Stokes (1963), although concluding there are elements of constituent control in Congress, notably found that constituents had only a “chemical trace” of knowledge about their representatives’ positions (p. 54). A little more than a decade later, Hurley and Hill (1980) concluded, “Retrospective control requires knowledge of a policy maker’s performance in office, and only a small minority of constituents have such knowledge on specific issues” (p. 447). Studies do, however, find that members of Congress take into account potential constituency opinion when deciding how to vote (e.g., Arnold, 1990; Miller & Stokes, 1963) and moderate their issue positions leading up to elections (Wright & Berkman, 1986). A common view is that senators’ and representatives’ collective policy positions and voting records do affect election outcomes (e.g., Canes-Wrone, Brady, & Cogan, 2002; Carson, Koger, Lebo, & Young, 2010; Erikson, 1971; Koger & Lebo, 2012).
When we use the terms awareness or knowledge, we mean simply that constituents are correctly identifying their senator’s position on a vote. Knowledge is an observed outcome and could theoretically be obtained through a variety of means, including (but likely not limited to) party cue usage, concrete information about a senator’s vote, or random guessing (see also Ansolabehere & Jones, 2010). Given the overall low levels of political knowledge in the electorate and the relatively low salience of roll call votes, we think it is unlikely that sizable numbers of citizens have knowledge due to concrete information about a senator’s vote. Rather, knowledge of senators’ votes likely reflects an inferential process in which citizens rely heavily on party cues.
At the individual level, scholars disagree about the extent to which citizens are able to overcome limited knowledge and cognitive biases to form correct impressions about the positions their representatives in Congress take. Constituents tend to project their own views onto representatives and senators who they like (Alvarez & Gronke, 1996; Feldman & Conover, 1983; Wilson, 2012; Wilson & Gronke, 2000). In addition, Dancey and Sheagley (2013) argue that the public relies extensively on party cues to identify their senators’ positions on key issues, and when senators defect from the party line, it causes high levels of misinformation, especially among the most politically engaged segments of the electorate. Importantly, Wolpert and Gimpel (1997) conclude that voters with incorrect information about their senators’ positions use these incorrect impressions in their voting decisions.
However, Ansolabehere and Jones (2010) find that members of Congress’ actual voting records are a robust predictor of constituents’ beliefs about how their members voted. Jones (2011) also shows that individuals base their vote choice on policy agreement with senators and that this policy-based voting is more common in competitive states (Jones, 2013). Critically, citizens can make accurate inferences even without knowing exactly how their senators voted on each issue by relying on shortcuts (heuristics), most notably senator party affiliation, to infer their senators’ positions on many issues (Conover & Feldman, 1989; Mitchell, 2009). There is even evidence that citizens have knowledge about the extent to which their senators toe or buck the line, which could lead them to rely on party cues only when it helps them form accurate impressions of senators’ votes (Fortunato & Stevenson, 2014).
Taken together, these bodies of research indicate that citizens utilize party cues to infer the positions taken by their elected officials. Thus, although some citizens may have concrete ideas about their senators’ votes, by and large, citizens act as if they are knowledgeable by utilizing heuristics. What is largely missing from the literature is an across-senator assessment of overall levels of political knowledge to better detect how well citizens perform in the aggregate when confronted with senators who present constituents with unclear party cues by not clearly adhering to party lines. For instance, Ansolabehere and Jones (2010) examine the effect of policy agreement between voter and representative on individual-level approval and vote choice and, at least implicitly, assume that this process is the same regardless of characteristics of representatives. Similarly, Dancey and Sheagley (2013) assume that the effect of senators deviating from the party line on knowledge of roll call votes is the same for all constituencies in their sample. We believe that focusing solely on the individual level misses important heterogeneity across senators in how these processes play out. Thus, a natural extension of work in this area is to explicitly account for how senator characteristics condition awareness and the impact of policy agreement.
One study that does take into account senator-level characteristics and specifically the extent to which a senator votes with her party is that of Fortunato and Stevenson (2014). Their findings suggest that constituents are less likely to use partisanship as a heuristic in inferring senators’ votes on specific issues if the senator is a less consistent partisan. Such a finding raises the possibility that constituents are able to draw relatively accurate inferences about the votes of less partisan senators. Fortunato and Stevenson (2014) do not, however, look at aggregate levels of knowledge across senators. Instead, their focus is on constituents’ use of heuristics to infer senators’ votes on individual issues, which they argue is mostly rational (in contrast to Dancey & Sheagley, 2013). We ultimately differ from both of these studies in a few key ways. First, we assess how variation in party line voting affects citizens’ perceptions of their senators’ collective records on multiple key votes. Second, we examine how policy (dis)agreement on multiple votes relates to how constituents evaluate their senators. Finally, our focus in both cases is on understanding how variation in senators’ behavior, namely, the tendency to vote with or against their political party, shapes these processes.
In sum, given that we expect citizens rely extensively on partisan cues to make inferences about elected officials’ positions, we expect lower levels of constituent knowledge when senators routinely vote against the party line. Research that suggests constituents form relatively accurate impressions of their senators’ votes (Ansolabehere & Jones, 2010) and are at least somewhat cognizant of variation in senators’ ideological or partisan commitments (e.g., Carson et al., 2010; Fortunato & Stevenson, 2014) might lead us to expect constituents can form relatively accurate beliefs about the votes of even the least partisan senators. Although it is reasonable to conclude that many constituents have some understanding of how committed their senators are to party principles, we are skeptical that such an understanding can easily translate into knowledge of where the senator stands on many of the major issues of the day. More simply, voters will have a harder time identifying their less partisan senators’ votes because those senators are less predictable in their voting behavior. We remain agnostic in this article about how much constituents know about the extent to which their senators deviate from the party line (see Fortunato & Stevenson, 2014, for a fuller treatment of that question), but we do argue that the lack of predictability in less partisan senators’ votes will limit constituents’ ability to make accurate inferences.
The Link Between Policy Agreement and Approval
If, as we expect, constituents have less accurate perceptions of less partisan senators’ votes, it naturally begs the following question: Do constituents have a harder time linking their policy views with their evaluations of less partisan senators? As legislators are tasked with crafting and voting on policy, it is desirable that citizens should use their own views on good public policy to assess their support for legislators. Although “projection effects” (Wilson & Gronke, 2000) and low levels of political awareness in the public (Delli Carpini & Keeter, 1996) almost certainly limit accountability to some extent, research has shown that accountability exists (Ansolabehere & Jones, 2010; Jones, 2011, 2013) and that both aggregate roll call vote records and positions on specific votes are related to members’ reelection prospects (e.g., Brady, Cogan, Gaines, & Rivers, 1996; Canes-Wrone et al., 2002; Carson et al., 2010; Jacobson, 2013, p. 231; Nyhan, McGhee, Sides, Masket, & Greene, 2012). We therefore expect that citizens have sufficient information about the policy preferences of their senators and representatives to use policy agreement as a basis for evaluation.
We do not, however, expect the link between roll call vote agreement and evaluations to be uniform across senators. Rather, we suspect that there should be a weaker link between policy agreement and evaluations of less loyal partisan senators than for those who are more loyal. We expect this will be the case because of the gap in knowledge of high- versus low-unity senators’ votes. In other words, citizens are less likely to be aware of policy agreement (or disagreement) with less loyal partisans due to their lower levels of awareness of those senators’ positions. Partisan senators, however, convey clear signals about their positions and thus should make it easier for citizens to connect policy positions to evaluations.
We use senators’ approval ratings as opposed to vote choice to measure evaluations of senators. Approval offers a more direct measure of how constituents evaluate their senator than vote choice because vote choice may come down to the relative evaluations of both the senator and challenger. Although vote choice is probably of foremost concern to senators themselves, approval has the potential to shape election outcomes through both voter decision making and elite-level decisions. Research shows a strong link between senators’ aggregate approval rates and reelection rates, even after controlling for state partisanship (Highton, 2008). Senators’ approval ratings in the years leading up to an election are also related to the decisions strategic politicians make, with senators more likely to retire when their approval ratings drop, and stronger challengers more likely to enter when a senator has low approval levels (Brown & Jacobson, 2008; Highton, 2008). In sum, senators have electoral reasons for caring about their approval levels.
Expectations and Hypotheses
To summarize, we have three expectations. First, we expect that constituents will know less about the collective voting records of their less partisan senators. Second, consistent with previous work showing a link between policy positions and evaluations of senators and representatives (e.g., Ansolabehere & Jones, 2010), we expect that the more a constituent agrees with a senator’s position on high-profile roll call votes, the more likely the constituent will be to approve of that senator. We do not, however, expect the link between roll call vote agreement and approval to be uniform across senators. Given our previous expectation that constituents will have a harder time accurately inferring the votes of their lower unity senators, we expect that this will lead voters to have a harder time linking their policy positions to their evaluations of their senators. This leads to three hypotheses:
It is important to note that in testing the final two hypotheses, we are not attempting to unpack the entire chain of mechanisms whereby citizens’ policy positions are translated into approval of their senator. Rather, our concern is with establishing if there are systematic differences in the relationship between policy agreement and approval across levels of senators’ party unity. The tests of our hypotheses are thus not tests of how constituents decide whether they approve of a senator but instead whether the link between support and agreement is stronger for some senators than for others.
Our hypotheses are also not meant to test the question of whether higher party unity scores are associated with lower vote shares in the aggregate. In House elections in particular, the negative relationship between party unity scores and incumbent vote shares is well documented (Carson et al., 2010; Koger & Lebo, 2012; see also Sulkin, Testa, & Usry, 2015, for the link between unity and approval). In light of these findings, we expect some degree of constituent awareness of senators’ levels of party unity, perhaps as a result of advertising (Koger & Lebo, 2012) or media coverage of senators’ behavior (Fortunato & Stevenson, 2014). In light of the public’s abstract preferences for bipartisanship (Harbridge & Malhotra, 2011) and dislike of heightened partisanship (Carson et al., 2010; Harbridge & Malhotra, 2011; Ramirez, 2009), it is sensible that more partisan voting records can damage incumbents’ reputations. Our aim is not to call that finding into question, but instead to test whether constituents can develop reasonably accurate impressions of low-unity senators’ positions on individual issues.
Knowledge and Unity
Data and Method
To test the knowledge hypothesis, we rely on the 2006 CCES (Ansolabehere, 2010a). For this survey, all respondents are asked what position they think each of their senators took on six roll call votes in the 109th Congress (2005-2006). The votes are withdrawing troops from Iraq, increasing the federal minimum wage, extending the capital gains tax cut, increasing funding for stem cell research, amnesty provisions for undocumented immigrants, and the Central American Free Trade Agreement (CAFTA). 1 Although respondents were only asked about six votes, most of these votes were on the Washington Post’s list of key votes in the 109th Congress 2 and two of the issues—Iraq and immigration—were the first and fourth most important problems facing the country in 2006, respectively, according to CCES respondents. We rely solely on the 2006 CCES here because the 2008, 2010, and 2012 CCES surveys did not ask respondents how they thought their senators voted on key issues, although they did ask how respondents would vote on issues and recorded senators’ actual votes in most years. The 2006 CCES is a nationally representative sample with more than 30,000 respondents. Overall, the 2006 CCES does a good job of mirroring the population of the United States; however, respondents do tend to be more educated and politically engaged than the general public (see Bafumi & Herron, 2010; Fridkin & Kenney, 2011).
For a given vote, respondents were given a brief description of the issue and then asked whether they thought their senator voted in support of the measure, against the measure, or if they did not know what position their senator took. Respondents could thus give either a correct, incorrect, or do not know response. There is variation across all three categories, with the proportion of correct responses ranging from 0.31 on the CAFTA vote to 0.57 on the Iraq vote, incorrect responses ranging from 0.09 on the capital gains vote to 0.16 on the immigration vote, and do not know responses ranging from 0.32 on the Iraq vote to 0.54 on the CAFTA vote. 3
To measure the percentage of time a member votes with his or her party, we use party unity scores available at http://voteview.org, which measure the percentage of the time a legislator votes with his or her party on votes that pit a majority of Republicans against a majority of Democrats.
Results—Knowledge
The knowledge hypothesis states that constituents will have a harder (easier) time correctly inferring the votes of senators with lower (higher) party unity scores. We begin with a series of aggregate analyses to test this hypothesis. The first panel of Figure 1 plots the relationship between party unity scores from the 109th Congress and the average level of knowledge across six issues from the 2006 CCES. The knowledge measure is simply the percentage of respondents able to correctly place their senator’s vote averaged across the six issues. 4 The scatterplot also includes a lowess curve with a bandwidth of 0.8. 5

Awareness of senators’ votes in 2006.
As the top panel of Figure 1 shows, there is a positive relationship between a senator’s party unity score and the average level of constituent knowledge on the six votes. At the low end, only about 30% of constituents are able to correctly identify how the least partisan senators—Ben Nelson (D-NE), Lincoln Chafee (R-RI), Susan Collins (R-ME), and Olympia Snowe (R-ME)—voted on the major issues in the previous Congress. For loyal partisans such as John Kerry (D-MA) and George Allen (R-VA), however, roughly 60% of constituents can correctly identify their votes. 6 On average, around 39% of constituents can identify a given vote by a senator in the lowest quartile of unity while 50% of constituents can identify a given vote by a senator in the highest quartile (note that remaining respondents are spread out over the “do not know” or incorrect answer categories). Finally, as can be seen from the graph, the result is not simply a function of the outlying moderate senators. Restricting the relationship to just the senators who have a party unity score of 80 or higher, the correlation between party unity and average knowledge is 0.44.
The second and third panels of Figure 1 show the relationship between senator party unity and the proportion of constituents offering incorrect and do not know responses, respectively. It is important to note that do not know responses appear only weakly related to senator party unity scores, with most of the decline in the proportion of do not know responses taking place at the high end of party unity. Incorrect knowledge, however, is much more strongly linked to levels of party unity. This finding suggests that the decline in knowledge of low-unity senators’ votes is not simply a result of respondents providing a do not know response. Instead, many respondents still offer an answer on how their senator voted but simply offer the incorrect answer. The high levels of misinformation offered up by respondents suggests that voters are not truly knowledgeable about the votes their senators take but instead are making inferences about how their senators vote (Dancey & Sheagley, 2013; Mitchell, 2009). The result is that with less predictable senators, which in this case is those with low party unity scores, constituents prove not only less “knowledgeable” but also more misinformed.
Multilevel Analysis
It is possible that the relationship between party unity and constituent knowledge is a result of differences in the constituencies that low- and high-unity senators represent. To account for this possibility, we next estimate an individual-level model predicting individual respondents’ knowledge of their senators’ votes. Because we expect that both constituent- and senator-level variables will affect constituents’ knowledge of their senators’ votes, we adopt a multilevel model framework. If we are correct in our argument that the clarity of party cues drives levels of knowledge, then we should find evidence that a powerful predictor of individuals’ knowledge of their senators’ votes is the senator’s party unity score.
Our dependent variable for these analyses is the proportion of correct responses a respondent offered in the 2006 CCES survey. 7 We model these responses in a multilevel framework using a random-intercept regression by treating respondents as nested within senators. 8 We stack the data by senator to take advantage of the fact that the CCES asks respondents about both of their senators. These models include controls at Level 1 for respondent education (six categories, ranging from a low of no high school degree to a high of postgraduate degree), age (in years), gender (0 = male, 1 = female), income (14 categories, ranging from an annual family income of less than US$10,000 to a higher of a family income greater than US$150,000), strength of party identification (four categories, 0 = independent, 1 = leaning Republican/Democrat, 2 = not very strong Republican/Democrat, 3 = strong Republican/Democrat), party agreement with the senator (0 = disagreement, 1 = agreement), if the respondent is an independent (0 = not independent, 1 = independent), and race (0 = non-White, 1 = White). For ease of presentation, we omit some of the demographic variables from the tables that follow, although full models are available in the online appendix.
At Level 2 (the senator level), we control for each senator’s electoral status (safe election, competitive election, or no election), 9 party identification, the number of years served in office, and a dummy variable for each issue indicating if a senator missed that vote. 10 We account for the competitiveness of the race because prior research leads us to expect that competition leads to richer information environments and voters who are more engaged in the race (Kahn & Kenney, 1999). In addition, a combination of campaign advertisements and media coverage of competitive elections should lead to greater opportunities for voters to learn about senators’ positions. We allow the intercepts in our model to vary randomly by senator and respondent, as each respondent appears in the model twice because respondents are asked about the votes of both senators from their state. We also include a measure of senator party unity, which is rescaled to run from 0 to 1 to ease interpretation (where 0 corresponds to the minimum observed party unity score and 1 to the highest observed score). 11
Table 1 shows the results from the multilevel model. The coefficient on party unity is positive (0.25) and statistically significant (p < .001), indicating that higher levels of party unity correspond to voters correctly identifying a larger proportion of their senators’ votes. The coefficient on unity demonstrates that going from the minimum observed unity score to the maximum observed unity score results in a roughly 0.25 increase in predicted proportion of correctly identified votes. This effect is substantively important. It is almost 3 times as large, for instance, as is the effect of going from no election to a senator being in a competitive election. In short, the tendency for a senator to vote with her political party is a substantial contributor to the ability of her constituents to correctly infer her votes.
Multilevel Regression Predicting Levels of Correct Knowledge.
Note. Model displays results from a multilevel regression predicting levels of correct knowledge by party unity and other covariates. Controls also included for respondent age, income, education, and sex. See the online appendix for full results. CAFTA = Central American Free Trade Agreement; AIC = Akaike information criterion. Standard errors in parentheses.
p < .05. **p < .01. ***p < .001.
Policy Agreement and Unity
Data and Method
Having established evidence of a link between senator party unity and how effectively citizens can infer those senators’ votes on key pieces of legislation, we now examine how senator party unity relates to the correspondence between constituent–senator policy agreement and evaluations of that senator. To test the agreement hypothesis and unity moderation hypothesis, we are able to draw on the 2006, 2008, 2010, and 2012 CCES surveys (Ansolabehere, 2010a, 2010b, 2012; Ansolabehere & Schaffner, 2013). Each survey asks voters for their positions on several high-profile roll call votes taken in the current Congress. Each year contains between five and seven policy questions that map onto roll call votes taken in the Senate. 12 For example, from the 2010 CCES we use how respondents would have voted on five key votes the Senate took in 2009-2010: health care reform, financial reform, the stimulus bill, the expansion of the State Children’s Health Insurance Program (SCHiP), and Elena Kagan’s appointment to the Supreme Court.
From these policy questions, we construct measures of policy agreement between a respondent and each of their senators on the signature roll call votes in the previous Congress. We construct our agreement variable by adding together the total number of issue positions on which a respondent agrees with each of their senators. Because the theoretical maximum of this count varies by respondent—with some respondents not offering policy positions on every issue—and senator—not all senators cast votes on each issue—we then divide this count by the total number of positions on which a respondent and senator both offered a position. 13 The result is a variable capturing the percent agreement between respondent and senator on the votes where both took a position. Policy agreement averages around 50% each year, with a range from 52% agreement on average in 2006 to 60% agreement on average in 2008. Thus, in 2008, for example, the average respondent agreed with her senator on 60% of the major votes asked about. See the online appendix for a histogram of the variable across all 4 years. Note that we choose not to pool our data because the votes used to construct our policy agreement variable differ across years.
One way to assess how well voters link their policy positions with their assessment of their senators would be to look at vote choice as a function of ideological agreement with the senator (e.g., Joesten & Stone, 2014; Simas, 2013). While understanding how voters make choices in an election, a vote choice model that accounts for the positions of both candidates, such as the proximity voting models cited above, is ideal. Vote choice is less ideal when measuring constituents’ evaluations of their senators, however, as a constituent might vote for a senator she dislikes (or against a senator she likes) depending on the quality of the opposition candidate. Furthermore, focusing on vote choice requires examining only incumbent senators running for reelection. We therefore eschew focusing on vote choice in favor of a more direct measure of how a constituent views her senator.
To measure evaluations of sitting senators, we rely on approval questions from the CCES. The approval variable is a 4-point scale that ranges from strongly disapproves to strongly approve. We use this measure to construct a binary approval variable, coded 1 if the respondent strongly or somewhat approved of their senator and 0 if they somewhat or strongly disapproved. 14 We chose to recode approval into a binary measure for ease of interpretation when graphing the result and find the same results when using the original four-level dependent variable and ordered logistic regression. We model this dependent variable using binary logistic regressions.
We include several demographic control variables in our agreement models (respondent gender, race, age, and education). We again include controls for party agreement with the senator and strength of respondent party identification. We also ran the models with a measure of the perceived ideological distance between the respondent and her senator. Inclusion of the distance measure does not change the relationship between policy agreement, unity, and approval (nor is there a significant interaction between ideological distance and party unity in any models). We chose to omit the distance measures because of high levels of respondent non-response. 15 In addition, because studies of vote choice in Senate elections suggest presidential approval matters (e.g., Atkeson & Partin, 1995; Carsey & Wright, 1998), we control for approval of the president (Bush in 2006 and 2008 and Obama in 2010 and 2012) in our approval models on the assumption that senators’ approval ratings may vary in accordance with the presidents’ approval ratings. The variable is a 4-point approval scale ranging from strongly disapprove to strongly approve, and is coded so that higher approval of the president is associated with higher values for senators who share the president’s party identification and lower values for senators who are from the opposite party. Unlike presidential approval, there is some disagreement in the literature (Atkeson & Partin, 1995; Carsey & Wright, 1998) over whether economic perceptions matter in Senate elections. We choose to include economic perceptions in the model through a 5-point scale asking respondents how the economy fared in the last year. Like the presidential approval variable, this variable is coded so that higher values correspond to better evaluations of the economy when the senator is from the president’s party and worse evaluations when the senator is from the opposite party.
We again control for whether or not the senator is in a competitive election or safe election because longitudinal studies of approval find that senators engaged in tough reelection campaigns see a drop in their approval ratings, though the relationship between approval and competitive elections is likely endogenous (Brown & Jacobson, 2008). We ultimately do not attempt to draw any conclusions from our cross-sectional data about the causal link between elections and approval.
We also include dummy variables for retiring senators and those senators who lost their primary election and did not run as an independent. 16 Binder, Maltzman, and Sigelman (1998) find retiring senators tend to get a boost in their approval ratings, although the fact that low approval ratings predict retirements (Highton, 2008) suggests a negative relationship between retirement and approval. We therefore remain agnostic about the anticipated relationship between retirement and approval. We expect lower levels of approval for senators who lost their primary election. The final two variables in the models are a dummy variable for whether or not a senator was named to Citizens for Responsibility and Ethics in Washington’s (CREW) most corrupt members of Congress list between 2005 and 2012. 17 Each year, CREW puts out a list of roughly 20 members who it deems the “most corrupt.” The lists from 2005-2012 include such names as David Vitter (R-LA; involved in a prostitution scandal), Conrad Burns (R-MT; implicated in the Jack Abramoff scandal), and Larry Craig (R-ID; accused of soliciting sex in an airport restroom). Scandals and controversy are associated with lower vote shares in Senate elections (Abramowitz, 1988), and we expect them to negatively affect approval ratings as well. Finally, Binder et al. (1998) and Schaffner, Schiller, and Sellers (2003) find a negative relationship between state population size and approval, so we include a variable for state population—expecting a negative coefficient.
Our primary independent variables are policy agreement between the senator and a respondent and a senator’s party unity score. Our results first report a model with the direct effects of policy agreement and unity on approval and then a model specifying an interaction between each. Note that in all our models, we rely on objective as opposed to perceived policy agreement between respondent and senator. We made this decision for several reasons. First, only the 2006 CCES asks respondents for their perceptions of how their senators voted on issues. Thus, it is only possible to estimate perceived policy agreement for 2006. Second, perceived agreement is likely endogenous to approval of the senator because of projection effects and partisan bias (Ansolabehere & Jones, 2010; Wilson & Gronke, 2000). Objective agreement, however, should not be contaminated by respondents’ approval of their senator because it is constructed from respondents’ views on specific roll call votes and their senators’ votes on those same issues (Ansolabehere & Jones, 2010). Finally, and perhaps most importantly, from a normative perspective, we should care more about the link between actual policy agreement and approval than the link between perceived policy agreement and approval given that constituents’ perceived policy agreement is not always accurate and may be affected by their preexisting evaluations of the senator.
To be clear, we do expect that perceived policy agreement mediates the link between actual policy agreement and evaluations of a senator (e.g., Ansolabehere & Jones, 2010). As we have demonstrated, constituents make less accurate inferences about a senator’s votes when the senator is not a loyal partisan, which we expect translates into a weaker link between actual policy agreement and perceived policy agreement. For the 1 year where we can test this link, 2006, we find support for this expectation, with the correlation between actual policy agreement and perceived policy agreement being weaker for constituents whose senator is in the lowest quartile of party unity (r = .48) than for constituents whose senator is in the highest quartile of party unity (r = .78). In other words, for low-unity senators, constituents have a harder time translating actual policy agreement into perceived policy agreement. 18 With less accurate impressions of the degree to which they agree with their senator, then, constituents should have a harder time linking actual agreement on policy to evaluations of the senator.
All models are weighted by the included CCES sample weights, and we cluster all standard errors by senator. The clustered standard errors allow us to estimate cross-level interactions while accounting for the potential non-independence across senators (Primo, Jacobsmeier, & Milyo, 2007). 19 There is some question about whether clustering standard errors in a situation like ours is the best way to model a cross-level interaction; in our case, the link between party unity and policy agreement. In particular, this strategy can bias downward the size of the standard errors on the coefficient for the cross-level interaction, especially when clustering on 30 or fewer units (Leoni, 2009). Although this particular worry should not apply to our models because we cluster on 100 senators in our interactive models, we have also replicated our results by modeling the four-level approval dependent variable using weighted multilevel regressions with random intercepts for senator and respondent. Our results are substantively unchanged.
One final point is that we made the decision to omit responses for Arlen Specter (R-PA) from our analyses in 2008. We explain this decision in depth in our supporting information. In brief, in April 2009, Specter switched from the Republican Party to the Democratic Party. At the time of the 2008 CCES, 62% of Pennsylvania Republicans disapproved of Specter while 60% of Pennsylvania Democrats approved of him. This pattern is unique and not merely a reflection of Specter’s moderate voting record or low party unity score (in 2008, his unity score is 52.5), as 76% of Republicans approved of Olympia Snowe (R-ME; unity score = 41.8). His poor standing among Republican constituents affects the estimated link between party unity scores and policy agreement in the 2008 model. Including Specter in our analyses results in an insignificant coefficient on the interaction term (p = .64) while omitting Specter from the analysis results in a significant interaction (p = .01). When we systematically re-estimate each interactive model in each year while iteratively omitting one senator from each model, we find that omitting Specter in 2008 is the only instance of the 400 models in which excluding a single senator changes the statistical and substantive significance of the link between policy agreement and party unity. Because of the unique partisan approval patterns and his impact on the link between policy agreement and unity, we omit Specter from the 2008 models. Thus, the 2008 results are the least robust of our four analyses.
Within each year, we first present a baseline model that controls for unity and then a model with an interaction between party unity and policy agreement. To remind readers, we theorize that there should be a significant interaction between unity and policy agreement such that the effect of policy agreement should be greater for higher compared with lower unity senators. Coefficients from these logistic regressions are presented in Table 2.
Binary Logistic Regressions of Senator Approval on Policy Agreement and Party Unity.
Note. Models show results from models predicting the log likelihood of approving of a senator by year of the CCES. Controls also included for respondent age, income, education, and sex. See the online appendix for full results. AIC = Akaike information criterion; CCES = Cooperative Congressional Election Study. Standard errors in parentheses. *p < .05. **p < .01. ***p < .001.
For all the baseline models (Models 1, 3, 5, 7), the coefficient on policy agreement is a significant and positive predictor of approval. This is consistent with the agreement hypothesis. In addition, the interaction between policy agreement and party unity is significant in all 4 years at p < .05 (Models 2, 4, 6, 8). Because of the inherently complicated nature of a continuous by continuous interaction, we turn to graphs to illustrate the substantive effect of the interaction between policy agreement and unity.
For each year, we simulate the probability of a respondent approving of their senator across levels of policy agreement for senators in the 5th percentile (low unity) and 95th percentile (high unity) of unity (see Figure 2). For the year 2012, the 5th percentile of party unity corresponds to a score of around 78, a senator like Claire McCaskill (D-MO), while the 95th percentile corresponds to a score of roughly 97, a senator like Charles Schumer (D-NY). The probabilities are simulated for someone who shares a party with their senator, has a Democratic senator who is not up for election, did not lose a primary, and who is not on the corruption list. The rest of the variables in the model are held at their sample mean or mode. Error bands around the estimates are 95% confidence intervals.

Predicted probability of senator approval (2006-2012).
Consistent with our hypothesis, the probability of approving of a senator increases as policy agreement increases. However, as the significant interaction indicates, the strength of the link between policy agreement and approval is conditioned by a senator’s party unity score. Specifically, the effect of policy agreement is greater for high-unity senators than it is for low-unity senators. For example, in 2006, a respondent who agrees with 20% of their senator’s votes has a predicted probability of around .38 of approving of a high-unity senator compared with a predicted probability of roughly .50 of approving of a low-unity senator. We find similar patterns at high levels of approval, with people who share 80% policy agreement with a low-unity senator having a predicted probability of approval of a little less than .64. For the high-unity senator, however, predicted approval is above .77.
It is worth emphasizing that even when comparing approval of senators in the 5th and 95th percentile, there is overlap in the confidence intervals across a range of values in the graphs. However, visual inspection is not sufficient to assess statistical differences (or lack thereof) between point estimates with overlapping confidence intervals (Long & Freese, 2014; Schenker & Gentleman, 2001). Formal tests of the differences in the predicted probabilities of approval between high- and low-unity senators reveal statistically significant differences at both low and high levels of policy agreement. Although the exact location of these differences varies by year, they are typically observed between 0% and 40% policy agreement, and again above 80% policy agreement. 20 Differences in approval are especially pronounced at low levels of policy agreement, suggesting that constituents have an easier time linking strong policy disagreement with evaluations of high-unity senators compared with low-unity senators.
Although we do not present these results graphically, it is also possible to simulate the marginal effect of policy agreement across the range of party unity scores. For all 4 years, we observe that the marginal effect of policy agreement is larger for higher levels of party unity. For example, in 2006, the marginal effect of policy agreement for a senator with a party unity score of 68 (about the 5th percentile of our sample), a senator like Mike DeWine (R-OH), is 0.22, while the effect is roughly double that (0.49) for a senator like Carl Levin (D-MI), whose party unity score is 95 (about 75th percentile of our sample). In 2006, the marginal effect of agreement on approval is not significant at or below a party unity score of 67, while in 2012, the marginal effect is not significant at or below a party unity score of 78. 21
The other coefficients in the model confirm results from previous studies, with state population size as a negative predictor of senator approval (Binder et al., 1998; Schaffner et al., 2003), senators in safe elections enjoying higher approval ratings than senators in competitive elections (Brown & Jacobson, 2008), and both presidential approval and economic approval are related to evaluations of incumbent senators (Carsey & Wright, 1998).
In sum, we find consistent evidence that objective policy agreement affects approval and that this relationship is conditioned by the tendency of a senator to vote with or against her political party. These substantive effects persist even after accounting for sharing a party with a senator, a respondent’s economic retrospections, presidential approval, and the electoral status of a senator.
Conclusion
Our results demonstrate that the potential for citizens to hold their elected officials accountable for their actions in office is partly rooted in the representatives’ characteristics and behaviors while in office. We find that voters are decidedly more adept at identifying the votes of senators who adhere to the party line. Across six high-profile issues, voters were consistently better able to identify the votes of more partisan members of the Senate and had substantially higher levels of inaccurate knowledge of the least partisan senators. We then showed that party unity also moderates the connection between policy congruence and approval. Constituents were better able to align their preferences on high-profile votes with approval of their senator when that senator votes consistently with the party line.
These findings have implications for research on political accountability. We find substantial variation in how knowledgeable constituents are about their elected officials’ positions, which may limit their ability to hold their elected officials accountable for the specific positions they take. Moreover, we have identified that this variation is not simply observed at the individual level. Instead, senators’ behavior can enhance or obscure their policy positions. Politicians may take vague positions for strategic reasons (Tomz & Van Houweling, 2009), but even when they go on the record with a specific vote, some positions will be more difficult for the public to discern than others (see also Conover & Feldman, 1989). When a senator develops a consistent record of bucking her party, the result is that constituents are left without a clear cue to decipher her vote on a given issue. This makes it not only less likely that a voter will be able to identify the senator’s vote but also more likely that the voter will be incorrect in her or his inference.
Critically, the gap in accurate knowledge we observe between high- and low-unity senators seems to affect how voters evaluate their elected officials as well. Voters are consistently less able to link their policy positions, the positions taken by their senators, and approval when senators have a tendency to vote against the party line. Although this article does not examine vote choice, low approval ratings are consequential because they can lower vote margins, prompt senators to retire, and encourage quality challengers to enter a race (Brown & Jacobson, 2008; Highton, 2008).
Of course, approval rates are likely not simply a function of policy agreement between respondent and senator. However, given the salience of the votes asked about in these surveys, we think most would agree that a constituent should approve of a senator with whom she has high levels of policy agreement or disapprove of a senator with whom she has low levels of agreement. For example, the predicted probability that a same-party respondent approves of a low-unity senator she agrees with on zero or one votes is close to .5 in 2010. Given that the 2010 CCES asks respondents how they would have voted on health care reform, the stimulus, financial reform, SCHiP, and Elena Kagan’s confirmation, it is hard to make a normative argument that a respondent who disagrees with her senator on these votes should still approve of that senator. Furthermore, the consistency of our findings across years helps to rule out the potential that our conclusions are based on a unique combination of votes and/or senators in a particular year.
The conclusions presented here are not meant to challenge the empirical finding that higher party unity scores are associated with lower vote shares for incumbent legislators (Carson et al., 2010; Koger & Lebo, 2012). These findings suggest that at least some constituents are able to pick up on the degree to which their representative or senator is a loyal partisan (see also Fortunato & Stevenson, 2014; Sulkin et al., 2015). Our findings complement these works by showing that this awareness does not seem to translate into people being able to identify the positions their less loyal senators take on high-profile votes. In other words, constituents may know that their senator is a “moderate” or at least a less loyal Democrat or Republican, but constituents cannot easily translate that awareness into an understanding of their senators’ positions on specific issues.
How then should we make sense of these two sets of findings? One explanation is that constituents are simply not as concerned with the specific positions legislators take but instead with their general willingness to reach across the aisle. However, if this is the case, then our findings raise concerns about the quality of these preferences. Although moderate candidates may be more favorably evaluated than their more loyal colleagues, our findings offer no reason to believe that these evaluations are rooted in constituents accurately identifying those officials’ positions or bringing objective policy agreement to bear in their evaluations. Ultimately, more research is needed to better understand when and why incumbents benefit from accumulating less partisan voting records. Future research should explore both how much constituents know about their senators’ and representatives’ party loyalty as well as the criteria by which voters evaluate legislators who are more or less partisan in their voting records.
Our findings do further our understanding of how accountable senators are for the positions they take on high-profile votes. While awareness of policy positions may not be necessary for voters to hold politicians accountable, accurate inferences are preferred to the alternative we observe: high levels of incorrect beliefs about some senators’ voting records. Similarly, policy agreement may not be the only (or even necessarily the best) criteria by which senators should be judged, but we suspect few would argue with the contention that it is desirable for citizens to connect their policy preferences with their evaluations of elected officials. In both cases, we show that senator characteristics matter for the ingredients of accountability.
More generally, we see our findings as complementing recent work on the potential benefits of polarized political elites. Scholars have shown that awareness of party differences increases with polarization (Hetherington, 2001; Lupu, 2014), and voters themselves develop more consistent positions (Levendusky, 2010). We complement this work by showing that in our currently polarized era, voters are best able to infer elected officials’ voting behavior when asked about the most partisan senators. Perhaps surprisingly, it is ultimately the moderate senators that are so frequently pined for in an era of elite polarization that seem to pose the most problems for accountability. Given the cross-sectional nature of our study, we cannot comment on the role polarization has played over time in shaping constituents’ awareness of their elected officials positions. What we can say, however, is that in the contemporary era, it is the most partisan senators who enhance the ability of citizens to discern their elected officials’ positions. Moreover, it is those same senators about whom voters seem best equipped to make policy-based evaluations. Polarization and party conflict are maligned for the difficulties they create in the governing process (Mann & Ornstein, 2012), but it is worth remembering that such conditions may also heighten the public’s ability to hold their elected officials accountable (American Political Science Association, Committee on Political Parties, 1950; Fiorina, 1980).
Footnotes
Acknowledgements
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
