Can People Use Party Cues to Assess Policymaker Positions? Ecological Rationality and Political Heuristics

Abstract

Scholars disagree about the ability of people to use heuristics to make political judgments, with some arguing that heuristics are easy-to-use pieces of information and others arguing that applying heuristics may require some degree of political expertise. We argue that these debates have been somewhat intractable because most prior work has not considered the ecological rationality of political judgments—that is, the potential for cues to yield accurate judgments about a clearly defined reference class. In this paper, we present the results of two studies exploring whether people use party labels to make judgments about a random sample of U.S. Representatives’ voting behaviors. We find that respondents consistently performed worse in guessing U.S. Representatives’ votes than if they had correctly used a simple partisan heuristic. There is also some evidence that people performed worse with the presence of more nonparty cues. Attention to politics had a positive relationship with accuracy in both studies, although the relationship was modest. The results suggest that party cues may be more difficult to apply than some research has suggested.

Keywords

party cues heuristics ecological rationality take-the-best

Introduction

Some scholars have proposed that heuristics can compensate for people’s lack of knowledge about politics (Delli Carpini and Keeter 1996) by providing easy-to-use shortcuts that people can use to make political judgments (Downs 1957; Lupia 1994; Popkin 1994; Sniderman et al. 1991) while critics have emphasized barriers to heuristic use, including contextual knowledge required to appropriately use cues (Delli Carpini and Keeter 1996; Kuklinski and Quirk 2000). Critics have also argued that heuristics can lead to inaccurate judgments in some contexts (Dancey and Sheagley 2013; Lau and Redlawsk 2001), and people appear to make less than optimal use of cues when multiple cues are present (Kuklinski and Hurley 1994).

In spite of the large literature on party cues in particular, scholars have lamented the lack of cumulative knowledge about partisan heuristics (Bullock 2020). We argue, echoing prior critiques of the literature (Kuklinski and Quirk 2000), that the normative criteria for evaluating heuristic use are unclear in much prior work on the topic. Like some recent studies of heuristic use in making political judgments (Fortunato et al. 2019; 2021; Fortunato et al. 2021), we propose that scholars consider the ecological rationality (Brunswik 1952; Todd and Gigerenzer 2012; Simon 1956) of judgments by considering whether commonly studied heuristics, such as party cues, lead to accurate judgments about long-run empirical frequencies.

According to Fortunato and colleagues’ formulation (2021, Supplementary Material), a heuristic is ecologically rational if it is (a) cheap, including indicators that are readily available; (b) accurate, implying that there is an empirical association between the cue and the population about which one is making a judgment, and (c) simple, requiring an easily applied rule rather than a complex synthesis of information.

The current work concerns using party labels to make judgments about legislators’ policy positions, a context where at least two out of these three criteria are met. Using party labels in this context is cheap, requiring information that is typically readily available in political contexts. Concerning the accuracy of party cue use, in the current research, we ask respondents to draw inferences about the policy positions of a representative sample of members of the U.S. House of Representatives from the 116th Congress (2019–2020). Having respondents make judgments about a random sample of Representatives is critical part of the studies, as it allows us to confirm the accuracy of using party by testing the long-run empirical relationship between party labels and policy positions for the broader population of all Representatives.

Whether using party labels is simple is another question—making judgments using party labels requires one to know the implications of party labels to use them to make inferences about policy positions (Delli Carpini and Keeter 1996; Fortunato and Stevenson 2021), but whether people have this ability has been a source of debate (Converse 1964; Freeder et al. 2019; Kinder and Kalmoe 2017). We aim to evaluate to what extent people are able to apply party labels to make legislators’ positions on a series of recent, salient bills.

We find, first, that while people are aware of the usefulness of party cues relative to other binary cues in making judgments, the vast majority of respondents perform considerably worse than they would have if they had used party cues alone to make judgments. In fact, on average, respondents perform little better than they would have if they had made their judgments randomly. Second, there is some evidence that the inclusion of more nonparty cues reduces the accuracy of judgments. Third, attention to politics was associated with accuracy, although the magnitude of this relationship was modest. We conclude with a discussion of the implications for normative and theoretical considerations about cue use and some directions for future research.

The Ecological Rationality of Political Heuristics

While people’s lack of substantive knowledge about political figures, processes, and policy details is well established (Delli Carpini and Keeter 1996), easy-to-use pieces of information may allow people to compensate for a lack of information when making political judgments (Downs 1957; Sniderman et al. 1991). Although political scientists have proposed a number of potentially useful political heuristics (Kuklinski and Quirk 2000; Mondak and McCurley 1994; Sniderman et al. 1991), many researchers are less sanguine about the ability of people to use heuristics to make accurate judgments. Critics have argued that scholars have underestimated the ease with which cues can be used, as some heuristics may require considerable contextual information to apply to a political context (Delli Carpini and Keeter 1996, 51–53).

A major barrier to resolving debates about the usefulness of heuristics for people with low levels of substantive knowledge is establishing a clear standard with which to evaluate the use of cues. Many criteria that have been used to evaluate judgments have relied on values or other subjective normative evaluations which are open to debate (Kuklinski and Quirk 2000). One way to evaluate the ability of voters to use cues effectively would be to observe the ecological rationality (Todd and Gigerenzer 2012) of political judgments in the presence of cues. This normative standard concerns how well heuristics work in producing accurate judgments in the environment where they are applied (Brunswik 1952; Fortunato and Stevenson 2021; 2021; Todd and Gigerenzer 2012; Simon 1956; for similar arguments in political contexts, see De la Cuesta et al. 2022; McDonald 2020). The advantage of this approach is that it concerns the accuracy of judgments rather than whether cues lead to a “good” judgment by some other debatable normative standard, such as voting for the “better” candidate (see Kuklinski and Quirk 2000 for detailed critiques).

Prior work on heuristics in political judgments has evaluated the ecological validity of partisan cues—specifically the heuristic that legislators vote with their party (Dancey and Sheagley 2013; Fortunato and Stevenson 2021). Applying the partisan cue in this way requires knowing both a figure’s party affiliation as well as which party supports the bill, although the latter can be inferred from other knowledge, such as the ideological positions of the major parties (Fortunato and Stevenson 2021). In a variety of studies, Fortunato and Stevenson (2021, 2021; Fortunato et al. 2021) find that the use of party cues depends on the broader political context. For example, people are more likely to apply the partisanship heuristic to policymakers who are loyal partisans than to mavericks who buck the party line (Fortunato and Stevenson 2021). Moreover, party cues are more useful when the parties in government are distinct and homogeneous (see also Levendusky 2010). Members of the public are more likely to be aware of the relative positions of the parties (and less about individual policymakers) when these conditions hold, evidence that the party cue is more valuable to members of the public depending on the context (Fortunato and Stevenson 2021).

While the evidence from this body of research suggests that people are more likely to acquire information relevant for applying partisan cues and to apply partisan cues when they can be used to make valid judgments, the authors, noting that prior work on political judgments has rarely applied the standard of ecological rationality, acknowledge that these results represent a starting point for evaluating the use of party cues (Fortunato et al. 2021, 16). The current study explores a different approach to evaluate people’s ability to apply party cues to judgments about legislative voting to explore the ecological rationality of heuristic use for two reasons.

First, prior work on party cues has evaluated judgments about named political figures (Ansolabehere and Jones 2010; Dancey and Sheagley 2013; Fortunato and Stevenson 2021, 2021). This raises the possibility that other information confounds the influence of party cues. For example, people may use the likeability of a policymaker as a cue to infer their position on a bill (Sniderman, Brody and Tetlock 1991), may project their own attitudes onto a liked policymaker (Ansolabehere and Jones 2010; Wilson and Paul 2000), or may use some other, subtler process, such as the recognition heuristic (Goldstein and Gigerenzer 2002), using even minimal knowledge about policymakers to make judgments about policymaker voting. Although Fortunato and Stevenson acknowledge and control for some of these possibilities (Fortunato and Stevenson 2021), we see value in using a slightly more abstract approach, removing specific characteristics of individual policymakers aside from a few binary cues to isolate the influence of party labels on judgments.

Second, we ask respondents to make judgments about a random sample of Representatives. There are a number of advantages to using a random sample of Representatives as the target of a decision-making task designed to evaluate the accuracy of judgments. First, informing respondents that they are being asked to make judgments about a random sample limits strategic behavior. Respondents might otherwise assume that target Representatives are selected based on other criteria, such as a set of atypical members selected to create a challenging task (see Gigerenzer et al. 1991). Second, a key feature of ecological rationality is that there is a relationship between judgments and long-run empirical frequencies (Fortunato et al. 2021, Supplementary Material). We can assess the predictive validity of party and other cues by exploring the relationship between cues and voting behavior for the population of all House members, as the target sample is representative of the broader population of all U.S. Representatives, and cues that are valid for the population can be assumed to be valid for the target sample as well.

The current study shares with prior work the goal of evaluating the ecological rationality of using party cues to make judgments about political figures (Dancey and Sheagley 2013; Fortunato and Stevenson 2019, 2021), a context where using party labels should lead to accurate judgments. The strategy of relying on only one valid cue rather than all available information performs well relative to more complex decision-making strategies (such as an optimal weighting of all available cues) when making binary judgments, when one cue has a strong predictive validity, and when the cues are highly correlated, making some of the cues redundant (Gigerenzer, Todd and the ABC Research Group 2000; Hogarth and Natalia 2006). Using only the cue with the highest predictive validity to make a binary judgment (the “Take the Best” strategy; Gigerenzer, Todd and the ABC Research Group 2000) is optimal when the cues are noncompensatory—that is, when cues are ordered according to predictive validity, the predictive validity of each cue is greater than the sum of the predictive validity of all subsequent cues (Martignon and Hoffrage 1999, 2002).

These criteria where simple heuristic judgments perform well characterize the task of making judgments about policymakers’ policy positions described below. Specifically, party labels have a strong predictive validity for many policy positions, while many other cues are either not valid (i.e., do not have a strong correlation with the voters’ criteria for selecting a candidate) or are redundant after accounting for party affiliation (e.g., knowing NRA vote scores will not add much additional information after accounting for party). Therefore, using party cues—and only party cues—in making judgments about policymakers’ positions on the selected bills is a valid strategy, a claim we support below.

Hypotheses¹

The current study will explore people’s ability to make inferences using heuristics. The decision-making context (making judgments about policy positions of a random sample of U.S. Representatives) is one in which the predictive validity of one cue (U.S. party affiliation) is much higher than the sum of the remaining cue validities (including demographic and readily available biographical cues such as gender and veteran status). This distribution of cue validities, as explained above, lends itself to using only the most valid cue to make judgments (Gigerenzer, Todd and the ABC Research Group 2000; Hogarth and Natalia 2006; Martignon and Hoffrage 1999, 2002). The first research question concerns how accurate people’s judgments are relative to using only party cues available and ignoring the remaining cues:

RQ: How will respondents’ accuracy scores compare to the accuracy score they would have received if they had used only party cues?

As explained below, respondents will be exposed to a longer or shorter set of cues, both of which include party cues. As using nonparty cues will lead to suboptimal performance, we predict that fewer available nonparty cues should lead to more accurate judgments, as there are fewer distractions from using party cues:

H1:

Respondents in the fewer cues condition will receive higher accuracy scores on average than respondents in the more cues condition.

The next hypothesis concerns the relationship between attention to politics (Delli Carpini and Keeter 1996; Zaller 1992) and judgments. People with higher levels of expertise should more effectively use heuristics (Lau, Andersen and Redlawsk 2008; Lau and Redlawsk 2001; Sniderman, Brody, and Tetlock 1991) to make accurate judgments:

H2:

Accuracy scores will be positively related to attention to politics.

In addition to predictions about the accuracy of judgments, we also make two predictions about what we will label respondent’s validity ratings, or subjective ratings of how useful cues are in making judgments. People who are more attentive to politics should correctly identify the relevance of cues (Delli Carpini and Keeter 1996) and should therefore be more likely to judge party cues as highly valid and other cues as less valid:

H3a:

Validity ratings of party cues will be positively related to attention to politics.

H3b:

Validity ratings of nonparty cues will be negatively related to attention to politics.²

Method

This paper presents the results of two studies with participants recruited from Amazon’s MTurk platform. Instructions explained that respondents would be asked to predict voting positions on five prominent bills from 2019–2020 for a random sample of six U.S. Representatives³. The bills were selected from a list of the most viewed bills for the 116th Congress (2022) to represent a diverse set of highly salient issues on which the parties were divided. The votes selected involved only final passage on bills, and bills were selected to avoid technical issues. The strategy of selecting salient, party-line votes was meant to facilitate the application of party heuristics to judgments about Representatives’ positions⁴. The number and order of cues were randomly assigned, although party cues appeared in all profiles. The full set of binary cues included Representative party (Democrat/Republican), gender (male/female), ethnicity (Latino/a, non-Latino/a), veteran (served in military/did not serve), religion (Catholic/non-Catholic), region (South/non-South), and tenure (serving first term/second term or higher). Biographical information about Representatives was collected from various sources (Congress 2022, Pew Research Center 2019, Shane 2018 and U.S. House of Representatives n.d.). All cues are binary to facilitate comparisons across cues. Respondents then rated the usefulness of all 7 cues and responded to a variety of demographic and political (attention to politics, party, and ideology) questions, followed by an open-ended request for feedback on the study.

After receiving a set of instructions, respondents were presented with the profile of the first Representative (the order of Representatives was randomized). The profiles did not include any identifying information other than a set of binary cues including party to prevent the confounding influence of other factors. Respondents were asked to make a judgment about the Representative’s position on the set of five bills and were encouraged to guess even if they were not certain. Respondents then completed the same task for the each of the remaining Representatives, for a total of 30 judgments (6 Representatives * 5 bills).

Our ability to evaluate the ecological rationality of judgments in this task stems from the fact that respondents are asked to make judgments about a representative set of legislators drawn from a known population. First, drawing a random sample from the population of all U.S. Representatives in the 116th U.S. Congress allows us to explore the ecological rationality of cues, as judgments are made not about nonstereotypical or hypothetical Representatives, but about a representative sample of Representatives drawn from a well-defined population. Moreover, previous work on general knowledge questions suggests that people’s judgments are influenced by what they perceive to be the reference class from which items were drawn. For example, people asked a trivia question about which of a pair of cities is larger might suspect that the pair was selected to create a challenging or misleading question, and may answer accordingly. On the other hand, if the cities were selected at random from all U.S. cities, this strategic thinking is unnecessary (Gigerenzer et al. 1991). Respondents were informed that they would be making judgments about a randomly selected set of U.S. Representatives to prevent the impression the target policymakers are in some way unrepresentative of the broader population. To randomly select six Representatives for stimulus materials for the two studies, we stratified on party (Democrats or Republicans), gender (male or female), and region (South or non-South) and randomly selected six Representatives. Representatives who did not vote on all five of the bills listed were excluded from the sample⁵.

Predictive Validity of Party Cues

Before proceeding to the two studies, we assess the predictive validity of the binary cues for the full population of U.S. Representatives for the set of selected bills. We regressed votes on each of the five bills on all binary cues included in the study⁶. The coefficients for the seven cues from the ordinary least squares regressions (estimated with clustered standard errors) are presented in Figure 1. The Figure shows that among the cues listed, party labels have by far the highest predictive validity. None of the other binary cues has a sizeable impact on voting behavior after controlling for party.

Figure 1.

Estimated predictive validity for seven binary cues. Note. Figure displays regression coefficients and 95% CIs (estimated with robust standard errors) from regression models predicting U.S. Representative’s voting behavior (1 = “Y”; 0 = “N”) with the seven binary cues.

Due to the noncompensatory nature of the structure of cues (Hogarth and Natalia 2006; Martignon and Hoffrage 1999, 2002) for the selected bills, the optimal strategy for respondents would be to use only party affiliation and ignore the rest of the cues. In fact, using this strategy would lead respondents to guess correctly in all judgments for the sample selected, although Figure 1 suggests that making judgments using only party cues would likely lead to mostly accurate judgments even if a different sample were selected from the population of U.S. Representatives.

An alternative approach supports this intuition. Estimating the mean number of votes compatible with party labels using 10,000 bootstrap samples of size N = 6 from the complete list of Representatives with nonmissing voting data on all five bills predicts that a mean of 29.5 (bootstrap SE = 1.2) votes out of 30 were compatible with party labels; 95% bootstrap CI = (26, 30), percentile method. The results show that regardless of the specific sample selected, party labels were highly valid predictors of voting for the bills in the study.

Study 1

Participants (N = 502) completed a short survey in which they were asked to predict the voting positions of United States Representatives on five prominent bills from 2019–2020. After completing the ratings, respondents rated the usefulness of all seven cues, provided demographic information, and were asked to provide feedback about the study.

Sample

Participants were recruited from Amazon’s MTurk to participate in a study of “current events.” Respondents were limited to the U.S. and were required to have completed more than 500 Human Intelligence Task (HIT)—tasks created by requesters on the MTurk platform—and had an HIT approval rate greater than 90%⁷. Descriptive statistics for the sample are presented in Supplementary Table 1.

Table 1.

Predicting the Number of Judgments (out of 30) Consistent With Party Cue Use (Study 1).

	Model 1			Model 2
Variable	B	SE B	95% CI	B	SE B	95% CI
Fewer cues	−0.23	0.56	[−1.34, 0.88]	−0.26	0.53	[−1.30, 0.78]
Party first	−0.54	0.56	[−1.65, 0.56]	−0.47	0.53	[−1.51, 0.58]
Attention to politics				1.09**	0.34	[0.41, 1.77]
Extreme ideology				−0.39	0.32	[−1.02, 0.24]
Party strength				−0.74*	0.30	[−1.33, −0.15]
Age				0.15***	0.02	[0.11, 0.19]
Female				1.87***	0.56	[0.77, 2.96]
Latino				1.40	2.39	[−3.30, 6.10]
African American				−0.72	0.96	[−2.61, 1.17]
Asian				0.50	1.11	[−1.68, 2.68]
Education level				0.04	0.18	[−0.31, 0.40]
Income level				0.46*	0.22	[0.02, 0.90]
(Constant)	20.52	0.48	[19.58, 21.46]	13.84	1.76	[10.37, 17.30]
N	502			480
R ²	0.002			0.184

Note: + p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001, two-tailed.

Stimulus

Participants were randomly assigned to either a long (7 cues) or short (3 cues) list of cues. Both sets included party labels along with two other cues (tenure and gender). Respondents were also randomly assigned to two separate orders of cues, with party either first or last (the order of cues was simply reversed in the party last condition). The order and number of cues remained the same for each respondent across the six Representatives. An example stimulus is displayed in Figure 2.

Figure 2.

Example of a stimulus profile.

Results

The Research Question concerns the accuracy of respondents’ inferences about Representatives’ positions. Accuracy scores were measured by summing the total number of correct responses, which, for the sample of Representatives selected, is equivalent to guessing that all Democrats supported and all Republicans opposed all 5 selected bills. The distribution of this variable is displayed in Figure 3 (upper panel). The results suggest that participants perform poorly in inferring policymaker positions. The mean number of correct scores out of 30 was 20.1 (SD = 6.3). Fewer than 15% of the sample made judgments consistent with party labels for all 30 judgments.⁸

Figure 3.

Distribution of judgments compatible with party cues.

The first set of hypotheses concerns which factors contributed to accuracy. Accuracy scores were regressed on an indicator for fewer cues, an indicator for party first, and an additive score of attention to politics based on three items (α = .81). Regressions were run with and without controls for party strength (the absolute value of 7-point party identification minus the midpoint of 4), extreme ideology (the absolute value of 7-point self-identification minus 4)⁹, education, income, indicators for African American and Latina/o identity, an indicator for female gender, and age in years.¹⁰

The results are presented in Table 1. Model 1 shows that, contrary to expectations, fewer cues do not improve accuracy; the coefficient is negative and not statistically significant. Likewise, the coefficient for party first is also negative and not statistically significant. These results are not substantively affected with the addition of controls (Model 2). The results do not support H1: receiving fewer cues did not improve accuracy scores.

Model 2 includes a control for attention to politics; this coefficient is positive, as expected. While statistically significant, the coefficient is small: a one-standard deviation increase in attention increases one’s accuracy score by only about one additional judgment on average, holding other variables constant. While the results support H2, the effect of attention to politics on accurate judgments is modest. While not a formal hypothesis, the coefficients for extreme ideology and party strength are both negative, although only the coefficient for party strength is statistically significant, suggesting that party strength surprisingly leads to less reliance on party cues.

Table 2 includes the interaction of attention and cue conditions (party first X attention, fewer cues X attention)^.11 There is some evidence (Model 4) of a positive, statistically significant interaction between attention and the shorter list of cues. Although the results are mixed, the results suggest that attentive people may make more accurate judgments when there were fewer irrelevant cues to distract them from using party cues. There is no other evidence of an interaction between attention and cue condition.

Table 2.

Interaction of Attention to Politics, Cue Order, and Number of Cues (Study 1).

	Model 3			Model 4
Variable	B	SE B	95% CI	B	SE B	95% CI
Attention to politics	0.35	0.56	[−0.73, 1.43]	0.44	0.54	[−1.33, 0.75]
Fewer cues	−0.19	0.79	[−1.29, 0.91]	−0.29	0.53	[−1.56, 1.37]
Fewer×attention	0.71	0.66	[−0.58, 2.01]	1.37*	0.63	[0.13, 2.60]
Party first	−0.58	0.56	[−1.67, 0,52]	−0.47	0.53	[−1.51, 0.57]
Party first×attention	0.63	0.66	[−0.66, 1.93]	0.06	0.62	[−1.17, 1.28]
Extreme ideology				−0.44	0.32	[−1.07, 0.19]
Party strength				−0.74*	0.30	[−1.33, −0.16]
Age				0.15***	0.02	[0.11, 0.187]
Female				1.89**	0.56	[0.80, 2.98]
Latino				1.89	2.40	[−2.82, 6.60]
African American				−0.71	0.96	[−2.60, 1.18]
Asian				0.71	1.11	[−1.48, 2.89]
Education level				0.03	0.18	[−0.32, 0.38]
Income level				0.49*	0.18	[0.06, 0.93]
(Constant)	20.52	0.47	[19.59, 21.45]	13.85	1.76	[10.39, 17.30]
N	502			480
R ²	0.024			0.192

Note: Ordinary least square regression estimates. Dependent variable is accuracy score (out of 30). + p < 0.10. *p < .05. **p < .01. ***p < .001, two-tailed.

The low levels of accuracy in rating Representatives’ voting suggest that people may not appreciate how useful party cues are relative to other, less diagnostic cues. Figure 4 (upper panel) displays the ratings of the usefulness of cues (1 = not at all useful, 5 = extremely useful). All respondents, regardless of treatment condition, rated the full set of cues (the order of the presentation of cues in these items was randomized). The results show that all respondents rated party cues higher than the remaining cues. However, the remaining nonparty cues are rated relatively high, placed close to the midpoint of the scale. The results suggest that people recognize the relative value of party cues over nonparty cues, but they may not appreciate how much more useful party cues are than the other cues listed.

Figure 4.

Mean respondent ratings of cue validities.

The next set of regressions concerns which factors predict ratings of cue usefulness. Two dependent variables, the rating of the usefulness of party cues and an additive scale of all six nonparty cues (α = 0.89), are each regressed on the treatment variables and the set of controls. The results are presented in Table 3. The coefficient for attention to politics in Model 5 is positive and statistically significant, indicating that as attention to politics increased, so did ratings of the usefulness of party cues. However, attention had no influence on ratings of less valid, nonparty cues (Model 6). The results provide support for H3a but not H3b.¹²

Table 3.

Predicting Ratings of the Usefulness of Party and Nonparty Cues (Study 1).

	Model 5			Model 6
	Party Cue Rating			Nonparty Cue Rating
Variable	B	SE B	95% CI	B	SE B	95% CI
Fewer cues	−0.06	0.08	[−0.20, 0.09]	0.16*	0.07	[0.03, 0.29]
Party first	−0.03	0.08	[−0.18, 0.12]	0.01	0.07	[−0.12, 0.14]
Attention to politics	0.27***	0.05	[0.18, 0.37]	−0.03	0.04	[−0.12, 0.05]
Extreme ideology	0.06	0.05	[−0.03, 0.15]	0.20***	0.04	[0.12, 0.28]
Party strength	0.01	0.04	[−0.07, 0.10]	0.00	0.04	[−0.07, 0.08]
Age	0.01+	0.00	[0.00, 0.01]	−0.02***	0.00	[−0.02, −0.01]
Female	0.13	0.08	[−0.03, 0.28]	−0.23***	0.07	[−0.36, −0.09]
Latino	−0.19	0.34	[−0.86, 0.48]	−0.11	0.30	[−0.70, 0.47]
African American	0.06	0.14	[−0.22, 0.33]	0.21+	0.12	[−0.02, 0.45]
Asian	0.46**	0.16	[0.15, 0.77]	0.16	0.14	[−0.11, 0.43]
Education level	0.02	0.03	[−0.03, 0.07]	0.01	0.02	[−0.03, 0.06]
Income level	0.02	0.03	[−0.04, 0.08]	0.00	0.03	[−0.06, 0.05]
(Constant)	3.69	0.25	[3.19, 4.18]	0.30	0.22	[−0.13, 0.74]
N	480			480
R ²	0.128			0.223

Note: Ordinary least square estimates. +p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001, two-tailed.

Discussion

In Study 1, respondents performed worse in making judgments about U.S. Representatives’ votes than if they had correctly used a simple partisan heuristic. We propose three possible caveats to these results that are addressed in Study 2. First, participants may expend minimal effort on the task. While participants may have the ability to apply party cues, they may lack sufficient motivation to use cues in making judgments. On the other hand, satisficing behavior could involve opting for the simple strategy of using only party cues instead of trying to optimize all available cues, suggesting that the effect of effort on party cue use is not straightforward. Second, respondents, being unfamiliar with this task, may perform in suboptimal ways that have little to do with their knowledge and application of cues. For example, in other laboratory studies, respondents have been shown to engage in probability matching (e.g., if they are informed that 80% of balls drawn from an urn are green and 20% are blue, they will guess that a ball drawn is green 80% of the time although guessing green 100% of the time will yield a higher expected value) (Shanks, Tunney, and McCarthy 2002; Vulkan 2000). Respondents may have engaged in probability matching if they thought that party was not related to voting behavior 100% of the time. This assumption would be incorrect for this sample, but political party may not have predicted voting behavior perfectly if a different sample of Representatives had been selected. Finally, the mechanism behind the results is unclear, as respondents may either unable to apply party labels correctly due to not knowing the relevance of parties for the votes listed or because they rely on other, less diagnostic cues¹³. The second study addresses these concerns by providing more careful screening of participants, adding an attention check and incentives to limit satisficing behavior, and adding a “party-only” condition in addition to the two lists of cues to assess the use of party labels on their own.

Study 2

Methods

The procedure and measurements were the same in Study 1. The criteria for the MTurk participants (N = 453) was adjusted based on best practices to avoid potential satisficing behavior (Bauer et al., 2020), including only MTurk respondents with an HIT approval rate of greater than or equal to 99% and at least 100 completed HITs. In addition, an incentive of $1 was offered to participants who guessed at least 20 out of 30 votes correctly. The threshold of 20 out of 30 correct answers was used rather than either awarding an incentive for each correct answer or for a very high threshold of correct responses, as these latter incentive structures may have still encouraged probability matching as respondents attempt to create the possibility (albeit with a small probability) of guessing all bills correctly, an outcome that would be impossible if party labels did not perfectly predict voting behavior. Participants were informed at the start of the survey that the incentive would only be available if they completed the survey in 15 min or less to prevent respondents from attempting to look up information on the bills.

An attention check item was also added to the study after the judgment task. A careful reading of the instructions was required to see that the item asked for respondents to check the boxes next to two specific colors rather than indicating their favorite color (Berinsky, Margolis, and Sances 2014). Fully 98% of the sample correctly answered this item, suggesting that few respondents were engaging in satisficing behavior. Descriptive statistics for the sample are included in Supplementary Table 2.

The design of Study 2 differs in another important way from Study 1: for each profile, respondents were assigned to one (party only), three (party, tenure, and gender), or all seven cues. The inclusion of party-only conditions allowed the possibility of exploring whether respondents can guess accurately based solely on party information without the distraction of other irrelevant information.

Results

How accurate were judgments with the modifications to the research design, including added incentives? Figure 2 displays the distribution of accuracy scores out of 30 (lower panel)—which, as in Study 1, is equivalent to assuming that Democrats supported and Republicans opposed all five bills. The mean accuracy score was 21.9 (SD = 5.9), slightly higher than Study 1. The modifications to the design improved performance, probably by reducing random guessing (observe the spike close to 50% correct in the top panel), although respondents fell well short of the perfect score that could have been achieved by using party labels. Figure 3 (lower panel) shows ratings of the usefulness of cues. The pattern is similar to Study 1, with ratings of the party cue rated as highly useful and the remaining cues rated close to the midpoint of the scale, although nonparty cues were rated slightly lower than in Study 1.

A multilevel model including random intercepts for respondent and Representative predicts accuracy scores for each Representative (out of a possible 5) with indicators of treatment category (indicators for three cues or seven cues; baseline = party cue only 14). The models are run without and with the full set of controls. Results are displayed in Table 4. In both Models 7 and 8, the coefficients for few cues are small and not statistically significant, although the coefficient for many cues is negative and statistically significant, suggesting that those exposed to more nonparty cues performed worse than those exposed to party cues alone. The results, unlike Study 1, provide some support for H1: a longer list of nonparty cues decreases the accuracy of judgments.

Table 4.

Multilevel Model: Predicting the Number of Judgments (out of 5) Consistent With Party Cue Use (Study 2).

	Model 7	Model 8
	B (SE)	B (SE)
Few cues	−0.05 (0.05)	−0.03 (0.05)
Many cues	−0.13 (0.05)*	−0.14** (0.05)
Attention to politics		0.18*** (0.06)
Extreme ideology		−0.02 (0.06)
Party strength		−0.13** (0.05)
Age		0.01*** (0.00)
Female		0.12 (0.09)
Latino		0.48* (0.24)
African American		−0.49*** (0.14)
Asian		0.40* (0.17)
Education level		−0.06* (0.03)
Income level		0.11*** (0.03)
Variance (representative)	0.32 (0.19)	0.32 (0.18)
Variance (respondent)	0.76 (0.06)	0.58 (0.05)
Variance (residual)	1.11 (0.03)	1.10 (0.03)
N	2718	2628

Notes: Random intercepts for respondent and Representative included.

+ p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001, two-tailed.

Attention to politics is positively related to accuracy: the coefficient for attention to politics in Model 2 is positive and statistically significant, although as in Study 1, the magnitude is modest: a one standard deviation increase in attention to politics leads to an increase of 0.18 out of five judgments. Although modest in magnitude, the results provide additional support for H2. Table 5 includes the interaction of attention and cue conditions (party first X attention, fewer cues X attention). Unlike Study 1, there is no evidence of an interaction between attention and cue condition.

Table 5.

Multilevel Model: Interaction of Attention to Politics and Number of Cues (Study 2).

	Model 9	Model 10
	B (SE)	B (SE)
Attention to politics	0.20**(0.06)	0.20**(0.07)
Few cues	−0.05 (0.05)	−0.03 (0.05)
Few X attention	−0.04 (0.06)	−0.05 (0.06)
Many cues	−0.13* (0.05)	−0.14** (0.05)
Many X attention	−0.02 (0.06)	0.00 (0.07)
Extreme ideology		−0.017 (0.06)
Party strength		−0.13** (0.05)
Age		0.01*** (0.00)
Female		0.12 (0.09)
Latino		0.49* (0.24)
African American		−0.49*** (0.14)
Asian		0.40* (0.17)
Education level		−0.06* (0.03)
Income level		0.11** (0.03)

Variance (representative)	0.32 (0.19)	0.32 (0.18)
Variance (respondent)	0.74 (0.06)	0.58 (0.05)
Variance (residual)	1.11 (0.03)	1.10 (0.03)
N	2718	2628

Notes: Random intercepts for respondent and Representative included.

+p < .10. *p < .05. **p < .01. ***p < .001, two-tailed.

Discussion

Study 2 added a number of modifications to the design, including screening of respondents and incentives for accurate judgments, but did not result in much higher accuracy beyond apparently eliminating some guessing. The overall pattern of results, including the distribution of accuracy judgments, the use of party labels, and the rating of cues, was similar to Study 1. The evidence suggests that people perform poorly both because they apply party cues incorrectly and because they overuse other cues relative to party cues, as respondents in the party-only condition performed poorly but were slightly more accurate than people exposed to more cues.

General Discussion

The current study uses a unique design, asking respondents to make judgments about a representative sample from the population of all U.S. Representatives. The design resolves a problem with prior studies, which have produced mixed evidence about the ability of people to use party cues to make judgments (Bullock 2020). The current study, in asking respondents to make judgments about representative samples, provides the ability to evaluate whether the use of cues leads to ecologically rational judgments—that is, judgments about a well-defined reference class. Unlike prior work on the ecological validity of cues, the current study removed other identifying information about legislators to isolate the influence of cues on judgments. The evidence about people’s ability to use cues is disappointing. While some prior work has suggested that party cues have the potential to compensate for people’s well-documented lack of knowledge about politics, the current work suggests that people lack the ability to use party labels to make accurate judgments about policy positions. Accuracy improves with attention to politics, although the gains are relatively small. Shortening the list of nonparty cues available has the potential to improve performance, although the evidence on this point was mixed.

Is it realistic to conclude that respondents are so inept at applying party cues to make policy judgments, or are the results merely an artifact of the design? While we would like to see more research on the topic, we conclude that the former interpretation is more likely. First, some features of the design, especially in Study 2, including careful screening of respondents, incentives for correct responses, and a challenging attention check, suggest that respondents exerted sufficient effort on the task. Moreover, the task required merely the application of party labels to judgments, a task that could be competently executed with low levels of effort, if only respondents had the ability to do so. Moreover, the bills were selected to be highly salient, and many of them, including votes relating to President Trump, should not have presented particular difficulty in applying party labels to judgments. Even respondents who were assigned to party labels alone in Study 2 did not perform particularly well on the task, suggesting that the application of party labels was the main barrier to accurate judgments rather than distraction by irrelevant information, although there is some evidence that both contributed to inaccurate judgments.

Other empirical work is compatible with this interpretation. Decades of research in political science suggest low levels of contextual knowledge about politics (Delli Carpini and Keeter 1996) and lack of familiarity with and use of ideological concepts in reasoning about politics (Converse 1964). Recent work has bolstered these findings in the current era of elite polarization (Kinder and Kalmoe 2017). For example, Freeder et al.’s (2019) estimate that fewer than 20% of the public can consistently estimate party and candidate positions are compatible with the current results, where about 15% of the public consistently used party labels to inform judgments about policy positions. In a recent striking example, Gunterman and Lenz (2022) find that the public is unaware of candidates’ stances on issues related to COVID—a pandemic that dominated news coverage for months and had important implications for public health, the economy, and daily life, and on which major party figures were starkly divided. Similarly, a survey on a variety of major issues found that people are not aware of major party positions (Pew Research Center 2019).

One practical implication of our findings is that the typical respondent did not recognize the relevance of political parties in making political judgments. This may stem in part from increased negativity toward political parties (Abramowitz and Webster 2016) or a desire to avoid overgeneralizing from party labels to individuals. Another possibility is that people are unaware of the level of partisan polarization on a number of salient issues (but see Fortunato and Stevenson 2021). Civic education and other forums in which political decision-making is discussed could place greater emphasis on the relevance of political parties in making reasonable judgments to facilitate decision-making (see Lupia 2016 for similar recommendations about civic education, broadly conceived). While there are downsides to the overuse of political parties in making judgments, it appears from the current study that people in fact underuse party labels in drawing conclusions about political figures.

Conclusion

Although some scholars have argued that voters could compensate for their ignorance about the substance of politics with easy-to-use cues, our results suggest that participants are not able to apply party cues to make judgments about Representatives’ voting on salient issues. People who are more attentive to politics are better able to use cues, although the relationship is small in magnitude. There is some evidence that the structure of the information environment, including the number of cues, could impact on accuracy judgments, although these results are mixed.

There are a number of limitations to the current study. The study relied on an MTurk sample of respondents, and while studies have validated these samples (Berinsky, Huber, and Lenz 2012; Clifford, Jewell, and Waggoner 2015), future work could explore the results with representative samples of respondents. Second, while we attempt to explore the ecological rationality of judgments, the exposure to cues in the current research is somewhat abstract, requiring judgments about unnamed Representatives, in order to remove the influence of confounding information. It is possible that cues available in real-world political contexts contribute to accurate judgments and make party cues redundant. Perhaps, for example, people rely on their attitudes about their Representatives to draw inferences about their issue positions (Sniderman, Brody, and Tetlock 1991), a strategy that is precluded in the current research design, although we note that in the current polarized environment, people tend to rely more on party cues than individuating information about specific political figures (Fortunato and Stevenson 2021).

Similarly, the sample of bills selected may limit the generalizability of the results. Specifically, we selected a set of bills on highly salient issues with clear partisan divides. We would expect that accuracy scores would be even lower if a set of bills were selected where partisan cues were not especially relevant, or for a set of bills dealing with less salient issues, suggesting that the modest accuracy rates in the current study may in fact overstate people’s ability to use cues to make accurate judgments.

Echoing recent work on party cues, we have made a case to use ecological rationality as the relevant benchmark for evaluating political judgments. The results show that considering the context of political judgments can help to resolve conflicting findings about the accuracy of heuristics, an area where there is much research with little accumulation of evidence (Bullock 2020). We find that people are not adept at using party cues to make accurate judgments, although future research using similar designs is needed to explore the generality and robustness of these results.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Daniel E. Bergan

Supplemental Material

Supplemental materials and replication materials for this article are available with the manuscript on the Political Research Quarterly (PRQ) website.

Notes

References

Abramowitz

Alan I.

Webster

Steven

. 2016. “The Rise of Negative Partisanship and the Nationalization of US Elections in the 21st Century.” Electoral Studies 41: 12-22. https://doi.org/10.1016/j.electstud.2015.11.001

Amazon Mechanical Turk . 2019. Qualifications and Worker Task Quality. https://blog.mturk.com/qualifications-and-worker-task-quality-best-practices-886f1f4e03fc

Ansolabehere

Stephen

Jones

Philip Edward

. 2010. “Constituents’ Responses to Congressional Roll-Call Voting.” American Journal of Political Science 54, 3: 583-597.

Bauer

Brian

Larsen

Kristy L.

Caulfield

Nicole

Elder

Domynic

Jordan

Sara

Capron

Daniel

. 2020. “Review of Best Practice Recommendations for Ensuring High Quality Data with Amazon’s Mechanical Turk.” https://psyarxiv.com/m78sf/download?format=pdf

Berinsky

Adam J.

Huber

Gregory A.

Lenz

Gabriel S.

. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon. com's Mechanical Turk.” Political Analysis 20, 3: 351-368. https://doi.org/10.1093/pan/mpr057

Berinsky

Adam J.

Margolis

Michele F.

Sances

Michael W.

. 2014. “Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self-Administered Surveys.” American Journal of Political Science 58, 3: 739-753. https://doi.org/10.1111/ajps.12081

Brunswik

Egon

. 1952. The Conceptual Framework of Psychology. Chicago: The University of Chicago Press.

Bullock

John G.

2020. “Party Cues. In: The Oxford Handbook of Electoral Persuasion, 129. Oxford University Press.

Clifford

Scott

Jewell

Ryan M.

Waggoner

Philip D.

. 2015. “Are Samples Drawn from Mechanical Turk Valid for Research on Political Ideology?” Research and Politics 2(4). https://doi.org/10.1177/205316801562207

10.

Converse

Philip E.

1964. “The Nature of Belief Systems in Mass Publics.” edited by David Apter , ed. Free Press. Ideology and Discontent.

11.

Dancey

Logan

Sheagley

Geoffrey

. 2013. “Heuristics Behaving Badly: Party Cues and Voter Knowledge.” American Journal of Political Science 57, 2: 312-325. https://doi.org/10.1111/j.1540-5907.2012.00621.x

12.

De la Cuesta

Brandon

Egami

Naoki

Imai

Kosuke

. 2022. “Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution.” Political Analysis 30, 1: 19-45. https://doi.org/10.1017/pan.2020.40

13.

Delli

Carpini

Michael

Scott

Keeter

. 1996. What Americans Know about Politics and Why it Matters. Yale University Press.

14.

Downs

Anthony

. 1957. An Economic Theory of Democracy, 20. Boston: Addison Wesley.

15.

Fortunato

David

Nick

Lin

Stevenson

Randolph T.

Wessel Tromborg

Mathias

. 2021. “Attributing Policy Influence under Coalition Governance.” American Political Science Review 115, 1: 252-268. https://doi.org/10.1017/S0003055420000763

16.

Fortunato

David

RandolphStevenson

. 2019. “Heuristics in Context.” Political Science Research and Methods 7, 2: 311-330. https://doi.org/10.1017/psrm.2016.37

17.

Fortunato

David

Stevenson

Randolph T.

. 2021. “Party Government and Political Information.” Legislative Studies Quarterly 46, 2: 251-295. https://doi.org/10.1111/lsq.12285

18.

Freeder

Sean

Lenz

Gabriel S.

Turney

Shad

. 2019. “The Importance of Knowing ‘What Goes with What’: Reinterpreting the Evidence on Policy Attitude Stability.” The Journal of Politics 81, 1: 274-290. https://doi.org/10.1086/700005

19.

Gigerenzer

Gerd

Ulrich

Hoffrage

Heinz

Kleinbölting

. 1991. “Probabilistic Mental Models: A Brunswikian Theory of Confidence.” Psychological Review 98, 4: 506-528.

20.

Goldstein

Daniel G.

Gigerenzer

Gerd

. 2002. “Models of Ecological Rationality: The Recognition Heuristic.” Psychological Review 109, 1: 75-90. https://doi.org/10.1037/0033-295X.109.1.75

21.

Guntermann

Eric

Lenz

Gabriel

. 2022. “Still Not Important Enough? COVID-19 Policy Views and Vote Choice.” Perspectives on Politics 20, 2: 547-561.

22.

Hogarth

Robin M.

Natalia

Karelaia.

. 2006. “‘Take-the-Best’ and Other Simple Strategies: Why and When They Work ‘Well’ with Binary Cues.” Theory and Decision 61, 3: 205-249. https://doi.org/10.1007/s11238-006-9000-8

23.

Kinder

Donald R.

Kalmoe

Nathan P.

. 2017. Neither Liberal nor Conservative: Ideological Innocence in the American Public. Chicago Studies in American Politics. Chicago; London: The University of Chicago Press.

24.

Kuklinski

James H.

Hurley

Norman L.

. 1994. “On Hearing and Interpreting Political Messages: A Cautionary Tale of Citizen Cue-Taking.” The Journal of Politics 56, 3: 729-751. https://doi.org/10.2307/2132190

25.

Kuklinski

James H.

Quirk

Paul J.

. 2000. “Reconsidering the Rational Public: Cognition, Heuristics, and Mass Opinion.” edited by Dennis Chong James H. Kuklinski , eds, 153-182. Cambridge University Press.Elements of Reason: Cognition, Choice, and the Bounds of Rationality.

26.

Lau

Richard R.

Andersen

David J.

Redlawsk

David P.

. 2008. “An Exploration of Correct Voting in Recent U.S. Presidential Elections.” American Journal of Political Science 52, 2: 395-411. https://doi.org/10.1111/j.1540-5907.2008.00319.x

27.

Lau

Richard R.

Redlawsk

David P.

. 2001. “Advantages and Disadvantages of Cognitive Heuristics in Political Decision Making.” American Journal of Political Science 45, 4: 951. https://doi.org/10.2307/2669334

28.

Levendusky

Matthew S

. 2010. “Clearer Cues, More Consistent Voters: A Benefit of Elite Polarization.” Political Behavior 32, 1: 111‐31. https://doi.org/10.1007/s11109-009-9094-0

29.

Lupia

Arthur

. 1994. “Shortcuts Versus Encyclopedias: Information and Voting Behavior in California Insurance Reform Elections.” American Political Science Review 88, 1: 63-76.

30.

Lupia

Arthur

. 2016. Uninformed: Why People Know so Little about Politics and What We Can Do about It. New York: Oxford University Press.

31.

Martignon

Laura

Ulrich

Hoffrage

. 2002. “Fast, Frugal, and Fit: Simple Heuristics for Paired Comparison.” Theory and Decision 52, 1: 29-71. https://doi.org/10.1023/A:1015516217425

32.

Martignon

Laura

Hoffrage

Ulrich

. 1999. “Why Does One-reason Decision Making Work?”, edited by Gigerenzer

Gerd

Peter

, eds, 119-140. New York: Oxford University Press.Simple Heuristics that Make us SmartTodd and ABC Research Group.

33.

McDonald

Jared

. 2020. “Avoiding the Hypothetical: Why ‘Mirror Experiments’ Are an Essential Part of Survey Research.” International Journal of Public Opinion Research 32, 2: 266-283. https://doi.org/10.1093/ijpor/edz027

34.

Mondak

Jeffery J.

McCurley.

Carl

. 1994. “Cognitive Efficiency and the Congressional Vote: The Psychology of Coattail Voting.” Political Research Quarterly 47, 1: 151-175.

35.

Pew Research Center . 2019. Faith on the Hill. https://www.pewforum.org/2019/01/03/faith-on-the-hill-116/

36.

Popkin

Samuel L.,

1994. The Reasoning Voter: Communication and Persuasion in Presidential Campaigns. 2nd ed. Chicago: University of Chicago Press.

37.

Shanks

David R.

Tunney

Richard J.

McCarthy

John D.

. 2002. “A Re-Examination of Probability Matching and Rational Choice.” Journal of Behavioral Decision Making 15, 3: 233-250. https://doi.org/10.1002/bdm.413

38.

Simon

H. A.

1956. “Rational Choice and the Structure of the Environment.” Psychological Review 63, 2: 129-38. https://doi.org/10.1037/h0042769

39.

Sniderman

Paul M.

Brody

Richard A.

Tetlock

Philip E.

, eds 1991. Reasoning and Choice: Explorations in Political Psychology. Cambridge [England]; New York: Cambridge University Press.

40.

Todd

Peter M.

Gigerenzer.

Gerd

. 2012. Ecological Rationality: Intelligence in the World. New York: Oxford University Press.

41.

Vulkan

Nir

. 2000. “An Economist’s Perspective on Probability Matching.” Journal of Economic Surveys 14, 1: 101-118. https://doi.org/10.1111/1467-6419.00106

42.

Wilson

J. Matthew

Paul

Gronke

. 2000. “Concordance and Projection in Citizen Perceptions of Congressional Roll-Call Voting.” Legislative Studies Quarterly 25, 3: 445. https://doi.org/10.2307/440415

43.

Zaller

John R.

1992. The Nature and Origins of Mass Opinion. Cambridge University Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.02 MB

0.00 MB

0.27 MB

Can People Use Party Cues to Assess Policymaker Positions? Ecological Rationality and Political Heuristics

Abstract

Keywords

Introduction

The Ecological Rationality of Political Heuristics

Hypotheses 1

Method

Predictive Validity of Party Cues

Study 1

Sample

Stimulus

Results

Discussion

Study 2

Methods

Results

Discussion

General Discussion

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

Supplemental Material

Notes

References

Supplementary Material

Hypotheses¹