Reconsidering the “Palin Effect” in the 2008 U.S. Presidential Election

Abstract

“The ‘Palin Effect’ in the U.S. 2008 Presidential Election” analyzes the effect of Sarah Palin on presidential vote choice. Two of the substantive conclusions are (1) Palin cost McCain votes among independents and moderates, and (2) Palin had the largest effect on vote choice of any recent vice-presidential nominee. Our analysis shows that the data do not support these findings. We find that respondent evaluations of Palin have a positive effect on McCain vote choice, even among independents and moderates, and Palin’s effect on the election outcome is comparable with ten of the last fifteen vice-presidential nominees.

Keywords

2008 presidential elections vice-presidential nominees elections

Introduction

In a recent issue of this journal, “The ‘Palin Effect’ in the 2008 U.S. Presidential Election” by Jonathan Knuckey (2012) addressed a substantively interesting question: Did the selection of Sarah Palin negatively affect John McCain’s share of the vote? In line with the conventional postelection narrative and other research on the “Palin Effect” (see Elis, Hillygus, and Nie 2010), the article concludes that Palin hurt McCain among key moderate and independent voters.¹ Specifically, the article makes three claims. First, Palin had a measurable, independent effect on the presidential popular vote in 2008. Second, she hurt the McCain campaign by driving away independent and moderate voters. Third, Palin is a uniquely divisive figure and her effect on the presidential vote was larger than any recent vice-presidential nominee.

We see great value in this research question, in terms of understanding the dynamics of the 2008 election and for the broader question of the effect of running mate choice on presidential elections (Holbrook 1991, 1994). However, our reading of the article suggests that the evidence presented does not support two of the conclusions reached. In this research note, we will argue the following:

The finding that Palin hurt McCain among moderate and independent voters is based on a flawed interpretation of the empirical analyses of the 2008 American National Election Studies (ANES) data used in the article. When the results are correctly interpreted using marginal effects in place of the predicted probabilities reported in the article, the data do not support this finding. Furthermore, the results in Table 1 of the original paper show that the coefficient for the Palin feeling thermometer variable is positive, and therefore the appropriate interpretation of the original model is that attitudes toward Palin are positively correlated with McCain vote choice. This relationship does not support the conclusion that Palin hurt McCain among the electorate as a whole or, as we will show, among independents (and moderates) in particular.

The finding that Palin’s effect on the 2008 race was larger than that of any recent vice-presidential nominee cannot be stated with confidence. Estimates of the effects of a variable within a model are inherently uncertain. When 95 percent confidence intervals are included, the results show that Palin’s effect overlaps many other vice-presidential nominees and may not have been the largest in recent history.

A simpler, re-specified model with greater predictive power than the original allows for easier interpretation and shows high evaluations of Palin had a positive effect on McCain vote choice. The model specification in the original paper, where ideology is treated as three separate dichotomous variables with “don’t know/no opinion” as the excluded category, is difficult to interpret. Furthermore, the creation of “Candidate Image” variables using factor analysis is not, we argue, the best way to address the issue of multicollinearity among candidate feeling thermometers. Our re-specified model shows that Palin had a conditionally positive effect on independent voters.

Like many readers, we find the analysis intriguing and topical and we appreciate the opportunity to engage in a dialogue with the author explaining our critique in detail. Due to the importance and visibility of this research, we feel strongly about coming to a clear understanding of the findings and their implications and hope the dialogue furthers our understanding of vice-presidential nominee effects and voter behavior. In the following section, we detail each of our three major arguments, offer an alternative analysis, and discuss how our analysis changes the substantive interpretation of Sarah Palin’s effect on the 2008 election.

Did Palin Cost McCain Independent and Moderate Voters?

We begin with the major finding in the article: Palin hurt McCain specifically by costing him votes among independents. More formally, the author claims that Palin had an independent, statistically significant and negative effect on McCain vote choice among independent and moderate voters. As the author notes, the counterfactual—a hypothetical different running mate—would likely not have altered the outcome of the 2008 election. However, if Palin did in fact cost McCain votes, it suggests that in a closer election, the choice of a running mate could affect the outcome. To this end, the article begins by demonstrating the robust positive effect of Palin evaluations (as measured with a feeling thermometer variable in the ANES) on McCain vote choice. An increase in a voter’s assessment of Palin has a positive and statistically significant effect on McCain vote choice, even controlling for party identification, ideology, assessment of national economic conditions, and a voter’s attitudes toward Obama and McCain. The substantive effect of the logit coefficient reported in Table 1 in the original is equal to a respondent being about 1.03 times more likely to vote for McCain given a one-unit increase in his or her assessment of Palin. The data in Table 1, specifically the positive coefficient on the Palin thermometer variable, undermine the conclusion that Palin cost McCain votes. The author then interacts ideology and party identification (separately) with Palin evaluation to calculate predicted probabilities of McCain vote choice (see Figures 6 and 7 in the original). Because the predicted probabilities for both interactions (Party identification × Palin evaluation and Ideology × Palin evaluation) are lower for moderates and independents, the author concludes that Palin cost McCain voters from these groups by saying, “[g]iven the concentration of moderates in many of the critical swing states in 2008, this ‘Palin Effect’ appears to have been very costly to McCain” (Knuckey 2012, 284).

We have two objections relevant to this finding. First, the interpretation of the results, leading to the primary finding that Palin cost McCain support among independent voters, is not supported by the evidence presented. Second, the model specification is complicated by the measurement of ideology as three separate dichotomous variables and the lack of appropriate controls due to multicollinearity among the closely related “feeling thermometer” variables. We will deal separately with the specification issues in the “Model Specification Issues” section.

Predicted Probabilities versus Marginal Effects

Our primary objection to the claim that Palin cost McCain independent votes is the use of predicted probabilities to demonstrate it. In Figure 6 in the original, Republicans appear to have a higher probability of voting for McCain as feelings toward Palin increase compared with independents and Democrats, while in Figure 7 conservatives appear to have a higher probability of voting for McCain as feelings toward Palin increase compared with moderates and liberals.² Interpreting these differences in the predicted probabilities answers a different question than the article asks. Predicted probabilities show the likelihood of voting for McCain given certain characteristics on party identification (Figure 6) and ideology (Figure 7) but, importantly, not whether independents were less likely to vote for McCain because of Palin. The figures show that independents had a lower probability of voting for McCain than Republicans, but this is substantively different from concluding that Palin cost McCain votes among independents and moderates. Furthermore, for all parties and ideologies and at all values on the Palin feeling thermometer, the predicted probability of voting for McCain is positive (though likely not different from zero at many of the lower feeling thermometer values).

To answer this question—did Palin cost McCain votes among independents?—a more straightforward approach using cross-sectional data is to show the marginal effects, not the predicted probabilities. Testing the hypothesis presented in the paper requires modeling the relationship between two variables conditional on a third variable—the marginal effects. Of interest here is the slope of the relationship between feelings toward Palin and McCain vote choice, conditional on party identification or ideology. In addition, we model the relationship between feelings toward Palin and McCain vote choice for Republicans and conservatives to determine whether there are significant differences between these groups and the groups of interest—independents and moderates. Significant differences would be consistent with the claims of the paper (see Figures 6 and 7). The results presented in our Figure A³ show that Palin had a positive effect on McCain vote choice, and this effect is not conditional on partisanship or ideology. Our results are produced with the same data used in the article, without changes to the model specification or measurement of any variable.

Figure A.

The effect of ideology and party identification on McCain vote conditional on Palin feeling thermometer.

The interaction term is not significant and there is no feeling thermometer rating for Palin that produces a negative and statistically significant slope on McCain vote choice for independents or moderates. In fact, the slope is positive, though not statistically significant for all Palin feeling thermometer values. For Republicans, any rating of Palin results in a statistically significant positive effect on McCain vote choice although there is no increase in effect size as a Republican rates Palin more positively. Excepting independents who are neutral toward Palin (near fifty on the thermometer), the positive effect of Palin rating on vote choice among independents is not statistically different from Republicans. The same is true for ideology. There is never a statistically significant negative effect of feelings toward Palin on McCain vote choice conditional on ideology. As before, there are no statistically significant differences between conservatives and moderates. The substantive interpretation is clear: the positive relationship between McCain vote choice and feelings for Palin is not conditional on party identification or ideology. Not only is there no negative effect for independent voters on feelings toward Palin but there is also no meaningful difference between Republicans and independents on how feelings toward Palin affected McCain vote choice. Our analysis reaches a different conclusion from the original paper; we find that the positive relationship between the Palin feeling thermometer and the likelihood of voting for McCain does not depend on a voter’s ideology or party affiliation. Therefore, the results call into question the major conclusions of the paper; Palin did not have a negative effect on McCain’s vote share overall, nor did she result in “eroded support for McCain among critical ‘swing voters’ such as Independents and moderates” (Knuckey 2012, 286–87).

Magnitude of Palin’s Effect on Vote Choice

A second major conclusion from the paper is that Sarah Palin’s impact on McCain was larger than any vice-presidential nominee has had on his or her running mate dating back to 1980. These results are depicted in Figure 8 in the original (Knuckey 2012, 285). Here our issue with the interpretation of the data is straightforward. Parameters estimated from samples are drawn from a distribution. They do not exactly estimate the population parameter, but instead provide a range of values around which the population parameter will fall a specified number of times given repeated sampling from the same population. As point estimates are inherently uncertain, it is appropriate to present such results with confidence intervals.

In our Figure B, we reproduce Figure 8 from the paper using the author’s original model specification with the addition of standard 95 percent confidence intervals. To determine which vice-presidential effect is largest, the relevant comparison is not to zero but to the effect sizes of all other vice presidents. Taking into account the confidence intervals, the estimated Palin effect overlaps with ten of the fifteen nominees included in the model. While the original model certainly shows that Palin’s effect on vote choice was large and statistically different from zero, we cannot conclude from these data that it was the largest of any recent vice-presidential nominee. Similarly, the claims that, “the only variable that had a stronger effect on vote choice in 2008 was the candidate image variable for Barack Obama” (Knuckey 2012, 285) and that Palin’s effect was greater than McCain’s and “over 1.5 times” larger than that for Biden (Knuckey 2012, 280) are undermined when 95 percent confidence intervals are taken into consideration.⁴ Moreover, the original Figure 8 shows the absolute effect size, not the direction of that effect. The paper, when discussing the size of the “Palin Effect,” concludes that it had a negative impact on McCain vote choice. However, this claim is undermined because Figure 8 in the original depicts the significant, unconditional positive effect on McCain vote choice reported in Table 1.

Figure B.

The effect of vice-presidential evaluations on vote choice with confidence intervals.

It is the nature of these data, not any flaw in the analysis, that make the results presented in the original Table 8 and elsewhere inherently uncertain. The accurate interpretation of point estimates is possible only when some measure of confidence is provided. Confidence intervals are particularly important in this instance because their absence alters the substantive conclusions drawn from the data.

Model Specification Issues

Up to this point, we have used a specification identical to the original paper to produce our results. However, there are three issues with that specification we wish to raise in this section, in which we will offer a re-specified model with more appropriate measures of ideology, partisanship, and respondent attitudes toward McCain and Obama. First, the approach to the issue of multicollinearity among candidate feeling thermometer variables is unnecessarily complex. Second, the use of three separate dichotomous variables to measure ideology where “other” is the excluded category is theoretically inappropriate and makes a meaningful interpretation of the ideology variables more difficult. Finally, the original specification of partisanship combines independent identifiers with other identifiers including “don’t know” and nonresponses.

Specification of Candidate Feeling Thermometers

The original article correctly notes that evaluations of the four candidates—Obama, Biden, McCain, and Palin—are closely related, “making multicollinearity a problem” (Knuckey 2012, 279). That including multiple feeling thermometers would create multicollinearity is expected and logical. To this end, the model in the original uses two “Candidate Image” variables one each for Obama and McCain—to deal with the fact that assessment of effects from either Biden or Palin is difficult to parse from respondents’ feelings toward the presidential candidates. The results from the model including these Candidate Image variables show that Palin has a large, significant effect on vote choice. We believe that this approach is unnecessarily complex and that a simpler alternative is more appropriate.

To demonstrate the independent effect of a variable, the theoretically appropriate approach is to include all relevant control variables regardless of their level of multicollinearity. In fact, the high correlation of two variables is a good reason to include both rather than exclude one. As Gujarati and Porter (2009, 326–44) note, multicollinearity does not bias coefficient estimates whereas dropping a variable that should be included results in specification error and biased coefficients. A simpler approach is to include the McCain, Palin, and Obama feeling thermometers where McCain vote choice (1 = McCain, 0 = other) is the dependent variable. While evaluations of these candidates are correlated, it is theoretically appropriate to include all three and, as our results that follow will show, the predictive power of the model is improved.

Specification of Respondent Ideology

The article states that ideology is measured with, “three dummy variables, liberals (1=liberal, 0=otherwise), moderates (1=moderate, 0=otherwise), and conservatives (1=conservative, 0=otherwise). Those who responded ‘don’t know’ or ‘haven’t thought’ are the excluded category” (Knuckey 2012, 278–79). The author justifies operationalizing ideology in this manner because ideology, like party identification, is “neither linear nor bounded at 7” (Knuckey 2012, 278). In addition, this allows the analysis to incorporate “don’t know” and “haven’t thought” respondents rather than dropping them, as would be necessary with an ordinal measure of ideology.

We are sympathetic to these arguments, but while ideology may theoretically be conceptualized as continuous, it is nearly impossible to measure it as such. In practice, ideology is almost always treated as an ordinal variable with a continuous specification (e.g., Ansolabehere and Jones 2010; Markus 1988; Wattenberg 1995). If there are strong theoretical reasons not to treat a variable as continuous, the appropriate specification would be to include a separate dummy variable for each value of the ordinal variable (excluding one as a reference category). In this specification, each dummy is allowed to predict the dependent variable in the form of a change in the intercept. The costs to including numerous dummy variables rather than one continuous variable are that their inclusion, though it ensures the coefficients are unbiased, reduces the degrees of freedom and increases the inefficiency of the model, and makes interpretation more difficult. Rather than specifying seven dummy variables or treating the variable as continuous, the practical effect of using a set of collapsed dummy variables to measure ideology is that it reduces variation within the variable and increases shared variation among the dummy variables, likely resulting in less shared variation between the Palin feeling thermometer variable and the ideology variables. Simply put, ideology is not explaining as much variation as it could if it were measured on a seven-point scale, either through the inclusion of six dummy variables or through one continuous variable. As a consequence, its explanatory power in the model is likely reduced.

The second problem with the original specification is that dummy variables are mutually exclusive categories, and therefore the dummy categories included in a model are interpreted with respect to the excluded (baseline) category. Consider a simple regression model with three mutually exclusive categories, such that D₁ + D₂ + D₃ = 1 for any observation i. If y = α + β₁ × D₁ + β₂ × D₂ + e, then the intercept α = β₃ when D₁ = 0 and D₂ = 0, or the value of y when D₃ = 1. This example is generalizable to any situation where y = α + β₁ × D₁ + β₂ × D₂ + β_i × X_i + e where X is a vector of covariates. In the context of this article, the effects of the dichotomous ideology variables—liberal, conservative, and moderate—are interpreted with respect to an excluded category (“don’t know” or “haven’t thought”) that has no substantive interpretation relative to ideology. The “don’t know” and “haven’t thought” categories are not ideologies but rather an articulated nonresponse. Even if a different category is excluded, the coefficients are interpreted as the independent effect controlling for respondents who could not articulate an ideology. Therefore, regardless of which of the dummy variables is treated as the excluded category, the inclusion of the “don’t know” and “haven’t thought” makes a meaningful interpretation of the coefficients relative to the other categories much more difficult. While we agree that dropping respondents who indicate no ideological self-identification is far from ideal, it is the theoretically appropriate approach to measuring the attitudes of respondents who can identify their ideological inclination.

The specification of party identification raises a similar issue. By coding identifiers for Republicans (1 = Republican, 0 = otherwise) and Democrats (1 = Democrat, 0 = otherwise), the excluded baseline category is a mixture of independents, nonresponses, and “don’t know” responses. As independent and moderate respondents are key to the research question, it is important to define this category as accurately as possible. We re-specify party identification by using the seven-point scale in the ANES to create three dummy variables: Republican (1 = yes, 0 = no), Democrat (1 = yes, 0 = no), and independent (1 = yes, 0 = no). Republicans are excluded in the logistic regression, which allows the most readily interpretable comparison of the effects on independents relative to the Republican baseline.

We reestimate the model used in Table 1 using the standard seven-point ideological scale, which avoids the necessity of using three dummy variables for ideology. We also specify partisanship with three dummy variables (Democrat, Republican, and independent), and replace the “Candidate Image” variables with the candidate feeling thermometers for McCain and Obama. The rest of the variables are specified as in the original. Our results appear in Table A.

Table A.

The Effect of Party ID on McCain Vote Conditional on Palin Feeling Thermometer.

Variable	Coefficient
Palin feeling thermometer rating	2.06* (0.878)
Palin thermometer × Independent party identification	4.88* (2.09)
Independent party identification	−3.53* (1.44)
Democratic party identification	−2.45* (0.487)
National economic conditions	1.47 (0.872)
Seven-point ideological scale (7 = conservative)	0.122 (0.149)
McCain feeling thermometer	0.059* (0.013)
Obama feeling thermometer	−0.094* (0.011)
Constant	−0.073* (1.33)
Pseudo-R²	.795
N	1,031

p < .05.

Importantly, evaluations of Palin remain a statistically significant and positive predictor of McCain vote choice. The interaction between Palin and the independent dummy variable is also significant but, contrary to the original findings, it is positive. The dummy variable can be interpreted relative to the excluded baseline category (Republican). The positive coefficient of the interaction term demonstrates that, relative to Republicans, positive feelings about Palin increase the likelihood of McCain vote choice. In Figure C, we show that this finding holds for independents but not for Republicans. The more an independent respondent liked Palin, the more his or her probability of voting for McCain resembled that of a Republican. For Republicans, evaluations of Palin were not a significant predictor of vote choice; they were always more likely to vote for McCain. These results are consistent with expectations and are not unique to Palin—positive evaluations of a vice-presidential nominee by those voters not firmly in one party’s camp increase the chances that voter will prefer that party’s presidential candidate.

Figure C.

The effect of party identification on McCain vote conditional on Palin feeling thermometer using re-specified model.

Discussion

The original paper represents the value of political science in untangling real-world political questions and the obstacles inherent in bringing the knowledge of our field to bear on these questions. While novel findings, especially ones that conform to the dominant narrative, are exciting and satisfying, conventional wisdom can be misleading or incorrect. Social science is useful and meaningful to the extent that it applies analytic rigor to better inform our understanding of the world, regardless of popular preconceptions. With respect to Sarah Palin, our analysis of the ANES data used in the original paper produces results that are more noteworthy for what they do not show—evidence of a negative “Palin Effect” on McCain’s vote share—than for what they do show. Our reanalysis shows that Sarah Palin did not have a unique or unprecedented influence on the race; at best, she had precisely the small effect on vote choice in 2008 that we would expect of any running mate—an important conclusion precisely because the null result still informs our understanding of presidential elections in general and 2008 in particular. Although this finding is important and useful to our understanding of presidential elections, it lacks the appeal of a finding that supports the dominant postelection narrative. Unfortunately, because null findings are difficult to publish in political science, there is a bias toward reporting only significant results (Gerber et al. 2010; Gerber, Green, and Nickerson 2001; Gerber and Malhotra 2008). Particularly in the literatures on elections, we feel that more openness to null findings will add value to the knowledge our field can contribute to practical, applied questions.

Conclusion

Sarah Palin was a highly visible and polarizing figure in the 2008 presidential election. She generated media attention and attracted praise and criticism beyond what is usually given to vice-presidential nominees. It is logical to assume, as popular postelection wisdom did, that her impact on the outcome of the election was also greater than previous running mates. “The Palin Effect in the 2008 Presidential Election” uses survey data to support that conclusion. Our reading of the article respectfully argues that the data do not support the key findings, which are:

That there is a negative conditional effect of feelings toward Palin on likelihood of a McCain vote among independents and moderates. We find that using marginal effects, as is appropriate for cross-sectional data, shows that Palin had a positive effect on McCain vote choice, and based on our model specification, may have had a positive, conditional relationship for independent voters.

That Palin’s impact on vote choice was the largest among all recent vice-presidential candidates. We find that when confidence intervals are included, Palin’s effect was not necessarily the largest among the nominees since 1972.

As this article is important to the literature on the 2008 election and the effects of vice-presidential selection on vote choice, we consider it crucial to understand exactly what the available data tell us about these phenomena. To that end, the original article makes a valuable contribution. Our goal with this response is to build upon it.

Footnotes

Acknowledgements

We thank Jonathan Knuckey for graciously providing data and engaging in this dialogue with us. We also thank John Hulsey for reading an earlier version of the manuscript, and the editors and two anonymous reviewers for their thoughtful suggestions.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Ansolabehere

Stephen

Jones

Edward

. 2010. “Constituents’ Responses to Congressional Roll-Call Voting.” American Journal of Political Science 54 (3): 583–97.

Elis

Roy

Hillygus

D. Sunshine

Nie

Norman

. 2010. “The Dynamics of Candidate Evaluations and Vote Choice in 2008: Looking to the Past or Future?” Electoral Studies 29 (4): 582–93.

Fiorina

Morris P.

1981. Retrospective Voting in American National Elections. New Haven: Yale University Press.

Gerber

Alan S.

Green

Donald P.

Nickerson

David

. 2001. “Testing for Publication Bias in Political Science.” Political Analysis 9 (4): 385–92.

Gerber

Alan S.

Malhotra

Neil

. 2008. “Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals.” Quarterly Journal of Political Science 3:313–26.

Gerber

Alan S.

Neil

Malhotra

Dowling

Conor M.

Doherty

David

. 2010. “Publication Bias in Two Political Behavior Literatures.” American Politics Research 38 (4): 591–613.

Gujarati

Damodar N.

Porter

Dawn C.

2009. Basic Econometrics. New York: McGraw-Hill.

Holbrooke

Thomas M.

1991. “Presidential elections in space and time.” American Journal of Political Science 35(1): 91-109.

Holbrooke

Thomas M.

1994. “The Behavioral Consequences of Vice-Presidential Debates: Does the Undercard Have Any Punch?“ American Politics Research 22(4): 469-482.

10.

Key

V. O.

Jr.

1966. The Responsible Electorate: Rationality in Presidential Voting 1936-1960. Cambridge, MA: Harvard University Press.

11.

Knuckey

Jonathan

. 2012. “The ‘Palin Effect’ in the U.S. 2008 Presidential Election.” Political Research Quarterly 65 (2): 275–89.

12.

MacKuen

Michael B.

Erikson

Robert S.

Stimson

James A.

1992. “Peasants or Bankers: The American Electorate and the U.S. Economy.” American Political Science Review 86 (3): 597–611.

13.

Markus

Gregory B.

1988. “The Impact of Personal and National Economic Conditions on the Presidential Vote: A Pooled Cross-Sectional Analysis.” American Journal of Political Science 32 (1): 137–54.

14.

Wattenberg

Martin P.

1995. “The Role of Vice Presidential Candidate Ratings in Presidential Voting Behavior.” American Politics Research 23 (4): 504–14.