More Equal than We Thought? Using Vote Validation to Better Understand Participation Inequality in the States

Abstract

Inequality in political participation is a well-known and often studied feature of American politics. An important study examining misreporting of voter turnout in opinion surveys, however, calls into question the true extent of participation inequality. Ansolabehere and Hersh’s first-ever 50-state vote validation project shows that those with more political resources are more likely to misreport turnout. That is, those with higher incomes are more likely than others to report that they had voted when in fact they did not. These findings suggest that income disparities in participation are not as large as opinion surveys have led us to believe. Moreover, studies using differences between voters and nonvoters as a key political indicator may also be biased. This article presents the first assessment of whether vote misreporting creates systematic bias in measures of state participation inequality. An index of economic inequality in participation for the 50 states is developed using the Ansolabehere and Hersh validated vote data and compares the measure of political inequality with a similar measure using traditional (i.e., nonvalidated) survey data. These state indices are used to determine the extent of bias produced by misreporting and whether this bias has implications for studies using these measures of participation inequality. The latter is assessed by examining the influence of the self-reported and validated inequality measures on state welfare programs and minimum wage policy.

Keywords

voting behavior political behavior political participation poverty/inequality public policy federal/state federalism

Introduction

As many observers have pointed out, political participants in the United States are particularly unrepresentative of the population as a whole. One well-known difference between the politically active and those who are less involved in the world of politics is that active citizens tend to be those on the upper end of the socioeconomic scale (Piven and Cloward 1988; Rosenstone and Hansen 1993; Verba, Schlozman, and Brady 1995; Wolfinger and Rosenstone 1980). Recognizing this bias in participation is especially important when considering the potential relationship between participation and political influence. Participation allows the public to reveal preferences and interests, with the goal of influencing the decision making that takes place throughout the policy process. This particular effect of participation has been a common theme for a number of democratic theorists. Schattschneider (1960), for instance, viewed low rates of political participation as one of the primary failures of American politics because he believed it would produce a bias between the general population and those who are represented. If elected officials are more responsive to particular groups compared with others, unequal political participation may be an important factor in understanding differences in who government represents. As cited in Lijphart’s (1997, 4) study of the consequences of unequal participation, “the old saw remains profoundly true: if you don’t vote, you don’t count” (Burnham 1987, 99).

Several studies examining the influence of unequal participation, particularly voter turnout, on policy outcomes provide evidence of the consequences of bias in political participation (Avery and Peffley 2005; Fellowes and Rowe 2004; Hill and Leighley 1992; Hill, Leighley, and Hinton-Andersson 1995). These scholars argue that greater participation of the wealthy, relative to the poor, leads to redistributive policies that favor the rich at the expense of low-income groups. They demonstrate that states with disproportionately low turnout of disadvantaged individuals produce less generous welfare policies and more restrictive welfare eligibility rules. These studies suggest inequalities in political voice, or disparities in participation of the rich and poor, lead to policy outcomes that are unfavorable to the disadvantaged. The poor, the group with less political voice, is more likely to benefit from state welfare policies but are unable to influence lawmakers to adopt more redistributive policies due to their disproportionate level of political power.

These studies of unequal participation (as well as other analyses using similar measures of bias in voter turnout), however, may have overestimated the degree of participation inequality in the United States. The problem stems from the use of public opinion surveys to measure voter participation. Opinion surveys universally overestimate the proportion of the population that actually turns out to vote, with the extent of this overestimation dependent on factors such as sampling techniques, mode of the survey, and question design. Why surveys routinely overestimate voter turnout is still debated, but an important study examining misreporting of voter turnout in opinion surveys brings us closer to an answer while also calling into question the true extent of income inequalities in voter turnout.

Ansolabehere and Hersh’s (2012) first-ever 50-state vote validation project shows that those with more political resources are more likely to misreport turnout. That is, those with higher incomes are more likely than others to report that they had voted when in fact they did not. These findings suggest that income disparities in participation are not as large as opinion surveys have led us to believe. Moreover, studies using economic differences between voters and nonvoters as a key political indicator (including the research cited above) may also be biased. As Ansolabehere and Hersh (2012, 441) argue, “When one is studying the differences between voters and nonvoters, misreporters do indeed bias results.” This critique suggests research examining measures of participation inequality might be flawed and, if true, the use of these measures may produce misleading conclusions.

While Ansolabehere and Hersh’s (2012) study shows higher socioeconomic status is one indicator of vote misreporting, much more can be done to better understand how misreporting affects measures of participation inequality. This is particularly true for scholars interested in unequal participation at the state level because Ansolabehere and Hersh mainly focus on the characteristics of individuals throughout the United States who misreport their turnout status. That is, it is still unclear whether vote misreporting creates a systematic bias across the American states when measuring levels of income inequality in participation. Uncovering any potential bias in measures of state participation inequality is vital for scholars examining the consequences of unequal participation largely because most of this research focuses on state politics.

This article presents the first assessment of whether vote misreporting creates systematic bias in measures of state participation inequality. The results of this study have important implications for both past and future research that focuses on economic biases in political participation, and will show whether measures of inequality using self-reported turnout data are accurately quantifying the concept of class bias in participation. To do this, I develop an index of economic inequalities in participation for each state using the Ansolabehere and Hersh validated vote data and compare the measure of political inequality with an identical measure using traditional self-reported (i.e., nonvalidated) survey data. These state indices are used to determine the extent of bias produced by misreporting and whether this bias has implications for studies using these measures of participation inequality. To assess the implications of any differences between self-reported and validated measures of participation inequality in an applied research context, two state-level policies—welfare programs and minimum wage—are analyzed using both inequality measures. The influence of unequal turnout on these two policies is examined to know whether the self-reported and validated measures of participation inequality lead to different substantive conclusions.

The next section of this article provides a brief overview of vote validation literature and gives readers a general sense of Ansolabehere and Hersh’s (2012) methodology and findings. Then, the concept of participation inequality is discussed, which includes some background on previous measures of class bias in participation and how the current study approaches the measurement of inequality. Finally, the self-reported and validated measures of participation inequality are examined and the analyses of state welfare and minimum wage are presented.

The 2008 Vote Validation Study

Vote validation studies have long been used to assess the accuracy of public opinion data, as well as to test established models of voting behavior. The objective of early validation studies was to understand why national surveys of voter turnout consistently overestimated participation rates when compared with results from official records. Initially, a common explanation for the inflated estimates of turnout was inadequate coverage of the population (e.g., those more likely to vote were also more likely to respond to surveys), but validation studies suggested vote misreporting was a legitimate issue (see Traugott and Katosh 1979). Matching its biannual political participation survey responses with official voting data, the American National Election Study (NES) produced a number of validated surveys beginning in the 1960s. The NES data have allowed scholars to examine the characteristics of those individuals who misreport, understand why vote overreporting occurs, and assess whether misreporting leads to biased inferences when studying political participation (Anderson and Silver 1986; Bernstein, Chadha, and Montjoy 2001; Hill and Hurley 1984; Katosh and Traugott 1981; Presser and Traugott 1992; Sigelman 1982; Silver, Anderson, and Abramson 1986; Traugott and Katosh 1979).

Broadly speaking, these validation studies have found both demographic characteristics and political attitudes are related to overreporting.¹ For instance, younger individuals (Hill and Hurley 1984; Traugott and Katosh 1979), those with higher levels of political interest (Hill and Hurley 1984; Sigelman 1982), and people with strong partisan ties (Silver, Anderson, and Abramson 1986) are all more likely to misreport. Particularly important for the study of economic inequalities in participation, research examining validated turnout suggests individuals who overreport generally have higher incomes and greater socioeconomic status (Anderson and Silver 1986; Hill and Hurley 1984; Silver, Anderson, and Abramson 1986).² As discussed above, if income is related to overreporting, measures of participation inequality based on income may potentially be biased.

While the NES validation studies have allowed researchers to gain a better understanding of the causes and consequences of vote misreporting, these data make it difficult to assess whether the correlates of overreporting are similar across the states. One reason the NES studies are not ideal for examining participation bias in the states is that the surveys use a cluster sample design intended only to be representative of the national population, meaning the respondents from subnational geographic areas will not be representative of the population living in those areas. In other words, the sample design used by the NES does not allow researchers to make inferences about political behavior at the state level. Second, even if the NES could be used to accurately estimate state-level opinion, the studies were only able to validate turnout for a small number of states. Fortunately, Ansolabehere and Hersh’s (2012) recent validation study does not suffer from either of these limitations. This makes it possible to examine overreporting in the states and, for the first time, determine whether measures of participation inequality based on self-reported turnout data are biased.

In collaboration with a private data vendor, Ansolabehere and Hersh (AH) use new technology and matching techniques to allow for a more robust and extensive validation procedure. The authors were able to link survey responses to the 2008 Cooperative Congressional Election Study (CCES)—a nationally representative, Internet-based public opinion survey of U.S. adults that regularly consists of more than 30,000 respondents—to government voting records in every state. Compared with the prominent NES studies, the AH validation procedure produced a far more accurate and complete picture of vote overreporting.³ Regarding the current study, the CCES sample design and considerable sample size of the survey makes it possible to examine self-reported and validated turnout at the state level.

By combining the 2008 survey data with official voting records, AH are able to analyze the differences between true voters and nonvoters, and better understand the factors associated with those that misreport voting—that is, those CCES respondents who report they voted in the 2008 election but did not vote according to voting records. Similar to the previous validation studies discussed above, one of the main findings from the AH data is that individuals who misreport look very similar to true (validated) voters. Those with higher levels of education, strong partisans, and those with high levels of political interest are more likely to misreport than others. This result is potentially problematic for studies of participation inequality, especially if overreporters also tend to have higher incomes. Discussed in more detail in the next section, research suggests that inequalities in political participation can lead to political outcomes that favor those who vote. As those who vote are wealthier than nonparticipants, public policy will be more representative of the priorities and preferences of the rich at the expense of the poor. But if the true economic differences between voters and nonvoters are exaggerated due to vote misreporting, the findings from income-based studies of participation inequality may be misleading or altogether incorrect.

Even with the likelihood that researchers have been overestimating the extent of the economic differences between voters and nonvoters, whether studies using participation inequality as a central concept are biased is an empirical question yet to be answered. In fact, it is possible this research has produced results that are substantively similar to results that would be produced with measures of participation inequality based on validated survey data. Several factors related to vote misreporting and the measurement of participation inequality are important to explore when determining the extent to which overreporting bias has affected studies of unequal participation.

First, the overall magnitude of misreporting based on an individual’s income will certainly have consequences for measures of participation inequality. At one extreme, income-based participation inequality may not exist at all if the income levels of true voters and nonvoters do not actually differ. In this case, studies of participation inequality would obviously be flawed. If, however, the influence of income on voter turnout is only slightly smaller when considering true turnout, then measures of participation inequality will exhibit very little bias. Fortunately for scholars studying participation inequality, the AH study demonstrates a relatively small relationship between income and vote misreporting (e.g., see Figure 3 in the AH study). While higher income respondents are more likely to misreport having voted, the degree of the relationship is small compared with characteristics such as education, political interest, and party identification.

The second factor that should be examined when analyzing differences between self-reported and validated measures of participation inequality is whether misreporting varies across the American states. Studying overreporting at the state level is important as most research with a focus on turnout inequality uses the states as the unit of analysis (Avery and Peffley 2005; Berinsky 2005; Fellowes and Rowe 2004; Franko 2013; Hill and Leighley 1992; 1994; Hill, Leighley, and Hinton-Andersson 1995; Jackson, Brown, and Wright 1998; Rigby and Springer 2011; Wichowsky 2012). Regardless of the overall magnitude of overreporting by the wealthy, measures of participation inequality will be problematic if the wealthy are more likely to overreport in some states compared with others. Alternatively, if income-based misreporting is equal (or very similar) in all states, differences in participation inequality across the states will not change when using validated vote data.⁴ While AH find that rates of vote misreporting do not systematically differ across the states on average, this does not necessarily rule out the possibility that income-based overreporting differs across the states.

Finally, the last factor that should be considered when assessing self-reported and validated measures of participation inequality is whether the two measures produce different substantive results when using them in applied research. This will help answer the question of whether vote overreporting leads to biased results when using measures of income inequality in turnout. The first two factors discussed above will certainly play a role in answering this question, but directly examining any practical substantive differences between self-reported and validated measures will provide researchers with some indication of what overreporting means for the study of participation inequality. The remainder of this article focuses on the measurement of participation inequality and examining any differences between measures of participation inequality based on self-reported and validated data.

Participation Inequality: Concepts and Measurement

Those who study the relationship between money and politics have argued that activities such as contributing to campaigns (Green 2004; Stern 1988) and lobbying (Richter, Samphantharak, and Timmons 2009) can buy political influence. While having the available resources necessary to use personal finances in an attempt to shape political decision making is certainly important, this is only one way that greater wealth produces inequalities in political influence. Making a campaign contribution or being part of an organization are specific types of political participation, and wealth is associated with a greater likelihood of taking part in a variety of political activities. It is also probable that the combination of both wealth and greater levels of political participation leads to inequalities in political influence (see, for example, Dahl 2006; Rosenstone and Hansen 1993; Schattschneider 1960; Wolfinger and Rosenstone 1980). Voting, contacting elected officials, and signing petitions, for example, are all approaches used to attempt to influence politics and are all done at higher rates by those who are economically advantaged (Verba, Schlozman, and Brady 1995).

The premise that disparities in political resources and participation give more influence and power over government than others has led to a number of studies that attempt to uncover the consequences of these inequalities. One segment of this research uses the well-known economic inequalities in participation as a way to capture variations in political influence. The underlying expectation of these studies is that participation inequality affects how elected officials govern as well as public policy outcomes. The strongest evidence linking unequal participation to disparities in policy outcomes comes from the welfare politics literature. For instance, Hill and Leighley (1992) find that American states with disproportionately low levels of turnout among poor individuals have lower levels of spending on welfare than those states less in participation bias. In a rigorous study of various components of state Temporary Aid to Needy Families programs, Fellowes and Rowe (2004) show that greater disparity in the turnout rates of the rich and poor leads to more restrictive state welfare policies in terms of eligibility and flexibility of the programs (also see Avery and Peffley 2005; Hill, Leighley, and Hinton-Andersson 1995).

Although much of the research connecting participation inequality to policy outcomes focuses on welfare policy, the influence of unequal participation is most likely not limited to this particular policy area. A variety of policy areas have disproportionate influences on the poor and rich, all of which may be shaped economic differences in participation. If future research is to shed light on the broader effects of unequal participation on the political process, it is essential that this particular concept of inequality is measured with as much precision as possible.

To properly analyze the differences between measures of unequal participation based on self-reported and validated vote data, it is important to use a robust measure of inequality that captures the relationship between economic resources and participation within the states. A number of scholars have developed measures of inequalities in political participation by examining levels of voter turnout by categories of income. While the term participation inequality is mainly used in this study, others have also referred to this concept and its related measures as indicators of electoral inequality, class bias, participation bias, or political inequality (Avery and Peffley 2005; Hill and Leighley 1992; 1994; Leighley and Nagler 1992; Rigby and Springer 2011; Rosenstone and Hansen 1993; Wolfinger and Rosenstone 1980).

The approach to measuring the concept of economic-based political inequalities has typically been consistent across these studies with the exception some subtle differences. In general, this research has measured participation bias by comparing the voter turnout rates of the rich and poor. In its most basic form, this measure is essentially a ratio of a ratio. That is, state inequality is typically measured by calculating the ratio of the turnout rate of the most affluent to the turnout rate of the poor for each state. This measure can then be used to assess the degree to which these two economic groups differ in their rates of political participation. When the turnout rate of the rich is divided by the rate of the poor, larger values indicate a greater bias in favor of the wealthy and values close to one suggest high- and low-income groups are equally active.

The measure of political inequality used in this study is in many ways similar to those used in prior research addressing participation bias. Participation inequality is evaluated by developing an absolute measure of turnout inequality for each state that builds on regression-based techniques recently used to assess economic biases in participation (see Blakely, Kennedy, and Kawachi 2001; Franko 2013; Mackenbach and Kunst 1997; Wichowsky 2012). The regression-based approach to measuring inequality provides a more robust measure of income-based biases in turnout when compared with the turnout ratio measure that has typically been used in studies of participation inequality (the advantages of this approach are discussed below in more detail). As the main goal of this study is to examine any differences in state measures of participation inequality due to voter overreporting, two measures of inequality are created for all states. The first uses traditional (nonvalidated), self-reported turnout data and the second uses the AH validated vote data. Both turnout variables used to create the measures of participation inequality—the self-reported vote and validated vote variables—come from the same 2008 CCES sample (Ansolabehere 2008), which is ideal for comparing the two measures.

Each participation inequality measure is created by first assigning all CCES respondents to a cumulative proportion distribution based on their family income when compared with the family incomes of all other respondents in their state, which ranges from a minimum of 0 to a maximum of 1. For instance, consider an individual that reports a family income of US$50,000 to $US59,999. If this income category consists of 10% of the state’s residents and 50% of the population falls below this income level, then this individual is assigned a midpoint of 0.55 on the state’s cumulative income distribution (0.50 + [0.10/2]). This adjustment allows for a comparable measure of income across states without eliminating any of the income categories used by the CCES. In other words, all available income information provided by the respondents is used without having to collapse the data into broader income categories (e.g., quintiles, quartiles, etc.) and without assuming the overall distribution of income in each state is the same.

This cumulative income scale is then used as a determinant of voter turnout in the following multilevel regression models:

S e l f - R e p o r t e d V o t e_{i j} = γ_{00} + γ_{10} I n c o m e_{i j} + u_{0 j} + u_{1 j} I n c o m e_{i j} + e_{i j},

V a l i d a t e d V o t e_{i j} = γ_{00} + γ_{10} I n c o m e_{i j} + u_{0 j} + u_{1 j} I n c o m e_{i j} + e_{i j},

where i indexes each respondent and j is an index for each state. Self-Reported Vote is coded 1 if the individual reported voting and 0 if the person did not vote, Validated Vote is coded 1 if the individual was verified as having voted and 0 if the person did not vote, and Income indicates each individual’s position on his or her state’s cumulative income distribution. The γ terms can be thought of as the fixed portion of the model while the u terms are the random components. By combining the fixed and random portions of the model, a unique value for the effect of income on voter turnout is obtained for each state. As both dependent variables (Self-Reported Vote and Validated Vote) and the Income variable are bounded between 0 and 1, the estimated effect for income is interpreted as the absolute difference in the probability of voting for the poorest and richest income group in each state.⁵

Hence, the participation bias measure ranges from −1 to 1, with negative values indicating low-income individuals are more likely to vote than the rich, 0 meaning the rich and poor have an equal probability of voting, and positive values meaning the rich vote at a higher rate than the poor. A value of 0.30, for example, indicates the rich are 30 percentage points more likely to vote than the poor. Like previous studies, the variable measures the disproportionate participation rates of individuals from upper and lower status groups, which assesses the degree to which income levels influence voter turnout in each state.

The participation inequality measure used here, however, improves upon previous analyses of class bias, particularly those using the ratio measure, in several ways. First, as mentioned above, unequal participation in each state is quantified using individuals from all income groups in the calculation. This is done to create a more robust measure of inequality that considers the participation rates of the entire distribution of income in a state, not just the top and bottom portions of the distribution. This becomes relevant, for instance, if middle-class citizens vote at different rates across the states, which is accounted for in the regression-based approach but not in the turnout ratio measure. In addition to including respondents from all income groups, this regression-based approach does not require the researcher to collapse individuals into broad income categories (e.g., rich and poor), which is necessary when calculating participation inequality using the turnout ratio measure.⁶ Instead, all income information available in the survey is used to assess the degree of income bias in participation.⁷

Second, the absolute measure of inequality is based on an income distribution that is relative to the overall wealth of each state.⁸ Many of the studies cited above use a single income threshold to determine which group is wealthy and which is poor. For instance, previous measures of inequality consider all individuals with family incomes below 150% of the federal poverty level as poor and those falling within the 80th percentile of income are considered rich. This strategy, however, ignores the fact that earning US$30,000 in Oklahoma is quite different than an income of US$30,000 in Connecticut. This variation in the cost of living from state to state is accounted for in the measure of participation inequality used here. This is accomplished by using the observed income distribution for each state to calculate economic biases in participation.⁹

Finally, a multilevel modeling approach (see equations 1 and 2) is used to estimate the extent of participation inequality in each state. The advantage of using multilevel estimation is that bias in participation can be modeled simultaneously for each state using all available information (i.e., all respondents in the data set), rather than estimating a measure of inequality separately for each state as previous studies using regression-based measures have done. This is particularly helpful when the survey being used has a relatively small number of respondents in some states, which is the case for the 2008 CCES.¹⁰ The “partial pooling” feature of multilevel regression allows one to obtain robust estimates for a geographic area even when the sample size is small (Gelman and Hill 2007).¹¹ The term partial pooling is used to describe the estimation technique as it does not separately model opinion for each geographic area, and the technique also does not completely pool all of the information (or responses) in the sense that it can account for geographic variation in opinion by way of the random state components included in the multilevel model. The result of this partial pooling strategy is that information from all respondents is used to varying degrees depending on the sample size within each state. For states where the sample size is quite large, the estimates of participation inequality will rely very little on information from respondents in other states. In smaller states where very few individuals are sampled, however, estimates of inequality will rely more on information from the entire sample.¹²

Overall, the strategy used here provides a general, efficient procedure for measuring participation inequality. The measure uses all available information from the survey being used to assess the degree to which income is associated with political participation rather than only focusing on limited segments of the population. In addition, this approach can easily be extended to measure participation bias at the local level (e.g., counties or cities) where small sample sizes are often problematic.

Self-Reported Versus Validated Participation Inequality

Before examining differences in the measures of participation inequality due to vote overreporting, Figure 1 presents the overall state turnout rates based on self-reported vote and validated vote. As the measures of participation inequality being analyzed are created at the state level, it is important that the states do not systematically overreport, which could potentially lead to biases in measures of income-based turnout inequality. As suggested in the previous section, this basic overview of voter turnout demonstrates that most states generally do not overreport more than other states. While it is clear that turnout rates based on validated votes is lower than the measure derived from self-reported votes, states with higher (lower) self-reported turnout rates also tend to have higher (lower) validated turnout rates.

Figure 1.

State self-reported and validated turnout rates (weighted).

An additional preliminary examination of the self-reported and validated vote data is shown in Figure 2. This summary figure illustrates the U.S. turnout rate for the self-reported and validated vote by income quartile for the entire CCES sample. As discussed in the last section, the magnitude of overreporting bias by income will have implications for state measures of participation inequality. Again, it is evident that the validated turnout rate is lower than the self-reported rate for all levels of income, with an average difference in rates of 21%. More importantly for the current study, the differences in turnout rates for the two measures (illustrated by the black bars in the plot) are not substantially different across levels of income. While the top-income quartile has a slightly larger difference between the self-reported and validated rates than the lower income groups—suggesting higher income individuals are slightly more likely to misreport—the difference is larger by only 4%. Consistent with the findings of AH, Figure 2 indicates a relatively small bias in misreporting by upper income individuals.

Figure 2.

U.S. self-reported and validated turnout rates by income quartile (weighted).

An initial examination of the measures of state-reported participation inequality and validated inequality provides evidence that bias in the inequality measures due to vote overreporting is limited. The average self-reported inequality score is 0.24 while the average validated score is 0.21. Similar to the general findings discussed above, some bias does exist between the two measures but the difference appears to be small. As an average of all state scores may conceal larger biases in the inequality scores for some states, analyzing differences in participation inequality across the states will provide a more nuanced understanding of the measures.

Figure 3 plots both inequality scores with a fitted regression line to demonstrate the relationship between the two measures. The Pearson’s correlation coefficient for the two inequality scores is moderately strong at .54, again suggesting the differences between the self-reported and validated vote measures are generally small. Although this comparison provides little indication of any substantial bias between the participation inequality measures based on self-reported and validated turnout, some states do have larger inequality score differences than others.

Figure 3.

State self-reported and validated participation inequality scores.

To examine the extent of the dissimilarity between the two participation inequality measures, Figure 4 shows the difference between the self-reported and validated scores. The plot highlights two important points about the comparison of the inequality measures. First, while some states do have relatively large differences between the self-reported and validated scores (e.g., WV, SC, AL, TN, and GA), the mean of all state differences is only 0.04. This suggests that on average we can expect a 4% difference in the extent of participation inequality with measures based on self-reported and validated data. This difference is certainly not something to ignore, but it is unlikely that this error will lead to invalid findings when the measures are used in practice. The second point to consider is that state measures of participation inequality based on self-reported turnout are not all biased upward when compared with the validated turnout measures. In a number of states, those at the bottom of the plot with negative differences, validated inequality scores are actually higher than the self-reported scores. It should be noted that while the difference between the two measures is quite small for most of the states with greater validated inequality scores, some states have differences hovering around 2% and 3%.

Figure 4.

Difference between state self-reported and validated participation inequality scores.

Comparing Self-Reported and Validated Participation Inequality in an Applied Context

Inequalities in political participation have important implications for the policy process, which is why it is vital to have a clear understanding of how vote overreporting influences the measurement of participation inequality. This section presents an analysis of state welfare and minimum wage policy with both measures of participation inequality as the explanatory variables. The examination of the self-reported and validated inequality scores to this point has focused on direct comparisons of each measure. Using the two measures in a practical application will give a stronger indication of the substantive consequences of using self-reported turnout data to measure participation inequality.

The combination of having both greater levels of political resources and being more politically active is expected to affect policy outcomes. This is due to the likelihood that those with higher incomes and higher rates of political participation will have greater influence over the decision making of elected officials. This disproportionate influence of upper class voters will lead to policies that are more favorable to the rich and less favorable to the poor. State welfare policy is one area where the advantages and disadvantages for different income groups are relatively straightforward. For the most part, more generous and less restrictive welfare policies will directly benefit lower income groups. Conversely, states with welfare policies that have greater restrictions and offer fewer benefits will be less desirable for disadvantaged groups. As discussed above, much of the research examining the influence of unequal participation on policy outcomes has focused on welfare policies with most concluding that states with higher levels of inequality have less generous welfare programs. Following a common strategy used by some of these previous studies, state welfare policy is operationalized as a Welfare Policy Index by combining several components of each state’s welfare program (see Avery and Peffley 2005; Fellowes and Rowe 2004). The welfare index ranges from 0 to 5 with higher values of the measure indicating a more flexible and generous welfare system (M = 1.82).¹³ Therefore, states with higher levels of participation inequality are expected to have less flexible and less generous welfare policies.

The relationship between state welfare policy and unequal participation is modeled using ordinary least squares (OLS) regressions for each inequality index. Two sets of results are presented in Table 1. The second and third columns of the table show the results for simple bivariate regressions of the welfare index on each measure of participation inequality. The last two columns of the table provide regression results when including several covariates that are commonly used in the state policy adoption literature.¹⁴ The independent variables added to the multiple regression models include the updated Berry et al. (1998) measures of Citizen Ideology and Government Ideology where higher values indicate a more liberal ideology.¹⁵ Also included are the percent of the population that is white (Percent White) and state Per Capita Income in thousands of dollars.¹⁶

Table 1.

OLS Regression of State Welfare Index on Participation Inequality.

	Bivariate regression		Multiple regression
	b (SE)	b (SE)	b (SE)	b (SE)
Self-Reported Inequality	−8.535* (3.247)		−5.660^† (3.231)
Validated Inequality		−9.146 (7.301)		−12.022^† (6.453)
Citizen Ideology			0.003 (0.010)	0.005 (0.010)
Government Ideology			0.020^† (0.012)	0.021^† (0.012)
Percent White			0.528 (1.092)	1.147 (1.047)
Per Capita Income			0.040 (0.024)	0.050* (0.024)
Constant	3.886** (0.788)	3.763* (1.536)	−0.066 (1.764)	0.030 (1.746)
N	49	49	49	49

Note. Entries are unstandardized regression coefficients with standard errors in parentheses. Virginia is not included in the analysis as inequality scores for the state could not be calculated due to access restrictions placed on voting records by the state government (see Ansolabehere and Hersh 2012). OLS = ordinary least squares.

†

p < .10. *p < .05. **p < .01.

The participation inequality variables in all four regression models are negative indicating that states with higher levels of participation inequality are less likely to have generous welfare programs. Only the coefficient on the self-reported inequality measure is statistically significant for the bivariate models, but both measures are significant at the .10 level for the multiple regression models. Examining the size of the coefficient produced by each inequality measure we can see that the validated inequality index does produce a larger coefficient than the validated inequality measure for both sets of models. This difference in the magnitude of the coefficients, however, does not necessarily mean that the self-reported measure leads to bias in the substantive interpretation of the relationship between participation inequality and welfare policy. As discussed above, the validated inequality measure is generally lower than the self-reported scores, which suggests it is crucial to examine the substantive effect of each inequality measure on a common scale.

The substantive influence of both participation inequality variables on the welfare index for both models is shown in Figure 5. The graph plots the change in the state welfare index when changing each inequality measure from its minimum to maximum value. The figure demonstrates that when adding some basic covariates to the model, the effect of the inequality variables on welfare policy is nearly identical for both inequality measures, as are the confidence intervals for the expected values. For the simple bivariate case, the self-reported inequality score seems to somewhat overestimate the effect of unequal participation on the welfare index while the validated measure underestimates the effect. The inclusion of some commonly studied state characteristics appears to limit some of the bias exhibited in the bivariate estimates.

Figure 5.

Expected change in state welfare index.

The second applied example explored here is the influence of unequal participation on state minimum wage policy. Like welfare programs, the minimum wage is a policy area that has a direct influence on disadvantaged groups. Having a strong minimum wage policy can dramatically shape the earnings of low-income families. Some evidence even suggests that the relatively low federal minimum wage in the United States has contributed to the growing income inequality that has been experienced over the past few decades (Bartels 2008; Card and Krueger 1995; Lee 1999; Morris and Western 1999). Although Congress has increased the minimum wage in recent years, the real value of the federal minimum wage has actually decreased when accounting for inflation. While the value of the federal minimum wage has deteriorated over time, a number of states have passed laws that set the minimum wage above the federal level. As a higher minimum wage is likely to have a considerable influence on low-income groups, those states with lower levels of participation inequality are expected to have higher minimum wage rates.¹⁷

The analysis of minimum wage policy and unequal participation follows the same strategy presented in the examination of state welfare programs. Once again, the self-reported and validated inequality scores are used as predictors of the dependent variable in bivariate and multiple OLS regressions. The covariates used in the multiple regressions are the same as those included in the models of welfare policy. Table 2 presents the results from all four regression models. The findings in Table 2 are very similar to those in the case of state welfare programs. As expected, the overall relationship between unequal participation and minimum wage is negative suggesting that higher levels of participation inequality lead to lower state minimum wage rates. As the two measures of inequality differ due to voter overreporting, it is important to examine the substantive effects of unequal participation on minimum wage for each model.

Table 2.

OLS Regression of State Minimum Wage on Participation Inequality.

	Bivariate regression		Multiple regression
	b (SE)	b (SE)	b (SE)	b (SE)
Self-Reported Inequality	−8.919** (2.817)		−6.922* (2.742)
Validated Inequality		−12.284^† (6.375)		−13.084* (5.546)
Citizen Ideology			0.019* (0.008)	0.021* (0.008)
Government Ideology			0.011 (0.010)	0.012 (0.010)
Percent White			−0.473 (0.927)	0.272 (0.899)
Per Capita Income			−0.008 (0.021)	0.003 (0.021)
Constant	9.019** (0.684)	9.462** (1.341)	7.564** (1.498)	7.383** (1.501)
N	49	49	49	49

†

p < .10. *p < .05. **p < .01.

Figure 6 plots the expected change in a state’s minimum wage when increasing participation inequality from its minimum to its maximum value. Again, the results are analogous to those presented in Figure 5. In the simple bivariate case, the self-reported inequality score seems to overestimate the influence of unequal participation when compared with the estimated effects from the multiple regression models, and the opposite is true for the validated measure in the bivariate model. In the multiple regression context, however, the estimated effects and confidence intervals are nearly identical for the self-reported and validated inequality scores.

Figure 6.

Expected change in state minimum wage.

The findings from both applied examples show that the self-reported measure of participation inequality appears to provide a somewhat stronger relationship with the two policy areas than is observed with its validated vote counterpart in the bivariate context. When including some common state characteristics as covariates, however, the two measures produce very similar results.

Conclusion

For many observers of American politics, resource-based disparities in influence over government is a significant feature of our political system. A number of scholars have studied inequalities in political influence by focusing on the well-known economic biases in voter participation and demonstrate that these differences in political activeness can shape policy outcomes. A potential limitation of these studies, however, is that measuring voter turnout is a difficult task. As understanding income inequalities in participation requires the use of public opinion surveys, respondent misreporting (i.e., individuals reporting that they have voted when they in fact did not) may create biases in aggregate measures of participation inequality. Ansolabehere and Hersh’s (2012) impressive vote validation study has heightened concern over issues related voter overreporting. Particularly important for researchers using measures of economic-based participation inequality, the authors show those most likely to misreport turnout look very similar to actual voters. That is, nonvoters who report having voted on opinion surveys tend to be those at the higher end of the socioeconomic scale, which implies measures of participation inequality may be biased.

This article examined the consequences of measuring state-level participation inequality using self-reported turnout data. Two similarly constructed turnout inequality indices were compared, one using traditional survey responses and the other using validated vote data from Ansolabehere and Hersh’s (2012) validation project. The results suggest that while some states do have higher levels of self-reported participation inequality when compared with the validated measures, the differences appear to be minimal. Both measures were also used in two practical applications that examined how unequal participation influences state welfare programs and minimum wage policy. The findings show that the substantive effects produced by the two measures are nearly equivalent when modeled with common state-level covariates.

Although the results of this study do show signs of bias in measures of participation inequality due to vote overreporting, the extent of the observed differences is minimal and should not discourage those studying the consequences of unequal political participation. Instead, researchers should use vote validation studies in addition to traditional survey data to continue to better understand how disparities in citizen participation can lead to differences in political influence. Our knowledge of voting behavior will certainly be strengthened as additional validation studies allow us to examine voters beyond the 2008 election.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biography

William W. Franko is Assistant Professor of Political Science at Auburn University. His research broadly focuses on the causes and consequences of political, economic, and social inequality in the American context.

References

Anderson

Barbara A.

Silver

Brian D.

1986. “Measurement and Mismeasurement of the Validity of the Self-Reported Vote.” American Journal of Political Science 30 (4): 771–85.

Ansolabehere

Stephen

. 2008. Cooperative Congressional Election Study, Common Content [Computer File]. Release 4: July 15, 2011. Cambridge, MA: Harvard University [producer]. http://cces.gov.harvard.edu (accessed December 30, 2014).

Ansolabehere

Stephen

Hersh

Eitan

. 2012. “Validation: What Big Data Reveal about Survey Misreporting and the Real Electorate.” Political Analysis 20 (4): 437–59.

Avery

James M.

Peffley

Mark

. 2005. “Voter Registration Requirements, Voter Turnout, and Welfare Eligibility Policy: Class Bias Matters.” State Politics and Policy Quarterly 5 (1): 47–67.

Bartels

Larry M.

2008. Unequal Democracy: The Political Economy of the New Gilded Age. Princeton, NJ: Princeton University Press.

Belli

Robert F.

Traugott

Michael W.

Beckmann

Matthew N.

2001. “What Leads to Voting Overreports? Contrasts of Overreporters to Validated Voters and Admitted Nonvoters in the American National Election Studies.” Journal of Official Statistics 17 (4): 479–98.

Berinsky

Adam J.

2005. “The Perverse Consequences of Electoral Reform in the United States.” American Politics Research 33 (4): 471–91.

Bernstein

Robert

Chadha

Anita

Montjoy

Robert

. 2001. “Overreporting Voting: Why It Happens and Why It Matters.” Public Opinion Quarterly 65 (1): 22–44.

Berry

William D.

Ringquist

Evan J.

Fording

Richard C.

Hanson

Russell L.

1998. “Measuring Citizen and Government Ideology in the American States, 1960-93.” American Journal of Political Science 12 (1): 327–48.

10.

Blakely

Tony A.

Kennedy

Bruce P.

Kawachi

Ichiro

. 2001. “Socioeconomic Inequality in Voting Participation and Self-Rated Health.” American Journal of Public Health 91 (1): 99–104.

11.

Burnham

Walter Dean

. 1987. “The Turnout Problem.” In Elections American Style, ed. Reichley

James

. Washington, DC: Brookings Institution Press, 97–104.

12.

Card

David

Krueger

Alan B.

1995. Myth and Measurement: The New Economics of the Minimum Wage. Princeton, NJ: Princeton University Press.

13.

Dahl

Robert A.

2006. On Political Equality. New Haven, CT: Yale University Press.

14.

Fellowes

Matthew C.

Rowe

Gretchen

. 2004. “Politics and the New American Welfare States.” American Journal of Political Science 48 (2): 362–73.

15.

Franko

William W.

2013. “Political Inequality and State Policy Adoption: Predatory Lending, Children’s Health Care, and Minimum Wage.” Poverty & Public Policy 5 (1): 88–114.

16.

Gelman

Andrew

Hill

Jennifer

. 2007. Data Analysis Using Regression and Multilevel/Hierarchical Models. New York, NY: Cambridge University Press.

17.

Green

Mark

. 2004. Selling Out: How Big Corporate Money Buys Elections, Rams through Legislation, and Betrays Our Democracy. New York, NY: HarperCollins.

18.

Hill

Kim Quaile

Hurley

Patricia A.

1984. “Nonvoters in Voters’ Clothing: The Impact of Voting Behavior Misreporting on Voting Behavior Research.” Social Science Quarterly 65 (1): 199–206.

19.

Hill

Kim Quaile

Leighley

Jan E.

1992. “The Policy Consequences of Class Bias in State Electorates.” American Journal of Political Science 36 (2): 351–65.

20.

Hill

Kim Quaile

Leighley

Jan E.

1994. “Mobilizing Institutions and Class Representation in U.S. State Electorates.” Political Research Quarterly 47 (1): 137–50.

21.

Hill

Kim Quaile

Leighley

Jan E.

1995. “Lower-Class Mobilization and Policy Linkage in the U.S. States.” American Journal of Political Science 39 (1): 75–86.

22.

Jackson

Robert A.

Brown

Robert D.

Wright

Gerald C.

1998. “Registration, Turnout, and the Electoral Representativeness of U.S. State Electorates.” American Politics Quarterly 26 (3): 259–87.

23.

Katosh

John P.

Traugott

Michael W.

1981. “The Consequences of Validated and Self-Reported Voting Measures.” Public Opinion Quarterly 45 (4): 519–35.

24.

Lee

David S.

1999. “Wage Inequality in the United States during the 1980s: Rising Dispersion or Falling Minimum Wage?” Quarterly Journal of Economics 114 (3): 977–1023.

25.

Leighley

Jan E.

Nagler

Jonathan

. 1992. “Socioeconomic Class Bias in Turnout, 1964-1988: The Voters Remain the Same.” American Political Science Review 86 (3): 725–36.

26.

Lijphart

Arend

. 1997. “Unequal Participation: Democracy’s Unresolved Dilemma.” American Political Science Review 91 (1): 1–14.

27.

Mackenbach

Johan P.

Kunst

Anton E.

1997. “Measuring the Magnitude of Socio-Economic Inequalities in Health: An Overview of Available Measures Illustrated with Two Examples from Europe.” Social Science & Medicine 44 (6): 757–71.

28.

Morris

Martina

Western

Bruce

. 1999. “Inequality in Earnings at the Close of the Twentieth Century.” Annual Review of Sociology 25 (1): 623–57.

29.

Piven

Frances Fox

Cloward

Richard A.

1988. Why Americans Don’t Vote. New York, NY: Pantheon.

30.

Presser

Stanley

Traugott

Michael

. 1992. “Little White Lies and Social Science Models: Correlated Response Errors in a Panel Study of Voting.” Public Opinion Quarterly 56 (1): 77–86.

31.

Richter

Brian Kelleher

Samphantharak

Krislert

Timmons

Jeffrey F.

2009. “Lobbying and Taxes.” American Journal of Political Science 53 (4): 893–909.

32.

Rigby

Elizabeth

Springer

Melanie J.

2011. “Does Electoral Reform Increase (or Decrease) Political Equality?” Political Research Quarterly 64 (2): 420–34.

33.

Rosenstone

Steven J.

Hansen

John Mark

. 1993. Mobilization, Participation, and Democracy in America. New York, NY: Macmillan.

34.

Schattschneider

E. E.

1960. The Semi-Sovereign People: A Realist’s View of Democracy in America. New York, NY: Holt, Rinehart & Winston.

35.

Sigelman

Lee

. 1982. “The Nonvoting Voter in Voting Research.” American Journal of Political Science 26 (1): 47–56.

36.

Silver

Brian D.

Anderson

Barbara A.

Abramson

Paul R.

1986. “Who Overreports Voting?” American Political Science Review 80 (2): 613–24.

37.

Stern

Philip M.

1988. The Best Congress Money Can Buy. New York, NY: Pantheon Books.

38.

Traugott

Michael W.

Katosh

John P.

1979. “Response Validity in Surveys of Voting Behavior.” Public Opinion Quarterly 43 (3): 359–77.

39.

Verba

Sidney

Schlozman

Kay Lehman

Brady

Henry E.

1995. Voice and Equality: Civic Voluntarism in American Politics. Cambridge, MA: Harvard University Press.

40.

Wichowsky

Amber

. 2012. “Competition, Party Dollars, and Income Bias in Voter Turnout, 1980–2008.” Journal of Politics 74 (2): 446–59.

41.

Wolfinger

Raymond E.

Rosenstone

Steven J.

1980. Who Votes? New Haven, CT: Yale University Press.