When Does Increasing Mobilization Effort Increase Turnout? Evidence from a Field Experiment on Reminder Calls

Abstract

When does increasing mobilization effort increase turnout? Recent experiments find second calls containing a reminder to vote increase turnout beyond an initial contact. We argue existing studies cannot explain why reminder calls are effective because they test bundled treatments including a late mobilization attempt, a late mobilization attempt given earlier contact, and potentially activating reciprocity established in earlier contact. We report results from a two-round voter mobilization field experiment that allows us to isolate these different mechanisms. We find that reminder calls increase turnout by 1.2% points among subjects contacted in an earlier attempt, but that enhancing reciprocity by providing a reminder call offer during an early call does not increase turnout beyond a second call. Additionally, we fail to find heterogeneous effects of reminder calls by stated preference for a reminder or by stated vote intention, suggesting certain mechanisms do not explain the effects of reminder calls.

Keywords

voter mobilization reminder calls reciprocity field experiment

Scholars and practitioners working in the domain of campaigns and elections share a growing interest in understanding which Get Out the Vote (GOTV) mobilization tactics are effective at increasing participation levels and for what reasons. Work on the timing of mobilization efforts (Nickerson, 2007; Panagopoulos, 2011) finds that mobilization closer to an election is more effective than earlier contact. The interventions in these studies typically involve a single mobilization attempt. In real campaigns, however, citizens may be inundated by multiple mobilization attempts. This has also led to growing interest in an important question about the marginal returns to increasing the intensity of mobilization efforts: Do additional attempts at contacting voters increase turnout beyond the effects of a single mobilization attempt, and if so, why?

Despite increased interest, there are few experimental studies that address these questions. The limited work that exists involves phone-based GOTV campaigns and offers mixed findings. Early research found that a second mobilization attempt had no effect on turnout relative to an initial attempt (Green & Gerber, 2001). More recent research has reported relatively large positive effects of a second round mobilization attempt on turnout beyond an initial successful call when the second call provides a reminder to vote (Michelson et al., 2009). These competing findings, coupled with the relatively small number of studies on the efficacy of reminder calls, means there is theoretical and empirical uncertainty.

Existing experimental evidence seems to suggest that a second round mobilization attempt occurring just prior to an election is effective at increasing turnout beyond a successful initial call, but only when the second call is a reminder call (and perhaps only among those who state they intend to vote). But the literature is unable to clarify, both theoretically and empirically, why second GOTV attempts containing a reminder to vote are effective at increasing turnout beyond an initial and early GOTV attempt. Nor has it fully tested whether these effects depend on a citizen’s prior interest in participating.

Most succinctly, existing experimental designs test bundled treatments and do not provide leverage to test alternative causal theories explaining observed effects. Additionally, scholars have observed the fact that reminder call effects in the literature are from studies that condition on subjects saying they intend to vote and have assumed that the effects would not exist in the absence of such conditioning, leading them to emphasize theories about the role of baseline preferences toward voting. However, these mechanisms are not tested in existing studies; doing so would require directly and formally testing whether there are heterogeneous second round reminder call effects by subjects’ prior stated vote intentions.

In this article, we address these concerns and make several important incremental contributions to the literature. We design and analyze data from a phone-based GOTV experiment conducted by a non-profit, non-partisan organization in Colorado during the 2014 midterm election. In the experiment, the number and timing of GOTV call attempts (an early or late GOTV call) and the contents of the initial call (specifically, whether a reminder call to vote is offered in a first round call) are randomly manipulated. This experimental design provides leverage to isolate the causal effects of varying the level of campaign effort as a function of the number of calls, varying the timing of calls, and varying the degree to which an expectation of reciprocity is established and primed by manipulating the behavioral interactions occurring between the citizen and the campaign caller. Our design also provides leverage to test whether attempting a second round reminder call has heterogeneous effects by subjects’ stated vote intention.

We report three main empirical findings. First, we find that attempting a late GOTV call increases turnout by approximately 1.2% points among subjects successfully reached in an early GOTV call. This effect is smaller than some estimates previously reported in the literature, but provides additional experimental evidence supporting the argument that second round GOTV calls are effective at increasing turnout levels conditional on a successful initial contact (in support of Michelson et al. (2009), but contra Green and Gerber (2001)). Second, we do not find support for the reciprocity mechanism. The effect of a second round call is just as large when a reminder call is offered in an early call as it is when an offer is not provided in an early call. Third, we find that the effect of a late GOTV reminder call does not vary by subjects’ stated vote intention. Thus, the result we observe in this study leads us to downgrade our belief that observed second round reminder call effects on turnout are attributable to either reciprocity or a class of explanations emphasizing the causative role of intrinsic motivations to vote or statements about future vote intentions that are enhanced by a subsequent reminder call.

Existing Research

Does increasing mobilization effort increase turnout? The literature suggests that a second GOTV attempt providing a reminder to vote increases turnout among those successfully contacted in an earlier GOTV attempt. We briefly describe three experimental studies relevant to this argument and identify two important questions about existing research. First, we show that existing work is unable to clarify why observed effects exist because existing experiments involve bundled treatments that could operate through multiple hypothesized mechanisms. Second, we argue that existing theoretical explanations are incomplete because they inadequately distinguish between theoretical mechanisms related to GOTV contact generally, repeated GOTV efforts, or repeat interactions with a specific group. We argue that these theoretical ambiguities should inform experimental design.

Reminder GOTV Calls Increase Turnout

Green and Gerber (2001, pp. 21–22) conducted the first field experiment to assess whether increasing mobilization effort increases participation by testing the effect of a second GOTV call relative to a single GOTV call on voter turnout among a sample of registered voters aged 18 to 30. One of the factors in their original $2 \times 2$ factorial design randomly assigned half of the subjects to be called on Monday, the night before the election ( $n = 674$ ), and the other half to be called on both Sunday and Monday ( $n = 673$ ). This manipulation provides leverage to identify the effect of additionally calling on Sunday versus only calling on Monday on turnout. They find no effect of the second call on turnout relative to one call: the estimated average intent-to-treat (ITT) effect of attempting two calls is −0.3% points (s.e. = 2.4, not significant) relative to a one call group mean turnout rate of 75% and the Complier Average Causal Effect (CACE) of two calls is −0.4% points (s.e. = 3.1, not significant; estimated proportion of Compliers = .845) (Green and Gerber, 2001, p. 37, Table 7). This study is unusual in that both calls are timed to occur proximate to the election and to each other.

In subsequent research, Michelson et al. (2009) argue instead that multiple mobilization attempts taking the form of targeted reminder calls increase turnout, and that this effect is particularly pronounced among registered voters who are reached by a phone canvasser and who state to the canvasser that they intend to vote in the upcoming election. Of the four studies reported by Michelson et al. (2009), two are relevant to the substantive quantities of interest we investigate in this paper. We briefly review the design and findings of each in turn.

In the first of these studies, Michelson et al. (2009) analyze data from a field experiment conducted in 2003 where the Public Interest Research Group (PIRG) targeted registered voters aged 18 to 24. The study involved 2,817 subjects, all of whom were targeted by PIRG in an initial round of mobilization attempts using both phone and door-to-door canvassers prior to Election Day. About half of subjects were randomly assigned to a treatment group for whom PIRG would attempt follow-up GOTV calls containing a reminder to vote on Election Day.¹ The remaining subjects were assigned to a control group that received no follow-up call from PIRG beyond the initial GOTV attempt. Michelson et al. (2009) report that attempting a follow-up GOTV call on Election Day increases turnout by 3.8% points ( $p < . 01$ ) versus a 13.2% turnout rate in the control group for whom only an initial GOTV attempt was made. They also report that among subjects who were successfully contacted in the initial call attempt and who stated during the initial call that they intend to vote in the upcoming election, the estimated ITT of attempting a follow-up reminder call on Election Day is 10.6% points ( $p < . 01$ ) above a 16.9% baseline turnout rate in the control group. In contrast, among subjects who were successfully contacted in the initial call attempt and who stated that they did not intend to vote or refused to disclose their intention to vote, attempting a follow-up reminder call on Election Day decreases turnout by 3.9% points (not significant) relative to a 7.3% baseline turnout rate in the control group.

The second relevant field experiment from Michelson et al. (2009) builds on the conditional ITT estimates from the PIRG youth mobilization experiment and assesses whether a follow-up GOTV call containing a reminder to vote on Election Day increases turnout among registered voters who are reached by a campaign canvasser in an initial contact attempt and who state that they intend to vote in the upcoming election. In this study, the Asian Pacific American Legal Center (APALC) targeted a sample of 1,901 Asian American registered voters who were successfully contacted in an initial round of GOTV calls and who had stated that they planned to vote in person at the polls in the upcoming June 2008 election. Thus, subjects randomly assigned to treatment or control all stated that they would vote prior to random assignment. Of the 1,901 subjects in the study, 79% were randomly assigned to receive a follow-up GOTV call containing a reminder to vote from APALC and the remaining 21% were assigned to a control group that was successfully contacted in an initial call but received no follow-up call. They estimate that attempting a follow-up call increases turnout by 5.5% points ( $p < . 01$ ) versus a mean turnout rate of 14% in the control group who were contacted only once.

Existing Experiments Test Bundled Treatments, Not Competing Mechanisms

Existing experiments reporting increases in turnout due to an attempted second round reminder call test bundled treatments that limit the set of quantities that can be identified. Consequently, it is difficult to test competing hypothesized mechanisms explaining why effects are observed.

For example, the treatment conditions tested in the two studies by Michelson et al. (2009) are simultaneously a standard GOTV mobilization attempt with a reminder to vote occurring just before the election, receiving a follow-up call after an initial call, and a mobilization attempt conditional on a history of social interactions between the subject and the canvasser (due to successful contact in an earlier GOTV attempt). Because it is not possible to distinguish between the effect of a standard GOTV reminder call attempted just before an election (i.e., a late GOTV attempt) and the effect of a reminder call following an initial call (i.e., a late GOTV attempt following an early GOTV attempt), it is not possible to adduce whether the reminder to vote delivered in the late GOTV attempt is effective because an initial GOTV contact was made or not. More generally, it is not possible to ascertain whether early GOTV interactions condition the effectiveness of reminders to vote delivered in late GOTV attempts.

Given the set of quantities identified by existing experimental designs, the difficulty associated with testing competing theoretical explanations becomes evident when we consider the four hypotheses offered by Michelson et al. (2009) to frame their results. First, they argue that cognitive dissonance theory (Festinger, 1957) may be applicable because subjects may feel the need to maintain internal consistency across their previous expression of their intention to vote and their subsequent behavior. Second, they conjecture that the psychological theory of the self-erasing nature of prediction (Sherman, 1980) may explain observed effects. This theory posits that asking an individual to predict their future behavior might cause them to generate a mental image of engaging in that behavior, which increases the likelihood of doing the behavior of interest and rationalizing their behavior.² For both the first and second theoretical arguments, a second call is believed to heighten the importance of the proposed psychological mechanism. Third, they argue that the theory of reasoned action (Fishbein & Ajzen, 1975) may be applicable because the GOTV reminder call may affect two psychological mediators that in turn affect behavior: one’s perceived norms about voting and one’s intention to vote. Lastly, the reminder call may be priming subjects to perceive the norm of voting as a socially valued act, which consequently leads subjects to vote in order to signal to others that they are compliant with norms of being a good citizen (Funk, 2006; Michelson et al., 2009).

The first two hypotheses proposed – cognitive dissonance theory and the theory of the self-erasing nature of prediction – both belong to a class of theories that posit the existence of heterogeneous effects by subjects’ prior commitment to vote and by subjects’ prior behavior in which they imagine their future behavior, respectively. Researchers may employ one of two inferential strategies to test theories of heterogeneous effects. We argue that both strategies generate unclear conclusions given existing experimental designs. One inferential strategy involves using the mean ITT effect to adduce whether a theory about the existence of heterogeneous effects is valid. To use cognitive dissonance theory as an illustrative example, employing this strategy substantively implies that the treatment (i.e., the second round GOTV attempt) causes subjects who would otherwise not form a prior commitment to vote under the control condition (where only an early GOTV contact was made) to form a prior commitment to vote.³ However, this strategy does not lead to clear inferences because we do not observe whether the treatment does, in fact, cause the specified prior to form and whether that prior is lacking among subjects assigned to control. A second inferential strategy involves assessing heterogeneous treatment effects by formally testing treatment-by-covariate interactions where the covariate of interest is a pre-treatment quantity. The substantive implication of this approach is that a second GOTV attempt beyond an initial GOTV contact is simply an increase in the dosage of mobilization effort that potentially primes different subgroups of subjects differently, and that the treatment variable does not qualitatively affect subjects’ beliefs about voting. Put differently, it is not clear whether effects are observed because people who can be contacted and state they want to vote are generally easier to mobilize or because stating that one will vote causes a second call to be better at mobilizing a person.

In both studies presented by Michelson et al. (2009), the authors fail to formally test for heterogeneous effects and instead point to statistically significant estimates of the conditional mean effect of the second round GOTV call attempt among the subset of subjects successfully contacted in the first round who state they intend to vote. However, even if a formal test of heterogeneous effects were conducted, we argue that their design would not be able to clarify which set of features of the bundled treatment, if any, were at work and why. The latter two hypotheses – the theory of reasoned action and the heightened salience of voting as a socially valued act – do not uniquely explain the effects of second round reminder call attempts on turnout and are not unambiguously operationalized by the treatments tested in prior research. These explanations are arguably applicable to a wide range of treatment effects, including the effects of reminder calls that are not preceded by any prior GOTV attempt, the effects of reminder calls that are preceded by a range of potential histories of social interaction between the caller and the subject, and the effects of GOTV treatments more generally.

It is also possible that reminder calls work because they are a reciprocal interaction between a citizen and a mobilizing group, a theoretical mechanism that is not developed in prior work. (This is separate from the possibility that the reminder call works by enhancing the perceived social cost of failing to vote (Michelson et al., 2009).) Generically, the norm of reciprocity is the belief that one is “morally oblige[d] . . . to give benefits to those from whom he has received them” (Gouldner, 1960, p. 174).⁴ Perhaps a campaign establishes a norm of reciprocity by credibly signaling goodwill to the citizen in earlier interactions, such that the citizen subsequently chooses to vote as an act of reciprocity toward the campaign canvasser with whom they interacted.⁵ In particular, it may be that offering a reminder call in an initial interaction with the citizen and following through by providing it enhances a norm of reciprocity, which the citizen can repay by doing something good for the caller, in this case voting. The experiment we implement below allows us to isolate this potential mechanism.⁶

Overall, bundled treatments such as those tested in prior experiments on the effectiveness of reminder GOTV calls provide leverage to assess whether, but not why, certain interventions induce behavioral change. In order to make progress in the literature and to assess the plausibility of competing explanations for why observed reminder call effects exist, we argue for the use of “mechanism experiments” (Ludwig et al., 2011) where the experimental conditions are operationalized to map onto clearly specified alternative mechanisms that affect behavior.

Experimental Design

We design and analyze data from a field experiment assessing the effectiveness of different multiple-attempt GOTV strategies, where the treatment arms are operationalized to capture the effect of different mechanisms affecting political participation. The field experiment was implemented during the 2014 midterm election by the Colorado Civic Engagement Roundtable (hereafter referred to as the Roundtable), a non-partisan, non-profit organization that works together with other charitable organizations in Colorado to achieve an inclusive, just, and equitable state for all Coloradans.⁷ Working with a campaign consulting firm hired to design and implement the experiment, the Roundtable randomly assigned subjects to receive mobilization calls where the number and content of calls were varied. We advised the Roundtable and the campaign consulting firm with the design of the field experiment.

Population and Subject Definition

The Roundtable targeted Latinos, African Americans, young voters between the ages of 17 and 34, and unmarried women in their mobilization campaign. Registrants who had voted in all of the last four elections and older, long-term registrants who did not vote in 2012 were excluded from the Roundtable’s population of interest.

Subjects were recruited using the following procedure. First, the campaign consulting firm provided a list of 225,717 eligible registrants belonging to the Roundtable’s target demographic groups. Second, households where registrants without a valid state identification number were excluded because it would be impossible to match administrative voter records back to these individuals. Registrants without a valid phone number were excluded because the mobilization attempts are conducted by phone. Finally, one subject was randomly sampled from each household remaining in the sampling frame.⁸ The sample eligible for randomization included 139,153 subjects.

In light of reported effect sizes from the two prior field experiments conducted by Michelson et al. (2009), we conduct power calculations demonstrating that our sample is adequately powered to achieve 95% power with a balanced design. For Michelson et al.’s experiment conducted during the 2003 election (n = 2817, 1399 in treatment, 1418 in control), where attempting a follow-up GOTV call on Election Day increases turnout by 3.8% points (vs. a 13.2% turnout rate in the control group where only an initial GOTV attempt was made), a balanced design would require 4602 subjects (2301 subjects per arm) to achieve 95% power. For Michelson et al.’s experiment conducted during the 2008 election (n = 1901, 1501 in treatment, 400 in control), where attempting a follow-up GOTV call increases turnout by 5.5 points (vs. a 14% turnout rate in the control group who were only contacted in the first round of calls), a balanced design would require 2384 subjects (1192 subjects per arm) to achieve 95% power. The sample size of this experiment, which contains at least 17,000 subjects per arm, is multiple times larger than these target sample sizes and sufficiently powered to detect the effects reported by Michelson et al. (2009).

Definition of Treatments

We manipulate three features of a multiple-attempt, phone-based GOTV campaign: whether subjects are assigned to an early GOTV call, whether subjects are assigned to a late GOTV call, and whether the early GOTV call offered a reminder call. Including an offer for a reminder call in the early GOTV call script serves as a signal of goodwill from the caller to the citizen that is intended to ensure that a norm of reciprocity is established between the subject and the caller that is fulfilled by the second call. Because reciprocity may be established simply by successfully reaching a subject in an early GOTV call and in a subsequent late GOTV call, we accordingly interpret the effect of additionally providing an offer for a reminder call during the early call as the effect of an enhanced effort to establish reciprocity between the caller and the subject.

Table 1 summarizes the six treatment conditions in the experiment. Subjects may be assigned to one of the following conditions:

an early GOTV call including a reminder call offer that is followed by a late GOTV call;

an early GOTV call including a reminder call offer that is not followed by a late GOTV call;

an early GOTV call without a reminder call offer that is followed by a late GOTV call;

an early GOTV call without a reminder call offer that is not followed by a late GOTV call;

a late GOTV call only that is not preceded by any early GOTV call; or

a pure control condition in which neither an early GOTV call nor a late GOTV call is attempted.

Table 1.

Definition of Treatment Arms by Treatment Components.

Treatment arm	Treatment components
	Attempt early GOTV call
	Standard GOTVMessage	ReminderCall offer	Attempt lateGOTV call
Early GOTV call including reminder call offer followed by a late GOTV call	✓	✓	✓
Early GOTV call including reminder call offer not followed by a late GOTV call	✓	✓
Early GOTV call without a reminder call offer followed by a late GOTV call	✓		✓
Early GOTV call without a reminder call offer not followed by a late GOTV call	✓
Late GOTV call only			✓
Pure control

Source: Authors’ summary of treatment arms.

To be clear about the inferences this design allows us to make, we draw sharp conceptual distinctions between three types of late GOTV calls. A standard late GOTV call is a GOTV call made just prior to Election Day that is not preceded by an early GOTV call. A late GOTV call is a reminder call if it is preceded by an early GOTV call that does not contain any offer for a reminder call. A late GOTV call is a follow-up call if it is preceded by an early GOTV call that contains an offer for a reminder call. In this case, the late GOTV call serves the purpose of following up on the reminder call offer made in the early GOTV call. We are therefore able to distinguish the effects of a reminder call (a late GOTV call that follows an early GOTV call) from the effects of a late GOTV call (that does not follow an early GOTV call). Additionally, by comparing the effects of follow-up calls to the effects of reminder calls, we are able to isolate the effect of reciprocity above and beyond providing a reminder to vote by exploiting experimental variation in whether an additional reminder call offer is included in the early GOTV call among subjects assigned to an early GOTV call followed by a late GOTV call. That is, does enhancing reciprocity increase the effectiveness of a reminder call?

Treatment Scripts

We briefly describe the contents of the treatment scripts below.⁹ Professional callers employed by the campaign consulting firm made the calls and delivered the treatment scripts assigned to each subject.

For both the early GOTV call and the late GOTV call, the caller first asks to speak to the subject by name and records whether the the target subject is successfully reached.¹⁰ In the early GOTV script, the caller then introduces herself and her affiliation to the subject and states, “I’m with Colorado Civic Engagement. We’re not asking for money. We’re a non-partisan community group working to get people voting in this upcoming election.”

Subjects assigned to any condition including an early GOTV call who are successfully reached are first read the following:

We wanted to remind you that Election Day is Tuesday, November 4th, and your ballot will be mailed to you next week. You can return your ballot by mail or by dropping it off at a Voter Service Center. If you prefer to vote in person, you can do so on Election Day at a polling station in your county. Remember, you can vote as early as October 20th all the way through Election Day on November 4th. Voting is one of the ways we can take control of our future and our community. Do you plan on voting in this election?

This portion of the script is similar to scripts used in standard GOTV mobilization phone campaigns that provide information about the date of the election and information about where and when to vote.¹¹ It also includes a persuasive sentence emphasizing the subject’s ability to “take control of our future and our community” by voting. The early GOTV call script continues by asking subjects if they intend to vote in the upcoming election, and the caller records the subject’s response.

At this point, the treatment script ends for subjects assigned to an early GOTV call without a reminder call offer. Subjects who are assigned to an early GOTV call containing a reminder call offer are then asked whether they would like a reminder call:

If we have time the day or two before the election, would you like us to call you to remind you about the upcoming election?

and the caller records the subject’s stated preference for a reminder call.¹²

For subjects assigned to receive a late GOTV call in the days prior to Election Day, callers are instructed to read the following script after asking for the target subject and recording whether the subject was successfully reached:

My name is [caller’s name]. I’m with Colorado Civic Engagement, a nonpartisan community group working to get people to vote. This is a reminder to vote in this November’s election.

The caller continues by asking whether the subject has voted. If the subject responds that they have not voted or refuses to answer, the caller concludes by reminding them of the date of the election and where they can drop off their ballot.

Timing of Calls Relative to Election Day

At the time of the experiment, Colorado employed a no-excuse mail-in (absentee) voting system where all registered voters receive a ballot in the mail and are allowed to choose between voting by mail by Election Day, dropping off their ballot at a Voter Service Center by Election Day, or voting in person at a polling place on Election Day.¹³ To allow subjects who prefer to vote by mail to do so, the late GOTV call attempts were conducted between October 28 and 30, 2014, or 5-7 days prior to Election Day. The early GOTV call attempts were conducted between October 10 and 13, 2014, or 22-25 days prior to Election Day and approximately 3 weeks before the late GOTV calls.¹⁴

Randomization and Measurement Procedures

Subjects were randomly assigned to one of the six treatment arms using a simple random assignment procedure where the probability of assignment to either of the conditions containing an early GOTV call with a reminder call offer is 0.25 and where the probability of assignment to each of the remaining treatment conditions is 0.125.

Table 2 summarizes the distribution of subjects by treatment arm. We perform a randomization check by modeling the vector of treatment assignments as a function of observed covariates using a multinomial logit regression. We fail to reject the null hypothesis that all of the covariates are jointly prognostic of treatment ( $L R χ^{2} (70) = 61.32$ , $p = . 76$ ) and infer that the randomization procedure did not fail.¹⁵

Table 2.

Distribution of Subjects by Treatment Arm.

Treatment arm	N	%
Early GOTV call including reminder call offer followed by a late GOTV call	34,987	25.1
Early GOTV call including reminder call offer not followed by a late GOTV call	35,099	25.2
Early GOTV call without a reminder call offer followed by a late GOTV call	17,411	12.5
Early GOTV call without a reminder call offer not followed by a late GOTV call	17,177	12.3
Late GOTV call only	17,275	12.4
Pure control	17,204	12.4
Total	139,153	100

Source: Authors’ calculations.

Following the election, we obtained voter files from the Colorado Department of State and matched administrative records containing turnout data to subjects in the analysis file.¹⁶ The outcome measure of interest is turnout in the November 4, 2014 general election. The turnout variable is coded 1 if the subject voted in the election and 0 otherwise. We additionally collect pre-treatment covariate data from the campaign consulting firm’s sampling frame, which was purchased from a political data vendor. The covariates collected for each subject include their gender, race, age, past vote history in the 2010 and 2012 primary and general elections, and the number of years since they last registered to vote.

Analysis and Results

Our analysis proceeds in three stages. First, we assess the intent-to-treat (ITT) effects of a late GOTV call on turnout after an early GOTV call.¹⁷ Second, we present a series of empirical tests of the reciprocity mechanism. As part of this analysis, we assess whether follow-up call effects are heterogeneous by subjects’ stated preference for a reminder call, in particular whether the effects of a follow-up call are stronger among those who state they want a reminder. Finally, given the importance placed on heterogeneous effects by stated vote intention in prior work, we conduct an exploratory analysis assessing whether follow-up calls and reminder calls have differential effects by subjects’ stated vote intention. For the sake of clarity, we discuss each set of empirical tests by specifying and justifying causal quantities of interest before presenting results. For all analyses, we estimate treatment effects parametrically using ordinary least squares. We employ both unadjusted and covariate adjusted estimators to assess robustness,¹⁸ specifying the covariate adjusted estimator as the primary specification to improve precision given expectations about small effect sizes. Raw turnout rates by treatment assignment, and contact rates are presented in Table 3.

Table 3.

Turnout and Contact Rates by Arm.

Treatment arm	Number assigned		Mean turnout		Early call contact rate		Late call contact rate
Treatment arm	N	Prop.	Mean	SD	N	Prop	N	Prop
Early GOTV call including reminder call offer followed by a late GOTV call	34987	0.251	0.458	0.498	7346	0.210	6979	0.199
Early GOTV call including reminder call offer not followed by a late GOTV call	35099	0.252	0.460	0.498	7380	0.210
Early GOTV call without a reminder call offer followed by a late GOTV call	17411	0.125	0.460	0.498	3745	0.215	3449	0.198
Early GOTV call without a reminder call offer not followed by a late GOTV call	17177	0.123	0.460	0.498	3649	0.212
Late GOTV call only	17275	0.124	0.453	0.498			3375	0.195
Pure control	17204	0.124	0.455	0.498
Total	139153	1.000

Intent-to-Treat Effects of Reminder GOTV Calls after an Early GOTV Call

We begin by assessing the average effect of attempting a reminder GOTV call among subjects who were randomly assigned to receive an early GOTV call, or $E [Y_{i} (Z^{L} = 1) - Y_{i} (Z^{L} = 0) | Z^{E} = 1]$ , where $Y_{i}$ is subject $i$ ’s potential outcome and equals 1 if they vote in the election and 0 otherwise, $Z^{L}$ denotes assignment to the late GOTV call and equals 1 if the subject is assigned to a late GOTV call and 0 if not, and $Z^{E}$ denotes assignment to the early GOTV call and equals 1 if the subject is assigned to the early call and 0 if not.¹⁹ This is the intent-to-treat effect that has been estimated in prior literature. To identify this quantity, we subset the sample to subjects assigned to receive an early GOTV call and estimate the following equation:

Y_{i} = α_{1} + β_{1} Z_{i}^{L} + γ_{1} X_{i} + ε_{1 i}

(1)

where $β_{1}$ is our estimate of the ITT effect of reminder calls, $X$ is a vector of controls, and $γ_{1}$ is a vector of coefficients on $X$ . We additionally assess the ITT effect of attempting a reminder GOTV call among subjects who were successfully reached during the early GOTV call, or E[Y_i(Z^L = 1) – Y(Z^L = 0) | D^E = 1], where $D^{E}$ denotes receiving the early GOTV call and equals 1 if the subject received the assigned early GOTV call treatment and 0 if the subject did not receive the early GOTV call. We operationalize the receipt of the early GOTV call as the caller reaching the subject and getting through the first question in the early GOTV phone survey that confirms that the individual on the line is the targeted subject.²⁰ Thus it excludes subjects who could not be reached during the early GOTV call. We identify this quantity by partitioning the sample to include only subjects who received an early GOTV call and re-estimating Equation 1 among this subgroup.

Table 4 Panel A presents our covariate adjusted estimates of the ITT effect of attempting a reminder call on turnout among subjects assigned to an early GOTV call attempt (Column 1) and the same effect among subjects who were successfully contacted in the early GOTV call (Column 2). Among subjects assigned to any early GOTV attempt, attempting a reminder call decreases turnout levels by 0.1% points on average, but this effect is indistinguishable from zero (s.e. = 0.003; $n = 104, 674$ ). Among subjects who were successfully reached in an early GOTV call, attempting a reminder call increases turnout by 1.2% points. This effect is statistically significant at the 5% level (s.e. = 0.006; $p < . 05$ , two-tailed; $n = 22, 120$ ). Given a control group voting rate of 54.1%, this translates into a 2.2% increase in the turnout rate.²¹ The unadjusted estimates in Panel B are highly similar to the adjusted estimates: –.1 points and 1.3 points in Columns (1) and (2), respectively.

Table 4.

Estimated ITT Effects of Attempting a Late GOTV Call on Turnout among Subjects Assigned to and Contacted in an Early GOTV Call Attempt.

Variable	Assigned to receive early GOTV call	Assigned to and contacted in early GOTV call
Variable	(1)	(2)
Panel A: Adjusted estimates
Attempted late GOTV call	−0.001[0.003]	0.012[0.006]**
Years since registration date	−0.004[0.000]***	−0.004[0.001]***
Years since registration date missing	−0.245[0.028]***	−0.322[0.058]***
Gender = Male (1 = Yes)	−0.004[0.003]	−0.01[0.006]*
Gender = Unknown (1 = Yes)	−0.059[0.107]	0.035[0.241]
Race = Black (1 = Yes)	−0.101[0.004]***	−0.089[0.009]***
Race = Latino (1 = Yes)	−0.091[0.004]***	−0.076[0.009]***
Race = Other (1 = Yes)	−0.055[0.008]***	−0.048[0.015]***
Age in years	0.010[0.001]***	0.008[0.001]***
Age missing	0.073[0.058]	0.066[0.110]
Age squared/100	−0.004[0.001]***	−0.003[0.001]*
Voted in 2012 general election (1 = Yes)	0.230[0.003]***	0.226[0.008]***
Voted in 2010 general election (1 = Yes)	0.230[0.004]***	0.214[0.008]***
Voted in 2012 primary election (1 = Yes)	0.180[0.008]***	0.182[0.015]***
Voted in 2010 primary election (1 = Yes)	0.090[0.007]***	0.092[0.014]***
Constant	0.037[0.012]***	0.131[0.026]***
Observations	104,674	22,120
Adjusted R-squared	0.157	0.144
Reference group mean	0.46	0.541
Panel B: Unadjusted estimates
Attempted late GOTV call	−0.001[0.003]	0.013[0.007]**
Constant	0.46[0.002]***	0.541[0.005]***
Observations	104,674	22,120
Adjusted R-squared	0	0
Reference group mean	0.46	0.541

Standard errors in brackets.

***

p < .01, **p < .05, *p < .1.

Testing the Reciprocity Mechanism

We now turn our attention to a series of estimates that allow us to evaluate whether a reciprocity mechanism plausibly explains the effectiveness of follow-up calls. Our inferential strategy involves testing a series of hypotheses about ITT effects among subgroups whose treatment assignment and behavior in response to early GOTV call attempts satisfy conditions that we argue would be necessary to attribute the effects of attempting a follow-up call on turnout to a reciprocity explanation. Specifically, we focus on assessing follow-up call effects under a set of conditions where subjects may credibly perceive goodwill from the caller, which then potentially induces the subject to engage in voting as reciprocal act. We then compare that estimate to the effect of attempting a reminder call on turnout among those contacted in the early call but who were not offered a reminder in the early call in order to assess the importance of the initial act of offering a reminder call in the early call. (Therefore we are able to difference out the effect of attempting a second call and isolate the effect of offering a reminder in the early call).

We begin this line of inquiry by examining whether, among subjects who were assigned to an early GOTV call containing a reminder call offer and who were successfully contacted in the early call, attempting a follow-up call increases participation levels. Formally we express this quantity as $E [Y_{i} (Z^{L} = 1) - Y_{i} (Z^{L} = 0) | D^{O} = 1]$ where $D^{O}$ denotes receipt of the reminder call offer (in the early GOTV call, i.e., if $D^{O} = 1$ then $D^{E} = 1$ ). We identify this effect by estimating Equation 1 among subjects contacted in an early GOTV call with an offer of a reminder. In other words, this is the combined effect of reciprocity plus a reminder call.

We then estimate the effect of a follow-up call attempt among subjects who received an early GOTV call containing a reminder call offer and who expressed that they wanted the reminder call. Formally we express this quantity as $E [Y_{i} (Z^{L} = 1) - Y_{i} (Z^{L} = 0) | D^{O} = 1, W_{i} = 1]$ where $W_{i}$ equals 1 if subject $i$ expresses that they want a reminder call and 0 otherwise. We identify this quantity by estimating Equation 1 among subjects offered a follow-up call who requested it. We argue that the subset of subjects satisfying these conditions can be interpreted as those who are most likely to perceive a follow-up call as an act of goodwill that induces voting as a reciprocal act. If wanting (valuing) something that is offered to you enhances reciprocity, this group should experience a larger treatment effect.

We complement our primary test of the reciprocity mechanism with two ancillary analyses that assess how participation levels vary when either of the conditions central to our primary test of the mechanism are relaxed. We first assess whether failing to deliver a follow-up call despite the initial offer of a reminder call in the early GOTV call decreases turnout.²² To do this, we compare average turnout rates between subjects assigned to an early GOTV call with a reminder call offer and subjects assigned to an early GOTV call without a reminder call offer, among those subjects not assigned to receive a late GOTV call. We identify this quantity, $E [Y_{i} (Z^{O} = 1) - Y_{i} (Z^{O} = 0) | D^{E} = 1, Z^{L} = 0]$ , by estimating the following equation among subjects assigned to receive any early GOTV call, are successfully contacted during the early GOTV call, and are assigned to receive no late GOTV call:

Y_{i} = α_{2} + β_{2} Z_{i}^{O} + γ_{2} X_{i} + ε_{2 i}

(2)

Finally we assess whether there is an effect of attempting reminder calls among subjects who were assigned to an early GOTV call without a reminder call offer and who were successfully contacted during the early GOTV call. In the absence of a reminder call offer in the early GOTV call, we would expect much less reciprocity to be established between the subject and the caller. We identify this quantity, $E [Y_{i} (Z^{L} = 1) - Y_{i} (Z^{L} = 0) | D^{E} = 1, Z^{O} = 0]$ , by re-estimating Equation 1 among this subset of subjects.

Table 5 Panel A reports our covariate adjusted estimates of these conditional ITT effects specified to test the reciprocity mechanism. Column 1 presents the estimated effect of attempting a follow-up call on turnout among subjects who were assigned to an early GOTV call with a reminder call offer and who were successfully contacted during the early call. We find that among this subgroup, attempting a follow-up call increases turnout levels by an average of 1% point, but we fail to reject the null hypothesis that the effect is equal to zero (s.e. = 0.008; $n = 14, 726$ ). Per Panel B, the unadjusted estimate is a similarly insignificant 1.2 points.

Table 5.

Estimated ITT Effects Testing the Reciprocity Mechanism.

Variable	Conditional on receiving early GOTV call
	Offered reminder	Offered and wanted reminder	Not assigned late GOTV call attempt	Not offered reminder in early call
	(1)	(2)	(3)	(4)
Panel A: Adjusted estimates
Attempted late GOTV call	0.010[0.008]	0.008[0.020]		0.016[0.011]
Assigned reminder call offer in early GOTV call			0.004[0.009]
Years since registration date	−0.004[0.001]***	−0.007[0.002]***	−0.005[0.001]***	−0.005[0.001]***
Years since registration date missing	−0.309[0.071]***	−0.314[0.122]**	−0.303[0.095]***	−0.339[0.099]***
Gender = Male (1 = Yes)	−0.018[0.008]**	0.033[0.020]*	−0.002[0.009]	0.005[0.011]
Gender = Unknown (1 = Yes)	0.050[0.282]		0.001[0.354]	0.015[0.480]
Race = Black (1 = Yes)	−0.104[0.012]***	−0.088[0.027]***	−0.077[0.013]***	−0.057[0.016]***
Race = Latino (1 = Yes)	−0.084[0.011]***	−0.055[0.026]**	−0.073[0.012]***	−0.062[0.015]***
Race = Other (1 = Yes)	−0.060[0.019]***	−0.027[0.052]	−0.043	−0.022[0.027]
Race = Other (1 = Yes)	−0.060[0.019]***	−0.027[0.052]	[0.022]**	−0.022[0.027]
Age in years	0.007[0.002]***	0.008[0.004]**	0.006[0.002]***	0.01[0.002]***
Age missing	0.063[0.150]	0.081[0.293]	0.062[0.168]	0.058[0.166]
Age squared/100	−0.001[0.002]	−0.002[0.005]	0[0.002]	−0.005[0.003]*
Voted in 2012 general election (1 = Yes)	0.236[0.010]***	0.234[0.026]***	0.232[0.012]***	0.207[0.014]***
Voted in 2010 general election (1 = Yes)	0.209[0.010]***	0.211[0.027]***	0.216[0.011]***	0.224[0.014]***
Voted in 2012 primary election (1 = Yes)	0.192[0.019]***	0.197[0.053]***	0.194[0.022]***	0.158[0.027]***
Voted in 2010 primary election (1 = Yes)	0.081[0.017]***	0.077[0.048]	0.097[0.020]***	0.115[0.025]***
Constant	0.153[0.032]***	0.091[0.081]	0.157[0.037]***	0.086[0.044]*
Observations	14,726	2,198	11,029	7,394
Adjusted R-squared	0.145	0.14	0.144	0.144
Reference group Mean	0.543	0.528	0.539	0.539
Panel B: Unadjusted estimates
Attempted late GOTV call	0.012[0.008]	−0.001[0.021]		0.017[0.012]
Assigned reminder call offer in early GOTV call			0.004[0.010]
Constant	0.543[0.006]***	0.528[0.015]***	0.539[0.008]***	0.539[0.008]***
Observations	14,726	2,198	11,029	7,394
Adjusted R-squared	0	0	0	0
Reference group mean	0.543	0.528	0.539	0.539

Standard errors in brackets.

***

p < .01, **p < .05, *p < .1.

Our primary test of the reciprocity mechanism is presented in Column 2, which reports the estimated effect of attempting a follow-up call among subjects who were assigned to an early GOTV call with a reminder call offer, who were successfully contacted in the early call, and who stated they wanted a reminder call. In other words this quantity captures the combined effect of a reminder call and the effect of reciprocity on turnout. We find that among those provided a reminder call offer and who wanted the reminder call, attempting a follow-up call increases turnout levels by 0.8% points (s.e. = 0.02; $n = 2, 198$ ). We draw attention to the fact that this estimate is weakly less than, and certainly no greater than, the estimated effect of a follow-up call among subjects reached in an early GOTV call with a reminder call who did not want the call (from Column 1). Thus we conclude that the effects of a follow-up call on turnout conditional on receiving an early call with a reminder call offer does not appear to depend on whether one wants a reminder call and, importantly, is not larger among those who want a reminder call.²³ We also note that while the estimated mean effects in these two analyses are comparable in magnitude to statistically significant effects in prior GOTV field experiments,²⁴ the estimated effects are not statistically distinguishable from zero in this study. The unadjusted estimate in Panel B is closer to 0 (–0.1 points with a similar standard error).

Columns 3 and 4 of Table 5 present two ancillary analyses assessing how turnout is affected when we relax different conditions under which we might expect reciprocity to affect turnout. In Column 3, we assess the effect of including an offer for a reminder call in the early GOTV call among those reached in the early call and not assigned to any late GOTV call. This contrast allows us to infer whether failure to follow-up conditional on providing a reminder call offer in the early GOTV call has an effect on turnout as well as if merely offering a reminder (but not carrying it out) increases turnout. The effect is 0.4% points (s.e. = 0.009; $n = 11029$ ). The magnitude of this difference is substantively small, which implies that failing to follow-up has no effect on turnout. Per Panel B, this estimate is unaffected by covariate adjustment. In Column 4, we assess the effect of attempting a late GOTV call on turnout among subjects assigned to and successfully contacted in an early GOTV attempt but who are not offered a reminder call in the early GOTV call. This quantity allows us to infer whether, in the absence of an act of goodwill in the early call, attempting a reminder call given successful contact in an early GOTV call increases turnout. The estimated effect of the reminder call attempt among this subgroup is 1.6% points (s.e. = 0.011; $n = 7394$ ). Thus the effect of attempting a second round call is not larger (and may be smaller) when it follows an offer for a reminder call in the early call than when it does not. (Per Panel B, the unadjusted estimate is a highly simlar, 1.7 points (s.e. = 0.012)). We therefore do not find evidence that enhancing reciprocity, operationalized as offering a reminder call in an early GOTV attempt, increases the turnout effects of reminder calls.²⁵

Heterogeneous Effects of Follow-Up and Reminder Calls by Stated Vote Intention

Finally, we assess whether follow-up call and reminder call effects vary by subjects’ stated vote intention. This analysis is motivated by prior work that has assumed that reminder calls are only effective among subjects who state they intend to vote in the upcoming election. However, the empirical claim that heterogeneous effects by stated vote intention exist has not yet been tested. Thus we conduct a series of exploratory analyses to assess whether heterogeneous effects exist. First, we assess whether follow-up calls are effective among subjects who state that they intend to vote during the early GOTV call. We identify the quantity $E [Y_{i} (Z^{L} = 1) - Y_{i} (Z^{L} = 0) | D^{E} = 1, D^{O} = 1, V_{i}^{Y} = 1]$ , where $V_{i}^{Y}$ denotes a subject stating she intends to vote in the upcoming election, by estimating Equation 1 on this subset of subjects and test the null hypothesis that the coefficient on the $Z_{i}^{L}$ variable is equal to zero. This conditional mean effect has been estimated in past studies and is of particular interest because it may be the case that a call to remind a subject to vote only works as a reminder if the subject already intends to vote.

We then formally test whether heterogeneous follow-up call effects exist among subjects successfully contacted in an early GOTV call and whether heterogeneous reminder call effects exist among subjects successfully contacted in the early call. For both of these subgroups, we estimate

\begin{array}{l} Y_{i} = α_{3} + β_{3} Z_{i}^{L} + β_{4} V_{i}^{Y} + β_{5} V_{i}^{U} + β_{6} Z_{i}^{L} \cdot V_{i}^{Y} + \\ β_{7} Z_{i}^{L} \cdot V_{i}^{U} + γ_{3} X_{i} + ε_{3 i} \end{array}

(3)

where $V_{i}^{Y}$ equals 1 if the subject was contacted during the early call and stated they intended to vote and 0 otherwise; and $V_{i}^{U}$ equals 1 if the subject was contacted during the early call but did not explicitly state whether they intended to vote and 0 otherwise. The omitted reference category is comprised of subjects who were successfully contacted in the early call and stated they did not intend to vote in the upcoming election. For each of these analyses, we test the null hypotheses that $β_{6} = β_{7}$ , $β_{6} = 0$ , and $β_{7} = 0$ to compare average follow-up and reminder call effects between subjects who were successfully contacted during the early call but who have different stated vote intentions.

Column 1 in Table 6 Panel A presents the estimated effect of attempting a follow-up call on turnout among subjects who received an early GOTV call with a reminder call offer and who stated during the early call that they intended to vote in the upcoming election. Among this subgroup, attempting the follow-up call increases turnout levels by 1.3% points (s.e. = 0.008, $n = 11794$ ) but this difference is not statistically distinguishable from zero. (The unadjusted estimate in Panel B is similar, 1.2 points.) In Columns 2 and 3, we report heterogeneous effects of attempting a follow-up call and a reminder call, respectively, on turnout by stated vote intention among subjects who were (Column 2) and were not (Column 3) offered a reminder call as part of the early GOTV call script. These analyses also condition on subjects who were successfully reached during the early call. As shown in Column 2, among subjects who were contacted and offered the reminder call offer in the early call, the effect of attempting a follow-up call is 3.0% points larger on average for subjects who stated they intend to vote than for subjects who stated they did not intend to vote, and 3.5% points larger for subjects who did not disclose their vote intention than for subjects who stated they did not intend to vote. However, neither of these mean effect estimates are statistically distinguishable from zero. Column 3 shows that among subjects who were contacted and not offered a reminder call in the early call, attempting a reminder call is 3.9% points larger on average for subjects who stated they intend to vote than for subjects who stated they did not intend to vote, and 5.4% points larger for subjects who did not disclose their vote intention than for subjects who stated they did not intend to vote. Similarly, these effects are not statistically distinguishable from zero. Thus while intending to vote seems to be a strong predictor of turnout²⁶ and despite the emphasis in prior work placed on claims suggesting the contrary, we do not find evidence that the effect of attempting a follow-up or reminder call varies by stated vote intention. Unadjusted estimates in Panel B are slightly different in magnitude, but also do not support the claim that turnout effects are larger among those who previously stated an intention to vote.

Table 6.

Estimated ITT Effects of Attempting a Late GOTV Call on Turnout, by Stated Vote Intention among Subjects Assigned to and Contacted in an Early GOTV Call Attempt.

Variable	Conditional on receiving early GOTV call
Variable	Offered reminder in early GOTV call and intends to vote	Offered reminder in early GOTV call	Not offered reminder in early GOTV call
Panel A: Adjusted estimates
Attempted late GOTV call	0.013[0.008]	−0.017[0.028]	−0.024[0.040]
Subject’s stated vote intention in early call = Yes		0.198[0.021]***	0.172[0.029]***
Subject’s stated vote intention in early call = Unknown		0.064[0.025]**	0.034[0.035]
Attempted late GOTV call * intends to vote		0.030[0.029]	0.039[0.042]
Attempted late GOTV call * vote intention unknown		0.035[0.035]	0.054[0.050]
Years since registration date	−0.005[0.001]***	−0.004[0.001]***	−0.005[0.001]***
Years since registration date missing	−0.339[0.093]***	−0.282[0.070]***	−0.340[0.098]***
Gender = Male (1 = Yes)	−0.020[0.009]**	−0.018[0.008]**	0.002[0.011]
Gender = Unknown (1 = Yes)	0.07[0.335]	0.062[0.279]	0.123[0.476]
Race = Black (1 = Yes)	−0.118[0.013]***	−0.105[0.011]***	−0.056[0.016]***
Race = Latino (1 = Yes)	−0.091[0.012]***	−0.082[0.011]***	−0.059[0.015]***
Race = Other (1 = Yes)	−0.04[0.022]*	−0.045[0.019]**	−0.018
			[0.027]
Age in years	0.006[0.002]***	0.006[0.002]***	0.009[0.002]***
Age missing	0.073[0.172]	0.040[0.149]	0.081[0.164]
Age squared/100	0[0.002]	−0.001[0.002]	−0.004[0.002]*
Voted in 2012 general election (1 = Yes)	0.245[0.011]***	0.224[0.010]***	0.198[0.014]***
Voted in 2010 general election (1 = Yes)	0.195[0.011]***	0.203[0.010]***	0.220[0.014]***
Voted in 2012 primary election (1 = Yes)	0.173[0.020]***	0.187[0.019]***	0.151[0.027]***
Voted in 2010 primary election (1 = Yes)	0.077[0.019]***	0.077[0.017]***	0.108[0.025]***
Constant	.208[0.036]***	0.005[0.037]	−0.034[0.051]
Observations	11,794	4,726	7,394
Adjusted R-squared	0.138	0.162	0.159
Reference group mean	0.311	0.311	0.347
Panel B: Unadjusted estimates
Attempted late GOTV call	0.012[0.009]	−0.009[0.030]	−0.02[0.043]
Subject’s stated vote intention in early call = Yes		0.272[0.022]***	0.232[0.031]***
Subject’s stated vote intention in early call = Unknown		0.102[0.027]***	0.058[0.037]
Attempted late GOTV call * intends to vote		0.022[0.032]	0.035[0.045]
Attempted late GOTV call * vote intention unknown		0.038[0.038]	0.054[0.053]
Constant	0.583[0.006]***	0.311[0.022]***	0.347[0.030]***
Observations	11,794	14,726	7,394
Adjusted R-squared	0	0.03	0.025
Reference group mean	0.311	0.311	0.347

Standard errors in brackets.

***

p < .01, **p < .05, *p < .1.

Discussion

Despite considerable interest in whether increasing targeted mobilization effort increases turnout, credible research on the topic remains sparse and open to competing theoretical interpretations. We address these concerns with a design, implemented in a field experiment conducted during the 2014 midterm election in Colorado, that allows us to isolate different theoretically relevant effects and, in turn, make inferences about the validity of competing hypotheses explaining observed reminder call effects.

Our findings make three contributions to the literature. First, we show that attempting a late GOTV call conditional on a successful early GOTV call increases turnout levels by 1.2% points. In contrast, when we condition on subjects assigned to an early GOTV call (pooling across successful and unsuccessful contact in early calls), attempting a reminder call has no effect on turnout. The substantive and practical implication of this result is that reminder calls should only be attempted among subjects who are successfully contacted in an initial call attempt and not among those who not contacted in the initial round of calls. This finding bolsters earlier experimental results showing that the direction of the effect of a reminder call on turnout levels is positive, but the magnitude of our effect is far more modest than estimates reported in prior experiments on the effectiveness of reminder calls.²⁷ The difference in the magnitude of the reported effects across studies may arise from multiple factors including sampling variability, differences in the demographic profile of the targeted populations across studies, characteristics of the mobilizing group, and differences in the salience of the elections associated with each study.

Second, we extend the literature by explicating and testing a reciprocity mechanism. To test this argument, we assess the effect of attempting a follow-up call among subjects offered a reminder call in an early call and among subjects who were offered the reminder call and wanted it. We interpret these subgroups as subjects whose interactions with the caller during the early call have the greatest potential to establish goodwill and a norm of reciprocity between the subject and the caller to induce them to vote as a reciprocal act. We find that attempting a follow-up call among subjects who are contacted in the early call and are provided with a reminder call offer generates a mean increase in the turnout rate by about 1% point. When further conditioning on subjects who state that they want the reminder when provided with a reminder call offer in the early call, we observe a similar result: attempting a reminder call among this subgroup increases turnout by 0.8% points. However, we fail to reject the null hypothesis that the mean effect is equal to zero for both of these groups. We additionally assess the importance of enhancing reciprocity (through the reminder call offer in the early GOTV call) by comparing the effect of attempting a second round call after an early call with a reminder call offer to the effect of attempting a second round call after an early call without a reminder call offer. Overall, we do not find that enhancing reciprocity by offering a reminder call increases the effect of a late GOTV call that follows an early call.²⁸

We offer two alternative interpretations of observed results that deserve further exploration in future research. The first interpretation is that a positive effect of the combined reminder call and reciprocity treatment exists but we are simply unable to detect that it is statistically distinguishable from zero in this study due to a lack of statistical power resulting from the relatively small sample sizes that remain after conditioning on successful early call contact, including a reminder call offer in the early call, and the subject stating that they wanted the reminder call. If this is the case, then a larger field experiment replicating the design of this experiment could provide more compelling evidence to test the reciprocity mechanism.

The second interpretation is that the reciprocity argument may be true, but that the treatments in this study were not optimally designed to test this mechanism. We provide two possible explanations why a null effect might be observed under the present design and describe alternative experimental designs that may provide better leverage to test the reciprocity mechanism. First, it may be the case that being offered a reminder call and wanting a reminder call did not establish a norm of reciprocity and goodwill between the subject and the campaign caller. This may be because subjects doubted the sincerity of the reminder call offer and, more generally, the intentions of the mobilization campaign’s agent. In the context of electoral campaigns, citizens likely understand that politicians and campaign organizations undertake a range of efforts to mobilize voters. In turn, subjects may not have believed that the reminder call offer was intended for their benefit, but rather for the benefit of the organization conducting the mobilization campaign.

A second potential explanation is that the combination of the early call with the reminder call offer and the follow-up call attempt was successful at establishing goodwill and a norm of reciprocity, but these conditions were not sufficient to compel subjects to reciprocate by changing their voting behavior.²⁹ As past work by Whatley et al. (1999) has shown, even though people may strongly internalize the norm of reciprocity, engaging in norm-compliant reciprocal behavior is more likely to occur in a public context than in a private context because people seek social rewards for engaging in reciprocal behavior and seek to avoid the social costs of failing to reciprocate when monitored by others. The treatments in this experiment may not have adequately primed social incentives to vote because the treatment conditions involved private interactions between the subject and the caller and because the subject may not have expected anyone to monitor whether they voted in the election.³⁰

Another fruitful direction for future research involves testing a placebo-controlled design that includes a treatment condition where subjects may be assigned to a first round call that has nothing to do with voter mobilization. Designs including a placebo first round call would provide leverage to assess conditional effects of attempting a second round call by varying prior interactions, to make credible inferences about the importance of prior interactions in multiple-attempt GOTV campaigns (rather than merely being the type of person one can contact in an early call), and to better isolate the effect of the late GOTV call on turnout.

Finally, we provide the first assessments of heterogeneous follow-up call and heterogeneous reminder call effects in the literature. Specifically, we examine whether reminder call effects vary by subjects’ stated demand for a reminder and by subjects’ stated vote intention. For both sets of analyses, we find no statistically significant differences in the effect of a follow-up or reminder call on turnout relative to an early call by subjects’ stated preference for a reminder or by subjects’ stated vote intention. While the present design cannot answer questions about the causal role of subjects’ stated vote intention and is unable to distinguish sorting from treatment (i.e., whether it is stating one’s vote intention or being the type of person who would state one’s vote intention that matters), our results cast doubt on the claim implicit in the literature that the effects of reminder calls vary by stated vote intention. However, the magnitudes of the estimates on the interaction terms are substantively large given the magnitude of other GOTV treatment effects in the literature, which suggests that the present design may be underpowered and that larger and better powered replication experiments may detect heterogeneous effects that are statistically distinguishable from zero.

Supplemental Material

ReminderCallsRR_SIOnly_AsSubmittedANON – Supplemental material for When Does Increasing Mobilization Effort Increase Turnout? Evidence from a Field Experiment on Reminder Calls

Supplemental material, ReminderCallsRR_SIOnly_AsSubmittedANON for When Does Increasing Mobilization Effort Increase Turnout? Evidence from a Field Experiment on Reminder Calls by Alan S. Gerber, Gregory A. Huber, Albert H. Fang and Catlan E. Reardon in American Politics Research

Footnotes

Author’s Note

Albert H. Fang is now an independent scholar.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Gregory A. Huber

Albert H. Fang

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Alan S. Gerber is Charles C. & Dorathea S. Dilley professor of Political Science and director of the Center for the Study of American Politics at Yale University.

Gregory A. Huber is Forst Family professor of Political Science and associate director of the Center for the Study of American Politics at Yale University.

Albert H. Fang is an independent scholar, was previously a postdoctoral associate at the Center for the Study of American Politics at Yale University, and holds a Ph.D. from Columbia University.

Catlan E. Reardon is a PhD candidate in the Travers Department of Political Science at the University of California, Berkeley.

References

Caporael

L. R.

Dawes

R. M.

Orbell

J. M.

van

Kragt

A. J. C.

(1989). Selfishness examined: Cooperation in the absence of egoistic incentives. Behavioral and Brain Sciences, 12(4), 683–699.

Cialdini

R. B.

(1984). Influence: The new psychology of modern persuasion. Quill.

Festinger

(1957). A theory of cognitive dissonance. Stanford University Press.

Fishbein

Ajzen

(1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Addison-Wesley.

Funk

(2006). Modern voting tools, social incentives and voter turnout: Theory and evidence. Working Paper. Presented at the Annual Meeting of the American Economic Association, Chicago, IL.

Gerber

A. S.

Green

D. P.

(2012). Field experiments: Design, analysis, and interpretation. W. W. Norton.

Gouldner

A. W.

(1960). The norm of reciprocity: A preliminary statement. American Sociological Review, 25(2), 161–178.

Green

D. P.

Gerber

A. S.

(2001). Getting out the youth vote: Results from randomized field experiments (Technical report). Yale University.

Green

D. P.

Gerber

A. S.

(2015). Get out the vote: How to increase voter turnout (3rd ed.). Brookings Institution Press.

10.

Kiesler

C. A.

(1971). The psychology of commitment: Experiments linking behavior to belief. Academic Press.

11.

Ludwig

Kling

J. R.

Mullainathan

(2011). Mechanism experiments and policy evaluations. Journal of Economic Perspectives, 25(3), 17–38.

12.

Michelson

M. R.

Bedolla

L. C.

McConnell

M. A.

(2009). Heeding the call: The effect of targeted two-round phone banks on voter turnout. Journal of Politics, 71(4), 1549–1563.

13.

Nickerson

D. W.

(2007). Quality is job one: Professional and volunteer voter mobilization calls. American Journal of Political Science, 51(2), 269–282.

14.

Panagopoulos

(2011). Timing is everything? Primacy and recency effects in voter mobilization campaigns. Political Behavior, 33(1), 79–93.

15.

Sherman

S. J.

(1980). On the self-erasing nature of errors of prediction. Journal of Personality and Social Psychology, 39(2), 211–221.

16.

Sobel

(2005). Interdependent preferences and reciprocity. Journal of Economic Literature, 43(2), 392–436.

17.

Whatley

M. A.

Matthew Webster

Smith

R. H.

Rhodes

(1999). The effect of a favor on public and private compliance: How internalized is the norm of reciprocity? Basic and Applied Social Psychology, 21(3), 251–259.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB