Abstract
Despite more than a million deaths and counting from the COVID-19 virus, fewer than 23% of U.S. adults have received the most recent bivalent booster vaccines, dramatizing the challenge of developing effective health promotion strategies in an era of broad distrust of science and authoritative expertise. This study compares the effects of directive and non-directive versions of a novel vaccination advocacy approach that combines the strategies of inoculation theory and narrative persuasion. The study also examines two critical underlying factors that can influence the effects of such advocacy efforts—message elaboration and psychological reactance—while also accounting for the effects of political identity. Using a survey-based pre-test/post-test experimental design, participants (N = 496) were randomly assigned to see either a non-directive or directive advocacy message delivered in either a static/print or video format. Participants were assessed for attitude and intention toward COVID-19 vaccination before and after message exposure and also provided demographic information including political identity. Results showed strong effects for political identity on attitude toward vaccination, message elaboration and reactance, but not on attitude change. Although the directive message produced greater levels of psychological reactance—which was negatively correlated with attitude change—than the non-directive message, it also produced greater levels of positive attitude change than the non-directive message. Results provide support for further application of narrative inoculation approaches for health promotion and persuasion strategies in general.
Keywords
Introduction
Despite more than a million U.S. deaths from COVID-19 and hundreds more every day, just 22.5% of U.S. adults had received the most recent bivalent booster vaccines as of May 22, 2024 (CDC, 2024). Few Americans now believe COVID-19 is a serious danger (Pew, 2024a) and most say they probably will not get an updated vaccine (Pew, 2024c), with vaccination intentions polarized along partisan political lines (Pew, 2024a, 2024c). These statistics highlight some of the challenges of developing effective health promotion strategies in an era of increasing distrust of science and authoritative expertise (Pew, 2023), which may be part of a decades’-long trend in declining trust in authoritative institutions of all types (Pew, 2024b).
The current study compares the effects of a directive versus a non-directive vaccination advocacy message on attitudes and intentions regarding staying up-to-date with COVID-19 vaccinations. The study also examines two critical underlying factors that can influence the effects of vaccination advocacy efforts: message elaboration (i.e., the degree to which a message-target pays attention to and thinks deeply about the message), and psychological reactance (i.e., the degree to which the message-target is stimulated to feel resentment or resistance as a consequence of the persuasive intent of the message). These underlying factors influence the effects of virtually all persuasive messages (Dillard & Shen, 2005; Gans & Zhan, 2023). They are significant in this context because health communications must navigate an environment in which people often: (1) make healthcare decisions based on low-elaboration mental shortcuts and (2) resent and actively avoid persuasive messaging (Compton et al., 2016; Kahan et al., 2011). Also included in the study’s analyses are the effects of political identification, which has been shown to be a significant factor in many individuals’ decisions on whether to get vaccinated (Gans, 2014; Kahan et al., 2010).
The study contributes to the literature of the fields of persuasion studies and health promotion by testing a cross-discipline messaging approach that combines the strategies of narrative persuasion and inoculation theory, methods that have been used both to reduce and enhance resistance to persuasion (Compton et al., 2016; Niederdeppe et al., 2015). The study is also one of the first to provide multivariate and mediation analyses of the interplay between elaboration, psychological reactance, and political identity with regard to health promotion in the form of vaccination advocacy. The study’s results underscore the challenges facing public health efforts to mitigate the ongoing COVID-19 crisis. They also identify key areas for exploration in developing more effective public health messaging strategies.
Message Construction: Narrative Inoculation
Concerted and repeated public attacks by conspiracy theorists and political actors (Kaiser Family Foundation, 2023; Pew, 2024c; Weisman, 2023) may partially account for the low compliance rates achieved by COVID-19 vaccination advocacy efforts. Inoculation theory (McGuire, 1961a, 1961b, 1964; McGuire & Papageorgis, 1962; Papageorgis & McGuire, 1962; Pfau, 1995) has been widely studied as a persuasive strategy to defend desired or advocated-for attitudes and behaviors from attacks (Banas & Rains, 2010; Compton et al., 2016), including those relating to vaccination (Ivanov et al., 2020; Wong & Harrison, 2014). Inoculative messages have been demonstrated to be effective when administered in the absence of or before such attacks, that is, “pre-bunking” (Cook et al., 2017) but also to be effective when administered as a “debunking” repair or support strategy after such attacks (Dillingham & Ivanov, 2017; Wigley & Pfau, 2010).
Use of Counterarguments and Refutations
Similar to a traditional sales strategy in which message recipients’ expected objections are raised and overcome through refutational arguments before the sales message is delivered (Claxton et al., 2001; Hunt & Bashaw, 1999), an inoculative message provides counterarguments to the desired attitudes or behaviors and then supplies refutations of those counterarguments (Ivanov et al., 2020)—in much the same way as the biological analog in which the recipient is administered a weakened dose of a disease to stimulate the recipient’s bodily defenses without overwhelming them (Pfau, 1995). For instance, in an intervention designed to inoculate adolescents against becoming tobacco smokers, the inoculation message started with a warning to the adolescent audience members that their current stance against smoking would be threatened by pressure from their peers and by their own curiosity: No matter how much you want to stay a non-smoker, the truth is that the pressure to smoke in junior high is greater than it will be at any other time of your life. Three out of four young adults…and that’s you—will pick up a cigarette and let curiosity take over. (Pfau, 1995, p. 107)
The intervention messaging then provided refutational support to prime the adolescents with strategies to overcome the threatened temptation: So, how do you stick to your guns when the odds are against you? Well, you might run into this crowd… Do they look cool to you? Putting a lit stick up to your mouth is not cool. It’s…“gross.” It causes bad breath, yellow teeth, and smelly clothes and hair. (Pfau, 1995, p. 108)
Messages that include such threatening counterarguments as well as supportive refutations have been shown to be more effective in protecting and thereby promoting desired attitudes and behavioral intentions than purely supportive messages (e.g., Banas & Rains, 2010; Compton et al., 2016; Dillingham & Ivanov, 2017; Wigley & Pfau, 2010).
Narrative Persuasion
Inoculation strategies have been widely used to protect desired attitudes or behaviors from attack that might lead to resistance against efforts to promote those attitudes or behaviors (Richards & Banas, 2015). Narrative persuasion has also been proposed by scholars as a way to counteract or avoid resistance to behavioral recommendations by inducing the target audience to vicariously experience the adoption of behaviors and attitudes they might never consider otherwise (Green & Brock, 2000; Hamby et al., 2017; Moyer-Gusé & Nabi, 2010; Slater et al., 2006; van Laer et al., 2014). When immersed in a narrative, message recipients can put themselves in the shoes of characters who express attitudes or make behavioral decisions they might ordinarily reject or argue against. By encouraging identification with characters within a narrative, this experience can impart information and promote behaviors and attitudes the message recipients might otherwise resist or oppose (Moyer-Gusé, 2008; Moyer-Gusé & Nabi, 2010). Persuasive narratives can take the form of scripted video dramatizations in which characters similar to the intended target audiences make decisions or perform behaviors that align with an advocated-for outcome (Moyer-Gusé & Nabi, 2010). They can also take the form of personal narratives from such figures (Gans & Zhan, 2023a, 2023b). Narratives have been shown to be effective in promoting vaccination (Krakow et al., 2017; Krishna & Amazeen, 2022), and to be effective in combination with inoculation strategies to address issues of obesity, smoking and painkiller addiction (Niederdeppe et al., 2015) as well as in other areas such as organizational policies (Gans & Zhan, 2023). These findings suggest the following hypothesis:
H1: Personal narrative advocacy messages incorporating counterarguments and refutations will produce positive attitude change in favor of vaccination.
Psychological Reactance and Resistance to Persuasion
The theory of psychological reactance (Brehm, 1989) is often cited by scholars describing scenarios in which behavioral recommendations produce resistance rather than compliance (e.g., Burgoon et al., 2002; Miller et al., 2006; Ringold, 2002). Reactance theory suggests such recommendations can be construed as threats to one’s personal freedom or autonomy, thereby eliciting reactions such as “ignoring the persuasive attempt, derogating the source, and even producing even more of the undesired behaviors as a means of demonstrating choice or restoring attitudinal freedom” (Burgoon et al., 2002, p. 215). Miller et al. (2006) have suggested that the stronger the recommendation, the greater the threat to freedom, noting that “psychological reactance theory predicts that the more directive and controlling a persuasive message is perceived to be, the more likely its position is to be rejected” (p. 223).
The current study compares the effects and effectiveness of directive and non-directive narrative advocacy messages. The research on psychological reactance suggests the following hypotheses:
H2: Directive advocacy messages will produce greater levels of psychological reactance than non-directive advocacy messages.
H3: Controlling for other factors, reactance will be negatively correlated with change in attitude toward vaccination.
Message Delivery: Rich Versus Lean Media and Effects of Elaboration
Media richness theory (MRT; Daft & Lengel, 1984) suggests that the “richness” of a communication medium influences its capacity to facilitate shared understanding, and that the greater the complexity of a communication-based decision, the greater the need and desirability for information richness. According to MRT, one of the key factors in a medium’s richness is its capacity to convey multiple informational cues (Daft & Lengel, 1984). The ability of video to convey body language, facial expression and tone of voice makes it a richer medium than static text. In this sense, a rich-medium presentation of a message would tend to give its audience more to think about than a lean-medium presentation of that message, with a greater potential to produce audience engagement.
Whether this would produce greater compliance with a persuasive message is open to question. The elaboration likelihood model (Petty & Cacioppo, 1984, 1986) suggests that expending more mental effort and elaborating deeply on a message will produce greater and more lasting attitude change based on the message but could also lead to a greater tendency to produce counterarguments to that message. This suggests the following research question:
RQ1: Controlling for other factors, will elaboration on the advocacy message be positively or negatively correlated with change in attitude toward vaccination?
Prior studies of video-based versus text-based teaching and learning suggest both these outcomes are possible. Participants in a teacher training course showed greater engagement, interest and retention of instruction over time with learning content in video format versus the same content in text format (Heemsoth et al., 2022), possible because video watching inspires engagement with less effort than reading. However, perhaps because of the effort involved in reading, nursing students in an online course reported greater engagement and interactivity in text-based formats rather than video-based formats (Swartzwelder et al., 2019). Thus, this uncertainty regarding the effects of rich versus lean media represents a gap in the research on processing of and compliance with persuasive advocacy messages, suggesting the following research question:
RQ2: Will advocacy messages presented using rich media (i.e., video) produce greater levels of (RQ1a) elaboration and/or (RQ1b) psychological reactance than messages presented using lean media (i.e., static text)?
Influence of Politics
Political partisanship remains one of the most powerful and well-publicized factors shaping attitudes and behaviors regarding COVID-19 vaccination, with a 27-point gap in vaccination rates between Democrats and Democratic-leaning Independents versus Republicans and Republican-leaning Independents, 42% to 15%, and Democrats and Democratic-leaners more than twice as likely to be concerned about unknowingly spreading COVID-19 than Republicans and Republican-leaners, 54% to 24% (Pew, 2024a).
Political ideology seems to play a predictable role in attitudes toward scientists and scientific information as well, with people who are politically more conservative less likely to trust scientists as a source of information about public policy issues such as climate change (Hamilton, 2011; Kahan et al., 2011) and more likely to actively resist and argue against science and the recommendations of scientists (e.g., Mooney, 2005, 2012). There seems to be a rural/urban divide in this regard, with rural areas more likely to mistrust the directions and recommendations of government-related institutions and scientific authorities (Larsson et al., 2006), as well as to identify as Conservative and Republican (Pew, 2018). While many people may claim to be non-aligned or independent, few are actually neutral, with most people—when they choose to vote—voting consistently either Republic or Democrat (Levendusky, 2009), and with political identity tending to remain stable over the course of a person’s lifetime (Hibbing et al., 2013).
Studies of other vaccination advocacy efforts have also reflected this pattern, with study participants who were identified as Conservative responding less favorably to vaccination advocacy messages than those who were identified as Liberal or Progressive (Gans, 2014; Kahan et al., 2010). These factors suggest the following hypothesis.
H4: Political identification will be correlated with attitude toward COVID-19 vaccination such that those identifying as Liberal or Democrat will exhibit more favorable attitudes than those identifying as Conservative or Republican.
While this hypothesis states that political identity will affect vaccination attitude, it is not clear how political identity will affect the effectiveness of the advocacy messages and attitude change. This suggests this final set of research questions:
RQ3: How will political identity affect underlying message processing factors of elaboration and psychological reactance?
RQ4: How will the relationships among message strategy (directive or non-directive), medium (rich or lean), elaboration, psychological reactance and political identity affect attitude change regarding vaccination?
Method
Overview
This study was conceived as a between-subjects 2 × 2 experiment comparing the effects of two different inoculation-inspired personal narrative advocacy messages (directive vs. non-directive) delivered in either a rich (video) or lean (static text) media format. Because of the political considerations attached to COVID-19 vaccination (Pew, 2024c), political identity has been included as an additional dichotomous variable, making this a 2 × 2 × 2 analysis. Data for this study were collected on April 12 and 13, 2023, using an online survey created on the QuestionPro platform and an online sample of adult residents of the United States recruited through the Prolific platform. The study, survey instrument, and Informed Consent components were approved for human subjects research by a research university’s Office of Regulatory Services (IRB Protocol Number 2023-0132.2) on April 11, 2023.
Sample
Participants for this study were recruited from Prolific.com, a research platform used by “more than 35,000 researchers across academia and industry” (Prolific.com, n.d.) that provides researchers with managed access to a diverse and customizable pool of pre-screened participants. To assure data quality, the platform assigns and verifies individual participant IDs each with a unique payment account, requires non-VOIP phone numbers, monitors participant IP/ISP quality, and includes bot detection and real-time monitoring of participant usage patterns (Hillman, 2023). Scholars have found the Prolific platform to be superior to another commonly used platform, Amazon’s Mechanical Turk (MTurk), in various aspects (Palan & Schitter, 2018; Peer et al., 2022). To maintain a high-quality pool of participants, the Prolific platform requires a payment rate of at least $8.00 per hour. In the current study, participants received a compensation of $1.20. With a median completion time of about 10 min, the average hourly rate was $10.19.
Acknowledging the potential issues of participant inattention and system-gaming often encountered when using online research participant pools, we followed recommended best practices for designing and implementing online experiments (Sheehan, 2018). After removing incomplete responses and those failing attention checks, the total number of valid responses was 496. Of these, 247 (49.8%) identified as female, 239 (48.2%) identified as male, and 10 (2.0%) chose other responses. Participants’ ages ranged from 18 to 93, with a median of 37 and a mean of 40.55 (SD = 13.85). As might be expected from a pre-qualified participant pool with requirements as stringent as those of the Prolific platform, participants were more well-educated than the U.S. adult population as a whole, with 60.0% (298) of the participants reporting having graduated from college, versus just 37.9% for the total U.S. adult population (U.S. Census Bureau, 2022). Regarding income, 54.2% reported earning less than $60,000 annually and 45.8% (227) reported earning more than $60,000 annually. For comparison, the U.S. median household income for 2023 was $80,610 (U.S. Census Bureau, 2024). Regarding race and ethnicity, the sample skewed disproportionately White, as shown in Table 1, below.
Demographic Identity of Survey Participants Versus U.S. Population.
Source. U.S. Census Bureau, 2023; New York Times, 2024*.
In terms of political identity, the sample skewed toward Liberal, with 22% (109) self-identifying as slightly to very strongly conservative, 19% (94) self-identifying as centrist, and 59% (293) self-identifying as slightly to very strongly liberal.
Study Procedures and Experimental Stimulus
A 140-word recruitment message was posted on the Prolific platform providing a general description of the study survey, the reward of $1.20, and an approximate completion time of 5 or 6 min. Prolific members who responded to the recruitment invitation were provided with a hyperlink to the QuestionPro survey, where they were required to agree to a detailed informed consent statement before proceeding.
The study employed a between-subjects pre-test/post-test design in which participants were assessed for their attitudes toward vaccination in general and toward COVID-19 vaccination specifically, then randomly assigned to view one of four different message conditions: a non-directive personal narrative or a directive personal narrative delivered as a video or as a static text message accompanied by a photo of the same spokesperson who delivered the message in the video. Participants were then re-assessed regarding their attitudes toward COVID-19 vaccination using the same assessment tool as before, and also assessed for their levels of elaboration on the message and psychological reactance resulting from the message. Participants provided demographic information after completing all the assessments.
Message Conditions
To test the relative efficacy and potential interaction effects of a directive and non-directive advocacy message delivered in a rich or lean medium, four messages were prepared using persuasion strategies adapted from studies in narrative persuasion (e.g., Moyer-Gusé, 2008; Moyer-Gusé & Nabi, 2010) and inoculation theory (e.g., McGuire, 1964; Pfau, 1995). The directive and non-directive messages each featured a personal narrative incorporating arguments against vaccination followed by refutations of those arguments leading to either (1) a recommendation why the message recipient should get vaccinated, in the directive condition or (2) a personal decision to get vaccinated without any accompanying behavioral recommendation to the message recipient, in the non-directive condition. The directive message was 232 words long. The non-directive message was 203 words long. The rich media versions of the messages were delivered as videos that ran 74 and 63 s for the directive and non-directive messages respectively. In the videos, both messages were presented by the same undergraduate student who had been hired expressly for this purpose. The lean-media/static-text versions were delivered as formatted text following a bold-text headline with an accompanying image of the same spokesperson featured in the video.
The directive and non-directive messages and the accompanying images are presented below in Figures 1 and 2.
Lean-media version of directive advocacy message.
Lean-media version of non-directive advocacy message.
Measures
The current study includes multi-item scale measures for attitudes toward vaccination in general as well as toward vaccination against the COVID-19 virus. It also includes multi-item scale measures of message elaboration and message-driven psychological reactance. Each of the scale items was assessed by asking for level of agreement with various statements using 7-point Likert-type scales with endpoints anchored at 1 (Strongly Disagree) and 7 (Strongly Agree).
Intention to Get Vaccinated
Behavioral intention regarding COVID-19 vaccination was assessed before and after message exposure with a single item, level of agreement with the statement: “I plan to stay up-to-date with the latest COVID 19 booster vaccines as they become available.” Pre-exposure mean (SD) = 4.32 (2.32). Post-exposure mean (SD) = 4.52 (2.32). Distribution of responses to these items resembled the opposite of a normal distribution and suggest asking for a simple “yes” or “no” answer may have been appropriate. The distributions, presented below in Figure 3, also suggest an effect for message exposure.
Behavioral intention regarding vaccination: distribution of responses.
Attitude Toward COVID-19 Vaccination and Attitude Change
There are few validated scales for measuring attitudes toward COVID-19 vaccination, and those we could find seemed inappropriate for the current study in that they focused on vaccination hesitancy and conspiracy theories (Phillips et al., 2022) or were too long to hold the attention of an online survey pool (Alam et al., 2022). Drawing on the health belief model (Rosenstock, 1974) assessment strategies from these and several general vaccination attitude studies (Kocoglu-Tanyer et al., 2020; Krakow et al., 2017; Okuhara et al., 2023) as well as on the fear and efficacy foci of the extended parallel processing model (Maloney et al., 2011; Witte, 1992), a 12-item assessment tool was developed with three items focused on participants’ perceptions of the severity of the COVID-19 disease, three items focused on their perceptions of their susceptibility to the disease, three items focused on their perceptions of the treatment efficacy of the COVID-19 vaccinations and boosters, and three items focused on their perceptions of their ability (i.e., personal efficacy) to get vaccinated. The 12 items are presented in Table 2, below.
12-Item HBM-EPPM Attitude Toward COVID-19 Vaccination Scale.
Note. Likert-type 7-point scale with anchors 1 = “Strongly Disagree”; 7 = “Strongly Agree”; (R) = reverse-coded.
As a validity check, correlation analyses were conducted between this COVID-specific scale and the measures of behavioral intention (r = .744, p < .001) and the vaccination in general VAX scale (r = .836, p < .001), suggesting acceptable levels of criterion-related and convergent validity. The Cronbach’s alpha reliability of the COVID-specific scale was .907 when administered before message exposure and .916 after message exposure. The McDonald’s omega measures were .919 before and .927 after. Attitude change was calculated by subtracting pre-exposure attitude from post-exposure attitude.
Attitude Toward Vaccination in General
To provide a comparison for validating the COVID-specific vaccination attitude scale developed for use in this study, the 12-item vaccination attitudes examination (VAX) scale developed by Martin and Petrie (2017) was administered to all participants prior to their assessment for COVID vaccination attitudes. The Cronbach’s alpha reliability measure for the VAX scale was .956. The McDonald’s omega was .957
Elaboration on Advocacy Message
The elaboration likelihood model (Cacioppo & Petty, 1984) suggests attitude change is more likely when elaboration levels are high. A five-item instrument adapted from a prior study on systematic processing of risk communication (Kahlor et al., 2003) was used to measure participants’ level of elaboration on the advocacy messages. The measure asked participants to rate their level of agreement or disagreement with statements such as “I found myself comparing the information to my own life experience,”“I tried to visualize the effects of this problem in my community,” and “I tried to use the information to puzzle out how I would address the problem.” Cronbach’s alpha for the five-item message elaboration scale was .874. McDonald’s omega was .875.
Psychological Reactance
The advocacy messages used in the current study explicitly or implicitly suggested message recipients should make specific behavioral actions and therefore had the potential to elicit psychological reactance in response. Following the example of prior research (Moyer-Gusé & Nabi, 2010), reactance was measured using Dillard and Shen’s (2005) 4-item threat-to-freedom scale, which asked for level of agreement with statements suggesting the persuasive message “…threatened my freedom to choose,”“…tried to make a decision for me,”“…tried to manipulate me,” and “…tried to pressure me.” Cronbach’s alpha for the four-item psychological reactance scale was .894. McDonald’s omega was .899.
Political Identity
Political partisanship continues to loom large in considerations regarding advocacy for and resistance to COVID-19 vaccination (Pew, 2024c). To explore its role in influencing the effects of pro-vaccination advocacy messages, political identity was measured both as a multi-categorical and dichotomous variable.
Ideology: Conservative or Liberal
Participants were asked to rate themselves on a scale of 1 to 9 in which 1 = Very strongly conservative, 2 = strongly conservative, 3 = moderately conservative, 4 = slightly conservative, 5 = centrist, 6 = slightly liberal, 7 = moderately liberal, 8 = strongly liberal, 9 = very strongly liberal. Mean for this measure was 6.12 (SD = 2.30), suggesting the sample was more liberal than conservative, as indicated by a one-sample t-test with a test value of 5, t (495) = 10.84, p < .001, Cohen’s d = .49.
Partisanship: Republican or Democrat
To create a dichotomous partisanship variable, participants were also asked for whom they would vote, an unnamed Republican or Democrat, in a far-future presidential election, with not voting not an option. This result aligned with the previous result, with 341 (68.8%) indicating they would vote for the Democrat and 155 (31.3%) for the Republican. With values of 1 = The Republican and 2 = The Democrat, the mean was 1.69 (SD = .46). Pearson correlation between the ideology (Conservative or Liberal) and partisanship (Republican or Democrat) measures was .752, p < .001, N = 496.
Results
Statistical Analyses
All statistical analyses were conducted using SPSS 29.0. Other than attitude change and change in intent, which had high kurtosis values indicating a robust central tendency reflecting no change, all measures’ skewness and kurtosis statistics were within the absolute value of 2 (Hancock et al., 2010) and original data were used without transformation. All assumptions of equal variances, independence, normality and linearity were met, except where specifically noted. Significance level was set at p < .05, but exact p-values are reported where appropriate.
Overview: MANOVA
This study employed multiple strategies to analyze the data. To provide an overview before testing specific hypotheses, a 2 × 2 × 2 multivariate analysis of variance (MANOVA) was used to examine the effects of persuasive message strategy (directive, non-directive), medium (print, video), and political identity (Republican, Democrat) on levels of elaboration, psychological reactance, and attitude change. Means and standard deviations included in the MANOVA are presented in Table 3.
Means and Standard Deviations for Attitude Change, Elaboration and Psychological Reactance by Message Type, Medium, and Political Identity.
The MANOVA omnibus test revealed significant main effects for message strategy, F (3, 486) = 5.46, p = .001, η2 p = .033, and for political identity, F (3, 486) = 60.82, p < .001, η2 p = .273 (NOTE: η2 p = partial Eta squared), as well as for the interaction between message strategy and political identification, F (3, 486) = 4.58, p = .004, η2 p = .028. This interaction is explored in greater detail in Figures 4 and 5, below.
Interaction effects of message strategy and political identity on psychological reactance.
Interaction effects of message strategy and political identity on attitude change.
The omnibus test identified no significant effects for medium F (3, 486) = 1.50, p = .212, η2 p = .009, or the interactions between medium and message, F (3, 486) = 1.84, p = .140, η2 p = .011, and between medium and political identity, F (3, 486) = 0.02, p = .998, η2 p < .001, or the three-way interaction between medium, message, and political identity, F (3, 486) = 0.63, p = .597, η2 p = .004. These findings suggest that political identification and message strategy each separately and together had significant effects on one or more of the dependent variables, but that medium and the interactions involving the medium variable did not.
Tests of between-subjects effects provided revelatory details of the effects identified in the omnibus test. As suggested by the omnibus tests, the between-subjects tests revealed no significant effects for medium or the interactions between medium, message and political identification on elaboration, psychological reactance or attitude change (all p > .08 or higher).
However, they did reveal significant main effects for message strategy on reactance, F (1, 488) = 11.71, p < .001, η2 p = .023, and on attitude change, F (1, 488) = 4.07, p = .044, η2 p = .008, but not on elaboration, F (1, 488) = 0.01, p = .988, η2 p < .001. Pairwise comparisons based on estimated marginal means revealed the directive message strategy produced greater levels of reactance, M = 12.83, SE = .37, than the non-directive strategy, M = 11.07, SE = .36, MD = 1.77, SE = .52; 95% CI [.75, 2.78]; p ≤ .001, as well as greater levels of attitude change, M = 1.73, SE = .25 versus M = 1.04, SE = .24, MD = 0.69, SE = .34, 95% CI [.18, 1.36], p = .044.
Between-subjects tests also revealed significant main effects for political identity on elaboration, F (1, 488) = 28.98, p < .001, η2 p = .056, and reactance, F (1, 491) = 173.50, p < .001, η2 p = .261, but not on attitude change, F (1, 491) = 1.64, p = .201, η2 p = .003. K matrix contrasts revealed that Democrats expressed greater levels of elaboration than Republicans, MD = 3.40, SE = .64, 95% CI [2.15, 4.65], p < .001, while Republicans exhibited greater levels of psychological reactance than Democrats, MD = 6.87, SE = 0.52, 95% CI [5.86, 7.89], p < .001.
The between-subjects tests also provided details of the interaction between message strategy and political identity revealed by the omnibus tests. This interaction had a significant effect on psychological reactance, F (1, 488) = 9.56, p = .002, η2 p = .019, and a near-significant effect on attitude change, F (1, 488) = 3.73, p = .054, η2 p = .008, but no effect on elaboration, F (1, 488) = 0.02, p = .882, η2 p < .001. Plots of the interaction effects on reactance and attitude change are presented in Figures 4 and 5. The plots in Figure 4 suggest that the directive message produced greater levels of reactance among Republicans than the non-directive message but made no difference in terms of reactance among Democrats. The plots in Figure 5 suggest the directive message produced greater positive attitude change among Democrats than the non-directive message but that the two persuasive conditions produced similar amounts of attitude change among Republicans.
Testing of Hypotheses and Exploration of Research Questions
H1: Narrative Advocacy Messages Will Produce Positive Attitude Change
H1 stated that the narrative-inoculative advocacy messages would produce positive attitude change in favor of vaccination. This was tested using a paired-samples t-test to compare the mean of the participants’ attitudes toward COVID vaccination after exposure to an advocacy message to their attitudes toward COVID vaccination before exposure. The t-test found that the mean of post-exposure attitude, M = 66.79 (SD = 13.59) is greater than the mean of pre-exposure attitude, M = 65.33 (SD = 13.36), MD = 1.47 (SD = 3.57), 95% CI [1.15, 1.78]; t (495) = 9.16, p < .001 (one-sided), Cohen’s d = .411, 95% CI [.320, .503]. This supports H1.
H2: Directive Advocacy Messages Will Produce Greater Levels of Psychological Reactance
H2 stated that the directive advocacy messages would produce greater levels of psychological reactance than the non-directive advocacy messages. As noted above in the summary of the MANOVA results, there was a significant main effect for persuasion strategy on reactance, F (1, 488) = 11.71, p < .001, η2 p = .023. Pairwise comparisons based on estimated marginal means revealed the directive message strategy produced greater levels of reactance, M = 12.83, SE = .37, than the non-directive message, M = 11.07, SE = .36; MD = 1.77, SE = .52; 95% CI [.75, 2.78]; p ≤ .001. This supports H2.
H3: Negative Correlation of Reactance with Change in Attitude Toward Vaccination
RQ1: Positive or Negative Effects of Elaboration on Attitude Change?
H3 stated that, controlling for other factors, psychological reactance would be negatively correlated with change in attitude toward vaccination after exposure to a vaccination advocacy message. RQ1 asked whether, controlling for other factors, elaboration would be positively or negatively correlated with attitude change. The hypothesis and research question were tested using a regression analysis with attitude change as the dependent variable. In addition to reactance and elaboration as predictor variables, pre-message-exposure attitude, message strategy, medium and political identity were included in the model as control variables. Means, standard deviations and bivariate correlations for this analysis are presented in Table 4. Regression coefficients are presented in Table 5.
Means, Standard Deviations, and Bivariate Correlations of Key Variables.
Note. N = 496.
p < .05. **p < .01. ***p < .001.
Regression Coefficients of Predictors of Attitude Change.
Note. DV = attitude change; § = partial correlation. R = .263, R2 = .069, SE = 3.46, F (6, 489) = 6.06, p < .001.
p < .05. **p < .01. ***p < .001.
As can be seen in Table 5, controlling for the other factors, psychological reactance and elaboration are both significant predictors of attitude change. With a correlation coefficient of B = −.09, SE = .04, p < .05, reactance is negatively correlated with attitude change such that for each one-unit increase in level of reactance, there is a negative change in attitude toward vaccination of .09 units. Thus, H3 is supported. In answer to RQ1, with a correlation coefficient of B = .07, SE = .03, p < .01, elaboration is positively correlated with attitude change such that for each 1-unit increase in level of elaboration, there is a positive change in attitude of .07 units.
H4: Political Identification Will be Correlated with Attitude Toward COVID-19 Vaccination
H4 stated that those identifying as Liberal or Democrat will exhibit more favorable attitudes toward COVID-19 vaccination than those identifying as Conservative or Republican. Bivariate correlations revealed positive Pearson correlations of r = .589 and r = .586 for the ideology (9-point Conservative to Liberal scale) and identity (Republican vs. Democrat dichotomy) variables with attitude toward vaccination.
A one-way analysis of variance (ANOVA) with pre-message-exposure attitude toward COVID-19 vaccination as the dependent variable found a significant effect for political ideology, F (8, 487) = 36.41, p < .001, η2 = .374. A similar one-way ANOVA also found a significant effect for political identity, F (1, 494) = 258.82, p < .001, η2 = .344. Means plots illustrating these relationships are presented below in Figures 6 and 7. These findings support H4.
Means of attitude toward COVID-19 vaccination (pre-message exposure) by political ideology.
Means of attitude toward COVID-19 vaccination (pre-message exposure) by political identity.
RQ2: Effects of Rich Versus Lean Media on Elaboration and Reactance
RQ2 asked whether advocacy messages presented using rich media (i.e., video) would produce greater levels of (RQ2a) message engagement (i.e., elaboration) and/or (RQ2b) psychological reactance than messages presented using lean media (i.e., static text). The MANOVA analyses detailed above identified no significant main or interaction effects for the media variable, indicating no significant differences in levels of elaboration, F (1, 488) = 2.63, p < .105, η2 = .005, or psychological reactance, F (1, 488) = 0.02, p < .890, η2 < .001, produced by the advocacy messages whether they were presented in video or static-text formats.
RQ3: Effects of Politics on Elaboration and Reactance
RQ3 asked how political identity would affect the underlying message processing factors of elaboration and psychological reactance. The between-subjects MANOVA analyses detailed above identified strongly significant effects of the dichotomous political identity variable on both, with Democrats exhibiting significantly greater levels of elaboration than Republicans, MD = 3.40, SE = .64, 95% CI [2.15, 4.65], p < .001, and Republicans exhibiting significantly greater levels of psychological reactance than Democrats, MD = 6.87, SE = 0.52, 95% CI [5.86, 7.89], p < .001. For additional exploration of this question, one-way ANOVAs were conducted using the political ideology variable as a nine-level categorical independent variable with elaboration and psychological reactance as the dependent variables. These tests showed significant effects for both elaboration, F (8, 487) = 2.75, p = .006, η2 = .043, and psychological reactance, F (8, 487) = 22.22, p < .001, η2 = .267. Plots of the means of elaboration and reactance produced by the different levels of the political ideology and political identity variables are provided in Figures 8 and 9.
Means of post-message exposure elaboration by political ideology and political identity.
Means of post-message exposure psychological reactance by political ideology and identity.
Mediation Analysis with Reactance as Mediator and Political Identity as Moderator
To test for other interactive effects of political identity on elaboration and reactance, an analysis of moderated mediation was conducted using the PROCESS 4.2 macro Model 7 in SPSS (Hayes, 2022) with elaboration as the dependent variable, message strategy as the primary independent variable, psychological reactance as a mediating factor and political identity as a moderating factor. The results showed that the effects of the different message strategies on elaboration was mediated by reactance. As shown in Table 6, the unstandardized coefficients between message strategy and elaboration were not significant, B = .47, SE = .57, p = .415, indicating no direct effect of message on elaboration. However, the unstandardized coefficients between message strategy and reactance were significant, B = 6.70, SE = 1.80, p < .001, as were the unstandardized coefficients between reactance and elaboration, B = −.35, SE = .05, p < .001, and the R2 value for the elaboration model was highly significant, R2 = .11, F (2, 493) = 29.04, p < .001, suggesting an indirect effect of message strategy on elaboration through reactance (message → reactance → elaboration). As indicated in Table 6 and illustrated below in Figure 10, the mediating effect of reactance was moderated by political identity such that Republicans exhibited significant differences in level of reactance conditional on receiving a directive or non-directive message, while Democrats exhibited no such differences. It is not coincidental that the moderated mediation effect shown in Figure 10 is the same interaction effect shown in Figure 4.
Moderated Mediation Analysis: Effects of Message on Elaboration Mediated by Reactance, Moderated by Political Identity.
Note. PID = political identity. Unstandardized coefficients.
p < .05. **p < .01. ***p < .001.
Moderated mediation: Psychological reactance by message strategy and political identity.
RQ4: Effects of Message, Medium, Politics, and Message Processing on Attitude Change
RQ4 asked how the complicated relationships among message strategy (directive or non-directive), medium (rich or lean), elaboration, psychological reactance and political identity would affect attitude change regarding vaccination. This was explored broadly through the MANOVA, regression, mediation and other analyses detailed above. Drawing on those analyses, the examination of these relationships was parsed as a series of more specific questions.
RQ4. 1: Does Medium Make a Difference?
A paired-samples t-test found that message exposure produced positive change in attitude toward vaccination, MD = 1.47 (SE = 3.57), t (495) = 9.16, p < .001 (one-sided), Cohen’s d = .411, which confirmed H1, but which did not reveal the factors influencing the change. The MANOVA detailed above revealed no main effects for medium on elaboration, F (1, 488) = 2.63, p = .101, η2 p = .005; or psychological reactance, F (1, 488) = 0.02, p = .890, η2 p < .001; or attitude change, F (1, 488) = 1.97, p = .161, η2 p = .004. Whether the advocacy messages were presented in video or static-text formats seemed to make no difference.
RQ4. 2: Does Message Strategy Make a Difference?
The MANOVA did identify main effects for message strategy on attitude change, F (1, 488) = 4.07, p = .044, η2 p = .008; and psychological reactance, F (1, 488) = 11.71, p < .001, η2 p = .023; but not on elaboration, F (1, 488) = 0.00, p = .988, η2 p < .001. Pairwise comparisons revealed that the directive message strategy produced greater positive attitude change than the non-directive message strategy, MD = .691, SE = 0.34, 95% CI [.18, 1.36], p = .044. The directive message strategy also produced greater levels of psychological reactance than the non-directive message, MD = 1.77, SE = 0.52, 95% CI [.75, 2.78], p < .001.
RQ4. 3: Does Political Identity Make a Difference?
The MANOVA did not identify any main effects for political identity on attitude change, F (1, 488) = 1.78, p = .183, η2 p = .004; but it did identify main effects on psychological reactance, F (1, 488) = 177.71, p < .001, η2 p = .267; and elaboration, F (1, 488) = 28.55, p < .001, η2 p < .055. Pairwise comparisons revealed that Republicans exhibited greater levels of psychological reactance than Democrats, MD = 6.87, SE = .52, 95% CI [5.86, 7.89], p < .001; while Democrats exhibited greater levels of elaboration than Republicans, MD = 3.40, SE = .64, 95% CI [2.15, 4.65], p < .001.
The MANOVA also found a significant interaction effect between political identity and message strategy on reactance, F (1, 488) = 9.56, p = .002, η2 p < .019, and a non-significant interaction effect with message strategy on attitude change, F (1, 488) = 3.73, p = .054, η2 p < .008. Plots of these interaction effects are shown above in Figures 4 and 5. They suggest that the directive message strategy produced greater levels of psychological reactance than the non-directive strategy among Republicans, but that there was no difference in levels of reactance stimulated by the different message conditions among Democrats. They also showed that the directive message strategy produced greater levels of positive attitude change than the non-directive strategy among Democrats, but that there was no difference in levels of attitude change stimulated by the different message conditions among Republicans.
Discussion
With less than 23% of U.S. adults having received the recommended bivalent COVID-19 vaccination (CDC, 2024), it is reasonable to suggest that current vaccination advocacy efforts have room for improvement. In pursuit of that goal, the current study tested the effectiveness of two variations of a novel narrative inoculation advocacy strategy that may provide a model for future vaccination advocacy and other health promotion efforts. Of equal or possibly greater value, the current study drilled down into the processes of attitude change by examining the effects and interactions of the message strategies on two potential mediators of attitude change, elaboration and psychological reactance, while also accounting for the influence of political identity, one of the strongest factors involved in COVID-19 vaccination decision-making (Pew, 2024a, 2024c).
The current study compared the relative effectiveness of directive and non-directive versions of personal narrative advocacy messages that used persuasion strategies adapted from studies in narrative persuasion (Moyer-Gusé, 2008; Moyer-Gusé & Nabi, 2010) and inoculation theory (McGuire, 1964; Pfau, 1995). Specifically, both the directive and non-directive message conditions followed the inoculation formula of prefacing any behavioral implications with presentations of counterarguments, that is, reasons to not get vaccinated, and refutations against those counterarguments. The directive version of the message then ended with an explicit recommendation why the message recipient should get vaccinated, while the non-directive message ended with a description of a personal decision to get vaccinated and no behavioral recommendation. The study also examined the influences of message-carrying medium, that is, video or static text, and political identity of the message recipients on the processing and effects of the advocacy messages.
Results of the analyses showed that, overall, the narrative inoculation advocacy message strategy produced significant positive change in attitude toward vaccination. Results also showed that the directive message strategy produced greater levels of psychological reactance than the non-directive message and—somewhat counterintuitively, because psychological reactance was negatively correlated with attitude change—the directive message strategy produced greater positive attitude change than the non-directive message strategy.
The results supported the notion that political identity must be regarded as an important factor in vaccination advocacy, considering its effects on attitude toward vaccination and on message processing factors such as psychological reactance and elaboration. Surprisingly, however, political identity had little direct effect on attitude change.
Theoretical Contributions and Practical Implications
The effects of narratives, inoculation theory, elaboration and psychological reactance have often been studied by health communication researchers, but rarely together (Compton et al., 2016; Miller et al., 2013; Niederdeppe et al., 2015). The current study breaks new ground by examining their combined effects as moderated by political identity and in the context of vaccination advocacy. The study makes several theoretical contributions to the ongoing research on health promotion and persuasion. By framing the counter-argument → refutation → persuasion message strategy of inoculation theory—and classical sales—in a personal narrative, the research demonstrates potential means for mitigating potentially negative effects of directive advocacy efforts. Testing the effects of message delivery in both rich and lean medium formats may not have identified any significant differences between the two, but the results suggest relatively equal effectiveness for both, which may help in media planning efforts. In addition, there is theoretical as well as practical value in finding that Republicans and Democrats processed the messages differently in terms of reactance and elaboration but that there were no differences in the effects of the messages in terms of attitude change. Furthermore, by identifying how political identity moderates the mediating role of reactance on message elaboration, the study contributes to the understanding of the underlying processes of persuasion.
Study Limitations and Implications for Future Research
Conducted as an online experiment with participants recruited from a pre-screened pool of experienced survey takers, the study’s findings must be considered in light of the ways in which the sample differs from the U.S. population as whole. In particular, the sample was more highly educated, with more than 60% having graduated from college compared to less than 40% for the nation as a whole (U.S. Census Bureau, 2022). Furthermore, with 68.8% of participants indicating preferences for an unnamed Democrat versus just 31.3% for an unnamed Republican, the sample was more Liberal than the nation as a whole as well. These factors may have led to more favorable attitudes toward vaccination, but the results suggested they had a limited effect on levels of attitude change in response to the vaccination advocacy messages.
The difference between an online experiment and the real world is a standard limitation for many studies, but it is of particular concern in the processing of persuasive messages. In an online experiment, the audience cannot avoid the experimenters’ messages and may feel somewhat obligated to pay attention to them. In the real world, people feel no such obligation and often pay regular monthly fees to avoid such messages in their streaming media services. The impact of any message strategy is likely to be dependent as much on its dissemination strategy as on its persuasive efficacy.
The message conditions themselves were a source of limitation in this study. With an infinite variety of message types, verbal arrangements, and spokesperson choices that could have been tested, it is possible the results of this study cannot be generalized beyond the specific combinations tested. Intended to provide similarity to an important target audience, the spokesperson used for the video message conditions and as the photo model for the static message conditions was a talented young student chosen because of his acting ability. His appearance, verbal skills and facial expressions may have produced positive or negative reactions unrelated to the persuasive messages he delivered. In addition, although the study was conducted as a pre-test/post-test experiment, inclusion of an additional non-persuasive control condition may have provided useful data.
Although there was no essential difference in the levels of attitude change produced by the advocacy messages between Republicans and Democrats, there were differences in the levels of elaboration and psychological reactance produced in the different partisans by the different message conditions, and elaboration and psychological reactance were both identified as significant predictors of attitude change. A future study could explore these factors further by testing separate messages tailored expressly for the different partisanships, both explicitly designed to reduce reactance and increase elaboration.
Conclusion: Support for Narrative Inoculation
This study provides support for further applications of combinations of inoculation theory and narrative persuasion for crafting health promotion strategies as well as for persuasion strategies in general. Regarding directive versus non-directive approaches and media planning, health communicators can infer that message exposure can be more important than message type or medium, and that directive messages—even at the risk of being construed as arm-twisting—may offer advantages in clarity and urgency despite the potential for reactance. The results suggest that narrative inoculation strategies can be used not just to defend against attitude change but to advocate for attitude change (Gans & Zhan, 2023). When it comes to promoting vaccination against the COVID-19 virus, and perhaps advocacy for a wide range of other worthy causes, inoculation appears to be a healthy strategy.
Footnotes
Author Note
All necessary permission and IRB approvals have been obtained.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Research Enhancement Program grant from the University of Texas at Arlington, but received no specific grant from any other funding agency in the public, commercial or not-for-profit sectors.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Data associated with this research is available upon reasonable request with written permission from the Office of Regulatory Services, University of Texas at Arlington. Email: