Estimating Vaccine Hesitancy in Colorado by Using Immunization Information System Data

Abstract

Objectives:

Vaccine hesitancy is a complex issue that threatens global health. We used data from the Colorado Immunization Information System (CIIS) to quantify vaccine hesitancy.

Methods:

We examined immunization records from CIIS for patients age 2 up to 9 months to estimate vaccine hesitancy by tabulating the number of doses received per visit and comparing it with the number of expected doses based on recommendations of the Advisory Committee on Immunization Practices. We calculated the percentage of patients in each vaccine hesitancy group who were up to date on the 7-antigen series by age 35 months. We examined the distribution of vaccine-hesitant populations among vaccination providers who report to CIIS to estimate the difference in vaccine-hesitant patient populations among vaccination providers in Colorado.

Results:

Of 201 450 patients, 5147 (2.6%) consistently limited the number of shots received at each visit as compared with recommendations from the Advisory Committee on Immunization Practices; 166 927 (82.9%) patients did not limit the number of shots received; 5693 (2.8%) limited the number of shots received at >1 visit but not all visits; and 23 683 (11.8%) limited the number of shots received at only 1 visit. We found differences in vaccine hesitancy distributions among certain Colorado vaccination providers.

Conclusions:

Immunization information system data, although sometimes incomplete, offer an opportunity to investigate state-level vaccine hesitancy. Areas of future research include performing similar analyses over time and determining geographic and socioeconomic factors that contribute to vaccine hesitancy.

Keywords

vaccines vaccine hesitancy record incompleteness shot limiting

Since their inception, vaccines have played a crucial role in preventing diseases. Creating immunity to a disease through vaccination is a powerful and cost-effective method to prevent morbidity and mortality among children and adults. Vaccinating children protects individuals and communities because certain people cannot receive vaccines, including people who cannot receive vaccines for medical reasons and children who are too young to be vaccinated. The vaccination schedule recommended by the Advisory Committee on Immunization Practices (ACIP) outlines immunization recommendations for children, adolescents, and adults.¹ From age 2 up to 9 months, ACIP recommends vaccinations for hepatitis B, rotavirus, DTaP (diphtheria, tetanus, and acellular pertussis), Haemophilus influenzae type b (Hib), pneumococcal conjugate 13 vaccine, and inactivated poliovirus vaccine (IPV), as well as an inactivated influenza vaccination at age ≥6 months. The Centers for Disease Control and Prevention publishes the ACIP schedule, which is endorsed by the American Academy of Pediatrics and the American Academy of Family Physicians.² ACIP recommendations are scientifically backed and evidence based.

Although the ACIP recommended schedule is followed by many parents, some choose alternate immunization schedules or choose not to vaccinate their children. Alternative immunization schedules are not developed with the same scientific standards as the ACIP schedule. Reasons given by parents for not following the ACIP schedule include a lack of confidence in the safety or effectiveness of the immunization process and feelings of complacency because vaccination has effectively eradicated some diseases.^3
-5 In the United States, these attitudes have led to consequences such as a recent decline in coverage for the measles, mumps, rubella (MMR) vaccine and a steep rise in measles cases.⁶ The World Health Organization categorized vaccine hesitancy as 1 of the top 10 threats to global health in 2019.⁷

State vaccination policies in Colorado allow parents to exempt their children from receiving vaccinations required to attend school and childcare for medical and nonmedical reasons.^8,9 Based on 2019 National Immunization Survey results, the percentage of children in Colorado who had received ≥1 MMR vaccine by age 19 months was 88.4%, which is below the 92%-94% community threshold needed for protection against measles.¹⁰

The Colorado Department of Public Health and Environment (CDPHE) maintains the Colorado Immunization Information System (CIIS), a confidential, population-based, secure computerized system that collects and consolidates person-level vaccine and exemption data for residents in Colorado of all ages from various sources.¹¹ Per an Internal Immunization Information Systems Annual Report submitted to the Centers for Disease Control and Prevention, the number of people in CIIS exceeded Colorado’s state population, according to 2020 US Census data (113%) (unpublished report, CDPHE, 2020).

Despite the relative maturity of CIIS, vaccine hesitancy is a poorly quantified issue in Colorado, in part because CIIS has historically been a voluntary reporting system.⁹ However, the importance of estimating hesitancy based on these data cannot be understated.¹² Immunization information system (IIS) data, although in some cases weakened by reporting gaps and missing data, have important benefits when compared with survey data.¹³ IIS data allow public health to learn important information about vaccination coverage across large areas without limitations related to time or resources. Historically, related research projects have primarily used qualitative methods such as self-reported survey data to quantify the prevalence of alternative vaccination schedule use.^14,15 Few studies that use IIS data to examine undervaccination exist, suggesting that this resource has been used infrequently.^16
-19 To our knowledge, quantifying vaccine hesitancy is a relatively novel use of IIS data.

The combination of IIS data and the recommended ACIP schedule provides a unique opportunity to evaluate vaccine hesitancy. Studying vaccine hesitancy in Colorado is especially important in the context of recent legislation. Senate Bill 20-163, which was signed in 2020, updated CIIS to a mandatory reporting system and standardized the nonmedical exemption process for school-required vaccines.⁹ Historically, no official process or standard form has been required to report exemption information to CDPHE.

Even with these improvements, public health agencies’ understanding of vaccine hesitancy based on IIS data may be affected by record incompleteness. Some vaccination records in CIIS are not up to date because the patient has not received recommended vaccines. Others are not up to date because the records are incomplete (ie, missing vaccines that the patient has actually received) or split between ≥2 records because of duplicate records. Even when a vaccination provider regularly submits vaccination data to CIIS, data transfer delays and failures are possible. While the extent of incomplete records in CIIS is poorly understood, it is likely that incomplete records affect CDPHE’s ability to accurately evaluate immunization coverage with CIIS data. Additionally, patient migration is a large source of change in Colorado data, resulting in incomplete, active CIIS records persisting after a patient has moved out of Colorado. These challenges in Colorado and other jurisdictions have contributed to limitations in conducting research about the use of immunization records to estimate vaccine hesitancy.

Although CDPHE is likely missing data on children exempted from vaccines, we are still able to make inferences about vaccine hesitancy by studying vaccination records in CIIS. In one of the first studies to use IIS data to quantify the proportion of children consistently delaying receipt of vaccines, Robison et al analyzed IIS data in Oregon to describe the prevalence of the use of alternative schedules, categorizing patients as non–shot limiters (ie, patients who did not limit the number of shots received), episodic shot limiters (ie, patients who limited the number of shots received during >1 visit but not all visits), or consistent shot limiters (ie, patients who limited the number of shots received at every visit).²⁰ A subsequent analysis reproducing these hesitancy categorizations with CIIS patients indicated an increase in the number of Colorado counties with ≥10% of their children following an alternative vaccination schedule in 2016 vs 2007, thus confirming vaccine hesitancy as an issue in Colorado requiring further study (unpublished report, CDPHE, 2016).

The quantitative classification of children into hesitancy categories is an important step toward increasing vaccination coverage across Colorado. Increased knowledge about vaccine-hesitant populations in Colorado would allow surveillance, provider outreach efforts, and public health education campaigns to be efficiently directed toward populations at higher-than-average risk of vaccine-preventable diseases. In the pursuit of these initiatives, our analysis used CIIS data to estimate the relative proportions of Colorado children age 2 up to 9 months in various categories of vaccine hesitancy. The primary objective was to build on work by Robison et al to establish a baseline method to evaluate relative levels of vaccine hesitancy in Colorado.

Methods

In June 2020, we selected a study population that included children with records in CIIS who were born from March 3, 2013, through March 1, 2017 (patients aged ≥3 to <7 years on March 1, 2020). We chose this date range because it was before the observed decrease in childhood vaccinations that occurred during the COVID-19 pandemic beginning mid-March 2020.^21,22 We excluded patients who had moved out of Colorado, died, or had residential addresses outside Colorado. The CDPHE Institutional Review Board determined that this project did not meet the definition of human subjects research and, thus, did not require prior review and approval.

We excluded patients without ≥2 non–influenza visits recorded in CIIS before age 9 months (274 days). We chose these criteria to decrease the likelihood that patients with incomplete records whose migration out of Colorado had not been captured in CIIS were included in the study. The American Immunization Registry Association’s Modeling of Immunization Registry Operations Workgroup recommends that IIS-based analyses exclude patients with no contact with their records in ≥10 years, reasoning that these patients have likely migrated out of the IIS jurisdiction.²³ Excluding patients without ≥2 noninfluenza visits recorded in CIIS before age 9 months builds on workgroup recommendations to be more relevant for a younger patient cohort.

We sourced complete immunization records for the population from age 2 months (61 days) up to but not including age 9 months (273 days) at vaccination from CIIS. We included data on doses of hepatitis B, rotavirus, DTaP, Hib, pneumococcal conjugate 13 vaccine, IPV, and inactivated influenza vaccination. We grouped the cohort into 4 vaccine hesitancy categories based on the number of unique shots that a patient received in a single visit, defined as a unique vaccination date in CIIS.²⁰ Because a child may be up to date on all recommended vaccines except influenza at a given visit, we excluded data on single-dose visits during which a patient received influenza vaccine.

Consistent shot limiters received ≤2 shots for all visits from age 2 up to 9 months. Episodic shot limiters received ≤2 shots for >1 visit but not all visits. Non–shot limiters received >2 shots for all visits. One-time shot limiters received ≤2 shots for only 1 visit from age 2 up to 9 months. The definitions of consistent shot limiters, episodic shot limiters, and non–shot limiters are based on work by Robison et al.²⁰

We analyzed the use of single-antigen vaccines compared with combination vaccines in each hesitancy group. This analysis controlled for the administering provider because providers may have only combination or single-antigen vaccines on hand for a given antigen.²⁴ We excluded data on patients who received vaccines from >1 provider from age 2 up to 9 months. We grouped hesitancy categories into “likely vaccine hesitant” (consistent and episodic shot limiters) and “likely not vaccine hesitant”' (one-time shot limiters and non–shot limiters). We excluded providers from this analysis who had not administered both single-antigen and combination vaccines to both likely vaccine-hesitant and likely non–vaccine-hesitant patients. To estimate the association between vaccine hesitancy group and dose type received, we fit a logistic regression model to a random sample of 25 vaccination providers (118 927 doses), controlling for the administering provider. We calculated odds ratios and 95% CIs from the model coefficient for hesitancy category.

We determined the percentage of patients in each hesitancy group who had received a complete recommended vaccination series by age 35 months to reproduce National Immunization Survey–Child methodology, which specifies a 7-antigen series (4 doses of DTaP, 3 doses of IPV, 1 dose of MMR, 3 doses of Hib, 3 doses of hepatitis B, 1 dose of varicella, and 4 doses of pneumococcal conjugate vaccine).¹⁰ We used the Pearson χ² test to determine the significance of the relationship between hesitancy category and up-to-date status.

To estimate the likelihood that certain vaccination providers in Colorado have a higher percentage of vaccine-hesitant patients than other vaccination providers in Colorado, we analyzed the distribution of the percentage of patients categorized as consistent shot limiters across 389 providers in CIIS. We excluded data on patients who received vaccines from >1 vaccination provider from age 2 up to 9 months and vaccination providers with <25 total patients. We calculated the proportion of providers whose patient population did not include any consistent shot limiters, the proportion whose patient population comprised >0% to 10% of consistent shot limiters, and the proportion whose patient population comprised >10% of consistent shot limiters.

We conducted all analyses using R version 4.0.3 (R Core Team).

Results

A total of 203 228 children in CIIS met the inclusion criteria. Of those, 201 450 children had received ≥1 immunization of DTaP, IPV, MMR, Hib, hepatitis B, varicella, or pneumococcal conjugate vaccine from age 2 up to 9 months. A total of 1 977 374 vaccines were administered to the cohort when they were age 2 up to 9 months. A mean of 9.8 and median of 11 vaccinations were administered per patient while they were age 2 up to 9 months.

The smallest number of patients (n = 5147, 2.6%) was categorized as consistent shot limiters, 5693 (2.8%) as episodic shot limiters, 23 683 (11.8%) as one-time shot limiters, and 166 927 (82.9%) as non–shot limiters (Table).

Table.

Patients in each vaccine hesitancy category, based on the number of vaccinations received per visit, children aged 2 up to 9 months, Colorado, 2020^a

Vaccine hesitancy category^b	No. (%)
Total	201 450 (100.0)
Consistent	5147 (2.6)
Episodic	5693 (2.8)
One-time	23 683 (11.8)
Non–shot limiting	166 927 (82.9)

Includes patients with ≥1 immunization of diphtheria, tetanus, and acellular pertussis; inactivated poliovirus; measles, mumps, rubella; Haemophilus influenzae type b; hepatitis B vaccine; varicella; or pneumococcal conjugate vaccine from age 2 up to 9 months. Patient vaccination records were sourced from the Colorado Immunization Information System.¹¹

Consistent shot limiters are patients who limited the number of shots received at every visit as compared with the recommendations of the Advisory Committee on Immunization Practices.¹ Episodic shot limiters limited the number of shots received at >1 visit but not all visits. One-time shot limiters limited the number of shots received at only 1 visit. Non–shot-limiting patients did not limit the number of shots received at any visit.

We found a significant association between vaccine hesitancy group and receipt of a single-antigen vaccine dose. The odds of receiving a single-antigen vaccine dose were 1.63 times higher among likely vaccine-hesitant patients than among likely non–vaccine-hesitant patients, controlling for the administering provider (odds ratio = 1.63; 95% CI, 1.45-1.85).

In examining the percentage of patients in each vaccine hesitancy category who had received the full 7-antigen series by age 35 months, consistent shot limiters had the lowest percentage (22.9%, n = 1179) and non–shot limiters had the highest percentage (78.7%, n = 131 431); 76.7% (n = 18 175) of one-time shot limiters and 67.3% (n = 3832) of episodic shot limiters were up to date (Figure 1). The relationship between hesitancy category and up-to-date status was significant (χ²₃ = 9015.8; n = 201 450; P < .001).

Figure 1.

Percentage of patients in each vaccine hesitancy category who had received the full 7-antigen series by age 35 months, Colorado, 2020: 4 doses of diphtheria, tetanus, and acellular pertussis; 3 doses of inactivated poliovirus; 1 dose of measles, mumps, and rubella; 3 doses of Haemophilus influenzae type b; 3 doses of hepatitis B vaccine; 1 dose of varicella; and 4 doses of pneumococcal conjugate vaccine.¹¹ The denominator for each percentage is the total number of patients categorized in each category. Consistent shot limiters are patients who limited the number of shots received at every visit as compared with recommendations from the Advisory Committee on Immunization Practices.¹ Episodic shot limiters limited the number of shots received at >1 visit but not all visits. One-time limiters limited the number of shots received at only 1 visit. Non–shot-limiting patients did not limit the number of shots received at any visit. Patient vaccination records were sourced from the Colorado Immunization Information System.¹¹

Of 389 providers, 116 (29.8%) had no patients categorized as consistent shot limiters, 257 (66.1%) had a patient population comprised of >0% to 10% of consistent shot limiters, and 16 (4.1%) had a population comprised of >10% of consistent shot limiters (Figure 2).

Figure 2.

Proportion of 389 vaccination providers whose patient populations were comprised of 0% of consistent shot limiters, >0% to 10% of consistent shot limiters, and >10% of consistent shot limiters, Colorado, 2020. Consistent shot limiters are patients who limited the number of shots received at every visit as compared with recommendations from the Advisory Committee on Immunization Practices.¹ Patients who received vaccines from >1 vaccination provider from age 2 to 9 months and vaccination providers with <25 total patients were excluded. Patient vaccination records were sourced from the Colorado Immunization Information System.¹¹

Discussion

This evaluation contributes to our ability to identify vaccine-hesitant populations by using IIS data. The combination of the ACIP schedule and IIS vaccination records provides a unique opportunity to analyze schedule adherence and vaccine hesitancy. Methods that identify populations and providers with higher-than-average levels of vaccine hesitancy allow health care providers to improve the up-to-date status of their patients, therefore decreasing the risk for vaccine-preventable diseases.^25
-27 This evaluation found that non–shot limiters and one-time shot limiters had the highest percentage of being up to date by age 35 months. This finding suggests that patients who fully or closely follow the recommended ACIP schedule are more likely to be up to date than those who follow alternative schedules.

This evaluation also found differences among providers in the proportion of their patient populations that fall into the consistent shot-limiting category. These findings support research suggesting that some providers may be more flexible than others in allowing parents to refuse recommended vaccinations for their children or follow an alternative immunization schedule.^26,27 A patient population consisting of >10% of consistent shot limiters could be used as a threshold to determine which providers would benefit most from outreach from Colorado public health agencies about communication strategies with vaccine-hesitant parents.

Limitations

There are limitations in interpreting analyses that rely on IIS data. First, it is difficult to confirm that IIS data contain complete immunization records because of patients who change health care providers and move out of state, resulting in unrecorded mobility within a state’s IIS. This unrecorded mobility can contribute to denominator inflation, a situation in which people who no longer reside in a geographic area are still included as active patients in the area.²⁸ Second, information about visits during which a patient refused all vaccines is not recorded in the system. Our analysis was limited to visits in which patients received ≥1 vaccine; therefore, it may be biased against the detection of shot-limiting patients. Third, while CIIS data are a powerful tool for evaluating immunization rates and improving public health, they may not represent all vaccinations that Colorado residents have received. Finally, because only data from CIIS were evaluated, the findings of this project are not generalizable to other states’ IIS data. Multistate studies would be required to determine if our findings are reproducible.

Conclusion

IIS data are an important and largely untapped resource that can give public health a better understanding of vaccine-hesitant populations in Colorado. The potential to find and provide outreach to vaccine-hesitant populations is especially critical in the context of increases in COVID-19 vaccine availability starting in 2021 and decreases in routine vaccination observed during the COVID-19 pandemic.²⁹

Our study details how the number of shots per visit during the time when childhood vaccinations are recommended can be used as an effective method for estimating vaccine hesitancy with IIS data. The significant association between likely vaccine-hesitant groups (consistent and episodic shot limiters) and the receipt of single-antigen vaccines could serve as additional validation for this method, because concern about the use of combination vaccines in vaccine-hesitant populations has been reported.³⁰ We additionally found that some providers in Colorado have higher percentages of shot-limiting patients than other providers. Public health commonly influences vaccine acceptance through strong provider–patient-level relationships; with this knowledge, public health may be able to offer targeted outreach to providers with a tiered approach corresponding to patient levels of hesitancy, potentially resulting in more efficient vaccine-preventable disease control.^31,32

Future work related to this project includes performing similar analyses over time and determining geographic and socioeconomic factors that contribute to vaccine hesitancy. To further public health’s understanding of how vaccine hesitancy has changed in Colorado over time, one aspect of this future work will examine the distribution of vaccine hesitancy categories in patient cohorts during a 10-year period. Additional work is necessary to understand the geographic distribution of vaccine hesitancy in Colorado, including analyzing how factors such as county population level and distance from the nearest vaccinating provider relate to vaccine hesitancy categorization. Future research will also focus on determining how record incompleteness interferes with our ability to estimate vaccine hesitancy from IIS data.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Centers for Disease Control and Prevention’s Immunization and Vaccines for Children Program (funding opportunity CDC-RFA-IP19-1901).

ORCID iD

Kimberly Campbell, BA

References

Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases. Child and adolescent immunization schedule: recommendations for ages 18 years or younger, United States, 2020. Accessed April 1, 2020. https://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolescent.html

Nadeau

Bednarczyk

Masawi

, et al. Vaccinating my way—use of alternative vaccination schedules in New York State. J Pediatr. 2015;166(1):151-156. doi:10.1016/j.jpeds.2014.09.013

Wheeler

Buttenheim

AM.

Parental vaccine concerns, information source, and choice of alternative immunization schedules. Hum Vaccine Immunother. 2013;9(8):1782-1789. doi:10.4161/hv.25959

Ventola

CL.

Immunization in the United States: recommendations, barriers, and measures to improve compliance: part 1—childhood vaccinations. P T. 2016;41(7):426-436.

Wang

Baras

Buttenheim

AM.

“Everybody just wants to do what’s best for their child”: understanding how pro-vaccine parents can support a culture of vaccine hesitancy. Vaccine. 2015;33(48):6703-6709. doi:10.1016/j.vaccine.2015.10.090

Vaccine hesitancy: a generation at risk. Lancet Child Adolesc Health. 2019;3(5):281. doi:10.1016/S2352-4642(19)30092-6

World Health Organization. Ten threats to global health in 2019. Spotlight. 2019. Accessed June 5, 2020. https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019

Ernst

Jacobs

ET.

Implications of philosophical and personal belief exemptions on re-emergence of vaccine-preventable disease: the role of spatial clustering in under-vaccination. Hum Vaccin Immunother. 2012;8(6):838-841. doi:10.4161/hv.19743

Colorado General Assembly. School entry immunization: concerning the modernization of the school entry immunization process, and, in connection therewith, making an appropriation. SB20-163 School Entry Immunization, 2020. Accessed November 8, 2020. https://leg.colorado.gov/bills/sb20-163

10.

Centers for Disease Control and Prevention. 2017 National Immunization Survey—Child. 2018. Accessed December 12, 2020. http://www.cdc.gov/vaccines/imz-managers/nis/datasets.html

11.

Colorado Department of Public Health and Environment. Colorado Immunization Information System. 2020. Accessed March 5, 2020. https://cdphe.colorado.gov/colorado-immunization-information-system-ciis

12.

Gianfredi

Moretti

Lopalco

PL.

Countering vaccine hesitancy through immunization information systems, a narrative review. Hum Vaccin Immunother. 2019;15(11):2508-2526. doi:10.1080/21645515.2019.1599675

13.

Hendrickson

Panchanathan

Petitti

Evaluation of immunization data completeness within a large community health care system exchanging data with a state immunization information system. J Public Health Manag Pract. 2015;21(3):288-295. doi:10.1097/PHH.0000000000000045

14.

Smith

Humiston

Parnell

Vannice

Salmon

DA.

The association between intentional delay of vaccine administration and timely childhood vaccination coverage. Public Health Rep. 2010;125(4):534-541. doi:10.1177/003335491012500408

15.

Dempsey

Schaffer

Singer

Butchart

Davis

Freed

GL.

Alternative vaccination schedule preferences among parents of young children. Pediatrics. 2011;128(5):848-856. doi:10.1542/peds.2011-0400

16.

Glanz

Newcomer

Narwaney

, et al. A population-based cohort study of undervaccination in 8 managed care organizations across the United States. JAMA Pediatr. 2013;167(3):274-281. doi:10.1001/jamapediatrics.2013.502

17.

Daley

Reifler

Shoup

, et al. Temporal trends in undervaccination: a population-based cohort study. Am J Prev Med. 2021;61(1):64-72. doi:10.1016/j.amepre.2021.01.037

18.

Newcomer

Freeman

Wehner

Anderson

Daley

MF.

Timeliness of early childhood vaccinations and undervaccination patterns in Montana. Am J Prev Med. 2021; 61(1):e21-e29. doi:10.1016/j.amepre.2021.01.038

19.

Michels

Freeman

Williams

, et al. Evaluating vaccination coverage and timeliness in American Indian/Alaska Native and non-Hispanic White children using state immunization information system data, 2015-2017. Prev Med Rep. 2022;27:101817. doi:10.1016/j.pmedr.2022.101817

20.

Robison

Groom

Young

Frequency of alternative immunization schedule use in a metropolitan area. Pediatrics. 2012;130(1):32-38. doi:10.1542/peds.2011-3154

21.

O’Leary

Trefren

Roth

Moss

Severson

Kempe

Number of childhood and adolescent vaccinations administered before and after the COVID-19 outbreak in Colorado. JAMA Pediatr. 2021;175(3):305-307. doi:10.1001/jamapediatrics.2020.4733

22.

Muhoza

Danovaro-Holliday

Diallo

, et al. Routine vaccination coverage—worldwide, 2020. MMWR Morb Mortal Wkly Rep. 2021;70(43):1495-1500. doi:10.15585/mmwr.mm7043a1

23.

AIRA Modeling of Immunization Registry Operations Work Group, eds. Management of Patient Active/Inactive Status in Immunization Information Systems: Replacement of 2005 Guidelines. American Immunization Registry Association; 2015.

24.

CT.

Combination vaccines: choices or chaos? A practitioner’s perspective. Clin Infect Dis. 2001;33(4):S367-S371. doi:10.1086/322575

25.

Szilagyi

Vann

Bordley

, et al. Interventions aimed at improving immunization rates. Cochrane Database Syst Rev. 2002;(4):CD003941. doi:10.1002/14651858.CD003941

26.

Bonville

Domachowske

Cibula

Suryadevara

Immunization attitudes and practices among family medicine providers. Hum Vaccin Immunother. 2017;13(11):2646-2653. doi:10.1080/21645515.2017.1371380

27.

Hurley

Bridges

Harpaz

, et al. Physician attitudes toward adult vaccines and other preventive practices, United States, 2012. Public Health Rep. 2016;131(2):320-330. doi:10.1177%2F003335491613100216

28.

Robison

SG.

Addressing immunization registry population inflation in adolescent immunization rates. Public Health Rep. 2015;130(2):161-166. doi:10.1177/003335491513000209

29.

SeyedAlinaghi

Karimi

Mojdeganlou

, et al. Impact of COVID-19 pandemic on routine vaccination coverage of children and adolescents: a systematic review. Health Sci Rep. 2022;5(2):e00516. doi:10.1002/hsr2.516

30.

Halsey

NA.

Safety of combination vaccines: perception versus reality. Pediatr Infect Dis J. 2001;20(11):S40-S44. doi:10.1097/00006454-200111001-00007

31.

Kestenbaum

Feemster

KA.

Identifying and addressing vaccine hesitancy. Pediatr Ann. 2015;44(4):e71-e75. doi:10.3928/00904481-20150410-07

32.

Eskola

Duclos

Schuster

MacDonald

NE.

How to deal with vaccine hesitancy?

Vaccine. 2015;33(34):4215-4217. doi:10.1016/j.vaccine.2015.04.043