Using an Emergency Department Syndromic Surveillance System to Evaluate Reporting of Potential Rabies Exposures,Illinois,2013-2015

Abstract

Objectives:

Mandatory reporting of potential rabies exposures and initiation of postexposure prophylaxis (PEP) allow local health authorities to monitor PEP administration for errors. Our objectives were to use an emergency department (ED) syndromic surveillance system to (1) estimate reporting compliance for exposure to rabies in suburban Cook County, Illinois, and (2) initiate interventions to improve reporting and reassess compliance.

Methods:

We queried ED records from 45 acute care hospitals in Cook County and surrounding areas from January 1, 2013, through June 30, 2015, for chief complaints or discharge diagnoses pertaining to rabies, PEP, or contact with a wild mammal (eg, bat, raccoon, skunk, fox, or coyote). We matched patients with ≥1 ED visit for potential rabies exposure to people with potential rabies exposure reported to the Cook County Department of Public Health. We considered nonmatches to have unreported exposures. We then initiated active surveillance in July 2015, disseminated education on reporting requirements in August and September 2015, and reassessed reporting completeness from July 2015 through February 2016.

Results:

Of 248 patients with rabies-related ED visits from January 2013 through June 2015, 63 (25.4%) were reported. After interventions were implemented to increase reporting compliance, 53 of 98 (54.1%) patients with rabies-related ED visits from July 2015 through February 2016 were reported. Patients with ED visits for potential rabies exposure were twice as likely to be reported postintervention than preintervention (risk ratio = 2.1; 95% CI, 1.6-2.8). The volume of potential rabies exposure cases reported to the health department from July 2015 through February 2016 increased by 252% versus the previous year.

Conclusions:

Potential rabies exposures and PEP initiation are underreported in suburban Cook County. ED syndromic surveillance records can be used to estimate reporting compliance and conduct active surveillance.

Keywords

rabies postexposure prophylaxis syndromic systems disease surveillance

Encephalitis associated with exposure to the rabies virus is fatal in almost all cases.¹ In 2014, a total of 6033 animal rabies cases (5.9% of animals tested) and 1 human rabies case were reported to the Centers for Disease Control and Prevention.² The low incidence of human rabies in the United States is attributed to elimination of the canine rabies variant, wildlife vaccination programs, and appropriate and timely rabies postexposure prophylaxis (PEP).^2,3 When administered in accordance with clinical guidelines, PEP is safe and highly effective.³ However, errors in administration are common.^4
–6

PEP should be administered as soon as possible after an exposure occurs, although brief delays may be warranted when the animal is available for testing or observation. On day 0, exposed people receive rabies immune globulin and the first dose of a 4-dose rabies vaccination series. Additional doses are administered on days 3, 7, and 14. A fifth dose is recommended for immunocompromised people on day 28. Rabies biologics should be administered in the proper body sites (ie, vaccine given in the deltoid area of adults or the anterolateral thigh of young children; rabies immune globulin infiltrated around the wound or at a site distant from the vaccination site).^3,7 Failure to adhere to this regimen can result in reduced immunologic response and, in worst-case scenarios, rabies illness and death.^8,9

In addition to errors in the timing and location of injections, PEP is often administered when prophylaxis is unnecessary (eg, when the animal is a low-risk species or available for testing).^4,10,11 In the United States, cost for the complete series of rabies biologics typically exceeds $3000.¹² Excessive PEP administrations are a substantial financial burden to the health care system¹³ and an inconvenience to the patient.

Mandatory reporting of potential rabies exposures allows public health authorities to monitor PEP administration and to encourage compliance with clinical guidelines. In 2013, PEP administration was reportable in 21 states, including Illinois.¹⁴ However, a study in King County, Washington, found that 71 of 189 (37.6%) instances of PEP administration were not reported from 2003 to 2006.¹⁵

In Illinois, 578 of 664 (87.0%) people reported with a potential rabies exposure in 2015 received at least 1 dose of PEP in an emergency department (ED) setting (Illinois Department of Public Health, unpublished data, 2015). The Cook County Department of Public Health operates an ED syndromic surveillance system that receives near–real-time data from all 20 acute care hospitals in the jurisdiction and 25 acute care hospitals in the surrounding area. Our objectives were to use ED syndromic surveillance records to (1) estimate reporting completeness for potential rabies exposures in suburban Cook County before and after an intervention to increase reporting compliance and (2) detect potential rabies exposures for active surveillance follow-up in conjunction with other initiatives to improve reporting.

Program Description

Potential exposures to rabies are reportable in Illinois.¹⁶ Reporting requirements are available on the websites of the Illinois Department of Public Health and the Cook County Department of Public Health^17,18 and are disseminated to health care providers electronically or through postal mail annually or biannually. Health care practitioners must notify local health authorities within 24 hours of treating a patient who meets the surveillance case definition. Most potential exposures to rabies are reported by infection preventionists at the hospital in which the patient was seen. Potential exposure to rabies is defined as any individual with ≥1 of the following criteria:

contact with a bat;

contact with a rabies-positive animal or person;

exposure to saliva, brain tissue, or cerebrospinal fluid of a suspected rabid animal or person;

any bite from a wild mammal, not including small rodents or rabbits;

any bite from a nonhuman primate;

any person who was in the same room as a bat and might be unaware that contact occurred; and

any person started on rabies PEP.¹⁶

Reported exposures are entered into the Illinois National Electronic Disease Surveillance System. Information collected includes patient demographic characteristics, circumstances of the exposure, and details of PEP administration, if PEP was provided. Communicable disease epidemiologists with the Cook County Department of Public Health review reported cases and determine if PEP was administered appropriately and correctly. If an error is detected (eg, PEP was administered in an incorrect body site, PEP was administered but not indicated, or PEP was indicated but not administered), an epidemiologist contacts either the patient or the provider and offers recommendations in accordance with current clinical guidelines. If PEP was indicated, patients are followed to ensure all doses are received.

Purpose of Evaluation

Although communicable disease reporting is mandatory for all health care providers in the United States, many do not report communicable diseases, because they assume that the laboratory confirming the diagnosis will notify the local health authority.^19,20 A potential rabies exposure is one of only a few notifiable conditions in Illinois for which no laboratory test can be ordered. Lack of reporting for potential rabies exposures prevents communicable disease epidemiologists from confirming the administration of PEP for appropriate circumstances, correcting errors in administration that could result in a reduced immunologic response, and ensuring that patients complete a PEP series when indicated. Underreporting also impedes local health authorities’ accurate assessment of the burden of disease and may create bias in identifying high-risk groups. To address these issues, we set up an evaluation with 2 goals: (1) estimate the extent of underreporting for potential rabies exposures by comparing reported cases with exposures detected in ED syndromic surveillance records and (2) determine whether interventions geared toward infection preventionists could improve reporting and, if so, by how much.

Methods

Evaluation of Preintervention Reporting Compliance

The Cook County Department of Public Health administers a local ED syndromic surveillance system using proprietary software developed by the Johns Hopkins Applied Physics Laboratory: the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE). All 20 acute care hospitals in suburban Cook County and 25 acute care hospitals in surrounding jurisdictions transmit ED records to ESSENCE daily. Required data elements include a patient identifier (eg, medical record number), age, sex, ZIP code, visit date and time, and chief complaint. Optional data elements include discharge diagnosis and disposition. Because the purpose of this study was to inform and evaluate public health practice, it was not considered to be human subjects research by the Cook County Health and Hospital Systems Office of Research and Regulatory Affairs and was exempt from institutional review board review.

We queried ESSENCE records for ED visits with a chief complaint or discharge diagnosis related to rabies, rabies PEP, or contact with a wild mammal (eg, bat, raccoon, skunk, fox, or coyote) from January 1, 2013, through June 30, 2015. We also included the following discharge diagnosis codes from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM): V01.5 (contact with or exposure to rabies), V04.5 (need for prophylactic vaccination and inoculation against rabies), 979.1 (poisoning by rabies vaccine), and E949.1 (rabies vaccine causing adverse effects in therapeutic use).²¹ We used word segments and wild card symbols (eg, *lac* for “laceration”) where possible and included alternative spellings of commonly misspelled words (Table 1). We then manually reviewed the chief complaints and discharge diagnoses of the ED visits identified by the query to eliminate visits unrelated to potential rabies exposure.

Table 1.

Search terms queried in ESSENCE to identify emergency department visits for potential rabies exposures, suburban Cook County, Illinois, January 1, 2013, through February 29, 2016^a

Inclusionary Terms	Exclusionary Terms
Rabies, rabid
Postexposure prophylaxis^b
Bat	Ball, batter, metal, plastic, laceration, contusion, combat, assault, bath, intubation, exacerbation
Raccoon, skunk, fox, coyote
ICD-9-CM discharge diagnosis codes²¹ V01.5: Contact with or exposure to rabies V04.5: Need for prophylactic vaccination and inoculation against rabies 979.1: Poisoning by rabies vaccine E949.1: Rabies vaccine causing adverse effects in therapeutic use
ICD-10-CM discharge diagnosis codes^22,c Z20.3: Contact with and (suspected) exposure to rabies Z29.14: Encounter for prophylactic rabies immune globin Z23: Encounter for immunization T50.B91: Poisoning by other viral vaccines

Abbreviations: ESSENCE, Electronic Surveillance System for the Early Notification of Community-Based Epidemics; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification.

^aWild card symbols and word segments were used in place of full terms wherever possible (eg, *lac* for “laceration”). Alternative spellings of commonly misspelled words were also included.

^bThe acronym PEP was not included.

^cUsed only for the period after interventions to increase reporting compliance (July 1, 2015, through February 29, 2016).

We matched patients with ≥1 rabies-related ED visit identified by the query to people with potential rabies exposures reported to the Cook County Department of Public Health during the same period. We matched records by patient demographic characteristics, visit date(s), and hospital. We allowed small amounts of discord (eg, age off by 1 year) if all other variables matched. We considered patients with matches to be reported and nonmatches to be unreported.

Intervention

Our intervention to increase reporting compliance consisted of 2 components: active surveillance and hospital outreach.

Beginning July 20, 2015, we queried ESSENCE records for rabies-related ED visits once per week. We used a onetime catch-up query to identify rabies-related ED visits from July 1 through July 19, 2015. Each week, we compared patients who had ED visits identified by the queries with people who had potential rabies exposures reported to the Cook County Department of Public Health within the past month. An epidemiologist then contacted hospitals that treated patients with rabies-related ED visits who had not yet been reported and asked the hospital’s infection preventionist to review the record of the unreported patient, determine if the patient met the surveillance case definition for a potential rabies exposure, and report the case if appropriate. The epidemiologist also asked the infection preventionist if he or she was familiar with the requirement to report potential rabies exposures and about barriers to compliance that existed at the hospital.

Local and state health authorities also intensified hospital communication efforts about potential rabies exposure reporting compliance. On August 26, 2015, we presented baseline reporting compliance estimates to a technical advisory group comprising primarily local infection preventionists. Infection preventionists participated in a peer-to-peer discussion of barriers to reporting potential rabies exposures and shared best practices to increase compliance. On September 16, 2015, the Illinois Department of Public Health distributed a statewide memorandum to infection preventionists, hospital EDs, and hospital pharmacists that noted the large proportion of unreported potential exposures to rabies in suburban Cook County and reminded hospital personnel of their obligation to report.

Postintervention Analyses and Statistical Methods

During the postintervention phase (July 1, 2015, through February 29, 2016), we repeated methods used during the preintervention phase of the evaluation. We added to our search terms the following International Classification of Diseases, Tenth Revision, Clinical Modification discharge diagnosis codes, which were implemented nationally in October 2015: Z20.3 (contact with and [suspected] exposure to rabies), Z29.14 (encounter for prophylactic rabies immune globin), Z23 (encounter for immunization), and T50.B91 (poisoning by other viral vaccines).²² We then compared the proportion of patients with rabies-related ED visits reported to the Cook County Department of Public Health postintervention with the proportion reported preintervention to determine whether reporting compliance had improved. We also compared the total number of potential rabies exposure cases reported to the health department, by any health care provider, during the postintervention period (July 1, 2015, through February 29, 2016) with the total number of cases reported during the same period in the previous year (July 1, 2014, through February 28, 2015). To assess whether the change in case reports that we observed locally may have resulted from statewide trends, we obtained data on the number of potential rabies exposure cases reported to other local health jurisdictions in Illinois, outside of suburban Cook County. We calculated frequencies and proportions for descriptive analyses and used risk ratios, including 95% CIs, to compare outcome measures from the preintervention period with outcomes from the postintervention period. We used the Pearson χ² test to determine significance, with α = 0.05 considered significant, and we conducted all analyses using SAS version 9.4.²³

Results

Preintervention Reporting Compliance

For the preintervention period (January 1, 2013, through June 30, 2015), we identified 248 patients with ≥1 ED visit with a rabies-related chief complaint or discharge diagnosis. Of these, 63 (25.4%) were reported to the Cook County Department of Public Health (Table 2).

Table 2.

Reporting compliance for patients with ≥1 ED visit for potential rabies exposure among 45 hospital EDs, pre- and postintervention,^a suburban Cook County, Illinois, January 1, 2013, through February 29, 2016

	Patients, No. (%)
Variable	Preintervention^b	Postintervention^c
Reported^d	63 (25.4)	53 (54.1)
Unreported	185 (74.6)	45 (45.9)
Total	248 (100.0)	98 (100.0)
Risk ratio (95% CI) [P value^e]	2.1 (1.6-2.8) [<.001]

Abbreviation: ED, emergency department.

^aInterventions to increase reporting compliance included education for hospital staff members and active surveillance by the Cook County Department of Public Health.

^bJanuary 1, 2013–June 30, 2015.

^cJuly 1, 2015–February 29, 2016

^dReported to the Cook County Department of Public Health.

^eBased on Pearson χ² test, with α = .05 considered significant.

Of the 28 hospitals that treated rabies exposures during the preintervention period, 3 had a reporting compliance of 100%, and 14 had a reporting compliance of 0%. Among the remaining 11 hospitals, the median proportion of rabies exposure cases reported was 33.3% (range, 4.5%-81.8%). Hospital infection preventionists cited the most common barriers to reporting as an assumption that ED providers would report exposures and a reliance on internal notification systems triggered by positive laboratory results.

Postintervention Reporting Compliance

For the postintervention period (July 1, 2015, through February 29, 2016), we identified 98 patients with ≥1 ED visit with a rabies-related chief complaint or discharge diagnosis. Of these, 53 (54.1%) were reported to the Cook County Department of Public Health (Table 2). Patients with ≥1 ED visit for a potential rabies exposure were twice as likely to be reported during the postintervention period than during the preintervention period (risk ratio = 2.1; 95% CI, 1.6-2.8; P < .001).

Nineteen hospitals treated potential rabies exposures during the pre- and postintervention periods. During the postintervention period, the median proportion of potential rabies exposures reported by these hospitals increased from 9.2% to 50.0%.

A total of 88 cases of potential rabies exposure were reported to the Cook County Department of Public Health during the postintervention period, compared with 25 potential exposures reported preintervention during the same period—an increase of 252.0%. During the same period, reporting of potential rabies exposures to all other local health jurisdictions outside of suburban Cook County increased by only 16.3%, from 349 cases to 406 cases. During the postintervention period, potential rabies exposures were 2.7 times more likely to be reported in suburban Cook County than in the rest of Illinois (95% CI, 1.7-4.1; P < .001; Table 3).

Table 3.

Total number of cases of potential rabies exposure reported to local health departments by all health care providers, pre- and postintervention^a, Illinois, July 1, 2014, through February 29, 2016

	Potential Exposures Reported,^b No. (%)
Location	Preintervention^c	Postintervention^d
Suburban Cook County, Illinois	25 (6.7)	88 (17.8)
Illinois, excluding suburban Cook County	349 (93.3)	406 (82.2)
Total	374 (100.0)	494 (100.0)
Risk ratio (95% CI) [P value^e]	2.7 (1.7-4.1) [<.001]

^aInterventions to increase reporting compliance included education for hospital staff members and active surveillance by the Cook County Department of Public Health.

^bUnpublished data, Illinois Department of Public Health, 2014-2016.

^cJuly 1, 2014–February 28, 2015.

^dJuly 1, 2015–February 29, 2016.

^eBased on Pearson χ² test, with α = .05 considered significant.

Lessons Learned

A large proportion of potential rabies exposures identified in ED records were not reported in suburban Cook County during the pre- or postintervention study period. The proportion of potential rabies exposure cases reported was 54%, even after introduction of an intervention to improve reporting compliance. This proportion is consistent with measures of reporting completeness previously documented for rabies exposures in the literature. The study in King County that compared reported PEP administrations with hospital pharmacy rabies immune globulin dispensing records from 2003 to 2006 found an average reporting completeness of 62%.¹⁵

Although low reporting completeness persisted postintervention, the proportion of patients with rabies-related ED visits that were reported more than doubled from the pre- to postintervention period, from 25.4% to 54.1%. The proportion of rabies exposure cases reported varied substantially by hospital, suggesting that targeted outreach to poor performers could increase compliance. The intervention consisted of multiple components, including active surveillance and hospital outreach. The effectiveness of active surveillance in increasing reporting completeness is well documented.^24

–27 Weekly follow-up of potential exposures also created opportunities for epidemiologists to discuss facility-specific reporting obstacles individually with infection preventionists. Epidemiologists reinforced the importance of reporting by following up each time that a potential exposure was missed.

In addition to our targeted outreach, we shared estimates of baseline reporting completeness locally and statewide in concert with a reminder of mandatory reporting requirements. A previous study investigating the effect of message type on reporting compliance showed that content related to the legal obligation to report was more effective at increasing reporting than was content related to the public health benefits of reporting.²⁸ Our results suggest that message content documenting the extent of underreporting might also be beneficial. Local dissemination of results also included a peer-to-peer discussion among infection preventionists about common barriers and solutions to reporting. Best practices to increase compliance included clarifying who is responsible for reporting and monitoring pharmacy dispensing for rabies immune globulin as a trigger to report. To our knowledge, this investigation is the first to describe obstacles to reporting potential rabies exposures as cited by hospital staff members.

Limitations

Our study had several limitations. First, some of the patients with rabies-related ED visits identified in ESSENCE and not reported to the Cook County Department of Public Health might not have met the surveillance case definition and therefore would not have required reporting. For example, a patient with an ED visit with the chief complaint “exposed to skunk” would require reporting if the exposure was a bite but would not require reporting if the patient was sprayed by a skunk. As a result, some of the patients with rabies-related ED visits who were not reported may have had their exposures misclassified. However, after removal of all patients with rabies-related ED visits who were not known to meet the surveillance case definition in the pre- and postintervention groups and then recalculating, the risk ratio was 1.8 (95% CI, 1.2-2.6; P = .006), which indicates that the intervention had a moderately strong effect on reporting of potential rabies exposure cases in the postintervention phase. By including only patients with rabies-related ED visits and known case definition status in the analysis, the specificity of outcome classification is perfect, and we would expect outcome sensitivity to be higher postintervention than preintervention. However, these effects biased the observed risk ratio away from the null, indicating that 1.8 was likely the floor.

Second, cases of potential rabies exposure detected in ESSENCE did not include all cases of potential rabies exposure in suburban Cook County. We did not include in our estimates of reporting compliance patients treated by a primary care provider or in a hospital not reporting to ESSENCE or patients with ED visits in ESSENCE with a chief complaint that did not contain 1 of our search terms. Third, it is possible that ESSENCE records were inaccurately matched to reported potential exposures, which might have under- or overestimated reporting completeness. However, several variables were used to make a match, and the probability of errors was low. Finally, because our interventions, active surveillance and hospital outreach, were simultaneous and ongoing, it was not possible to isolate the individual effect of each intervention on reporting compliance. We could associate our combined efforts only with an increase in reporting of potential rabies exposures.

Conclusions

Mandatory reporting of potential rabies exposures in suburban Cook County needs to be improved. However, the near–real-time nature of syndromic surveillance facilitated timely detection of potential rabies exposures in ED records and rapid intervention by epidemiologists when potential exposures were not reported. As a result, errors in PEP administration could be corrected before patients received additional vaccine. Interventions that use ED syndromic surveillance to monitor reporting show great promise for increasing reporting compliance.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Kimberlin

Brady

Jackson

. Red Book: 2015 Report of the Committee on Infectious Diseases. Elk Grove Village, IL: American Academy of Pediatrics; 2015.

Monroe

Yager

Blanton

. Rabies surveillance in the United States during 2014. J Am Vet Med Assoc. 2016;248(7):777–788.

Manning

Rupprecht

Fishbein

; Advisory Committee on Immunization Practices Centers for Disease Control and Prevention. Human rabies prevention—United States, 2008: recommendations of the Advisory Committee on Immunization Practices. MMWR Morb Mortal Wkly Rep. 2008;57(RR-3):1–28.

Moran

Talan

Mower

. Appropriateness of rabies postexposure prophylaxis treatment for animal exposures. JAMA. 2000;284(8):1001–1007.

Jerrard

. The use of rabies immune globulin by emergency physicians. J Emerg Med. 2004;27(1):15–19.

Robertson

Feldman

. Clinician outreach to improve the quality of rabies postexposure prophylaxis administration: Maryland’s experience, 2010-2011. Vector Borne Zoonotic Dis. 2014;14(6):454–460.

Rupprecht

Briggs

Brown

; Centers for Disease Control and Prevention. Use of a reduced (4-dose) vaccine schedule for postexposure prophylaxis to prevent human rabies: recommendations of the Advisory Committee on Immunization Practices. MMWR Morb Mortal Wkly Rep. 2010;59(RR-2):1–9.

Wilde

. Failures of post-exposure rabies prophylaxis. Vaccine. 2007;25(44):7605–7609.

Ichhpujani

Mala

Veena

. Epidemiology of animal bites and rabies cases in India: a multicentric study. J Commun Dis. 2008;40(1):27–36.

10.

Kreindel

McGuill

Rupprecht

. Rabies postexposure prophylaxis: when is it appropriate? Infect Dis Clin Pract (Baltim Md). 1998;7(6):274–279.

11.

Conti

Wiersma

Hopkins

. Evaluation of state-provided postexposure prophylaxis against rabies in Florida. South Med J. 2002;95(2):225–230.

12.

Centers for Disease Control and Prevention. Cost of rabies prevention. http://www.cdc.gov/rabies/location/usa/cost.html. Accessed May 18, 2016.

13.

Kreindel

McGuill

Meltzer

. The cost of rabies postexposure prophylaxis: one state’s experience. Public Health Rep. 1998;113(3):247–251.

14.

Vora

Clippard

Stobierski

. Animal bite and rabies postexposure prophylaxis reporting—United States, 2013. J Public Health Manag Pract. 2015;21(3):e24–e27.

15.

Thiede

Koepsell

. Completeness of reporting of rabies postexposure prophylaxis in King County, Washington. J Public Health Manag Pract. 2008;14(5):448–453.

16.

Illinois General Assembly, Joint Committee on Administrative Rules. Illinois administrative code, title 11, part 690: control of communicable diseases code. http://www.ilga.gov/commission/jcar/admincode/077/07700690sections.html. Accessed May 18, 2016.

17.

Cook County Department of Public Health. Reportable diseases. http://www.cookcountypublichealth.org/communicable-diseases/reportable-diseases . Accessed April 3, 2017.

18.

Illinois Department of Public Health. Infectious disease reporting. http://www.dph.illinois.gov/topics-services/diseases-and-conditions/infectious-diseases/infectious-disease-reporting. Accessed April 3, 2017.

19.

Schramm

Vogt

Mamolen

. The surveillance of communicable disease in Vermont: who reports? Public Health Rep. 1991;106(1):95–97.

20.

McClean

Silk

Buehler

. Disease reporting among Georgia physicians and laboratories. J Public Health Manag Pract. 2010;16(6):535–543.

21.

Centers for Disease Control and Prevention. International classification of diseases, ninth revision, clinical modification (ICD-9-CM). https://www.cdc.gov/nchs/icd/icd9cm.htm. Accessed March 23, 2017.

22.

Centers for Disease Control and Prevention. International classification of diseases, tenth revision, clinical modification (ICD-10-CM). https://www.cdc.gov/nchs/icd/icd10cm.htm. Accessed March 23, 2017.

23.

SAS Institute, Inc. SAS: Version 9.4. Cary, NC: SAS Institute, Inc; 2013.

24.

Silk

Berkelman

. A review of strategies for enhancing the completeness of notifiable disease reporting. J Public Health Manag Pract. 2005;11(3):191–200.

25.

Doyle

Glynn

Groseclose

. Completeness of notifiable infectious disease reporting in the United States: an analytical literature review. Am J Epidemiol. 2002;155(9):866–874.

26.

Vogt

LaRue

Klaucke

. Comparison of an active and passive surveillance system of primary care providers for hepatitis, measles, rubella, and salmonellosis in Vermont. Am J Public Health. 1983;73(7):795–797.

27.

Modesitt

Hulman

Fleming

. Evaluation of active versus passive AIDS surveillance in Oregon. Am J Public Health. 1990;80(4):463–464.

28.

Brissette

Gelberg

Grey

. The effect of message type on physician compliance with disease reporting requirements. Public Health Rep. 2006;121(6):703–709.