Erroneous Reporting of Deaths Attributed to Pneumonia and Influenza at 2 New York City Teaching Hospitals,2013-2014

Abstract

Objectives

Cause-of-death information, reported by frontline clinicians after a patient’s death, is an irreplaceable source of public health data. However, systematic bias in cause-of-death reporting can lead to over- or underestimation of deaths attributable to different causes. New York City consistently reports higher rates of deaths attributable to pneumonia and influenza than many other US cities and the country. We investigated systematic erroneous reporting as a possible explanation for this phenomenon.

Methods

We reviewed all deaths from 2 New York City hospitals during 2013-2014 in which pneumonia or influenza was reported as the underlying cause of death (n = 188), and we examined the association between erroneous reporting and multiple extrinsic factors that may influence cause-of-death reporting (patient demographic characteristics and medical comorbidities, time and hospital location of death, type of medical provider reporting the death, and availability of certain diagnostic information).

Results

Pneumonia was erroneously reported as the underlying cause of death in 163 (86.7%) reports. We identified heart disease and dementia as the more likely underlying cause of death in 21% and 17% of erroneously reported deaths attributable to pneumonia, respectively. We found no significant association between erroneous reporting and the multiple extrinsic factors examined.

Conclusions

Our results underscore how erroneous reporting of 1 condition can lead to underreporting of other causes of death. Misapplication or misunderstanding of procedures by medical providers, rather than extrinsic factors influencing the reporting process, are key drivers of erroneous cause-of-death reporting.

Keywords

pneumonia influenza death certificates cause of death reporting mortality statistics

Death certificates record a causal chain of events in the death of a patient. The person (typically a physician) completing the death certificate lists an immediate cause of death (COD), any intermediate CODs, and the underlying COD, or the condition that predisposed the patient to that immediate COD. For example, in a patient with HIV infection who has Pneumocystis pneumonia and who subsequently dies, the pneumonia would be the immediate COD and HIV infection would be the underlying COD. The information reported on death certificates serves as an invaluable data source for public health surveillance and evidence-based decision making in public health. Systematic errors in COD reporting can undermine the reliability of disease-specific mortality statistics and negatively affect public health decision making that may rely on these statistics.¹

Pneumonia and influenza have together ranked as the third leading underlying COD in New York City for more than a decade,² with a higher death rate than in aggregate national statistics and other major metropolitan areas.^3
-5 In 2005 and 2014, the age-adjusted rates of death attributable to pneumonia and influenza in New York City were 34 and 25 per 100 000 people,² respectively, compared with age-adjusted national rates of 20.3 and 15.1 per 100 000 people.³ Rates of death attributable to pneumonia and influenza were 10.5 per 100 000 people in San Francisco County in 2014-2016 and 14.5 per 100 000 people in Philadelphia in 2014.^6,7 The reasons for the higher rate of pneumonia and influenza-related deaths in New York City are not clear, but they may be explained by systematic errors in COD reporting. Erroneous COD reporting is relatively common, both in New York City⁸ and elsewhere.⁹ A study in 2017 examining the quality of COD reporting found that death certificates for pneumonia-related deaths were more likely than death certificates for cancer and diabetes to be incomplete or have other quality-related issues.¹⁰ The reasons for this discrepancy are unclear but may be related to uncertainties inherent to pneumonia diagnosis,¹¹ particularly in patients with multiple concurrent medical problems. Furthermore, limited data exist on how logistical factors (such as a relative unfamiliarity with a patient after a change in physician schedule) may contribute to misreporting on death certificates.

This study aimed to evaluate the accuracy of death certificates reporting pneumonia or influenza as the underlying COD in 2 teaching hospitals in New York City. We also sought to identify patient characteristics, clinical data, and logistical factors associated with accuracy of COD reporting for pneumonia and influenza.

Methods

Data Source and Study Population

The New York City health code¹² mandates that a trained medical provider reports each death occurring in its 5 boroughs to the New York City Department of Health and Mental Hygiene (DOHMH). We compiled a list of all decedents at Columbia University Medical Center (CUMC) and Allen Hospital for whom pneumonia or influenza was listed as the underlying COD on the death certificate from January 1, 2013, through December 31, 2014 (n = 188). CUMC is a 535-bed tertiary care hospital, and Allen Hospital is a 196-bed community hospital, both of which are in the same hospital system and located in northern Manhattan. Graduate medical trainees (resident physicians) are the primary physician-level medical providers in both hospitals, most of whom rotate through both hospitals.

Study Approval

This study was determined by CUMC and DOHMH to be not human subjects research and by the Centers for Disease Control and Prevention to be public health nonresearch.

Medical Record Review and COD Assignment

Two resident physicians trained in death certificate reporting by DOHMH staff members reviewed the hospital electronic medical records and assigned underlying, intermediate, immediate, and contributing CODs for each decedent (Table 1). Each record was reviewed by 1 resident physician; a subset was reviewed jointly with a senior DOHMH physician. Training for resident physicians conducting the secondary medical record review involved multiple in-person sessions with DOHMH senior physicians and staff members in the Division of Vital Statistics at DOHMH. After training, a senior DOHMH physician assessed the competency of the resident reviewers via joint review of example cases. Autopsy reports were available for 11 decedents (5.9%) and, when available, were used to inform COD assignments. A senior DOHMH physician who oversees trainings in New York City on completion of death certificates reviewed 20% of the decedent hospital electronic medical records, and any discrepancies between the senior physician review and resident physician review were reconciled via consensus. At both hospitals in the study, all clinical data (including vital signs, clinician and nursing notes, radiology results, and laboratory data) are recorded in the hospital electronic medical record, and these data were available to both the resident physicians performing the secondary review and the senior DOHMH physician.

Table 1

Definitions used for classifying deaths attributable to pneumonia or influenza at 2 New York City hospitals, 2013-2014

Term	Definition
Underlying COD	The disease or injury that initiated the chain of morbid events that led directly and inevitably to death
Immediate COD	The final disease, injury, or complication directly causing death
Intermediate COD	A disease, injury, or complication related to the underlying and immediate COD
Contributing COD	A disease or injury that was also present at the time of patient’s death but not directly involved in the causal chain of events leading from underlying to immediate COD
Reported pneumonia death	A death in which pneumonia was listed as the underlying COD on the original hospital-reported death certificate
Reported influenza death	A death in which influenza was listed as the underlying COD on the original hospital-reported death certificate
Clinical pneumonia case	A reported pneumonia or influenza death in which the decedent has a new lung infiltrate and meets at least 3 of 4 additional clinical criteria (fever or hypothermia other than induced therapeutic hypothermia, leukocytosis or leukopenia, productive cough or increased secretions from tracheostomy or endotracheal tube, decreased oxygenation) during the terminal hospitalization
Clinical influenza case	Reported pneumonia or influenza death in which the decedent had microbiological diagnosis of influenza virus infection during the terminal hospitalization (a clinical influenza case can also be a clinical pneumonia case)
Influenza-associated pneumonia	Clinical pneumonia case in which the decedent had microbiological diagnosis of influenza virus infection within 7 days before or after meeting criteria for a clinical pneumonia case
Confirmed pneumonia death	Reported pneumonia death in which pneumonia was determined to be the underlying COD per secondary review of the medical record
Confirmed influenza death	Reported influenza death in which influenza was determined to be the underlying COD per secondary review of the medical record
Erroneously reported pneumonia death	Reported pneumonia death in which the decedent was determined per secondary review of the medical record to have an underlying COD other than pneumonia
Erroneously reported influenza death	Reported influenza death in which the decedent was determined per secondary review of the medical record to have an underlying COD other than influenza
Confirmed underlying COD	The underlying COD assigned among reported pneumonia or influenza deaths per secondary review of the medical record

Abbreviation: COD, cause of death.

Trained medical nosologists from DOHMH converted COD information assigned by the study team to International Classification of Diseases, 10th Revision (ICD-10) codes.¹³ Consistent with standardized COD reporting practices,¹⁴ the study team applied the Automated Classification of Medical Entities (ACME) decision tables¹⁵ to the ICD-coded data (from both the resident physicians and the senior DOHMH physician) to yield the final underlying COD information used in the analysis. Briefly, ACME decision tables apply a set of consensus-based rules to identify the most likely sequence of events from ICD-10–coded COD information and thereby enable selection of the most likely underlying COD. Importantly, ACME decision tables prioritize influenza as an underlying COD if it is listed anywhere in the COD information (including as an immediate or intermediate COD).

We obtained demographic data on decedents, including age, sex, race/ethnicity, borough of residence, and date and time of death, from the original death certificate data that were previously transmitted to DOHMH. We abstracted or calculated from the electronic medical record the following additional information for each decedent: (1) whether the decedent resided in a nursing home before the terminal hospitalization, (2) the decedent’s Charlson Comorbidity Index at the time of admission,¹⁶ (3) a Quick Sepsis Related Organ Failure Assessment (qSOFA) score for sepsis in the 48 hours before the decedent’s death,¹⁷ and (4) whether the decedent received certain microbiologic tests (sputum culture, blood culture, respiratory pathogen polymerase chain reaction [PCR], Streptococcus pneumoniae urine antigen, Legionella pneumophila urine antigen, and Legionella pneumophila sputum culture) and radiographic tests (chest x-ray and computed tomography scan) for diagnosis of pneumonia.

Lastly, we reviewed the electronic medical record, hospital personnel records, and resident physician work schedules to determine (1) the type of medical provider who determined the original COD (resident physician, attending physician, physician assistant, or nurse practitioner), (2) the location in which the decedent died (emergency department, hospital ward, or intensive care unit), (3) the hospital service reporting each death, and (4) for deaths reported by a resident physician, whether the patient’s death occurred within 48 hours of a change in the resident block schedule (ie, scheduled dates on which primary responsibility for the care of a patient was transferred from 1 team of residents to another).

Data Analysis

We made counts and calculated percentages for patient characteristics, clinical data, and logistical factors for the entire sample as well as by confirmed and erroneously reported pneumonia or influenza deaths. We defined erroneously reported pneumonia deaths as reported pneumonia deaths in which the decedent was determined per secondary review of the medical record to have an underlying COD other than pneumonia. We defined clinical pneumonia cases by using a set of 5 clinical criteria (Table 1). According to these definitions, we classified clinical pneumonia cases as erroneously reported pneumonia deaths if pneumonia was not determined to be the underlying COD via secondary medical record review. We used separate bivariable relative risk regressions to calculate risk ratios for erroneous reporting of pneumonia or influenza deaths by various factors. In addition, among erroneously reported pneumonia or influenza deaths, we described the top 11 leading underlying CODs verified by medical record review. We performed all statistical analyses in SAS version 9.4 (SAS Institute, Inc).

Results

We identified 188 deaths at CUMC and Allen Hospital during the 2-year study period with a reported underlying COD of pneumonia or influenza, 13 (6.9%) of which were reported as influenza and 175 (93.1%) as pneumonia. Our secondary review confirmed only 25 (13.3% of 188) pneumonia or influenza deaths, of which 9 (4.8% of 188) were confirmed pneumonia deaths and 16 (8.5% of 188) were confirmed influenza deaths. All 16 confirmed influenza deaths were also clinical pneumonia cases (Figure). The remaining 163 pneumonia or influenza deaths (86.7%) were erroneously reported pneumonia or influenza deaths (Table 2), deaths in which secondary review indicated that pneumonia or influenza was not the underlying COD. We did not find significantly increased odds of erroneously reported pneumonia or influenza deaths in any of the sociodemographic characteristics examined (Table 2).

Figure

Secondary review of medical record and assignment of confirmed underlying causes of death at 2 New York City teaching hospitals, 2013-2014. Each box indicates the total number and percentage (of all 188 reported pneumonia and influenza deaths) of decedents in each category.

Table 2

Sociodemographic characteristics of decedents with a confirmed versus erroneously reported underlying cause of death of pneumonia or influenza at 2 New York City teaching hospitals, 2013-2014^a

Characteristic	Reported pneumonia or influenza deaths,no. (%)	Erroneously reported pneumonia or influenza deaths,no. (%)	Confirmed pneumonia or influenza deaths,no. (%)	Risk ratio (95% CI) for erroneously reported pneumonia or influenza deaths^b
Total	188 (100.0)	163 (100.0)	25 (100.0)	—
Sex
Male	97 (51.6)	85 (52.2)	12 (48.0)	1 [Reference]
Female	91 (48.4)	78 (47.9)	13 (52.0)	1.0 (0.9-1.1)
Age, y
18-64	41 (21.8)	32 (19.6)	9 (36.0)	1 [Reference]
≥65	147 (78.2)	131 (80.4)	16 (64.0)	1.1 (1.0-1.4)
Race/ethnicity
Hispanic	63 (33.5)	50 (30.7)	13 (52.0)	1 [Reference]
Non-Hispanic Asian or Pacific Islander	6 (3.2)	5 (3.1)	1 (0.5)	1.1 (0.7-1.5)
Non-Hispanic White	90 (47.9)	82 (50.3)	8 (32.0)	1.1 (1.0-1.3)
Non-Hispanic African American	27 (14.4)	24 (14.7)	3 (12.0)	1.1 (0.9-1.3)
Other/unknown	2 (1.1)	2 (1.2)	0	—^c
Type of residence
Nursing or rehabilitation facility	71 (37.8)	65 (39.9)	6 (24.0)	1 [Reference]
Community residence	115 (61.2)	96 (58.9)	19 (76.0)	0.9 (0.8-1.0)
Unknown	2 (1.1)	2 (1.2)	0	—^c
Borough of residence
Manhattan or Bronx	132 (70.2)	111 (68.1)	21 (84.0)	1 [Reference]
Brooklyn, Queens, or Staten Island	23 (12.2)	21 (12.9)	2 (8.0)	1.1 (0.9-1.3)
Non–New York City resident	33 (17.6)	31 (19.0)	2 (8.0)	1.1 (1.0-1.3)

^aDeath certificate data, including information on demographic characteristics of decedents (age, sex, race/ethnicity, and borough of residence), were obtained from the New York City Department of Health and Mental Hygiene. All other data were obtained from hospital records.

^bSeparate bivariable relative risk regression models.

^cExcluded from calculations of risk ratio.

Of the 188 decedents with a reported underlying COD of pneumonia or influenza, clinical pneumonia cases occurred in 164 (87.2%), according to medical record review (Figure). However, we deemed 139 (84.8%) of these 164 cases to have a confirmed underlying COD other than pneumonia or influenza (Table 3). That is, the decedents met criteria for having a clinical case of pneumonia, but we found them to have a different underlying COD. Heart disease and dementia/Alzheimer disease were the most commonly identified confirmed underlying CODs among erroneously reported pneumonia or influenza deaths. Of 38 cases reviewed by the senior DOHMH physician, we found 14 discrepancies between the senior DOHMH physician and the resident in the assignment of pneumonia as the underlying COD.

Table 3

Confirmed, by secondary analysis of medical records, underlying causes of death among erroneously reported pneumonia or influenza deaths at 2 New York City teaching hospitals, 2013-2014^a

Confirmed underlying cause of death	No. (%)
Total	163 (100.0)
Heart disease	34 (20.9)
Dementia/Alzheimer disease	27 (16.6)
Cancer	19 (11.7)
Chronic lower respiratory diseases	14 (8.6)
Cerebrovascular disease	9 (5.6)
Viral hepatitis	6 (3.7)
Other disorders of the brain	4 (2.5)
Interstitial pulmonary disease, unspecified	4 (2.5)
Other viral infections of unspecified site	3 (1.8)
Benign or uncertain cancer	3 (1.8)
Diabetes mellitus	3 (1.8)
All other causes	37 (22.7)

^aDeath certificate data were obtained from the New York City Department of Health and Mental Hygiene. All other data were obtained from hospital records.

We found 24 (12.8%) of 188 cases in which pneumonia was listed as the underlying COD, but pneumonia was not confirmed to be a clinical case in medical record review (Figure). Eleven of the 24 cases (45.8%) were in patients who had heart disease, including valvular disease, cardiomyopathy, coronary artery disease, or heart failure, as the confirmed underlying COD after secondary review of the medical record.

The Charlson Comorbidity Index and qSOFA were not associated with significantly increased odds of erroneously reported pneumonia or influenza death (Table 4). We found no correlation between time of death, location of death, or type of medical provider reporting death and odds of erroneously reported pneumonia or influenza death. Of the microbiologic and radiographic tests studied, only not using respiratory pathogen PCR was associated with a significantly increased risk of erroneously reported pneumonia or influenza death (risk ratio = 1.2; 95% CI, 1.1-1.3).

Table 4

Decedent comorbidity, medical provider characteristics, and use of microbiologic and radiographic tests among confirmed versus erroneously reported pneumonia or influenza deaths at 2 New York City teaching hospitals, 2013-2014^a

Characteristic	Reported pneumonia or influenza deaths,no. (%)	Erroneously reported pneumonia or influenza deaths,no. (%)	Confirmed pneumonia or influenza deaths,no. (%)	Risk ratio (95% CI) for erroneously reported pneumonia or influenza deaths^b
Total	188 (100.0)	163 (86.7)	25 (13.3)	—
Clinical pneumonia case	—	139 (73.9)	—	—
Not a clinical pneumonia case	—	24 (12.8)	—	—
No. of comordities^c
<4 Comorbidities	90 (47.9)	75 (46.1)	15 (16.0)	1 [Reference]
≥4 Comorbidities	98 (52.1)	88 (54.0)	10 (24.0)	1.1 (1.0-1.2)
Sepsis score within 48 h of death^d
<2	19 (10.1)	18 (11.0)	1 (4.0)	1 [Reference]
≥2	69 (36.7)	58 (35.6)	11 (44.0)	0.9 (0.8-1.0)
Not applicable^e	100 (53.2)	87 (53.4)	13 (52.0)	0.9 (0.8-1.0)
Time of death^f
8:00 am-8:00 pm	91 (48.4)	81 (49.7)	16 (64.0)	1 [Reference]
8:01 pm-7:59 am	97 (51.6)	82 (50.3)	9 (36.0)	0.9 (0.8-1.0)
Location of death in hospital at time of death
Emergency department	13 (6.9)	10 (6.1)	3 (12.0)	1 [Reference]
Hospital ward	79 (42.0)	71 (43.6)	8 (32.0)	1.2 (0.9-1.6)
Intensive care unit	95 (50.5)	81 (49.7)	14 (56.0)	1.1 (0.8-1.5)
Other^g	1 (0.5)	1 (0.6)	0	—^g
Medical provider reporting death
Attending physician	31 (16.5)	25 (15.3)	6 (24.0)	1 [Reference]
Resident or intern physician	110 (58.5)	97 (59.5)	13 (52.0)	1.1 (0.9-1.3)
Physician assistant	16 (8.5)	15 (9.2)	1 (4.0)	1.2 (0.9-1.4)
Cross-coverage medical physician, physician assistant, or nurse practitioner	19 (10.1)	16 (9.8)	3 (12.0)	1.0 (0.8-1.4)
Nurse practitioner	12 (6.4)	10 (6.1)	2 (8.0)	1.0 (0.8-1.4)
Hospital service reporting death
Internal medicine—resident	103 (54.8)	89 (54.6)	14 (56.0)	—^h
Internal medicine—hospitalist/physician assistant	36 (19.2)	32 (19.6)	4 (16.0)	—
Emergency medicine	8 (4.3)	6 (3.7)	2 (8.0)	—
Surgery	11 (5.9)	11 (6.8)	0	—
Nurse practitioner	21 (11.2)	17 (10.4)	4 (16.0)	—
Neurology	7 (3.7)	6 (3.7)	1 (4.0)	—
Other^g	2 (1.1)	2 (1.2)	0	—
Time of death within 48 h of resident schedule change
No	149 (79.3)	127 (77.9)	22 (88.0)	1 [Reference]
Yes	22 (11.7)	20 (12.3)	2 (9.1)	1.1 (0.9-1.2)
Not applicable	17 (9.0)	16 (9.8)	1 (5.9)	1.1 (1.0-1.3)
Use of microbiologic and radiographic testing
Sputum culture
Yes	110 (58.5)	95 (58.3)	15 (60.0)	1 [Reference]
No	78 (41.5)	68 (41.7)	10 (40.0)	1.0 (0.9-1.1)
Blood culture
Yes	179 (95.2)	155 (95.1)	24 (96.0)	1 [Reference]
No	9 (4.8)	8 (4.9)	1 (4.0)	1.0 (0.8-1.3)
Respiratory pathogen polymerase chain reaction
Yes	107 (56.9)	85 (52.2)	22 (88.0)	1 [Reference]
No	81 (43.1)	78 (47.9)	3 (12.0)	1.2 (1.1-1.3)
Streptococcus pneumoniae urine antigen
Yes	90 (47.9)	76 (46.6)	14 (56.0)	1 [Reference]
No	98 (52.1)	87 (53.4)	11 (44.0)	1.1 (0.9-1.2)
Legionella pneumophila urine antigen
Yes	89 (47.3)	74 (45.4)	40 (60.0)	1 [Reference]
No	99 (52.7)	89 (54.6)	10 (40.0)	1.1 (1.0-1.2)
Legionella pneumophila sputum culture
Yes	35 (18.6)	28 (17.2)	7 (28.0)	1 [Reference]
No	153 (81.4)	135 (82.8)	18 (72.0)	1.1 (0.9-1.3)
Chest x-ray
Yes	187 (97.9)	159 (97.6)	25 (100.0)	—^h
No	4 (2.1)	4 (2.5)	0	—
Chest computerized tomography
Yes	57 (30.3)	52 (31.9)	5 (20.0)	1 [Reference]
No	131 (69.7)	111 (68.1)	20 (80.0)	0.9 (0.8-1.0)

^aDeath certificate data were obtained from the New York City Department of Health and Mental Hygiene. All other data were obtained from hospital records.

^bSeparate bivariable relative risk regression models.

^cDetermined by Charlson Comorbidity Index.¹⁶

^dDetermined by Quick Sepsis Related Organ Failure Assessment (qSOFA) score.¹⁷

^eNot applicable: qSOFA scores cannot be calculated if a patient is using mechanical ventilation or vasopressors.

^fData obtained from the death certificate, not hospital medical records.

^gExcluded from calculations of risk ratios.

^hCould not be calculated.

Discussion

Rates of death attributable to pneumonia and influenza reported in New York City are consistently higher than in aggregate national statistics and many other major metropolitan areas, raising the possibility that systematic error exists in reporting pneumonia and influenza deaths in New York City. Given the central importance of death certificate data in public health surveillance and allocation of health resources, identifying and addressing sources of systematic reporting errors is important for local and national public health efforts. Overall, we observed that a high proportion of deaths in which pneumonia was listed as an underlying COD were erroneously reported, but influenza listed as an underlying COD was often accurately reported. Nearly 9 of 10 decedents for whom pneumonia or influenza was reported as the underlying COD had a different underlying COD identified after secondary review of the medical record.

Our study showed that more than one-third of deaths with underlying CODs erroneously reported as pneumonia and influenza deaths were actually due to underlying heart disease or dementia/Alzheimer disease. Although we are unable to generalize our results for all of New York City, we demonstrated that deaths caused by heart disease and dementia/Alzheimer disease are not being reported in New York City, and we suggest that erroneous reporting of certain conditions can lead to underreporting of other important public health problems.

We also found that among all microbiologic and radiographic testing examined in our analyses, only respiratory pathogen PCR testing was associated with significantly lower odds of misreporting of deaths caused by pneumonia or influenza. Higher concordance between initial COD reporting by the medical provider and the confirmed underlying COD per our secondary medical record review in decedents in whom a respiratory pathogen PCR was performed was due at least in part to methodologic factors. Specifically, we included influenza testing as a required criterion in our definition of a clinical influenza case, such that all cases with a positive respiratory pathogen PCR result for influenza were considered clinical influenza cases. In addition, ACME decision tables, which assign influenza as the underlying COD whenever influenza is included as an immediate, intermediate, or underlying COD on the death certificate, were applied to both the initial COD reporting data on the death certificate and to the COD information generated during secondary medical record review. Thus, if a decedent had a positive influenza PCR result during the terminal hospitalization, and the reporting clinician identified influenza on the death certificate (whether as an immediate, intermediate, or underlying COD), this would result in automatic concordance between the reported underlying COD and confirmed COD after secondary medical record review, with both identifying influenza as the underlying COD. However, it is also reasonable to expect that the additional objective clinical data provided to clinicians by respiratory pathogen PCR can help improve the accuracy of COD reporting in this situation.

We found no evidence that patient-related factors or factors related to timing of reporting, clinician type, or physician schedules influenced the accuracy of underlying COD reporting for pneumonia or influenza. Interestingly, contrary to a previous study suggesting that resident physicians might be a common source of misreported COD,¹⁸ we found no association between the type of medical provider submitting a death certificate and odds of erroneous reporting.

Most decedents with an erroneously reported underlying COD of pneumonia or influenza met clinical criteria for pneumonia during their terminal hospitalization. Therefore, although pneumonia was correctly documented on the death certificate as part of the chain of events leading to death, the death certificate did not document the underlying COD accurately. Several possible explanations exist for this type of error in reporting. Clinicians filling out death certificates may misunderstand or misapply the definitions and procedures used in COD reporting, such that they fail to correctly identify or list the underlying CODs that led decedents to develop pneumonia.¹⁹ Alternatively, clinicians may not be able to ascertain, given the clinical data available at the time of reporting, the complete causal chain of events leading to the death (eg, a patient who dies quickly in the emergency department before laboratory data are available). Our study investigators had access to more complete information than the clinicians completing the death certificates, including autopsy reports and other clinical data that may not have been available to reporting clinicians at the time of death. That said, autopsy reports were available for a small proportion of all cases reviewed, and it is unlikely that this additional information explains the reporting discrepancies identified in our study. In addition, although New York City DOHMH guidelines state that an amendment to the death certificate should be submitted if more information on the circumstances surrounding the decedent’s death becomes available after the death certificate is initially submitted, this practice does not likely often occur. It is possible that 2 clinicians presented with the same clinical evidence could report a different COD, or disagree on an underlying COD, on the basis of different but equally valid clinical judgment. However, this situation would not explain the omission from the death certificate of the chain of events that led a decedent to develop a fatal case of pneumonia.

Our analysis also highlighted a subset of cases in which pneumonia was listed as the underlying COD but secondary review demonstrated no clinical evidence of pneumonia. Nearly half of these patients had known heart disease (coronary artery disease, valvular disease, heart failure, or cardiomyopathy). Given that equivocal findings on radiographic imaging and possible respiratory distress or a new oxygen requirement could be attributable to cardiac or pulmonary disease, it is not surprising that we found a sizeable percentage of patients who had underlying heart disease but did not meet our rigorous definition of a clinical case of pneumonia.

Limitations

Our study had several limitations. First, we evaluated data from only 2 hospitals, which are closely affiliated and staffed largely by the same resident physicians. Thus, we were unable to generalize our results beyond these 2 hospitals and the group of frontline clinicians reporting the deaths included in this study, although training for residents included in this study is likely typical of training for residents at other hospitals in New York City. Second, our small sample size limited the statistical power for subgroup analyses, and the study was not powered to evaluate the observational comparisons in our analysis; the small sample size also limited our ability to assess for confounding. Third, the physicians conducting the secondary review of medical records may have been limited in their ability to clearly identify the underlying COD (eg, in situations in which the medical documentation and data were limited), and incomplete documentation by frontline medical providers is itself an important potential limitation to our study. Fourth, we acknowledge differences in subjective clinical judgment between the frontline and reviewing medical provider as a well-established and unavoidable source of interobserver variability. However, we underscore that in most cases in which we found pneumonia as an erroneously reported underlying COD, the error was due to the frontline provider not including a well-documented chronic medical condition that should be implicated as an underlying process that caused the decedent to acquire or have lethal complications of pneumonia (eg, dementia with recurrent aspiration episodes). Lastly, a senior DOHMH physician reviewed only 20% of the decedent medical records, whereas the remainder were reviewed by resident physicians with less expertise in death certificate completion; furthermore, we did identify discrepancies in the senior DOHMH physician’s review and the residents’ review.

Conclusion

Our results underscore the critical role that physicians and other medical providers have in public health surveillance. We emphasize that misreporting underlying CODs by clinicians can lead to not only overreporting of certain underlying causes of mortality but also underreporting and underrecognition of other important public health priorities. In at least 2 hospitals in New York City, deaths due to heart disease and dementia/Alzheimer disease were frequently misreported as attributable to pneumonia. Our results underscore the need for expanded policy and funding support for quality improvement efforts in COD reporting, with the aim of improving the accuracy and reliability of this essential public health data source. More research is needed to understand the potentially wider extent of erroneous COD misreporting and its effect on public health data, and additional strategies are needed to better use objective clinical data (eg, respiratory pathogen PCR) and standardized clinical case definitions, both of which could be incorporated as automatic checks in the COD reporting process. Lastly, our work highlights the central role that clinicians have in correctly reporting COD information and the importance of training and support for clinicians in this role.

Footnotes

Acknowledgments

Tyler S. Brown and Kathryn Dubowski contributed equally to this work. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the New York City Department of Health and Mental Hygiene.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support with respect to the research, authorship, and/or publication of this article.

ORCID iDs

Tyler S. Brown, MD

Neil M. Vora, MD

References

Stark

DE.

Shah

. Funding and publication of research on gun violence and other leading causes of death. JAMA. 2017;317(1):84-85.doi:10.1001/jama.2016.16215

http://www.ncbi.nlm.nih.gov/pubmed/28030692

New York State Department of Health . Leading causes of death, New York City, 2005-2014. Accessed June 2, 2020. https://www.health.ny.gov/statistics/leadingcauses_death/nyc_by_year.html

Kochanek

KD.

Murphy

SL.

Arias

. Mortality in the United States, 2016. NCHS Data Brief, no 293. National Center for Health Statistics; 2017.

Corrado

RE.

Lee

Lucero

DE.

Varma

JK.

Vora

. Burden of adult community-acquired, health-care–associated, hospital-acquired, and ventilator-associated pneumonia: New York City, 2010 to 2014. Chest. 2017;152(5):930-942.doi:10.1016/j.chest.2017.04.162

http://www.ncbi.nlm.nih.gov/pubmed/28455128

Cordoba

Maduro

Huynh

Varma

JK.

Vora

. Deaths from pneumonia—New York City, 1999-2015. Open Forum Infect Dis. 2018;5(2):ofy020. doi:10.1093/ofid/ofy020

29955618

California Department of Public Health . County health status profiles 2018. Accessed June 2, 2020. https://www.cdph.ca.gov/Programs/CHSI/CDPH%20Document%20Library/CHSP-County%20Profiles%202018.pdf

Philadelphia Department of Public Health . Vital statistics report Philadelphia: 2014. Accessed June 2, 2020. https://www.phila.gov/media/20181105161044/2014-Vital-Statistics-Report-FINAL.pdf

Agarwal

Norton

JM.

Konty

et al. Overreporting of deaths from coronary heart disease in New York City hospitals, 2003. Prev Chronic Dis. 2010;7(3):A47.

http://www.ncbi.nlm.nih.gov/pubmed/20394686

Johansson

LA.

Westerling

. Comparing hospital discharge records with death certificates: can the differences be explained? J Epidemiol Community Health. 2002;56(4):301-308.doi:10.1136/jech.56.4.301

http://www.ncbi.nlm.nih.gov/pubmed/11896139

10.

Falci

Lee Argov

EJ.

Van Wye

Plitt

Soto

Huynh

. Examination of cause-of-death data quality among New York City deaths due to cancer, pneumonia, or diabetes from 2010 to 2014. Am J Epidemiol. 2018;187(1):144-152.doi:10.1093/aje/kwx207

http://www.ncbi.nlm.nih.gov/pubmed/28595293

11.

Welker

JA.

Huston

McCue

. Antibiotic timing and errors in diagnosing pneumonia. Arch Intern Med. 2008;168(4):351-356.doi:10.1001/archinternmed.2007.84

http://www.ncbi.nlm.nih.gov/pubmed/18299488

12.

City of New York . New York City Health Code. 2017. Accessed June 29, 2020. https://www1.nyc.gov/site/doh/about/about-doh/health-code-and-rules.page

13.

World Health Organization . ICD-10: International Statistical Classification of Diseases and Related Health Problems. 10th rev, 2nd ed. Accessed June 29, 2020. https://apps.who.int/iris/handle/10665/42980

14.

National Center for Health Statistics . Part 2—instructions for classifying underlying and multiple causes of death. 2017. Accessed June 29, 2020. https://www.cdc.gov/nchs/nvss/instruction_manuals.htm

15.

. Using ACME (Automatic Classification of Medical Entry) software to monitor and improve the quality of cause of death statistics. J Epidemiol Community Health. 2003;57(6):470-471.doi:10.1136/jech.57.6.470

http://www.ncbi.nlm.nih.gov/pubmed/12775799

16.

Charlson

ME.

Pompei

Ales

KL.

MacKenzie

. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-383.doi:10.1016/0021-9681(87)90171-8

http://www.ncbi.nlm.nih.gov/pubmed/3558716

17.

Seymour

CW.

Liu

VX.

Iwashyna

et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762-774.doi:10.1001/jama.2016.0288

http://www.ncbi.nlm.nih.gov/pubmed/26903335

18.

Wexelman

BA.

Eden

Rose

. Survey of New York City resident physicians on cause-of-death reporting, 2010. Prev Chronic Dis. 2013;10:E76. doi:10.5888/pcd10.120288

http://www.ncbi.nlm.nih.gov/pubmed/23660118

19.

Smith Sehdev

AE.

Hutchins

. Problems with proper completion and accuracy of the cause-of-death statement. Arch Intern Med. 2001;161(2):277-284.doi:10.1001/archinte.161.2.277

http://www.ncbi.nlm.nih.gov/pubmed/11176744