Ranked Severe Maternal Morbidity Index for Population-Level Surveillance by Mode of Delivery

Abstract

Objective:

To determine which severe maternal morbidity indicators identify the most in-hospital mortality during delivery hospitalization by delivery mode.

Materials and Methods:

We obtained data from the 1993–2015 Healthcare Cost and Utilization Project’s National Inpatient Sample. Separate analyses were conducted for cesarean and vaginal deliveries. We ranked 22 severe maternal morbidity indicators by their overall population-attributable fraction of in-hospital mortality.

Results:

We identified 87,864,173 delivery hospitalizations; 27.9% were cesarean deliveries and 72.1% were vaginal deliveries. There were 6,686 records with a discharge disposition of “died,” with the most occurring for cesarean deliveries (71.2%). Most deaths had a severe maternal morbidity indicator (cesarean deliveries = 94.2%; vaginal deliveries = 73.5%). Among cesarean deliveries, the top five ranked indicators were as follows: cardiac arrest/ventricular fibrillation, conversion of cardiac rhythm, ventilation, temporary tracheostomy, and aneurysm. Among vaginal deliveries, the top five ranked indicators were as follows: conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation, temporary tracheostomy, and amniotic fluid embolism. The top three ranked indicators identified 78.8% of in-hospital mortality among cesarean deliveries and 66.0% of in-hospital mortality among vaginal deliveries.

Conclusion:

Severe maternal morbidity indicator rankings for cesarean and vaginal deliveries were similar; however, there were differences by delivery mode in the performance of the SMM indicators in identifying in-hospital deaths. Our findings underscore the need for the improved documentation and measurement of severe obstetric complications during pregnancy and the postpartum period at the population level.

Keywords

maternal mortality maternal morbidity population surveillance cesarean delivery vaginal delivery

Introduction

Surveillance of maternal mortality is important for monitoring temporal trends and identifying where maternal health outcomes could be improved¹; however, pregnancy-related deaths are rare.² Severe maternal morbidity (SMM) is defined as unexpected outcomes of birth that result in significant short- or long-term health consequences.³ SMM encompasses a range of serious conditions that may or may not result in death, and it is widely used in public health to monitor trends.⁴ Because pregnancy-related deaths are comparatively rare, SMM indicators are used to monitor severe complications that may lead to mortality.^5–7 In this study, we examine the effectiveness of SMM indicators in identifying in-hospital mortality as one critical, but not exclusive, dimension of severe maternal outcomes. Surveillance of SMM offers a larger number of adverse health events to monitor maternal health and inform delivery of care in health care institutions.⁸ In the United States, hospital discharge data are an important population-based source for SMM surveillance. To identify SMM cases using hospital discharge data, indicators were developed based on International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes.⁹

Validation studies of the indicators as a composite measure show the suboptimal positive predictive value (0.44 and 0.48) of ICD-9-CM codes correctly identifying adverse health events compared with a medical record review.^10,11 The precision and accuracy of the indicators and codes for severe events during delivery hospitalization could be improved. Recently, a ranked SMM index was developed based on in-hospital mortality, concluding that 15 indicators identified 80% of in-hospital mortality, and the remaining 7 indicators identified only 2%.⁶ The remainder of in-hospital deaths (18%) had no SMM indicators.

The ranked SMM index determines the indicators that identify the most in-hospital mortality; however, it is unknown if the ranking of the indicators varies by delivery mode. Previous studies show that the risk of maternal complications is higher in cesarean deliveries than in vaginal deliveries.^12–16 A study assessing the associations of maternal characteristics and delivery mode with SMM found that the largest percentage of population-attributable risk for SMM was for cesarean delivery (37%).¹⁷ However, it is important to note that cesarean delivery may lead to SMM and vice versa. In hospital discharge data, this temporal relationship between cesarean delivery and SMM cannot be determined. Cesarean deliveries may occur for various life-threatening indications (e.g., placenta previa, obstructed labor, and severe preeclampsia), which may lead to confounding by indication when looking at the associations between mode of delivery and SMM. Women who experience cesarean delivery may represent a different population, with different risk factors and preexisting comorbidities, than women who experience vaginal delivery.

Therefore, understanding how the ranking of the indicators varies by delivery mode is important given that the risk of maternal complications is higher in cesarean deliveries compared with vaginal deliveries.^12–16 The objective of this study is to determine which SMM indicators identify the most mortality during delivery hospitalization by delivery mode. As a secondary objective, we sought to identify the possible other factors related to in-hospital death, stratified by delivery mode. Based on the current literature of complications associated with cesarean delivery, we conducted a review of the discharge record for deaths that did not have an SMM indicator code. We sought to determine if discharge records for deaths without codes for an SMM indicator were proportionately greater in one delivery mode compared with the other.

Materials and Methods

Study sample

We obtained data from the 1993–2015 Healthcare Cost and Utilization Project’s (HCUP) National Inpatient Sample (NIS),¹⁸ an all-payer database of hospital discharge records. The NIS is obtained annually from all states participating in HCUP, representing >97% of the U.S. population. Discharge records contained ICD-9-CM diagnosis and procedure codes, Medicare severity diagnosis-related group (MS-DRG) codes, and external cause of injury codes (E codes). We used data from the 1993–2015 period to be consistent with the study period used to develop the ranked SMM index. This period allowed for comparability of codes across years and the ability to obtain a sufficient sample size to produce reliable estimates of SMM and death, as these are rare events We did not examine the ICD transition because this was previously assessed for the ranked SMM index for all deliveries, indicating comparable SMM prevalence before and after the transition.⁶ Delivery hospitalizations were identified using a previously described algorithm.¹⁹ Analyses were restricted to hospitalizations for females aged 12–55 years. The cross-sectional nature of NIS data prevented the identification of multiple hospitalizations for the same woman during the study period. Therefore, the unit of analysis was the delivery hospitalization and not the woman. We suppressed estimates according to HCUP reporting guidelines (e.g., estimates based on cell size <11).²⁰

Study measures for ranking of SMM indicators

In-hospital mortality was identified based on the disposition of the patient at discharge (died). Cesarean deliveries were identified using diagnosis codes (669.70, 669.71), procedure codes (740, 741, 742, 744, 749.9), and MS-DRG codes (370–371 for MS-DRG version 24 or lower; 765–766 for MS-DRG versions 25–32). We examined the 22 SMM indicators described by Kuklina et al. (Supplementary Table S1).⁶ Hospitalization characteristics included patient demographics (age, race/ethnicity, primary type of expected payer of the hospital stay, year of hospitalization) and hospital characteristics (urban/rural location and teaching status, region). Covariates were selected a priori based on the prior literature and clinical significance.⁶ Within our study sample, race/ethnicity was missing for approximately 24.1% of the sample in 1993. Race/ethnicity coding improved in more recent years, with 7.0% missing in 2015. Accordingly, for each year in the study sample, we used a fully conditional specification (FCS) method to generate five sets of imputed race/ethnicity values for delivery hospitalizations missing this information.²¹ Using a discriminant function, imputed race/ethnicity was modeled on patient age, primary type of expected payer of the hospital stay, NIS stratum, and a dichotomous indicator of a patient having at least 1 of the 22 SMM indicators included in the NIS discharge record (yes/no). This method adheres to the recommendations provided by HCUP for handling missing racial/ethnic data.²² The primary purpose of the imputation was to normalize population-based estimates and not to reliably identify the racial/ethnic classification of patients for any single delivery hospitalization. Observations for all other variables with missing data were excluded; each individual variable had <0.3% missingness, and the overall proportion of delivery hospitalizations excluded due to missingness in any covariate was 0.7%.

Ranking of indicators

We conducted separate analyses by delivery mode. Statistical analysis of the ranking of the SMM indicators is described elsewhere.⁶ Briefly, for each SMM indicator, we calculated the prevalence, in-hospital mortality rate, and population-attributable fraction (PAF) for all in-hospital deaths attributable to that indicator. We calculated the PAF using the formula: $p_{d} (\frac{R R - 1}{R R})$ , where p_d is the proportion with the indicator among all the records with death, and RR is the relative risk of indicator-specific death.²³ The indicator-specific RR was estimated using log-linear models, adjusting for patient demographics (age, race/ethnicity, primary type of expected payer of the hospital stay, year of hospitalization), hospital characteristics (urban/rural location and teaching status, region), and all other SMM indicators. Patient and hospital characteristics were included in each adjusted model to account for potential confounding. Each SMM indicator was modeled separately in a hierarchical manner, resulting in slight variations in the adjusted RR for these covariates. As they were not the primary focus, the covariate results are not shown; however, their inclusion improved the robustness of estimates and did not meaningfully change the relative ranking of the indicators. Parameter estimates from the five sets of imputed race/ethnicity values were pooled into a single set of estimates for the log-linear models.

For each delivery mode, an iterative process hierarchically ranked the indicators in order of their importance in identifying in-hospital mortality.⁶ To account for the indicators with relatively large PAFs due to high prevalence, we used a signal-to-noise ratio (SNR), described elsewhere, to determine the ranking of the indicators.^6,24 A higher SNR value suggests that an indicator’s higher PAF is due to a higher mortality rate rather than a higher prevalence in the population. After selecting the indicator with the highest SNR, we forced the selected indicator to be mutually exclusive of the remaining indicators by removing all as-yet-unselected indicators from a record, if that record contained the selected indicator. This enabled the identification of how many additional deaths that specific indicator captures. The resulting PAF was recalculated and designated as the hierarchical PAF (hPAF). A cumulative hPAF was calculated by summing the hPAF values up to, and including, that iteration. The cumulative hPAF at each iteration represents the proportion of in-hospital deaths identified using the indicator selected on that iteration and all the previously selected indicators. We repeated this iterative ranking process, selecting the indicator with the next highest SNR until all the indicators were selected.

Secondary analysis: Discharge record review of in-hospital deaths

We conducted a secondary analysis for each delivery mode to review in-hospital deaths without an SMM indicator to (1) confirm that no SMM code was available, and (2) identify the possible other factors related to these deaths (e.g., trauma-related, metastatic cancer; Supplementary Table S2). We defined SMM as a patient having at least one of the 22 SMM indicators included in the NIS discharge record (yes/no). NIS data for each delivery hospitalization death without an SMM indicator were reviewed by two of three obstetricians (R.R.G., E.A.C., L.K.A.) to identify diagnosis, procedure, MS-DRG, or E codes for the documentation of the possible other factors related to the death.

All analyses were conducted using SAS 9.4 (SAS Institute Inc) and SAS-callable SUDAAN version 11.0 (RTI International, Research Triangle Park, NC, USA) to account for the survey design and weights. Because the NIS is de-identified, this study did not require approval by an institutional review board at the Centers for Disease Control and Prevention.

Results

Among the weighted 87,864,173 delivery hospitalizations (unweighted number of records = 18,198,934) identified in the NIS, 24,541,451 (27.9%) were cesarean deliveries and 63,322,722 (72.1%) were vaginal deliveries. There were 6,686 in-hospital deaths, with most deaths occurring among cesarean deliveries (4,758 [71.2%]). Most deaths had an SMM indicator; however, cesarean deliveries had a higher proportion of deaths with an SMM indicator (4,483 [94.2%]) compared with vaginal deliveries (1,417 [73.5%]).

Prevalence of indicators by delivery mode

Prevalence, in-hospital mortality rate, and PAF for the SMM indicators are summarized for cesarean and vaginal deliveries (Table 1). Severe preeclampsia; hemolysis, elevated liver enzymes, and low platelet count (HELLP) syndrome; or eclampsia was the most prevalent indicator for both cesarean (365.4 per 10,000 delivery hospitalizations) and vaginal (81.7 per 10,000) deliveries; however, the in-hospital mortality rate for this indicator was the lowest among all the indicators (0.1% for both cesarean and vaginal deliveries). For both delivery modes, indicator-specific mortality rates were the greatest for hospitalizations with conversion of cardiac rhythm (39.0% for cesarean deliveries; 31.5% for vaginal deliveries), cardiac arrest/ventricular fibrillation (37.5%; 32.8%, respectively), and amniotic fluid embolism (16.4%; 14.5%, respectively).

Table 1.

Prevalence, in-Hospital Mortality, and PAF for SMM Indicators among Cesarean Deliveries (Weighted Sample = 24,541,451) and Vaginal Deliveries (Weighted Sample = 63,322,722) Using Existing SMM Indicators,⁶ NIS, 1993–2015

SMM indicators^a	Cesarean deliveries						Vaginal deliveries
	Prevalence among delivery hospitalizations		In-hospital mortality		RR^b	PAF^c	Prevalence among delivery hospitalizations		In-hospital mortality		RR^b	PAF^c
	n ^d	Per 10,000 deliveries(95% CI)	n ^d	% (95% CI)	RR^b	%	n ^d	Per 10,000 deliveries(95% CI)	n ^d	% (95% CI)	RR^b	%
Severe preeclampsia, HELLP syndrome, or eclampsia	896,621	365.4 (356.0, 374.7)	1,011	0.1 (0.1, 0.1)	1.1	1.9%	517,529	81.7 (78.2, 85.3)	368	0.1 (0.1, 0.1)	3.3	12.3%
Blood transfusion without DIC	349,430	142.4 (137.6, 147.2)	845	0.2 (0.2, 0.3)	5.3	18.0%	227,499	35.9 (34.6, 37.3)	220	0.1 (0.1, 0.1)	26.0	14.5%
DIC without blood transfusion	83,931	34.2 (32.8, 35.6)	630	0.8 (0.6, 0.9)	4.7	8.9%	86,713	13.7 (13.0, 14.4)	322	0.4 (0.3, 0.5)	29.3	15.5%
Hysterectomy	58,008	23.6 (22.8, 24.4)	514	0.9 (0.7, 1.1)	0.9	−1.3%	15,170	2.4 (2.3, 2.5)	265	1.8 (1.3, 2.2)	4.5	16.3%
Acute respiratory distress syndrome	35,492	14.5 (14.0, 14.9)	1,804	5.1 (4.6, 5.6)	1.4	13.0%	10,577	1.7 (1.6, 1.8)	597	5.6 (4.6, 6.7)	1.4	10.8%
Ventilation	31,801	13.0 (12.5, 13.4)	2,544	8.0 (7.3, 8.7)	39.4	57.2%	8,518	1.4 (1.3, 1.4)	807	9.5 (8.0, 10.9)	12.0	42.4%
Pulmonary edema/acute heart failure	30,072	12.3 (11.7, 12.8)	267	0.9 (0.6, 1.1)	0.7	−2.1%	12,184	1.9 (1.8, 2.1)	105	0.9 (0.5, 1.2)	2.9	3.8%
DIC and blood transfusion	26,689	10.9 (10.4, 11.4)	750	2.8 (2.4, 3.2)	7.4	17.7%	12,366	2.0 (1.8, 2.1)	328	2.7 (2.0, 3.3)	51.9	27.2%
Renal failure	25,688	10.5 (10.1, 10.9)	835	3.3 (2.8, 3.7)	1.2	3.8%	9,721	1.5 (1.4, 1.6)	377	3.9 (3.0, 4.8)	1.8	11.5%
Sepsis	20,320	8.3 (7.9, 8.7)	632	3.1 (2.5, 3.7)	3.7	15.1%	13,082	2.1 (1.8, 2.3)	344	2.6 (2.0, 3.3)	5.2	20.3%
Puerperal cerebrovascular disorders	16,860	6.9 (6.6, 7.1)	741	4.4 (3.7, 5.1)	5.3	11.3%	11,855	1.9 (1.8, 2.0)	227	1.9 (1.4, 2.4)	3.4	9.2%
Severe anesthetic complications	14,686	6.0 (5.6, 6.4)	172	1.2 (0.8, 1.6)	0.8	−0.4%	7,393	1.2 (1.1, 1.3)	DS	DS	DS	DS
Shock	12,733	5.2 (5.0, 5.4)	714	5.6 (4.7, 6.5)	0.9	−2.1%	7,557	1.2 (1.1, 1.3)	313	4.1 (3.2, 5.1)	1.1	2.0%
Heart failure/arrest during surgery or procedure	8,824	3.6 (3.3, 3.9)	116	1.3 (0.8, 1.8)	0.3	−2.3%	1,330	0.2 (0.2, 0.2)	DS	DS	DS	DS
Air and thrombotic embolism	8,374	3.4 (3.2, 3.6)	459	5.5 (4.4, 6.6)	2.8	7.1%	5,492	0.9 (0.8, 0.9)	69	1.3 (0.6, 1.9)	0.7	−1.9%
Sickle cell crisis	4,545	1.9 (1.7, 2.0)	53	1.2 (0.5, 1.9)	1.5	0.4%	3,815	0.6 (0.5, 0.7)	DS	DS	DS	DS
Cardiac arrest/ventricular fibrillation	4,179	1.7 (1.6, 1.8)	1,568	37.5 (34.3, 40.8)	7.9	33.4%	1,168	0.2 (0.2, 0.2)	384	32.8 (26.8, 38.9)	6.2	21.6%
Conversion of cardiac rhythm	3,830	1.6 (1.4, 1.7)	1,495	39.0 (35.4, 42.6)	6.5	31.1%	1,390	0.2 (0.2, 0.3)	438	31.5 (25.9, 37.0)	10.0	25.8%
Amniotic fluid embolism	2,735	1.1 (1.0, 1.2)	450	16.4 (13.3, 19.6)	1.0	0.0%	1,123	0.2 (0.2, 0.2)	163	14.5 (9.7, 19.3)	1.9	4.4%
Temporary tracheostomy	1,692	0.7 (0.6, 0.8)	217	12.8 (9.5, 16.2)	0.6	−3.3%	412	0.1 (0.1, 0.1)	DS	DS	DS	DS
Acute myocardial infarction	1,121	0.5 (0.4, 0.5)	99	8.8 (5.2, 12.5)	0.7	−0.7%	512	0.1 (0.1, 0.1)	DS	DS	DS	DS
Aneurysm	771	0.3 (0.3, 0.4)	DS	DS	DS	DS	306	0.1 (0.0, 0.1)	DS	DS	DS	DS

Sorted by prevalence among delivery hospitalizations for cesarean deliveries.

The estimated RR associated with a specific indicator was obtained from log-linear regression models, adjusting for patient age, patient race/ethnicity, primary expected payer, hospital location and teaching status, hospital region, year, and all other SMM indicators.

PAFs for all SMM indicators were calculated using the formula p_d ((RR − 1)/RR), where p_d is the proportion with the indicator among all hospital discharge records with in-hospital death, and RR is the estimated relative risk of indicator-specific in-hospital death. The estimated RR associated with a specific indicator was obtained from log-linear regression models, adjusting for patient age, patient race/ethnicity, primary expected payer, hospital location and teaching status, hospital region, year, and all other SMM indicators.

Weighted count (n).

PAF, population attributable fraction; SMM, severe maternal morbidity; NIS, National Inpatient Sample; CI, confidence interval; HELLP, hemolysis, elevated liver enzymes, and low platelet count; DIC, disseminated intravascular coagulation; DS, data suppressed because of cell count of 10 or less.

Ranking of indicators by delivery mode

Among cesarean deliveries, the top five ranked indicators were as follows: cardiac arrest/ventricular fibrillation, conversion of cardiac rhythm, ventilation, temporary tracheostomy, and aneurysm (Table 2). The lowest ranked indicators (ranked 13, 14, 15) were puerperal cerebrovascular disorders; blood transfusion without disseminated intravascular coagulation (DIC); and severe preeclampsia, HELLP syndrome, or eclampsia. In addition, 7 of the 22 SMM indicators did not identify any additional death for cesarean deliveries (acute myocardial infarction, shock, sickle cell crisis, heart failure/arrest during surgery or procedure, severe anesthetic complications, pulmonary edema/acute heart failure, DIC without blood transfusion). The top five ranked indicators among vaginal deliveries were as follows: conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation, temporary tracheostomy, and amniotic fluid embolism (Table 2). The lowest ranked indicators (ranked 11, 12, 13) were sepsis; DIC without blood transfusion; and severe preeclampsia, HELLP syndrome, or eclampsia. Moreover, 9 of the 22 SMM indicators did not identify any additional death for vaginal deliveries (aneurysm, acute respiratory distress syndrome, acute myocardial infarction, air and thrombotic embolism, puerperal cerebrovascular disorders, sickle cell crisis, heart failure/arrest during surgery or procedure, severe anesthetic complications, blood transfusion without DIC).

Table 2.

Results of the SMM Indicator Ranking, by Delivery Mode, Using Methods Described by Kuklina Et al., ⁶ NIS, 1993–2015

Overall ranking for all delivery hospitalizations from Kuklina et al.⁶	SMM indicators	Cesarean deliveries						Vaginal deliveries
	SMM indicators	Iteration/rank	SNR^a	In-hospital mortalityn^b,c	Cumulative in-hospital mortalityn^b,d	hPAF^e	Cumulative hPAF^f,g	Iteration/ rank	SNR^a	In-hospital mortalityn^b,c	Cumulative in-hospital mortalityn^b,d	hPAF^e	Cumulative hPAF^f,g
1	Conversion of cardiac rhythm	2	55.9	750	2,318	15.1%	48.5%	1	50.5	438	438	25.8%	25.8%
2	Cardiac arrest/ventricular fibrillation	1	25.0	1,568	1,568	33.4%	33.4%	2	111.1	191	629	12.6%	38.3%
3	Ventilation	3	14.0	1,438	3,756	30.3%	78.8%	3	43.4	536	1,165	27.7%	66.0%
4	Temporary tracheostomy	4	DS	DS	DS	DS	DS	4	DS	DS	DS	DS	DS
5	Amniotic fluid embolism	6	DS	DS	DS	DS	DS	5	DS	DS	DS	DS	DS
6	Aneurysm	5	DS	DS	DS	DS	DS	—	—	—	1,417	—	76.4%
7	Acute respiratory distress syndrome	10	1.5	89	4,203	0.6%	87.3%	—	—	—	1,417	—	76.4%
8	Acute myocardial infarction	—	—	—	4,483	—	90.4%	—	—	—	1,417	—	76.4%
9	Shock	—	—	—	4,483	—	90.4%	8	DS	DS	DS	DS	DS
10	Air and thrombotic embolism	7	4.1	62	3,925	1.6%	81.6%	—	—	—	1,417	—	76.4%
11	Puerperal cerebrovascular disorders	13	DS	DS	DS	DS	DS	—	—	—	1,417	—	76.4%
12	Sepsis	8	3.3	87	4,012	2.3%	83.9%	11	DS	DS	DS	DS	DS
13	DIC and blood transfusion	9	3.2	101	4,114	2.8%	86.7%	10	DS	DS	DS	DS	DS
14	Acute renal failure	11	DS	DS	DS	DS	DS	6	DS	DS	DS	DS	DS
15	Hysterectomy	12	DS	DS	DS	DS	DS	7	DS	DS	DS	DS	DS
16	Sickle cell crisis^h	—	—	—	4,483	—	90.4%	—	—	—	1,417	—	76.4%
17	Heart failure/arrest during surgery or procedure^h	—	—	—	4,483	—	90.4%	—	—	—	1,417	—	76.4%
18	Severe anesthetic complications^h	—	—	—	4,483	—	90.4%	—	—	—	1,417	—	76.4%
19	Pulmonary edema/acute heart failure^h	—	—	0	4,483	—	90.4%	9	DS	DS	DS	DS	DS
20	DIC without blood transfusion	—	—	—	4,483	—	90.4%	12	DS	DS	DS	DS	DS
21	Blood transfusion without DIC^h	14	DS	DS	DS	DS	DS	—	—	—	1,417	—	76.4%
22	Severe preeclampsia, HELLP syndrome, or eclampsia	15	DS	DS	4,483	DS	90.4%	13	DS	DS	1,417	DS	76.4%

SNR is the selection criterion used to select the indicator that would maximize the ratio of the expected number of SMM cases detected (signal) to the increase in the standard deviation of the cumulative sum of indicators (noise).

Weighted count (n).

The number of additional deaths that the indicator identifies at this iteration of the ranking.

The cumulative count is based on the iteration/rank and not the position in the table rows. The row order is presented using the overall ranking for all delivery hospitalizations to enable comparison across the mode of delivery.

The hPAF value represents the PAF that was calculated for an indicator at the specified iteration/rank after previously selected indicators were made mutually exclusive. hPAFs for all SMM indicators were calculated using the formula p_d ((RR − 1)/RR), where p_d is the proportion with the indicator among all hospital discharge records with in-hospital death, and RR is the estimated relative risk of indicator-specific in-hospital death. The estimated RR associated with a specific indicator was obtained from log-linear regression models, adjusting for patient age, patient race/ethnicity, primary expected payer, hospital location and teaching status, hospital region, year, and all remaining (in hierarchical order) SMM indicators.

The cumulative hPAF is calculated as the sum of hPAF for the individual indicator and those above in ranking.

The cumulative hPAF is based on the iteration/rank and not the position in the table rows. The row order is presented using the overall ranking for all delivery hospitalizations to enable comparison across the mode of delivery.

Indicator fell out the ranking for vaginal deliveries because the indicator did not capture any additional death.

hPAF, hierarchical population attributable fraction; SNR, signal-to-noise ratio.

The top three ranked SMM indicators (conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation) identified 78.8% of in-hospital mortality among cesarean deliveries and 66.0% of in-hospital mortality among vaginal deliveries. These indicators had the three highest hPAFs for both cesarean deliveries (conversion of cardiac rhythm = 15.1%; cardiac arrest/ventricular fibrillation =33.4%; ventilation = 30.3%) and vaginal deliveries (conversion of cardiac rhythm = 25.8%; cardiac arrest/ventricular fibrillation = 12.6%; ventilation = 27.7%). Among cesarean delivery hospitalizations, 15 SMM indicators accounted for 90.4% of in-hospital mortality. Among vaginal delivery hospitalizations, 16 indicators accounted for 76.4% of in-hospital mortality.

Secondary analysis: Possible other factors related to in-hospital death for discharge records without an indicator

While most in-hospital deaths had an SMM indicator, the possible other factors related to death could not be determined for most deaths without an SMM indicator. Among in-hospital cesarean delivery deaths, 275 deaths (5.8%; unweighted number of records = 56) did not include coding for an SMM indicator. After a review of each discharge record by obstetricians, the possible other factors related to death could not be determined based on the available data for 205 deaths (74.5%) without an SMM indicator. Among in-hospital vaginal delivery deaths, 511 deaths (26.5%; unweighted number of records = 107) did not have coding for an SMM indicator based on the available data; the possible other factors related to death could not be determined for 456 deaths (89.4%). Among the deaths without an SMM indicator, the possible other factors related to death that could be determined based on the available data included: possible trauma, possible complications of metastatic cancer, possible cardiac condition, possible hemorrhage, multiple high-risk conditions, possible surgical complications, possible infectious cause, and possible DIC (data suppressed because cell count was <11). A list of all ICD-9-CM codes documented among the deaths without an SMM indicator is included in Supplementary Table S3.

Discussion

The top four ranked indicators were the same for both delivery modes. The top three ranked indicators identified the most in-hospital deaths for both cesarean (78.8%) and vaginal (66.0%) deliveries. While most deaths during delivery hospitalization had documented SMM indicators, vaginal deliveries (26.5%) had a higher proportion of deaths without an SMM indicator compared with cesarean deliveries (5.8%).

Our study provides rankings of the SMM indicators in identifying in-hospital mortality for cesarean and vaginal deliveries. These rankings are similar to the ranked SMM index for all delivery types described previously⁶; however, there were differences by mode of delivery in the performance of the SMM indicators in identifying in-hospital deaths. Specifically, 15 of the 22 SMM indicators used in the ranking process accounted for 90.4% of in-hospital mortality among cesarean delivery hospitalizations; 7 of the indicators did not identify any additional death. In addition, 13 of the 22 SMM indicators used in the ranking process accounted for 76.4% of in-hospital mortality among vaginal delivery hospitalizations; 9 of the indicators did not identify any additional death.

In our study, 94.2% of cesarean delivery in-hospital deaths and 73.5% of vaginal delivery in-hospital deaths were identified as having SMM. This finding is consistent with previous population-level studies from the United States and Canada that identified 78.7% and 68.0% of deaths at delivery as having SMM, respectively.^5,25 Our study also aligns with earlier studies examining the association between maternal mortality and delivery mode, which show a higher risk of maternal mortality associated with cesarean versus vaginal deliveries.^26,27

Data derived using ICD codes to identify SMM cases are routinely used to monitor maternal health at the population level and evaluate the impact of clinical quality improvement initiatives during delivery hospitalization.^28–30 However, the measurement of SMM is limited by how accurately the indicators identify morbidity that could contribute to maternal deaths. Our study demonstrated that several indicators (e.g., DIC without blood transfusion, blood transfusion without DIC, severe preeclampsia, HELLP syndrome, or eclampsia) do not identify a substantial proportion of in-hospital deaths for either delivery mode after accounting for the other indicators, suggesting that these specific indicators alone may not be as effective for surveillance in capturing severe events such as death. Using only higher ranked indicators to identify morbidity that could contribute to maternal deaths could potentially improve the measurement of SMM by improving the positive predictive value for in-hospital death. However, using only these indicators could reduce the sensitivity of SMM measurement because fewer indicators and codes will be used to identify morbidity.

While death represents the most severe adverse outcome, it is not the only meaningful consequence of SMM.^31–33 Surveillance of SMM aims to monitor and reduce severe pregnancy and delivery complications that can have lasting health impacts and increase health care use, regardless of mortality.^31,32 In this study, we focused on mortality to assess how individual SMM indicators contribute to in-hospital death, recognizing that some indicators may have limited validity. For example, a validation study of ICD-10-CM codes for SMM at delivery hospitalization using 21 indicators reported low sensitivity (26%) and positive predictive value (28%) for identifying true clinical events.³⁴ Our study along with the findings from this validation study suggests that there is a need for the improved documentation and measurement of severe complications at delivery. Future studies could examine SMM measurement compared with other adverse outcomes, including intensive care unit admission. As ICD-11-CM implementation advances, surveillance systems, in turn, will adapt to coding changes.

These rankings are intended to inform the refinement of population-level SMM surveillance strategies by identifying which indicators identify the most in-hospital mortality. The lower ranked indicators may contribute minimally to mortality in our analysis because of both limitations in ICD-coded administrative data and true differences in mortality risk across indicators. Some SMM indicators are associated with lower mortality and would remain lower-ranked. Still these indicators remain important for comprehensive SMM surveillance. Further studies could assess whether these conditions are accurately captured and determine their relevance in surveillance definitions or in facility-level morbidity reviews.

In our secondary analysis, we sought to identify the possible other factors related to deaths without an SMM indicator, particularly because the SMM indicators used in the ranking process accounted for <80% of deaths among vaginal delivery hospitalizations. After a discharge record review of deaths without an SMM indicator, the possible other factors related to death could not be determined for most deaths, especially among vaginal deliveries (89.4%). It is possible that these records could be missing codes that could have provided more information about why these deaths occurred. However, it is also possible that these records may have discharge dispositions that were miscoded. For instance, some records could have discharge dispositions coded as “died” instead of “routine” or vice versa. Our findings suggest that more work may be needed to identify the ranked indicators and the possible other factors related to in-hospital death among vaginal deliveries. Specifically, once there are enough years of data for a sufficient sample size to produce a reliable estimate, it will be important to re-examine deaths with ICD-10-CM codes because ICD-10-CM codes capture a higher level of detail than ICD-9-CM codes.

A major strength of the study is that the NIS is a national all-payer database that provides generalizable estimates and is widely used for health research in the United States. The NIS provided an opportunity to determine which SMM indicators, although rare outcomes, contributed the most to in-hospital mortality at delivery hospitalization by delivery mode.

Our study has several limitations. First, we were limited to using data with ICD-9-CM codes to compare codes across years and to obtain a sufficient sample size to produce reliable estimates of SMM and death. However, predictive modeling, using the ranked SMM index for all deliveries, determined that the SMM prevalence remained comparable after the transition to ICD-10-CM coding.⁶ Second, only SMM events during delivery hospitalization are analyzed in this study. The SMM indicators are combined to form an index generally used to capture severe events during delivery hospitalization and do not capture severe events that may contribute to mortality in the late postpartum period (43–365 days postpartum), such as mental health conditions and substance use disorders.⁹ Other metrics may be needed to capture SMM postpartum events after delivery hospitalization. Third, temporality cannot be determined from the data. The timing of the SMM events—either before or after delivery—cannot be established in the database; thus, this aspect of our results should be interpreted with caution. The purpose of this study was to assess SMM measurement for surveillance and not to identify the causes of individual indicators or causes of maternal death. Additional analyses are needed to examine the ranked SMM index and causes of morbidity and mortality, which include the factors contributing to death, using other data sources, such as data from maternal mortality review committees. Fourth, race/ethnicity values were missing for nearly a quarter of our study population in the earlier years, which are known to not be missing completely at random. To decrease the risk of bias, we performed multiple imputation using a FCS method according to the recommendations from HCUP.²² While this approach improves the representativeness of population-level estimates, imputed data are not equivalent to the observed values and may still introduce residual bias, particularly given the differences in maternal health outcomes across racial and ethnic groups that may not be fully captured in administrative data.³⁵ Finally, after a discharge record review, our study was unable to determine the possible other factors related to deaths without coding for an SMM indicator for either delivery mode. Misclassification is possible because some records could have discharge disposition codes that were miscoded (e.g., coded “died” instead of “routine” or vice versa). Additionally, misclassification is also possible because some records may have SMM codes that were miscoded. The presence of in-hospital death may prompt a more thorough documentation of complications, particularly in cesarean deliveries, or conversely, under-coding of complications in vaginal deliveries, especially in cases without death, may underestimate the utility of certain indicators in that group. These differential documentation patterns, if present, could bias our results and complicate the interpretation of differences in indicator performance by delivery mode.

Conclusions

The rankings for cesarean and vaginal deliveries are similar to the ranked index for all deliveries. However, there were differences by mode of delivery in the performance of the SMM indicators in identifying in-hospital deaths. Most in-hospital deaths for both cesarean and vaginal deliveries were identified by the top three ranked SMM indicators (conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation). Future studies of the ranked index are needed and may inform its utility for public health surveillance. Our findings underscore the need for the improved documentation and measurement of severe obstetric complications during pregnancy and the postpartum period at the population level.

Authors’ Contributions

L.S.W.: Conceptualization, methodology, formal analysis, data curation, writing—original draft, writing—review and editing; R.R.G.: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing; L.K.A.: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing; E.A.C.: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing; A.C.E.: Conceptualization, methodology, validation, formal analysis, data curation, writing—original draft, writing—review and editing; G.A.S.: Conceptualization, methodology, writing—original draft, writing—review and editing, supervision; J.Y.K.: Conceptualization, methodology, writing—original draft, writing—review and editing, supervision; C.D.F.: Conceptualization, writing—original draft, writing—review and editing, supervision; C.D.K.: Conceptualization, resources, writing—original draft, writing—review and editing, supervision, project administration.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

The author(s) received no specific funding for this work.

Supplemental Material

References

World Health Organization. Maternal and perinatal death surveillance and response: Materials to support implementation; 2021. Available from: https://www.who.int/publications/i/item/9789240036666

Trost

, Beauregard

, Chandra

et al. Pregnancy-Related Deaths: Data from Maternal Mortality Review Committees in 36 US States, 2017–2019; 2022. Available from: https://www.cdc.gov/reproductivehealth/maternal-mortality/erase-mm/data-mmrc.html [Last accessed: September 19, 2022]

Obstetric Care Consensus No. 5. Severe maternal morbidity: Screening and review. Obstet Gynecol, 2016; 128(3):e54–e60; doi: 10.1097/AOG.0000000000001642

Snowden

, Lyndon

, Kan

, et al. Severe maternal morbidity: A comparison of definitions and data sources. Am J Epidemiol, 2021; 190(9):1890–1897; doi: 10.1093/AJE/KWAB077

Ray

, Park

, Dzakpasu

, et al. Prevalence of severe maternal morbidity and factors associated with maternal mortality in Ontario, Canada. JAMA Netw Open, 2018; 1(7):e184571; doi: 10.1001/JAMANETWORKOPEN.2018.4571

Kuklina

, Ewing

, Satten

, et al. Ranked severe maternal morbidity index for population-level surveillance at delivery hospitalization based on hospital discharge data. PLoS One, 2023; 18(11):e0294140; doi: 10.1371/JOURNAL.PONE.0294140

Kayem

, Kurinczuk

, Lewis

, et al. Risk factors for progression from severe maternal morbidity to death: A national cohort study. PLoS One, 2011; 6(12):e29077; doi: 10.1371/journal.pone.0029077

Callaghan

, MacKay

, Berg

. Identification of severe maternal morbidity during delivery hospitalizations, United States, 1991–2003. Am J Obstet Gynecol, 2008; 199(2):133.e1–133.e8; doi: 10.1016/J.AJOG.2007.12.020

Centers for Disease Control and Prevention. How does CDC identify severe maternal morbidity? Available from: https://www.cdc.gov/reproductivehealth/maternalinfanthealth/smm/severe-morbidity-ICD.htm [Last accessed: August 24, 2022].

10.

Main

, Abreo

, McNulty

, et al. Measuring severe maternal morbidity: Validation of potential measures. Am J Obstet Gynecol, 2016; 214(5):643.e1–643.e10; doi: 10.1016/J.AJOG.2015.11.004

11.

Himes

, Bodnar

. Validation of criteria to identify severe maternal morbidity. Paediatr Perinat Epidemiol, 2020; 34(4):408–415; doi: 10.1111/ppe.12610

12.

Schutte

, Steegers

EAP

, Schuitemaker

NWE

, et al.; Netherlands Maternal Mortality Committee. Rise in maternal mortality in the Netherlands. BJOG, 2010; 117(4):399–406; doi: 10.1111/J.1471-0528.2009.02382.X

13.

Zwart

, Richters

, Öry

, et al. Severe maternal morbidity during pregnancy, delivery and puerperium in the Netherlands: A nationwide population-based study of 371,000 pregnancies. BJOG, 2008; 115(7):842–850; doi: 10.1111/J.1471-0528.2008.01713.X

14.

Liu

, Liston

, Joseph

, et al.; Maternal Health Study Group of the Canadian Perinatal Surveillance System. Maternal mortality and severe morbidity associated with low-risk planned cesarean delivery versus planned vaginal delivery at term. CMAJ, 2007; 176(4):455–460; doi: 10.1503/CMAJ.060870

15.

Pallasmaa

, Ekblad

, Gissler

, et al. The impact of maternal obesity, age, pre-eclampsia and insulin dependent diabetes on severe maternal morbidity by mode of delivery-a register-based cohort study. Arch Gynecol Obstet, 2015; 291(2):311–318; doi: 10.1007/S00404-014-3352-Z

16.

Pallasmaa

, Ekblad

, Aitokallio-Tallberg

, et al. Cesarean delivery in Finland: Maternal complications and obstetric risk factors. Acta Obstet Gynecol Scand, 2010; 89(7):896–902; doi: 10.3109/00016349.2010.487893

17.

Leonard

, Main

, Carmichael

. The contribution of maternal characteristics and cesarean delivery to an increasing trend of severe maternal morbidity. BMC Pregnancy Childbirth, 2019; 19(1):16; doi: 10.1186/S12884-018-2169-3

18.

HCUP Databases. Healthcare Cost and Utilization Project (HCUP). HCUP-US NIS overview. Agency for Healthcare Research and Quality, Rockville, MD. September 2021. Available from: https://www.hcup-us.ahrq.gov/nisoverview.jsp [Last accessed: August 23, 2022].

19.

Kuklina

, Whiteman

, Hillis

, et al. An enhanced method for identifying obstetric deliveries: Implications for estimating maternal morbidity. Matern Child Health J, 2008; 12(4):469–477; doi: 10.1007/S10995-007-0256-6

20.

Healthcare Cost and Utilization Project (HCUP). Publishing with HCUP Data.; 2023. Available from: https://hcup-us.ahrq.gov/db/publishing.jsp [Last accessed: February 9, 2024].

21.

Berglund

, Heeringa

. Multiple Imputation of Missing Data Using SAS. SAS Institute; 2014.

22.

Houchens

. Missing Data Methods for the NIS and the SID. HCUP Methods Series Report # 2015-01 ONLINE.; 2015. Available from: http://www.hcup-us.ahrq.gov/reports/methods/methods.jsp

23.

Rockhill

, Newman

, Weinberg

. Use and misuse of population attributable fractions. Am J Public Health, 1998; 88(1):15–19; doi: 10.2105/AJPH.88.1.15

24.

George

, Kibria

BMG

. Confidence intervals for estimating the population signal-to-noise ratio: A simulation study. J Appl Stat, 2012; 39(6):1225–1240; doi: 10.1080/02664763.2011.644527

25.

Friedman

, Ananth

, Huang

, et al. Hospital delivery volume, severe obstetrical morbidity, and failure to rescue. Am J Obstet Gynecol, 2016; 215(6):795.e1–795.e14; doi: 10.1016/J.AJOG.2016.07.039

26.

Hall

, Bewley

. Maternal mortality and mode of delivery. Lancet, 1999; 354(9180):776; doi: 10.1016/S0140-6736(05)76016-5

27.

Deneux-Tharaux

, Carmona

, Bouvier-Colle

, et al. Postpartum maternal mortality and cesarean delivery. Obstet Gynecol, 2006; 108(3 Pt 1):541–548; doi: 10.1097/01.AOG.0000233154.62729.24

28.

The Alliance for Innovation on Maternal Health. AIM data overview. Available from: https://saferbirth.org/aim-data/overview/ [Last accessed: September 26, 2022].

29.

Main

, Chang

, Dhurjati

, et al. Reduction in racial disparities in severe maternal morbidity from hemorrhage in a large-scale quality improvement collaborative. Am J Obstet Gynecol, 2020; 223(1):123.e1–123.e14; doi: 10.1016/J.AJOG.2020.01.026

30.

Conrey

, Manning

, Shellhaas

, et al. Severe maternal morbidity, a tale of 2 states using data for action-Ohio and Massachusetts. Matern Child Health J, 2019; 23(8):989–995; doi: 10.1007/S10995-019-02744-1

31.

Ukah

, Dayan

, Potter

, et al. Severe maternal morbidity and long-term risk of cardiovascular hospitalization. Circ Cardiovasc Qual Outcomes, 2022; 15(2):e008393; doi: 10.1161/CIRCOUTCOMES.121.008393

32.

Vesco

, Ferrante

, Chen

, et al. Costs of severe maternal morbidity during pregnancy in US commercially insured and Medicaid populations: An observational study. Matern Child Health J, 2020; 24(1):30–38; doi: 10.1007/s10995-019-02819-z

33.

Mengistu

, Turner

, Flatley

, et al. The impact of severe maternal morbidity on perinatal outcomes in high income countries: Systematic review and meta-analysis. J Clin Med, 2020; 9(7):2035; doi: 10.3390/jcm9072035

34.

Boulet

, Stanhope

, Valdez-Sinon

, et al. Validation of ICD-10 codes for severe maternal morbidity at delivery in a public hospital. Epidemiology, 2024; 35(4):506–511; doi: 10.1097/EDE.0000000000001743

35.

Zhang

, Kissin

, Liao

, et al. Multiple imputation of missing race/ethnicity information in the National Assisted Reproductive Technology Surveillance System. J Womens Health (Larchmt), 2024; 33(3):328–338; doi: 10.1089/jwh.2023.0267