Validating Imputation Procedures to Calculate Corrected Opioid-Involved Overdose Deaths,Marion County,Indiana,2011-2016

Data Sources and Sample

Through a collaboration with the MCCO, we extracted death certificate data and accompanying toxicology records for all accidental poisoning deaths in Marion County, Indiana, from 2011 through 2016. ¹¹ Marion County is the largest county in Indiana (with a 2017 population of 940 000), and its county seat, Indianapolis, is the state capital. From death certificate data, we extracted data on the cause of death from ICD-10 codes that were recorded by the Indiana Death Registration System and that indicated primary and contributing causes of death. We determined accidental drug poisoning for deaths that were assigned ICD-10 codes X40-X44. In addition, we determined opioid-involved overdose deaths through ICD-10 codes indicating opioids as a contributing cause of death, including ICD-10 codes T40.0-T40.4 or T40.6. Finally, we recorded whether an accidental drug poisoning death was coded as T50.9 (“unspecified”); this code indicates a death caused by polysubstance use, but no single drug is specified on the death certificate. Marion County, similar to Indiana, has a high rate of drug-unspecified accidental poisoning deaths, resulting in an undercounting of opioid-involved overdose deaths. ^12,13 In 2018, Indiana passed legislation to address the high number of drug-unspecified overdose deaths through the provision of training and education for county coroners and clearer standards for death investigation and reporting in cases of accidental overdose. ¹⁴ However, this legislation was not in effect at the time of our investigation. Death certificates also provided data on age, race/ethnicity, sex, marital status, education, ZIP code, and time (ie, week, month, year) of death.

From toxicology screening reports, we recorded substances detected at threshold levels. The procedures for manual abstraction and coding of toxicology reports are detailed elsewhere. ^12,15,16 Two researchers (P.H. and E.M.) coded data from death certificates and toxicology reports. A third senior reviewer (B.R.R.) reviewed entries at random to check for accuracy. This study was exempt from review by the Indiana University Institutional Review Board, per the university’s policy on research on decedents. The final sample included 1169 matched death certificates and toxicology reports from accidental overdose deaths occurring during 2011-2016.

Measures

Our primary outcome measure was whether an overdose death was specified to have opioid involvement. We defined a drug-specified overdose death as a death for which ICD-10 codes on the death certificate specified at least 1 drug. We defined a drug-unspecified overdose death as a death for which no single drug was listed on the death certificate as a primary or contributing cause of death. However, we still coded these deaths as accidental drug poisoning deaths based on death investigations conducted by the MCCO. We determined opioid involvement by positive detection of opioids on toxicology tests using the DetectiMed (American Institute of Toxicology, Inc, Indianapolis, IN) blood panel thresholds set by the testing agencies (AIT Laboratories, Denton, TX; NMS Labs, Horsham, PA). We based opioid identification on detection of the following substances: codeine, hydrocodone, hydromorphone, morphine, oxycodone, and oxymorphone. These drugs composed 86% of total opioid prescription dispensing during 2011-2016 in Indiana. ¹⁷ We used data on the decedent’s demographic characteristics, including age, sex, race/ethnicity, and circumstances of the death (ie, place of death), to predict opioid involvement among drug-specified overdose deaths and then to extrapolate the number of opioid-involved overdose deaths among drug-unspecified overdose deaths.

Statistical Analysis

We broadly replicated Ruhm’s approach by estimating a probit model (ie, regression for binary data with normally distributed error terms) predicting opioid involvement in drug-specified overdose deaths. Our replication differed from the approach by Ruhm in 4 ways. First, because our data were limited to 1 county, we did not include county-level characteristics. Accordingly, our predictors included binary controls for the decedent’s sex, race/ethnicity, marital status, age, education, place of death, and time (ie, day of the week, month, and year) of death. Second, because few decedents were classified as Hispanic or “other non-white,” we included interactions only between black race and sex instead of the more detailed sex and race/ethnicity interactions conducted by Ruhm. Third, we conducted our estimates at the county level rather than at the state level, as Ruhm did. Fourth, because our sample size was smaller than Ruhm’s, we included year-specific interactions for selected variables that were most predictive of opioid involvement, instead of year-specific probits estimated by Ruhm. Compared with Ruhm’s approach, our sparser imputation model performed well and provided further evidence of its suitability for within-single-county corrections as well. Next, we used estimated probit coefficients to predict the probability of opioid involvement in overdose deaths in the drug-unspecified sample. We assigned 1 to predicted probabilities ≥0.5, indicating the presence of the specific drug, and we assigned 0 to predicted probabilities <0.5, indicating the absence of the drug. Adding the ones, we calculated the estimated number of opioid-involved overdose deaths in the drug-unspecified sample.

We tested 2 specifications of Ruhm’s imputation approach. The first specification included year-fixed effects but pools effects across years (Imputation A). The second specification allowed the effect of these characteristics to vary by year (Imputation B). Imputation B is in line with Ruhm, who estimated his model separately for each year, essentially introducing interactions by year for all predictors.

We also replicated the simple imputation approach used by Buchanich et al, ¹⁰ which imputed the number of opioid-involved overdose deaths in the drug-unspecified sample based on the share of opioid-involved overdose deaths in the drug-specified sample. We compared estimates based on Ruhm’s imputation with this imputation procedure.

We used coefficients produced by these 3 imputations to predict the number of opioid-involved overdose deaths in the 672 unspecified overdose deaths. ¹⁸ We compared the total predicted number of overdose deaths with the true number of opioid-involved overdose deaths (determined by data in toxicology reports) and with the number of opioid-involved overdose deaths from the drug-specified sample only, which is the commonly reported measure of opioid-involved overdose deaths that the imputation procedure seeks to correct.

In the absence of toxicology reports confirming involvement of specific drugs for drug-unspecified overdose deaths, it is difficult to know whether the assumptions underlying the imputation are accurate. As Ruhm ^3,7 noted, the approach assumes that the predictors of opioid involvement in overdose deaths are the same in the drug-specified sample as in the drug-unspecified sample. For example, a violation of this assumption would occur if males were more likely than females to have opioid-involved overdose deaths in the drug-specified sample but not in the drug-unspecified sample. Consistent differences in characteristics of persons in the drug-specified and drug-unspecified sample could, in turn, lead to errors in predicting the number of opioid-involved overdose deaths in this group.

Similarly, the simple imputation approach by Buchanich et al ¹⁰ assumes that the share of opioid-involved overdose deaths is the same in the drug-specified and drug-unspecified samples. This assumption would be violated if, for example, persons who died of an overdose and whose death certificate specified no drugs were systematically less likely to have opioid involvement than persons who died of an overdose and whose death certificate specified at least 1 drug. Our data set enabled us to validate these imputation strategies by comparing substances detected through toxicology reports for all overdose deaths, not just those in which at least 1 drug was specified on the death certificate. We reported the estimated number of opioid-involved overdose deaths using this imputation and compared it with the true number of opioid-involved overdose deaths from toxicology data for each year.

Finally, we conducted 3 sensitivity tests to ensure the robustness of the imputation procedures. First, both Imputation A and Imputation B were estimated using a logistic regression instead of a probit regression. Second, we calculated the number of opioid-involved overdose deaths by adding the predicted probabilities, without first assigning 1 to predicted probabilities if ≥0.5 and 0 otherwise. Third, we tested 2 additional optimal cut points to classify an overdose death as involving opioids—derived by maximizing the sum of the model specificity and sensitivity, as summarized in the Youden J statistic: predicted probabilities >0.82 (from Imputation A) or >0.66 (from Imputation B). We used Stata version 16.0 for all analyses and considered t tests with P < .10 to be significant. ¹⁸

Limitations

Our analysis had several limitations. Our matched sample of death certificates and toxicology reports, which was smaller than the sample used by Ruhm, did not allow year-specific probits such as those conducted by Ruhm. ^3,7 We addressed this limitation by including by-year interactions on key variables. Estimates showed that our approach performed well, which provides confidence in our alternative. A second implication of our smaller sample size was that our study was underpowered, thereby preventing us from formally testing for differences in the estimated number of opioid-involved overdose deaths across imputation approaches. For example, the 3% difference we observed may not have been significant. Significance testing requires accounting for sampling error in means across samples and standard errors in estimating the probits required for imputation. The most straightforward approach would be a bootstrap procedure that, within each bootstrap replication, first estimates a probit model to predict the number of opioid-involved overdose deaths in the drug-specified sample and then applies the estimated coefficients to estimate the number of opioid-involved overdose deaths in the drug-unspecified sample. In our study, even if the 3% difference was significant using a bootstrap procedure, the small magnitude of difference would have little practical implication. Finally, we validated the imputation procedure in a single region of the United States; as such, our results may not extend to other regions. Validation efforts should be extended to broader geographic regions.