Abstract
Objective:
Several states are building infrastructure and data collection methods for longitudinal, population-based surveillance systems for selected hemoglobinopathies. The objective of our study was to improve an administrative case definition for sickle cell disease (SCD) to aid in longitudinal surveillance.
Methods:
We collected data from 3 administrative data sets (2004-2008) on 1998 patients aged 0-21 in Georgia who had ≥1 encounter in which an SCD International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code was recorded, and we compared these data with data from a laboratory and medical record review. We assessed performance (sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) of case definitions that differed by number and type of SCD-coded encounters; addition of SCD-associated treatments, procedures, and complications; and length of surveillance (1 vs 5 years). We identified correct diagnoses for patients who were incorrectly coded as having SCD.
Results:
The SCD case definition of ≥3 SCD-coded encounters in 5 years simplified and substantially improved the sensitivity (96.0% vs 85.8%) and NPV (68.2% vs 38.2%) of the original administrative case definition developed for 5-year, state-based surveillance (≥2 encounters in 5 years and ≥1 encounter for an SCD-related treatment, procedure, or complication), while maintaining a similar PPV (97.4% vs 97.4%) and specificity (76.5% vs 79.0%).
Conclusions:
This study supports an administrative case definition that specifies ≥3 ICD-9-CM–coded encounters to identify SCD with a high degree of accuracy in pediatric patients. This case definition can be used to help establish longitudinal SCD surveillance systems.
Sickle cell disease (SCD) describes a group of genetic blood disorders characterized by chronic hemolytic anemia and various acute and chronic complications, including episodes of pain, strokes, and premature death. SCD severity and treatment vary, depending in part on whether a person has homozygous SCD (HbSS), compound heterozygous forms of SCD (eg, HbSC), or various forms of S β-thalassemia. SCD is a rare disease: according to state-based newborn screening data, SCD primarily affects persons of African ancestry, or approximately 1 in 360 black or African American births in the United States. 1 -4 However, the prevalence of SCD in the United States is not known because many persons affected were born outside of the United States or before implementation of universal newborn screening in 2006. 5 Prevalence is likely increasing because of increased life expectancy among persons living with SCD. 6,7
Since 2008, multiple stakeholders have identified the need for longitudinal, population-based surveillance of SCD to establish the true prevalence of the disease and define how current management and novel therapies improve patient outcomes and survival. 8 -10 From 2010 through 2012, the Centers for Disease Control and Prevention (CDC) and the National Heart, Lung, and Blood Institute funded 7 states (California, Florida, Georgia, Michigan, New York, North Carolina, and Pennsylvania) to participate in 5 years of SCD surveillance through the Registry and Surveillance System for Hemoglobinopathies (RuSH) project. 11,12 Since 2016, California and Georgia have leveraged RuSH data and methods to expand the SCD surveillance period from 5 years of data (2004 through 2008) to 15 years of data (2004 through 2018) through the Sickle Cell Data Collection program. 13 Determining the best case definition to use for expanded surveillance and informing surveillance systems developing in other states, such as Michigan and North Carolina, 14,15 may improve surveillance.
The RuSH case definition for identification of SCD required ≥2 encounters in which a sickle cell diagnosis code was recorded in an administrative data set and ≥1 code from a predetermined list of SCD-associated treatments, procedures, and complications (Box). In 2009, a national workgroup of clinicians with expertise in SCD developed this list because no algorithms had been validated. 19 Since completion of the RuSH project, at least 2 SCD validation studies have reported several high-performing case definitions. 20,21 Using Medicaid claims data for children aged ≤18 with a diagnosis of SCD, Reeves et al 20 developed and compared 37 alternative case definitions for the disease and found that the definition of ≥3 paid SCD claims within a 1-year period, regardless of the type of service, was a straightforward definition to implement and identify SCD in children with the most accuracy (sensitivity, 89.7%; positive predictive value [PPV], 95.8%). Michalik et al 21 did not compare algorithms but found that another high-performing algorithm in the study by Reeves et al (≥1 inpatient SCD claim or 2 SCD outpatient visits at least 30 days apart) performed well (sensitivity, 99.4%; PPV, 99.4%) when electronic health record data from 2 hospital systems were used. However, neither study assessed the usefulness of adding SCD-associated treatments, procedures, and complications to the case definition or the effect of varying periods of surveillance on performance.
Sickle cell disease (SCD)–associated treatments, procedures, and complications included in the RuSH probable case definitiona based on a consensus panel of SCD experts, Atlanta, Georgia, 2009
Treatments Hydroxyurea Parenteral analgesics (morphine, meperidine, hydromorphone, ketorolac, butorphanol) Iron chelators (deferasirox, deferoxamine) Erythropoietin Folic acid
Procedures Red cell transfusion Red cell exchange Splenectomy Cholecystectomy Transcranial Doppler
Complications Chronic renal failure/proteinuria Pneumonia, acute chest syndrome Pulmonary hypertension Stroke (ischemic or hemorrhagic), transient ischemic attack, seizures Intracranial bleeding Priapism Iron overload Gallstones/cholelithiasis, cholecystitis Avascular necrosis Retinal disease Splenomegaly, splenic sequestration, hypersplenism Leg ulcers Dactylitis Osteomyelitis
Abbreviation: RuSH, Registry and Surveillance System for Hemoglobinopathies.
aThe RuSH definition of a probable case required ≥2 SCD International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes 16 on ≥2 separate health care encounters plus ≥1 hemoglobinopathy-associated complication, treatment, or procedure from this list identified through ICD-9-CM codes, Current Procedural Terminology codes, 17 or National Drug Codes 18 but no laboratory-based disease confirmation available.
Our study further examined and compared the performance of the RuSH administrative case definition with elements from these newly reported, high-performing case definitions across varying periods of surveillance to recommend the optimal administrative case definition for future longitudinal SCD surveillance.
Methods
Data Sources
Georgia’s RuSH surveillance system collected claims data from 3 administrative data sets (Medicaid, the Children’s Health Insurance Program, and the State Health Benefit Plan) and Georgia hospital discharge data for patients who had ≥1 encounter in which an SCD International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) 16 code (282.6, 282.60-282.64, 282.68-282.69, 282.41-282.42) was recorded in any position from January 1, 2004, through December 31, 2008. Each claim could contain as many as 10 diagnosis codes. Each data set had previously been de-duplicated and matched in the RuSH surveillance system to state newborn screening data and clinical data from 3 large SCD treatment centers in Georgia. 11
The RuSH definition for identifying confirmed cases required newborn-screening laboratory documentation of SCD or clinical data provided by the SCD treatment centers. The RuSH definition for identifying probable SCD cases in administrative data required ≥2 encounters in which an SCD ICD-9-CM code was recorded plus ≥1 encounter in which an ICD-9-CM code, Current Procedural Terminology 17 code, or National Drug Code 18 from a predetermined list of SCD-associated treatments, procedures, and complications was recorded (Box). In total, the Georgia RuSH project identified 4288 confirmed cases of SCD and 2721 probable cases of SCD from 2004 through 2008.
Validation Sample
We studied a cohort of 1998 patients aged 0-21 who (1) had ≥1 health care encounter in which an SCD ICD-9-CM code was recorded in any of the 3 RuSH administrative data sets, (2) had ≥1 encounter at Children’s Healthcare of Atlanta from 2004 through 2008, and (3) had a medical record available for review. Of these 1998 patients, 1560 were previously confirmed SCD cases, according to linkage with newborn screening confirmatory testing or clinical data from an SCD treatment center. For the 438 patients who were not confirmed cases, 1 of 2 pediatric hematologists (P.A.L. and M.O.Q.) with expertise in SCD reviewed Children’s Healthcare of Atlanta medical records to confirm case status. The pediatric hematologists confirmed the SCD diagnosis through a review of clinical and laboratory data, complete blood count, reticulocytes, and hemoglobin separation by electrophoresis, isoelectric focusing, and high-performance liquid chromatography. Criteria used to exclude the diagnosis of SCD were the absence of any documentation of SCD in provider notes and laboratory results inconsistent with SCD. Cases in which the data in medical records were inadequate to confirm or exclude SCD were categorized as indeterminate (Figure).
Process flow to determine sickle cell disease (SCD) case status of Children’s Healthcare of Atlanta (CHOA) patients (n = 1998) used to compare the performance of alternative case definitions to the Registry and Surveillance System for Hemoglobinopathies (RuSH) probable case definition, Atlanta, Georgia, 2016. The RuSH definition of a probable case required ≥2 SCD International Classification of Diseases, Ninth Revision (ICD-9-CM) codes 16 on ≥2 separate health care encounters plus ≥1 hemoglobinopathy-associated complication, treatment, or procedure identified through ICD-9-CM codes, Current Procedural Terminology codes, 17 or National Drug Codes 18 but no laboratory-based disease confirmation available.
Testing of Alternative Case Definitions
We chose alternative case definitions based on the results of other published SCD validation studies that found that simple counts of all SCD-related claims outperformed other algorithms. In our study, we counted the number of SCD-related encounters by using all medical claims data that contained an SCD-related ICD-9-CM code; we included inpatient and outpatient (emergency department, laboratory, and radiology) visits and excluded pharmacy visits. We counted consecutive inpatient claims as 1 encounter and similarly counted any outpatient claims that occurred during an inpatient episode as an encounter. We de-duplicated all encounters by patient identification, service date, and provider identification. Furthermore, we examined the additional treatment, procedure, and complication-related ICD-9-CM codes (Box) and length of the surveillance period (1 vs 5 years) to refine the RuSH case definition.
We examined alternative case definitions in 3 phases. First, we examined the effect of adding ≥1 SCD-associated treatment, procedure, or complication (Box) to the number of SCD-related medical encounters (1 to ≥6) during the 5-year surveillance period. We tabulated descriptive statistics, including the number of patients in which SCD was confirmed, excluded, or indeterminate, by number of encounters and stratified identified patients by the presence of ≥1 SCD-associated treatment, procedure, or complication defined by RuSH. We used the Mantel-Haenszel χ2 test for trend, with P < .05 considered significant. We reported true diagnoses for persons miscoded as SCD (excluded cases). We removed indeterminate cases from further analysis.
Second, we compared differences in the surveillance period (1 vs 5 years) and retained only 1 definition that included the additional SCD-related treatments, procedures, and complications—the RuSH case definition—because that addition did little to improve the identification of confirmed cases during phase 1. We compared the RuSH case definition with 11 alternative case definitions that included ≥2 to ≥6 encounters within the 5-year surveillance period and within any 1 year of the 5-year surveillance period. We defined the 1-year period on a rolling basis rather than as a calendar year. That is, for definitions requiring ≥2 SCD diagnosis codes within 1 year, the year was determined by the gap of time between ≥2 SCD encounters. We calculated sensitivity, specificity, PPV, and negative predictive value (NPV) for each case definition tested, using newborn screening, clinical, and medical record data as the reference standard. We calculated 95% confidence intervals (CIs) for these statistics and assumed significance for metrics that had nonoverlapping CIs.
Finally, to test whether including a single SCD ICD-9-CM code from an inpatient claim would improve performance of the preferred SCD case definition from phase 2 (≥3 encounters within 5 years), we subdivided the claims for patients with only 1 or 2 encounters within the 5-year surveillance period by whether or not ≥1 of the encounters was an inpatient admission. We conducted all analyses by using SAS version 9.4. 22 We received institutional review board approval under a public health practice exemption from participating institutions (Georgia State University, Emory University, and Augusta University).
Results
Of 1998 cases initially identified, we confirmed SCD in 1763 (88.2%) cases, excluded SCD in 197 (9.9%) cases, and declared SCD indeterminate in 37 (1.9%) cases (Figure). Most indeterminate cases were patients who had few encounters in Children’s Healthcare of Atlanta medical records, with no mention or suspicion of SCD, and insufficient laboratory results to exclude the possibility of SCD. Of 197 patients miscoded for SCD by ICD-9-CM coding, 65 (33.0%) were carriers (usually HbAS or HbAC), 11 (5.6%) had non-sickle hemoglobin disorder (eg, HbCC), 9 (4.6%) had thalassemias, 18 (9.1%) had other nonmalignant hematologic disorder, 19 (9.6%) had malignancy, and 75 (38.1%) had other or unknown diagnosis.
The likelihood of having SCD significantly increased with an increasing number of SCD encounters (P < .001) (Table 1). Fifty of 105 (47.6%) patients with ≤2 SCD encounters during the 5-year period did not have SCD, whereas three-quarters of patients with ≥3 encounters had SCD. For patients with ≥3 encounters, the addition of ≥1 SCD-associated treatment, procedure, or complication did little to improve the accuracy of case identification, adding <50% of true cases, but did serve to exclude more than 60% of non-SCD cases. Of 1998 patients, 1607 (80.4%) had ≥6 encounters during the 5-year period.
Relationship between the total number of encounters with a sickle cell disease (SCD) ICD-9-CM code and presence of an SCD-associated treatment, procedure, or complication in administrative data and known SCD status, Atlanta, Georgia, 2004-2008
Abbreviation: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification.
aData source: Centers for Disease Control and Prevention. 16
bAny treatment, procedure, or complication from the Registry and Surveillance System for Hemoglobinopathies (RuSH) probable case definition. The RuSH definition of a probable case required ≥2 SCD ICD-9-CM codes 16 on ≥2 separate health care encounters plus ≥1 hemoglobinopathy-associated complication, treatment, or procedure identified through ICD-9-CM codes, Current Procedural Terminology codes, 17 or National Drug Codes 18 but no laboratory-based disease confirmation available.
The performance of case definitions applied within the 1-year rolling period was similar to the same encounter counts applied within the 5-year period (Table 2). Small differences favored the 5-year period, but these differences were significant only in the case definitions that included ≥4 encounters. For the 5-year period, the PPV of ≥2 encounters (94.8%; 95% CI, 93.6%-95.7%) increased to 97.4% (95% CI, 96.4%-98.1%) with the addition of ≥1 encounter with an SCD-associated treatment, procedure, or complication (RuSH case definition), but the number of missed cases increased from 28 (1.5%) to 251 (16.1%) SCD cases identified. The RuSH case definition substantially underperformed on both sensitivity (85.8%) and NPV (38.2%) compared with case definitions that included ≥3 SCD encounters (96.0% sensitivity, 68.2% NPV) and ≥4 SCD encounters (93.4% sensitivity, 58.7% NPV).
Performance measures for sickle cell disease (SCD) case definitions, including RuSH probable case definition,a within 1 year and 5 years of administrative data, Atlanta, Georgia, 2004-2008
Abbreviations: CPT, Current Procedural Terminology 17 ; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification 16 ; RuSH, Registry and Surveillance System for Hemoglobinopathies.
aThe RuSH definition of a probable case required ≥2 SCD ICD-9-CM codes 16 on ≥2 separate health care encounters plus ≥1 hemoglobinopathy-associated complication, treatment, or procedure identified through ICD-9-CM codes, Current Procedural Terminology codes, 17 or National Drug Codes 18 but no laboratory-based disease confirmation available.
Upon further review of inpatient admissions for the 128 patients with 1 encounter, 9 had an SCD diagnosis recorded during an inpatient admission, 4 of whom had confirmed SCD. For the 92 patients with 2 encounters, 7 had an SCD diagnosis recorded during an inpatient admission, 4 of whom had confirmed SCD.
Discussion
Given the expansion of SCD surveillance to include 15 years or more of data, we sought to determine the best administrative case definition to use for longitudinal SCD surveillance. By taking elements from recent validation studies and examining different lengths of surveillance, we found an administrative case definition that improves the accuracy of case identification and is straightforward to implement.
The preferred SCD case definition suggested by our analysis (≥3 encounters within a 5-year period) substantially improved the sensitivity (96.0% vs 85.8%) and NPV (68.2% vs 38.2%) for identifying SCD, while maintaining a PPV (97.4% vs 97.4%) and a specificity (76.5% vs 79.0%) that were similar to those resulting from use of the RuSH case definition. The higher specificity and lower sensitivity of administrative case definitions used to identify more common conditions, such as adult hypertension 23 and pediatric diabetes, 24 might be expected when compared with administrative case definitions used to identify rare diseases, such as SCD. Because of a trade-off between sensitivity and specificity in choosing an administrative case definition for any condition, the optimal definition depends in part on the purpose for which the data are used. To define the burden of SCD for the purposes of public health resource allocation and planning, the number of cases identified should be maximized by choosing a definition that optimizes sensitivity. Conversely, for quality and outcome studies of SCD interventions, it is important to limit the number of non-SCD patients in the study sample by maximizing specificity. Thus, the performance measures we reported should be useful to health services planners and researchers who need to identify the best case definitions for their needs.
Findings from our study confirm and extend the results of recent studies that examined the validity of using administrative data to identify cases of SCD. Reeves et al 20 identified ≥3 SCD encounters within a 1-year period as the highest-performing and straightforward definition after testing 37 algorithms. The Reeves et al study reported a sensitivity of 89.7% and a specificity of 92.4%, compared with 96.0% sensitivity and 76.5% specificity in our study. Our lower specificity can be partially explained by including only patients with ≥1 SCD claim.
Addition of SCD-Associated Codes
Like Reeves et al, 20 who explored a wide array of services, pharmacy claims, and coding combinations in the tested algorithms, we examined the addition of SCD-associated treatments, procedures, and complications defined by the RuSH project to improve specificity. When we added treatments, procedures, and complications to the case definition of ≥2 SCD encounters, specificity increased to 79.0% but sensitivity decreased to 85.8%, because a substantial number of confirmed SCD cases were missed. Furthermore, 6 of the tested case definitions that relied only on the number of encounters outperformed the RuSH case definition on all 4 measures of performance, including specificity.
Contribution of Inpatient Codes
Recent studies also found high-performing SCD case definitions that give higher weight to SCD diagnosis codes from inpatient admissions than from outpatient encounters, for example, requiring only 1 inpatient SCD diagnosis code to identify a case. 25,26 Based on our results, giving a higher weight to SCD diagnosis codes from inpatient admissions rather than from other health care encounters did little to improve the performance of the case definition, because an equal number of cases and noncases would be identified for patients with few health care encounters.
Length of Surveillance Periods
Our study also compared the performance of case definitions applied during various surveillance periods. Results suggest that for the purposes of SCD surveillance, the performance of case definitions that include any number of encounters occurring in a 5-year period was similar to the performance of the same number of encounters clustered in any 1-year period during the 5 years of observation. This finding may reflect the fact that children with SCD in this validation cohort, followed in an SCD treatment center, had high rates of health care use—90% of confirmed SCD cases had ≥6 encounters during the 5-year period—which is consistent with children receiving at least annual follow-up care for their disease. Results for these children may not be representative of the general population of persons with SCD, which includes numerous adults with more limited access to SCD specialty care.
Miscoding of SCD
Lastly, we sought to determine the correct diagnosis for cases miscoded for SCD. Of 197 patients for whom an SCD ICD-9-CM code was misused at least once during the 5-year period, 10.2% had non-sickle hemoglobinopathy (eg, HbCC) or non-sickle thalassemia syndrome (eg, HbE-, α- or β-thalassemia syndrome). Such miscoding may have been due in part to the potentially confusing terminology and paucity of ICD-9-CM codes for these other hemoglobinopathies. Another 33% of miscoded cases were patients with the sickle cell trait or another hemoglobinopathy carrier state (eg, HbAC, HbAD). Sickle cell trait should be coded as 282.5, but coding for other carrier states may be less clear. We found that 18.7% of inaccurate SCD ICD-9-CM coding occurred for patients with other clinically important nonmalignant hematology disorders (eg, acquired hemolytic disorders) and malignant disorders. Many of these conditions require complex specialty services at the same location and from the same group of specialists that provide services for patients with SCD.
Limitations
This study had several limitations that might minimize its generalizability to other settings. First, we examined administrative claims data from public payers only. Studies that use claims data from private payers might generate different results, although children with SCD are more often covered by Medicaid than by private insurance. 27 Second, all patients included in our analysis were children and young adults who had received some health care services at a large academic tertiary health care system. Thus, the accuracy of using ICD-9-CM coding for disease surveillance in adult populations or for patients who receive care in other settings might be different. It is unclear how results might differ at a smaller health care system without a comprehensive SCD program. It is possible that a higher volume of SCD patients leads to better coding of confirmed cases and similarly leads to miscoding of children without SCD seen by SCD specialists. Finally, we conducted this analysis by using ICD-9-CM codes. Coding and surveillance using data from 2016 and later will need to use ICD-10 coding, which has the potential to better distinguish SCD from thalassemias, thereby potentially improving case identification.
Conclusion
This study supports an administrative case definition that includes ≥3 ICD-9-CM codes to identify SCD with a high degree of accuracy in pediatric patients and suggests that this definition can be useful for establishing longitudinal SCD surveillance systems. Results can also be used by public health practitioners and epidemiologic researchers to estimate the prevalence of SCD and to determine overall rates of disease-specific complications, emergency department use, and inpatient hospital use.
Footnotes
Acknowledgments
The authors acknowledge the contributions of other valued members of the Sickle Cell Data Collection program in Georgia and our colleagues in California.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Centers for Disease Control and Prevention (CDC) through a cooperative agreement with the Association of University Centers on Disabilities (5U38OT000140) and through the help and support of the CDC Foundation and Pfizer, Inc. Support was also provided by a grant from the Abraham J & Phyllis Katz Foundation.