Comparative Performance of Comorbidity Indices in Predicting Health Care-Related Behaviors and Outcomes among Medicaid Enrollees with Type 2 Diabetes

Abstract

No single gold standard of comorbidity measure has been identified, and the performance of comorbidity indices vary according to the outcome of interest. The authors compared the Charlson Comorbidity Index, Elixhauser Index (EI), Chronic Disease Score (CDS), and Health-related Quality of Life Comorbidity Index (HRQL-CI) in predicting health care-related behaviors (physicians' concordance with diabetes care standards and patients' oral antidiabetic drug [OAD] adherence) and outcomes (health care utilization and expenditures) among Medicaid enrollees with type 2 diabetes. A total of 9832 diabetes patients who used OAD were identified using data from the MarketScan Medicaid database from 2003 to 2007. Predictive performance of the comorbidity index was assessed using multiple regression models controlling for patient demographics, diabetes severity, and baseline health care characteristics. Among the 4 indices, the CDS was best at predicting physician's concordance with care standards. The CDS and HRQL-CI mental index performed better than other indices as predictors of medication adherence. The EI was best at predicting health care utilization and expenditures. These results suggest that, for these low-income diabetes patients, the CDS and HRQL-CI mental index were relatively better risk-adjustment tools for health care-related behavior data evaluation and the EI was the first choice for health care utilization and expenditures data. (Population Health Management 2012;15:220–229)

Introduction

A comorbidity measurement is a tool for quantifying the burden of coexisting medical conditions that are distinct from the primary condition under investigation.¹ Accurately measuring comorbidity is essential to the validity of the findings in epidemiological and health services research. Assessing comparative predictive abilities of comorbidity indices is critical because no single gold standard has been identified, and the predictive performance of a comorbidity index varies according to the outcome and population of interest.^2,3 Hence, the selection of comorbidity index in studies that require clinical risk adjustment should be specific to the outcome of interest, population, and source data. However, in most research, the choice of a comorbidity index often is based on the convenience of the data source for measuring comorbidities or simply on the most convenient method of measurement, rather than considering the relative performance of alternative comorbidity indices for a given outcome. Also, most studies that focus on the comparative performance of comorbidity indices were conducted for only 1 outcome (ie, mortality,^4

–11 health care utilization,^12,13 costs¹⁴). Health care-related behaviors, such as whether the physician's treatment is concordant with care standards or a patient's medication-taking behaviors, have seldom been used to compare various indices.

The Charlson Comorbidity Index (CCI),¹⁵ Elixhauser Index (EI),¹⁶ and Chronic Disease Score (CDS)¹⁷ are 3 commonly used comorbidity indices. The CCI and EI are diagnosis-derived measures and often are used by investigators focusing on mortality or health care utilization outcomes.^18
–20 The CDS is based on the patient's medications and has demonstrated better predictive ability in analyses of health care expenditures when compared to diagnoses-derived indices.^21,22 The Health-related Quality of Life Comorbidity Index (HRQL-CI) is a more recently developed, diagnosis-derived measure originally designed to predict HRQL as measured by the Short Form-12 using the Medical Expenditure Panel Survey database.²³ The HRQL-CI has been demonstrated to outperform CCI in predicting HRQL outcome in a general population and an asthma subsample.²³ Further research is needed to compare the HRQL-CI with other existing indices, other types of populations, and using other types of data sources to assess its performance for other types of health care outcomes. To date, no study has compared the CCI, EI, CDS, and HRQL-CI directly and comprehensively across different health care outcomes.

Medicaid plans cover lower income, ethnically diverse populations in the United States, and have higher percentages of participants with multiple and severe chronic medical conditions than commercial plans.²⁴ In terms of chronic disease, diabetes prevalence is almost twice as high among Medicaid beneficiaries compared to the general population.²⁵ Medicaid beneficiaries with diabetes tend to be older or disabled and typically have comorbid conditions that complicate their management. Coexisting medical conditions contribute to high costs and service use rates for these patients.²⁶ Therefore, accurately capturing comorbidity burden in this population is critical to identify populations that need improvements in care quality or access, and those that use resources disproportionately. However, there has been little research comparing the performance of comorbidity measures in this population.

The present study aimed to compare the predictive performance of commonly used comorbidity indices across different critical outcomes and to identify the most predictive measure specific to each outcome for Medicaid patients with diabetes.

Methods

Data source and study cohort

This observational cohort study used data extracted retrospectively from the Thomson's MarketScan Medicaid database from 2003 to 2007.²⁷ This database contained the medical, surgical, and prescription drug experiences of nearly 22 million Medicaid enrollees from 8 de-identified and geographically dispersed states across the United States. Although the states were de-identified, the data included at least 1 state from each US region. It included records of inpatient services, inpatient admissions, outpatient services, and prescription drug claims, as well as information on long-term care and other medical care. In addition to standard demographic variables such as age and sex, the database included variables of particular value to researchers who investigate Medicaid populations such as aid category (blind/disabled, Medicare eligible), race, service use, and provider type.²⁷

We defined the years 2004 to 2006 as the index drug identification period; within these 2 years, the index date was identified as the date of the first oral antidiabetic drug (OAD) claim. The pre period and post period included the time periods 1 year before and 1 year after the index date, respectively.

Patients included in the analytic cohort met the following criteria: (1) A diagnosis of type 2 diabetes (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM]=250.0x-250.9x). (2) Age 18 to 64 years. Individuals age 65 years and older were excluded because they are likely to be dual-eligible beneficiaries (Medicare and Medicaid) with Medicare utilization events not captured in the Medicaid files. (3) Continuous Medicaid enrollment for 12 months before and 12 months after the index date. (4) At least 1 filled prescription for OAD during the index period. (5) No OAD prescription in the pre-index period. New OAD users were targeted because patients on long-term therapy likely have different service use patterns. (6) Continuous medication therapy with the OAD index drug in the post-index period. Continuous therapy was defined as no gap in refilling of 60 or more days, and at least 2 prescriptions of the index OAD.

Study variables

As guided by Andersen's Behavioral Model of Health Services Use,²⁸ 3 types of covariates were included as predictors for health care outcomes, representing predisposing, enabling, and need factors in determining service use and treatment quality. Predisposing characteristics were patient's age, sex, and race/ethnicity; enabling characteristics included the type of health plan, type of provider (general practitioner vs. endocrinologist), and the number of therapeutic classes and number of medications prescribed; need factors were diabetes disease severity and comorbidity. Diabetes severity was defined using 3 common diabetes-related complications as indicators: nephropathy, neuropathy, and retinopathy. The presence of each complication was recorded as a dichotomous variable. All predictors were measured in the pre-index period. Measurements of comorbidity and health care-related outcomes of interest (ie, physician concordance with diabetes care standards, patient OAD medication adherence, health care utilization, costs) are specified in the following sections. All outcomes were measured in the post-index period.

Comorbidity index scores

Four types of comorbidity index scores were constructed: CCI, EI, CDS, and HRQL-CI. We used a modified version of the Romano-adapted CCI for use with administrative data²⁹; this comprises 17 disease items weighted according to disease severity as 1, 2, 3, or 6. Severity weights are based on the adjusted relative risks of death from the Cox proportional hazard regression model used in the original development of the index.³⁰ The final CCI score is the sum of weights assigned to all of a patient's comorbidities. The EI is originally measured as 30 dichotomous variables, each representing one of the comorbidity groups.¹⁶ It has been condensed to a single numeric score that summarizes disease burden and is adequately discriminative for death in hospital when analyzing administrative data.³¹ We used this condensed single EI score for comparison to other single index scores such as the CCI. The RxRisk ³² is a revised and expanded version of the original CDS.¹⁷ For adults, the RxRisk identifies 25 distinct comorbid conditions by linking them to medications used during treatment. Weights for the RxRisk will be taken directly from originally published prospective cost coefficient estimates.³³ The HRQL-CI consists of 2 lists: 20 physical and 15 clinical conditions for mental aspects of illness burden. Each condition is assigned a point weight based on its relative influence on the HRQL outcomes (Short Form-12 Physical Component Score or Mental Component Score).²³ The final HRQL-CI physical or mental index score is the sum of weights assigned to each condition presented by the patient. The scores derived from the CCI, EI, and HRQL-CI indices were based on a list of selected ICD-9-CM diagnosis codes from inpatient and outpatient claims, while the CDS was estimated utilizing selected National Drug Code numbers from pharmacy prescription claims. We excluded the diagnostic codes of type 2 diabetes because of the disease population studied. The time period for constructing comorbidity score was the pre-index period.

Physician concordance with diabetes care standards

The American Diabetes Association recommends that each of the following services be performed for adults with type 2 diabetes³⁴: semiannual hemoglobin (HbA1c) testing, and annual cholesterol tests, eye examinations, and mircoalbuminuria tests to detect kidney disease. These performance measures also have been used to evaluate the quality of diabetes care.^35,36 Researchers have evaluated the relationship between the comorbidity burden, measured using such tools as the CCI,³⁷ and the quality of diabetes care,^37
–39 and found that patients with a higher comorbidity burden received better care. In the current study, a summary measure was calculated representing the number of measures performed as recommended. A person who received more than 2 HbA1c tests during the year was given credit for only 1. Similarly, a person who received more than the recommended number of cholesterol tests, eye exams, or microalbumin tests was given credit for only the recommended number of tests. The summary score ranged from 0 to 4 standard concordant areas of care.

Patient medication adherence

Medication adherence focused on 3 common OADs as index drugs, including sulfonylureas, metformin, and thiazolidinediones and their fixed-dose regimens: Glucovance (glyburide plus metformin), Avplusamet (rosiglitazone plus metformin), Metaglip (glipizide plus metformin), and ActaoplusMet (pioglitazone plus metformin). Adherence was measured using medication possession ratio (MPR),⁴⁰ a common measure calculated based on administrative claims for medication fills.⁴¹ The MPR is defined as the proportion of days within an observation period for which a specific drug or class of drugs is supplied. In this study, the time window for measuring adherence began with the first date of OAD dispensing (index date) and ended with the dispensing date of the last prescription within the post-index period. The formulas for computing the MPR for each OAD drug or regimen were as follows.

For monotherapy or fixed-dose regimen: MPR=total days supply obtained/(date of the last claim − date of the first claim+days supply of the last claim);

For combination therapy (eg, using more than 1 type of index drug or switching drugs): MPR=total days supply obtained/n*(date of the last claim − date of the first claim+days supply of the last claim)

(n=no. of OAD combined, [eg, for dual therapy, n=2])

If the MPR exceeded 1.0 (indicating overstocking) it was truncated at 1.0.

Using standard conventions, we classified patients with an MPR of less than 0.8 (eg, less than 80% adherent) as nonadherent.⁴² This threshold has been used in previous studies of OAD adherence among Medicaid beneficiaries and found to discriminate well in terms of patients' medication-taking behaviors.⁴³

Health care utilization and costs

Health care utilization included total number of hospitalizations, emergency room (ER) visits, and outpatient visits. Health care costs included total costs and diabetes care-related costs in the post-index period. Total cost was the sum of the costs submitted from both inpatient and outpatient claims files. Diabetes-related cost was the sum of all medical claims with a primary diagnosis of type 2 diabetes

Statistics

Descriptive statistics of population characteristics were calculated, including means, standard deviations, and proportions as appropriate. The correlations between comorbidity indices were assessed using Spearman rank correlations.

Predictive ability of comorbidity indices for health care outcomes was assessed using multiple regression techniques controlling for patient demographics, type of health plan, type of provider, number of therapeutic classes, and number of medications prescribed, diabetes disease severity, and baseline health care-related characteristics (eg, health care costs in the pre-index period). Specific regression analyses were chosen based on the property of each outcome. Scores of physician concordance with diabetes care standards, a count variable, was modeled using standard Poisson regression. We modeled medication adherence as a dichotomous variable based on an MPR value of 0.8 (MPR≥0.8=adherent; MPR<0.8=not adherent) and applied logistic regression analysis. Zero-inflated binomial regression analysis was applied for each type of health care utilization data. Medical expenditures data were skewed considerably to right. To deal with skewed data, a generalized linear model with gamma family and log link function was used.⁴⁴

The likelihood ratio for goodness of fit, deviance, and adjusted R² were reported as statistical evidence of model fit, compared to the nested intercept only model. The deviance is the value that compares a given model to a fully saturated one so that it reflects error associated with the model even after the predictors are included in the model. The smaller the deviance, the better the model fits the data. Two common information criterion measures, the Akaike's information criterion (AIC) and Bayesian information criterion (BIC),⁴⁵ were used to compare non-nested models; the model with the smallest AIC or BIC value was the best model.⁴⁶ The AIC difference (ΔAIC) was then calculated following Anderson and Burnham's method to select the best approximating model.⁴⁷ The ΔAIC is the difference between the AIC and the minimum of AIC over all candidate models, which provides a quantitative measure of model plausibility. A zero difference between 2 models indicates the best model, models having ΔAIC<2–3 are nearly tied, models having ΔAIC between 7 and 10 are considered fair, and models having ΔAIC>7 to 10 are substantially inferior.⁴⁷

Data management was conducted using SAS software version 9.1 (SAS Institute Inc., Cary, NC) and then data were converted into STATA software (StataCorp LP, College Station, TX) format for the analyses. This study protocol was approved by the University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board.

Results

Characteristics of study population

The study population characteristics are displayed in Table 1.

Table 1.

Characteristics of Study Population (n=9832)

Type	Variables	Mean (S.D.)	Frequency (%)
Predisposing (pre-index period)	Age (years)	44.81 (11.64)	–
	Sex (female)	–	7183 (73.06)
	Race	–
	White		5139 (52.27)
	Black		3096 (31.49)
	Hispanic		151 (1.54)
	Others		1239 (12.60)
	Multiracial		207 (2.11)
Need (pre-index period)	Diabetes severity	–
	Nephropathy		152 (1.55)
	Neuropathy		506 (5.15)
	Retinopathy		152 (1.55)
	Charlson Comorbidity Index Score (range: 0–35)	0.709 (1.27)	–
	Elixhauser Index Score (range: 0–30)	1.73 (1.55)	–
	Chronic Disease Scores (range: 0–18)	4.64 (2.96)	–
	HRQL-CI Scores		–
	Physical domain (range: 0–35)	4.15 (3.69)
	Mental domain (range: 0–25)	3.65 (3.39)
Enabling (pre-index period)	Type of health plan	–
	Fee-for-service		5448 (55.41)
	Capitated plan		3203 (32.58)
	Both		1181 (12.01)
	Type of provider: at least 1 endocrinologist visit (yes/no)	–	31 (0.32)
	Total no. of therapeutic classes	10.82 (10.82)	–
	Total no. of drugs supplied	557.35 (48.64)	–
Physician's adherence to diabetes care (post-index period)	At least 2 HbA1c tests/year		6950 (70.69)
	At least 1 LDL test/year		6209 (63.15)
	At least 1 nephropathy screening/year		2326 (23.66)
	At least 1 eye examination/year		4297 (43.70)
	Total physician treatment adherence score (range=0–4)	2.51 (1.11)
Patient's adherence to diabetes medication (post-index period)	Overall adherence (MPR) for 3 selected OADs (Met, Sulfa, TZD) (n=9832)	0.81 (0.26)	–
	MPR for monotherapy (n=7888)	0.78 (0.23)
	Met (n=62)	0.70 (0.21)
	Sulfa (n=5949)	0.77 (0.24)
	TZD (n=1877)	0.81 (0.21)
	MPR for fixed dose regimens^a (n=290)	0.78 (0.21)
	MPR for switching or combination regimens (n=1645)	0.95 (0.12)
Health care utilization	Total no. of hospital admissions
	Pre-index period	0.38 (0.09)
	Post-index period	0.35 (0.096)
	Total no. of emergency room visits
	Pre-index period	0.23 (0.09)
	Post-index period	0.21 (0.74)
	Total no. of outpatient visits
	Pre-index period	24.47 (14.85)
	Post-index period	27.43 (17.45)
Health care costs	Total costs ($)
	Pre-index period	8318.34 (24,051)
	Post-index period	8807.67 (27,204)
	Diabetes care-related costs ($)
	Pre-index period	1282.93 (8381)
	Post-index period:	2257.99 (9968)

HRQL-CI, Health-related Quality of Life Comorbidity Index; LDL, low-density lipoprotein; Met, metformin; MPR, medication possession ratio; OAD, oral antidiabetic medication; Sulfa, sulfonylurea; TZD, thiazolidinediones; ^a:4 fixed-dose regimens are Glucovance, Avplusamet, Metaglip, and Actaoplus Met.

The study population of 9,832 patients was on average 44.81 years (SD, 11.64). The majority was female (73%), White (52%), on the fee-for-service plan (55%), and with average 11 (SD, 10.82) different types of therapeutic classes and average 557 (SD, 48.64) drugs prescribed during the pre-index period. Two percent of individuals were diagnosed with nephropathy, 5% of individuals with neuropathy and 2% of individuals with retinopathy.

Over half of the population had HbA1c tests at the recommended frequency. More than half had the recommended one LDL-c test and one eye examination per year. Less than one quarter of the population had at least one nephropathy screening per year. The average score for physician's concordance with care standards was 2.51 (SD, 1.11). The average MPR was 0.81 (SD, 0.26). The average MPR for patients having switching or combination regimens was higher than patients on monotherapy or fixed-dose regimens. Within patients on monotherapy, those treated by thiazolidinediones had highest average MPR scores.

The average of health care utilization per year in pre-index period was generally higher than in post-index period across different types of health care services. However, the average of medical costs or diabetes care-related costs per year in the post-index period was higher than in the pre-index period.

Correlations between comorbidity indices

The correlations between any 2 of the diagnosis-based indices (ie, CCI, EI, HRQL-CI) were fair (P>0.5), with the exception of a small correlation between the CCI and HRQL-CI mental index (Table 2). Also, there were fair correlations between any 1 of the diagnosis-based indices and medication-based index (CDS), with the exception of a slightly low correlation between the CCI and CDS.

Table 2.

Spearman Rank Correlations of Comorbidity Indices

	Charlson Comorbidity Index	Elixhauser Index	HRQL-CI-Physical Aspect	HRQL-CI-Mental Aspect	Chronic Disease Score
Charlson Comorbidity Index	1.00
Elixhauser Index	0.56	1.00
HRQL-CI-Physical Aspect	0.55	0.65	1.00
HRQL-CI-Mental Aspect	0.39	0.59	0.68	1.00
Chronic Disease Score	0.41	0.59	0.60	0.52	1.00

HRQL-CI, Health-related Quality of Life Comorbidity Index; All correlations between any 2 different indices were statistically significant (P<0.0001).

Predictive performance of comorbidity index

When predicting physician concordance with diabetes care standards, the model with CDS comorbidity scores had the best fit (smallest deviance, AIC and BIC values, and higher adjusted R² ) (Table 3). In predicting patient's medication adherence, the HRQL-CI mental index-based model had the best fit (smallest deviance, AIC and BIC values, higher R² ). The difference in the AIC (ΔAIC) between the HRQL-CI mental index- and CDS-based models was close to 3, indicating that these 2 index-based models were tied (2 approximating models, in terms of model fit). However, the ΔAIC between the HRQL-CI mental index and CCI, EI, or HRQL-CI physical index-based model was over 7, implying that the fit in the models with other comorbidity index scores were substantially inferior compared with the HRQL-CI mental index-based model.

Table 3.

Predictive Validity of Comorbidity Indices in Health Care-Related Behaviors

Response Variable: Physician Concordance with Care Standard
	Goodness of Fit for overall model ^a
Predictor of interest	Likelihood ratio ^b	Deviance	McFadden's Adjusted R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in physicians' care standard concordance score ^c
CCI	202.81	32031.18	0.14	32075.18	32233.41	−0.2, p=0.644
EI	210.96	32023.03	0.15	32067.03	32225.27	1.4, p=0.000
CDS	219.71	32014.28	0.15	32058.28	32216.52	1.7, p=0.000
HRQL-CI–physical	209.70	32024.29	0.15	32068.29	32226.53	0.6, p=0.00
HRQL-CI–mental	205.23	32028.73	0.14	32072.73	32230.97	0.4, p=0.015

Response Variable: Patient Oral Antidiabetic Medication Adherence
	Goodness of Fit for overall model ^a
Predictor of interest	Likelihood ratio ^b	Deviance	Max-rescaled R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in the odds of being medication adherent ^d
CCI	495.94	12394.62	0.17	12430.62	12560.08	0.6, p=0.7596
EI	496.77	12393.79	0.17	12429.79	12559.25	1.7, p=0.3366
CDS	502.95	12387.61	0.17	12423.61	12553.08	3.7, p=0.0078
HRQL-CI–physical	496.23	12394.32	0.17	12430.32	12559.79	−0.5, p=0.5335
HRQL-CI–mental	506.16	12384.40	0.19	12420.40	12549.87	−2.5, p=0.0013

AIC, Akaike's information criterion; BIC, Bayesian information criterion; CCI, Charlson Comorbidity Index; EI, Elixhauser Index; CDS, Chronic Disease Score; HRQL-CI, Health-related Quality of Life Comorbidity Index. ^aThe variables included in the model were comorbidity score, patient's age, race, sex, type of health plan, type of provider, number of therapeutic classes and number of medications prescribed, diabetes disease severity, and baseline health care-related characteristics (eg, health care costs in pre-index period). ^b X² test for the likelihood ratio for each individual model was statistically significant (P=0.000). ^c The analysis is based on standard Poisson regression, the value in the column was calculated as: 100^*b%, where b=beta-coefficient. ^dThe analysis based on logistic regression, the value in the column was computed by the formula: 100^*[exp]c-1]%, where exp(c)=exponentiated coefficient.

Comorbidity scores measured by the CDS or HRQL-CI-mental index were statistically significantly associated with medication adherence. However, these indices indicated different directions of comorbidity impact.

When predicting hospitalization and ER visits, the models with diagnosis-based comorbidity index scores (CCI and EI) demonstrated better model fit (smaller deviance, AIC and BIC values), compared to medication-based comorbidity index scores (CDS) (Table 4). Because the ΔAIC between the CCI and EI-based models was close to 3, these 2 models were tied. However, the ΔAIC between the CCI (or EI) and CDS-based models was over 10, implying that the model with a medication-based comorbidity index score was substantially inferior to one with diagnosis-based scores. In predicting outpatient visits, the model with HRQL-CI mental index scores had the best fit (smallest deviance, AIC and BIC values, higher R² ).

Table 4.

Predictive Performance of Comorbidity Index in Health Care Utilization

Response Variable: Number of Hospitalizations
	Goodness of Fit for overall model ^a
Predictor of interest	Likelihood ratio ^b	Deviance	McFadden's Adjusted-R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in hospitalizations ^c
CCI	1122.29	13604.65	0.07	13706.65	14073.48	10.22^***
EI	1120.30	13606.64	0.07	13708.64	14075.46	10.23^***
CDS	1110.06	13616.87	0.07	13718.87	14085.70	6.67^***
HRQL-CI–physical	1104.80	13622.14	0.07	13724.14	14090.97	3.21^***
HRQL-CI–mental	1111.11	13615.82	0.07	13717.82	14084.65	3.83^***

Response Variable: Number of Emergency Room (ER) Visits
	Goodness of Fit for overall model ^a
Predictor of interest	Likelihood ratio ^b	Deviance	McFadden's Adjusted-R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in ER ^c
CCI	1075.97	9394.46	0.09	9496.46	9863.29	9.2^***
EI	1074.87	9395.57	0.09	9497.57	9864.39	9.5^***
CDS	1072.65	9397.78	0.09	9499.784	9866.61	7.4^***
HRQL-CI–physical	1067.27	9421.12	0.09	9505.17	9872.0	3.2^***
HRQL-CI–mental	1072.03	9398.41	0.09	9500.41	9867.232	4.0^***

Response Variable: Number of Outpatient Visits
	Goodness of Fit for overall model ^a
Predictor of interest	Likelihood ratio ^b	Deviance	McFadden's Adjusted-R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in outpatient visits ^c
CCI	8117.51	76861.95	0.09	76961.95	77321.58	6.6^***
EI	8298.86	76680.59	0.10	76780.59	77140.22	9.8^***
CDS	8131.51	76847.94	0.09	76935.94	77252.42	5.2^***
HRQL-CI–physical	8245.29	76734.17	0.10	76836.17	77202.99	3.9^***
HRQL-CI–mental	8438.71	76540.74	0.10	76640.74	77000.37	5.3^***

AIC, Akaike's information criterion; BIC, Bayesian information criterion; CCI, Charlson Comorbidity Index; EI, Elixhauser Index; CDS, Chronic Disease Score; HRQL-CI, Health-related Quality of Life Comorbidity Index. ^a The variables included in the model were comorbidity score, patient's age, race, sex, type of health plan, type of provider, number of therapeutic classes and number of medications prescribed, diabetes disease severity, and baseline health care-related characteristics (eg, health care costs in pre-index period). ^b X² test for the likelihood ratio for each individual model was statistically significant (P=0.000). ^c The analysis is based on zero-inflated negative binomial regression, the value in the column was calculated as: 100^*b%, where b=beta-coefficient; ^*** P value=0.000.

The models with EI comorbidity scores had better overall fit in predicting health care expenditures (smallest deviance, AIC and BIC values, higher R² ) (Table 5).

Table 5.

Predictive Performance of Comorbidity Index in Health Care Expenditures

Response Variable: Total Medical Costs
	Goodness of Fit for overall model^a
Predictor of interest	Likelihood ratio ^b	Deviance	Pseudo-R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in the cost^c
CCI	5382.44	18567.38	0.27	19.63	−71474.98	11.51^***
EI	5509.74	18452.98	0.28	19.61	−71589.37	15.32^***
CDS	5374.54	18576.07	0.27	19.63	−71466.28	8.37^***
HRQLCI–physical	5391.14	18562.69	0.27	19.63	−71479.67	4.99^***
HRQL-CI–mental	5498.35	18460.49	0.28	19.61	−71581.87	6.48^***

Response Variable: Diabetes Care-Related Costs
	Goodness of Fit for overall model^a
Predictor of interest	Likelihood ratio^b	Deviance	Pseudo-R²	AIC	BIC	For 1-unit increase in comorbidity score, % change in the cost^c
CCI	5515.87	27111.63	0.32	16.89	−62939.91	14.93^***
EI	6235.12	26661.73	0.36	16.82	−63389.82	28.67^***
CDS	5808.91	26924.47	0.34	16.86	−63127.08	16.12^***
HRQL-CI–physical	5989.68	26829.87	0.35	16.84	−63221.68	9.89^***
HRQL-CI–mental	5824.17	26879.12	0.34	16.86	−63172.42	8.55^***

AIC, Akaike's information criterion; BIC, Bayesian information criterion; CCI, Charlson Comorbidity Index; EI, Elixhauser Index; CDS, Chronic Disease Score; HRQL-CI, Health-related Quality of Life Comorbidity Index. ^a The variables included in the model were comorbidity score, patient's age, race, sex, type of health plan, type of provider, number of therapeutic classes and number of medications prescribed, diabetes disease severity, and baseline health care-related characteristics (eg, health care costs in pre-index period). ^b X² test for the likelihood ratio for each individual model was statistically significant (P=0.000). ^c The analysis is based on the Generalized Linear Model with gamma family and log link, the value in the column was computed by the formula: 100^*[exp(c)-1]%, where exp(c)=exponentiated coefficient; ^*** P value=0.000.

Discussion

To our knowledge, this is the first study to evaluate the comparative predictive performance of the CCI, EI, CDS, and HRQL-CI indices across different critical outcomes among a Medicaid-insured sample with chronic diseases. Overall, our findings showed that the predictive performance of a comorbidity index varied depending on the outcome of interest and, therefore, we suggest that the selection of comorbidity index should be specific to a given health care outcome (as summarized in Table 6). Our results also demonstrate the potential direction and magnitude of comorbidity impact on health care outcome.

Table 6.

Summary of Predictive Performance of Comorbidity Index in Health Care-Related Behaviors and Outcomes

Health Care-Related Outcomes	Comorbidity Index With Best Predictive Ability	Potential Comorbidity Impact
Physician concordance with diabetes care standard	Chronic Disease Score	Physician concordance with diabetes care standard score increased by 1.7%
Patient oral antidiabetic medication adherence	Chronic Disease Score	For increase in overall comorbidity burden: odds of being medication adherent increased by 3.7%
	HRQL-CI Mental Index	For increase in mental aspect of comorbidity burden: odds of being medication adherent decreased by 2.5%
Hospitalizations	Elixhauser Index	Hospitalizations increased by 10.2%
Emergency room visits	Elixhauser Index	ER visits increased by 9.5%
Outpatient visits	HRQL-CI Mental Index	Outpatient visits increased by 5.3%
Total costs	Elixhauser Index	Total costs increased by 15.3%
Diabetes care-related costs	Elixhauser Index	Diabetes care-related costs increased by 28.7%

ER, emergency room; HRQL-CI, Health related Quality of Life Comorbidity Index.

First, when analyzing physician concordance with diabetes care standards as the outcome, the CDS comorbidity score had best predictive ability. According to the CDS comorbidity score, comorbidity positively influenced physicians' care behavior, which supports previous findings that diabetes patients who have a greater comorbidity burden are more likely to receive recommended diabetes care.^37,38 This may be because patients with multiple conditions have a greater number of clinic visits scheduled, have lower no-show rates at scheduled appointments, or receive more medication prescription treatment and greater attention to comprehensive diabetes management standards during scheduled encounters.

When predicting patients' medication adherence, the CDS and HRQL-CI mental index were relatively better comorbidity measurements. However, these indices indicated different directions of comorbidity influence: as the CDS comorbidity score increased, medication adherence increased; however, as the HRQL-CI mental index score increased, adherence decreased. This difference may be because the CDS and HRQL-CI mental index measure different underlying dimensions of comorbidity burden. The CDS index, which is based on a list of medications representing 30 underlying disease conditions,^32,33 attempts to provide a comprehensive picture of the patient's overall comorbidity burden, while the HRQL-CI mental index originally was developed to capture the mental aspect of illness burden.²³ Because these 2 indices might represent different underlying disease profiles, our results suggest that mental illness burden could have a deterrent effect on medication adherence, while overall illness burden may enhance patients' adherence to treatment. Consistently, previous research that focused on patients with diabetes showed that the mental aspect of illnesses, particularly depression, had a negative impact on their OAD adherence.^48
–50 Research also has found that increasing overall illness burden was associated with higher medication adherence.⁵¹ It is conceivable that patients who have a higher number of chronic conditions could be better informed about diabetes and its complications and, therefore, would maintain higher rates of adherence despite their greater medication burden and numerous comorbidities. Also, increased perceived susceptibility and severity resulting from comorbid condition burden may motivate patients to improve their medication-taking behavior.

Moreover, analyzing health care utilization data in the context of hospitalization and ER visits, the diagnosis-based index (ie, EI) had better predictive performance than the medication-based index (CDS). This confirmed previous research findings, which showed that diagnosis-based comorbidity scores had better ability to predict health care utilization compared to a medication-based index.¹² Interestingly, we also found that the mental illness aspect of the comorbidity index (HRQL-CI mental index) had the best predictive ability for health care use in the context of outpatient visits, compared to other comorbidity indices that measure overall comorbidity burden (ie, EI, CDS). Further research is needed to validate this finding in other populations and health care settings.

Furthermore, regarding medical payment data, our findings support previous research focused on Medicaid enrollees, which demonstrated that the diagnosis-based index had better predictive ability than the medication-based index among Medicaid beneficiaries.⁵² However, this finding may be specific to the context of the Medicaid population because some previous literature demonstrated that the medication-based index had better performance in predicting health care expenditures (for elderly patients diagnosed with hypertension,⁵³ community dwelling elderly,⁵⁴ and migraine patients⁵⁵).

These study findings should be interpreted in light of the following limitations. First, because the analyses were based on claims data, information on services not billed to Medicaid was not available (ie, patients may have received treatment that was not submitted to their health plan for reimbursement and thus was not included in the claims data). Health care cost in our analysis, which was based only on the claim for reimbursement, may not reflect actual payments for services, although our results should show the same trends in general. Second, correct categorization of insurance database information depends on correct coding by clinicians and other medical staff. The accuracy of diagnostic coding cannot be evaluated in a claims-based study. When coding each ICD-9-CM claims-based measurement, the possibility exists that diagnoses that were designated to be ruled out for billing purposes were misclassified as existing comorbidities.¹ Third, data on comorbidities were limited to the conditions coded on medical claims within the time frame studied. Fourth, caution should be used when generalizing results beyond the study population of continuously enrolled Medicaid type 2 diabetes patients aged 18 to 64 who use OAD. Also, our sample was predominantly female and white.

In conclusion, while more work is warranted to evaluate whether these findings can be supported in other populations, these results are nevertheless important for epidemiological and health services researchers in the selection and use of existing alternative comorbidity indices to assess and control for comorbidity burden on health care outcomes.

Footnotes

Author Disclosure Statement

Drs. Ou, Mukherjee, Erickson, Piette, Bagozzi, and Balkrishnan disclosed no competing financial interests.

References

Klabunde

, Potosky

, Legler

, Warren

. Development of a comorbidity index using physician claims data. J Clin Epidemiol, 2000; 53:1258–1267.

Byles

, D'Este

, Parkinson

, O'Connell

, Treloar

. Single index of multimorbidity did not predict multiple outcomes. J Clin Epidemiol, 2005; 58:997–1005.

Holman

, Preen

, Baynham

, Finn

, Semmens

. A multipurpose comorbidity scoring system performed better than the Charlson index. J Clin Epidemiol, 2005; 58:1006–1014.

Miskulin

, Martin

, Brown

et al. Predicting 1 year mortality in an outpatient haemodialysis population: A comparison of comorbidity instruments. Nephrol Dial Transplant, 2004; 19:413–420.

Soares

, Salluh

, Ferreira

, Luiz

, Spector

, Rocco

. Impact of two different comorbidity measures on the 6-month mortality of critically ill cancer patients. Intensive Care Med, 2005; 31:408–415.

Yan

, Birman-Deych

, Radford

, Nilasena

, Gage

. Comorbidity indices to predict mortality from Medicare data: Results from the national registry of atrial fibrillation. Med Care, 2005; 43:1073–1077.

Schneeweiss

, Wang

, Avorn

, Maclure

, Levin

, Glynn

. Consistency of performance ranking of comorbidity adjustment scores in Canadian and U.S. utilization data. J Gen Intern Med, 2004; 19:444–450.

Wang

, Walker

, Tsuang

, Orav

, Levin

, Avorn

. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol, 2000; 53:571–578.

Martins

, Blais

. Evaluation of comorbidity indices for inpatient mortality prediction models. J Clin Epidemiol, 2006; 59:665–669.

10.

Rius

, Perez

, Martinez

et al. An adaptation of Charlson comorbidity index predicted subsequent mortality in a health survey. J Clin Epidemiol, 2004; 57:403–408.

11.

Chaudhry

, Jin

, Meltzer

. Use of a self-report-generated Charlson Comorbidity Index for predicting mortality. Med Care, 2005; 43:607–615.

12.

Dominick

, Dudley

, Coffman

, Bosworth

. Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis. Arthritis Rheum, 2005; 53:666–672.

13.

Wahls

, Barnett

, Rosenthal

. Predicting resource utilization in a Veterans Health Administration primary care population: Comparison of methods based on diagnoses and medications. Med Care, 2004; 42:123–128.

14.

Farley

, Harley

, Devine

. A comparison of comorbidity measurements to predict healthcare expenditures. Am J Manag Care, 2006; 12:110–119.

15.

Charlson

, Pompei

, Ales

, MacKenzie

. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chronic Dis, 1987; 40:373–383.

16.

Elixhauser

, Steiner

, Harris

, Coffey

. Comorbidity measures for use with administrative data. Med Care, 1998; 36:8–27.

17.

Von Korff

, Wagner

, Saunders

. A chronic disease score from automated pharmacy data. J Clin Epidemiol, 1992; 45:197–203.

18.

, Evans

, Faris

, Dean

, Quan

. Risk adjustment performance of Charlson and Elixhauser comorbidities in ICD-9 and ICD-10 administrative databases. BMC Health Serv Res, 2008; 8:12.

19.

Southern

, Quan

, Ghali

. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care, 2004; 42:355–360.

20.

Stukenborg

, Wagner

, Connors

Jr.

Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations. Med Care, 2001; 39:727–739.

21.

Farley

. FDA's Rx for better medication information. FDA Consum, 1995; 29:5–10.

22.

Schneeweiss

, Maclure

. Use of comorbidity scores for control of confounding in studies using administrative databases. Int J Epidemiol, 2000; 29:891–898.

23.

Mukherjee

, Ou

, Wang

, Erickson

. A new comorbidity index: The health-related quality of life comorbidity index. J Clin Epidemiol., 2011; 64:309–319.

24.

Adelmann

. Mental and substance use disorders among Medicaid recipients: Prevalence estimates from two national surveys. Adm Policy Ment Health, 2003; 31:111–129.

25.

Smith

, Gifford

, Ellis

, Wiles

. Low Medicaid Spending Growth Amid Rebounding State Revenues: Results from a 50-State Budget Survey. State Fiscal Years 2006-2007. Washington, DC: Kaiser Commission on Medicaid and the Uninsured, October 2006.

26.

Cohen

. Kaiser Commission on Medicaid and the Uninsured: An Overview of Medicaid Enrollees with Diabetes in 2003. Washington, DC: Kaiser Family Foundation, October 2007.

27.

Adamson

, Chang

, Hansen

. Health Research Data for the Real World: The MarketScan Databases. Ann Arbor, MI: Thomson Healthcare, 2008.

28.

Aday

, Andersen

. A framework for the study of access to medical care. Health Serv Res, 1974; 9:208–220.

29.

Romano

, Roos

, Jollis

. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: Differing perspectives. J Clin Epidemiol, 1993; 46:1075–1079discussion 1081–1090.

30.

Charlson

, Szatrowski

, Peterson

, Gold

. Validation of a combined comorbidity index. J Clin Epidemiol, 1994; 47:1245–1251.

31.

van Walraven

, Austin

, Jennings

, Quan

, Forster

. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care, 2009; 47:626–633.

32.

Fishman

, Goodman

, Hornbrook

, Meenan

, Bachman

, O'Keeffe Rosetti

. Risk adjustment using automated ambulatory pharmacy data: The RxRisk model. Med Care, 2003; 41:84–99.

33.

Clark

, Von Korff

, Saunders

, Baluch

, Simon

. A chronic disease score with empirically derived weights. Med Care, 1995; 33:783–795.

34.

American Diabetes Association. Standards of medical care in diabetes—2008. Diabetes Care, 2008; 31,suppl 1:S12–S54.

35.

National Committe for Quality Assirance. HEDIS 2007 Volume 2: Technical Update. Washington, DC: National Committee for Quality Assurance, 2006.

36.

North Carolina Division of Medical Assistance. Quality, evaluation, and health outcomes (QEHO) initiatives. http://www.ncdhhs.gov/dma/quality/qehoinitiatives.html. 2009 September 27.

37.

Halanych

, Safford

, Keys

et al. Burden of comorbid medical conditions and quality of diabetes care. Diabetes Care, 2007; 30:2999–3004.

38.

Clark

, Weir

, Ouellette

, Zhang

, Baxter

. Beyond health plans: Behavioral health disorders and quality of diabetes and asthma care for Medicaid beneficiaries. Med Care, 2009; 47:545–552.

39.

Millett

, Bottle

, Ng

et al. Pay for perfomance and the quality of diabetes management in individuals with and without co-morbid medical conditions. J R Soc Med, 2009; 102:369–377.

40.

Steiner

, Prochazka

. The assessment of refill compliance using pharmacy records: methods, validity, and applications. J Clin Epidemiol, 1997; 50:105–116.

41.

Andrade

, Kahler

, Frech

, Chan

. Methods for evaluation of medication adherence and persistence using automated databases. Pharmacoepidemiol Drug Saf, 2006; 15:565–574, discussion 575–577.

42.

Sikka

, Xia

, Aubert

. Estimating medication persistency using administrative claims data. Am J Manag Care, 2005; 11:449–457.

43.

Cheong

, Barner

, Lawson

, Johnsrud

. Patient adherence and reimbursement amount for antidiabetic fixed-dose combination products compared with dual therapy among Texas Medicaid recipients. Clin Ther, 2008; 30:1893–1907.

44.

Manning

, Mullahy

. Estimating log models: To transform or not to transform? J Health Econ, 2001; 20:461–494.

45.

Akaike

. Information theory and an extension of the maximum likelihood principle. Petrov

, Csaki

. 2nd International Symposium on Information Theory. Budapest, Hungary: Akademiai Kiado, 1973; 267–281.

46.

Burnham

, Anderson

. Model Selection and Multi-Model Inference: A Practical Information-Theoretic Approach, 2nd. New York: Springer-Verlag, 2002.

47.

Anderson

, Burnham

. General strategies for the analysis of ringing data. Bird Study, 1999; 46:261–270.

48.

Kilbourne

, Reynolds

3rd , Good

, Sereika

, Justice

, Fine

. How does depression influence diabetes medication adherence in older patients? Am J Geriatr Psychiatry, 2005; 13:202–210.

49.

Lin

, Katon

, Von Korff

et al. Relationship of depression and diabetes self-care, medication adherence, and preventive care. Diabetes Care, 2004; 27:2154–2160.

50.

Nau

, Chao

, Aikens

. The relationship of guideline-concordant depression treatment and patient adherence to oral diabetes medications. Res Social Adm Pharm, 2005; 1:378–388.

51.

Kerr

, Heisler

, Krein

et al.

Beyond comorbidity counts: How do comorbidity type and severity influence diabetes patients' treatment priorities and self-management?

J Gen Intern Med, 2007; 22:1635–1640.

52.

Gilmer

, Kronick

, Fishman

, Ganiats

. The Medicaid Rx model: Pharmacy-based risk adjustment for public programs. Med Care, 2001; 39:1188–1202.

53.

Schneeweiss

, Seeger

, Maclure

, Wang

, Avorn

, Glynn

. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol, 2001; 154:854–864.

54.

Perkins

, Kroenke

, Unutzer

et al. Common comorbidity scales were similar in their ability to predict health care costs and mortality. J Clin Epidemiol, 2004; 57:1040–1048.

55.

Baser

, Palmer

, Stephenson

. The estimation power of alternative comorbidity indices. Value Health, 2008; 11:946–955.