Abstract
Background:
The Veterans Affairs (VA)-developed Care Assessment Need (CAN) score, a risk-stratification tool used to identify complex high-risk patients and guide VA care coordination, was designed to predict hospitalization or death. Little is known about its utility in predicting primary care utilization or if gender differences in this metric are detectable. Our objective was to determine association of CAN score quintiles with high primary care visit (PCV) utilization among Veterans, the impact of adding reproductive health and psychosocial variables to the model and the difference between men and women Veterans.
Methods:
The sample included men and women from the post-9/11 cohort receiving VA care for at least 1 year, 2010–2017 (N = 665,379). PCV data for each year were collected from national Corporate Data Warehouse. A cumulative count ≥6 visits in a year was used as an indication for high PCV utilization in the analyses.
Results:
After accounting for potential confounding factors, women were associated with 42% higher odds of heavy PCV utilization (adjusted odds ratio: 1.42, 95% confidence interval: 1.37–1.46) than men. However, there was a significant interaction between sex and CAN quintiles (p < 0.001). After adjusting for all the covariates, CAN score quintiles appeared to have stronger associations and better predictive accuracy on the risk of 1-year heavy PCV utilization for men than for women.
Conclusion:
Further research is needed to understand sex differences in Veterans Health Administration clinical complexity measures and whether they can be successfully used to identify high-risk, high-utilizing women Veterans.
Introduction
Women Veterans comprise a growing proportion of Veterans Health Administration (VHA) users. There are now >700,000 women enrolled, 1 and >500,000 women Veterans actively using Veterans Affairs (VA) care. 2 Women Veterans who use VA care have high rates of mental health conditions, 2 trauma exposure, 3 and chronic medical conditions 4 resulting in complex intersecting comorbidities. The complex comorbidities among women Veterans challenge overburdened health care providers to meet their needs. Women Veterans have more visits, 2 longer visits, and more between-visit communications. They require gender-specific care and care coordination including health screenings for breast and cervical cancer, care for reproductive needs (contraception/pregnancy), and coordination between VA and community care providers for some services not available on site in VA. These care coordination and provider challenges are evidenced by high rates of burnout and attrition among women's health providers. 5 In addition, because of higher utilization, women Veterans may experience longer waits for appointments than men Veterans at some VA facilities. 6,7
The current VA measure, the Care Assessment Need (CAN) score (CAN 2.0), 8 used to identify complexity and guide care coordination was built using chronic medical conditions codes. It was designed as a risk-stratification tool to systematically identify and to improve the care management of high-risk patients and to predict hospitalization or death in a population with significant medical comorbidities, including end-stage heart and lung disease, among others, and validated on a primarily older, male Veteran population. This type of score, although useful, may fail to identify other high utilizing patients who may also benefit from higher levels of care coordination. This limitation may be particularly salient for high-utilizing women Veterans who are generally younger than their male counterparts, and who have complex needs related to psychosocial issues such as mental health conditions, chronic pain, sexual trauma, or unstable housing. 9 –13
We hypothesized that the CAN score, although designed to predict hospitalization or death, would predict also primary care utilization. Because the CAN score may not account for trauma and psychosocial conditions more common in women Veterans, we hypothesized that it would predict utilization better for men than women Veterans. We undertook this study to determine association of CAN score quartiles with primary care utilization for male compared with female Veterans, and to determine if addition of reproductive health and psychosocial variables to the model would mitigate the differences by sex.
Methods
Study population and data sources
The study sample included men and women from the post-9/11 cohort 14 receiving VHA care for at least 1 year, as defined by the presence of two primary care visits (PCVs) at least 1 year apart between the years of 2012 and 2017. The analytic sample included 665,379 Veterans, their demographic and deployment history data and electronic health record (EHR) data of their first year in VHA care. In addition to the demographic and deployment history data, EHR data were extracted from the Corporate Data Warehouse (CDW) that includes the VHA's extensive EHR, with information on health care utilization, pharmacy, laboratory, vital signs, coded diagnostic and procedural data (International Classification of Diseases, 9th and 10th Revision, Clinical Modification [ICD-9-CM and ICD-10-CM] and Current Procedural Terminology) associated with all VA inpatient and outpatient encounters. CDW data were included from 2010 to 2017. The study was approved by the VA Connecticut Healthcare System Institutional Review Board.
Study measurements
Data on PCVs over the year were collected through outpatient visit data in the CDW using defined stop codes (
Independent variables
CAN 2.0 score was developed as a risk-stratification tool for primary care clinicians to systematically identify Veterans at high risk for adverse outcomes (i.e., hospitalization and/or death for 90-day and 1-year prediction periods) and to improve the care of these high-risk patients. Primary care providers can access their patient panel CAN score report on a VA Intranet Site (Primary Care Almanac) that is accessed from the tool bar in the EHR. The report provides recommendations to engage high-risk patients in the Home TeleHealth Program and provides information about whether they are engaged in other care management resources such as palliative care. The CAN score reflects the estimated probability of hospital admission or death within a specified timeframe (90 days or 1 year). The score is expressed as a percentile, ranging from 0 (lowest risk) to 99 (highest risk) and indicates how a given patient compares with other VA patients in terms of likelihood of hospitalization or death. Patients with a very high score (e.g., 99) have a risk of admission or death that approaches 74% in the next year, whereas for those with a low score (e.g., 5) that risk is only ∼2%. The tool uses Veteran-specific information pulled from various domains of EHR information (i.e., demographics, health care utilization, laboratory, medical conditions and comorbidities, medications, and vital signs).
The inclusion criteria for patients are as follows: Veterans who are alive, who are assigned to a Primary Care Patient Aligned Care Team and are not hospitalized in a VA facility. The exact composition of the tool changed slightly between the first version and the one used in these analyses. Figure 1 shows the components of CAN 2.0. There are six distinct outcomes/events modeled: 90-day and 1-year prediction periods for hospitalization, death, and a combination of hospitalization or death. All are available in the CDW. CAN scores are calculated weekly with Friday as the risk date and the Monday after as the day the latest scores are pushed to production. 15 For this study, we used the CAN 2.0 and used the last calculated CAN score within the first year of care Veterans received in the VA. After examining the distribution in the cohort, we defined a categorical variable based on population quintiles as the primary exposure variable in the analyses.
Components of care assessment need score.
Under study hypothesis, sex is another important independent variable of interest, and serves as a potential effect modifier of association between CAN score and heavy PCV service utilization.
Potential confounding factors included demographic and health-behavioral characteristics, including age at first PCV, race/ethnicity, marital status, education, and body mass index (BMI). Military service factors included branch (Army, Costal Guard, Air force, Marines, and Navy) and multiple (vs. single) deployment to recent conflicts. In addition, we identified six additional health-behavioral or clinical covariates/conditions that were not used in the original CAN score development and hence, may be explored to improve predictive capacity of clinical outcome of the CAN score, especially in women. These potential CAN “enhancers” included current smoker, military sexual trauma, traumatic brain injuries (determined from health factors from VA's universal screening), chronic pain (defined as ≥3 monthly pain intensity ratings ≥4 over the year), 13 and any indication of homelessness (assessed using stop codes, inpatient bed section codes, health factors from VA's universal screening) and presence of common reproductive health conditions including sexually transmitted infections, sexual dysfunctions, contraceptive use, and infertility from ICD-9-CM and ICD-10-CM diagnoses.
Statistical analysis
Characteristics of study population at baseline were summarized as means (±standard deviations) or frequencies (%) and compared between men and women using Student's t-test or chi-square test, as appropriate. Distributions of CAN score among the total sample and by men and women, respectively, were examined using univariate procedure and histogram.
Association between CAN score quintiles and clinical outcomes was examined using a logistic regression model. After inspecting the distribution of cumulative PCV counts over 1-year, we defined heavy PC service utilization by cumulative PCV counts ≥6, capturing approximately the top 5% of the study population, as a stringent and clinically interpretable binary outcome for logistic model. The analyses were performed in a hierarchical manner. First, we examined whether there were independent associations between CAN score quintiles and sex with the likelihood of heavy PC service utilization in the overall sample of 665,379 Veterans. In the unadjusted model, we simultaneously included sex (men vs. women) and CAN score quintiles (bottom 20% as reference) as sole independent variables. Next, we fit a “baseline model” by introducing demographic, military service and health-behavioral variables that were used as predictors in the original CAN development, including age, race/ethnicity, marital status, and education above high school, service branch, multiple deployment, and BMI. Then, we fit an “enhancer model” by adding a priori defined clinical or psychosocial covariates that were not used in the original CAN development model and hence, may potentially enhance (or improve) the predictive capacity of CAN score on clinical outcome, including current smoker, chronic pain, military sexual trauma, traumatic brain injuries, homelessness, and presence of any of the four common reproductive disorders.
Adjusted odds ratio (aOR) and 95% confidence interval (CI) were estimated as evidence for the strength of association. To test our study hypothesis that CAN score may be inadequate for predicting clinical outcome for women than for men, we also included interactions between CAN score quintiles and sex in the fully adjusted enhancer model.
After estimating multivariable logistic regression in the whole sample with a statistically significant CAN quintiles by sex interaction, next, we refit the “baseline” and the “enhancer” models among men and women, respectively. aOR and 95% CI were derived for men and women separately, on each corresponding model. To reduce influence of missing data on intro-model comparison, we assigned missing observations to a nuisance category for clinical covariates with missing data (see footnotes to Table 1).
Baseline Characteristics of 665,379 U.S. Post-9/11 Veterans
Values represent mean ± SD or frequency (%).
Based on Student's t-test for continuous variables and chi-square test for categorical variables, except otherwise indicated.
Defined using an algorithm that identifies evidence of homelessness from EHR data including use of inpatient or outpatient healthcare services designated for individuals who are homeless, relevant ICD-9-CM, ICD-10-CM codes and health factors that indicate current/recent homelessness.
Defined by three or more monthly pain intensity ratings ≥4 over the 1 year follow-up period.
23,036 Veterans with missing observations on this variable.
36,597 Veterans with missing observations on this variable.
CAN score is based on the last one of calendar year for each Veteran.
BMI, body mass index; CAN, Care Assessment Need; PTSD, posttraumatic stress disorders; SD, standard deviation.
We examined model performance using different model fitting statistics, including Hosmer and Lemeshow goodness-of-fit test 16 and likelihood ratio test 17 between nested models, and calculated percent incremental difference between the unadjusted, the baseline, and the enhancer models. For comparing model performance between men (N = 579,248) and women (N = 86,131), we focused on the receiver operating characteristic (ROC) curves as the primary indices for model performance, and used a nonparametric Whitney rank sum test to compare area under ROC curve on each corresponding model (i.e., the baseline and the enhancer models, respectively). SAS version 9.4 software (2011, SAS Institute, Inc., Cary, NC) was used in all analyses. Values of p < 0.05 were interpreted to be statistically significant.
Results
Study sample
Of the 1,063,813 post-9/11 Veterans in the cohort, 665,379 had a PCV, complete demographics, and a CAN score within a year of the first PCV. Table 1 provides the baseline characteristics of the sample with women accounting for 12.9% (N = 86,131) of the sample. Men and women differ in most baseline and clinical characteristics (p < 0.0001; Table 1). The CAN score differs between men and women with women (0.06 ± 0.06) having slightly higher mean score than men (0.05 ± 0.06) (p < 0.001; Table 1), and higher proportions falling within upper quintiles (p < 0.001; Table 1). Over the 1-year period, 7,623 (8.9%) women and 29,296 (5.1%) men had six or more PCV visits.
Table 2 summarizes the logistic model results of the association between CAN score, sex, and heavy primary care service utilizations (≥6 visits) during first year of care in the whole sample. Results show that overall, both sex and CAN quintiles are independently associated with this outcome after accounting for potential confounding factor in the baseline (Supplementary Table S1) and the enhancer (Table 2, Model 2) models. Specifically, women had 42% higher odds of heavy PCV utilization (aOR: 1.42, 95% CI: 1.37–1.46) compared with men. CAN score quintiles were associated with 84% to 8.5-fold increased likelihoods of heavy PCV utilizations, with aORs ranging from 1.84 to 8.47 for Veterans falling within upper four CAN quintiles in comparison with those in the bottom quintile.
Association Between Care Assessment Need Score, Sex and Heavy Primary Care Service Utilizations (≥6 Visits) During First Year of Care Among 665,379 Post-9/11 U.S. Veterans
There was a statistically significant CAN quintiles by sex interaction (type 3 p < 0.001).
Model 1 (unadjusted model) included only sex (women vs. men) and CAN quintiles as independent variables; Model 2 (enhancer model) adjusted for baseline covariates (age at first primary care visit, race, marital status, education above high school, service branch, multiple deployment, and BMI categories, and additional psychosocial or clinical predictors that were not a component of CAN score, including current smoker, military sexual trauma, chronic pain, traumatic brain injuries, homeless, and presence of common reproductive health conditions (≥1 of sexually transmitted infections, sexual dysfunctions, contraceptive use, and infertility).
OR, estimated using a logistic regression model of heavy primary care service utilization over 1 year.
OR, odds ratio; CI, confidence interval.
Because of statistically significant sex by CAN score quintiles interactions (Type 3, p < 0.001), we refit the enhancer model among men and women, separately. As given in Table 3, after accounting for all the confounding factors, women tend to have slightly lower aORs (ranging from 1.75 to 6.94) for each of the four CAN score quintiles than men (aOR ranging from 1.84 to 8.40). However, the 95% CIs of the aORs overlapped between men and women, except for the top quintiles. The aORs for the top CAN score quintile (vs. the lowest 20%) was 8.40 (95% CI: 7.92–8.92) for men and 6.94 (95% CI: 6.02–8.00) for women, suggesting a stronger association with 1-year heavy PCV utilization for men than for women.
Adjusted Odds Ratio (95% Confidence Interval) and Model Performance of Care Assessment Need Score in Predicting Heavy Primary Care Service Utilizations (≥6 Visits) During First Year of Care Among 665,379 Post-9/11 U.S. Veterans Men and Women—the Enhancer Model
Model adjusted for baseline covariates (age at first primary care visit, race, marital status, education above high school, service branch, multiple deployment, and BMI categories), plus six potential enhancer covariates that were not a component of CAN score, including current smoker, military sexual trauma, chronic pain, traumatic brain injuries, homeless, and presence of common reproductive health conditions (≥1 of sexually transmitted infections, sexual dysfunctions, contraceptive use and infertility). OR estimated using a logistic regression model of heavy primary care service utilization over 1 year.
Also known as receiver operating characteristic or ROC curve. Comparison of the area under ROC curves between men and women with a nonparametric Whitney rank sum test: p < 0.001.
Adding the six additional enhancer covariates improved the model fit significantly to the baseline model (Supplementary Table S2) in the whole sample, and among men and women, respectively. The areas under ROC curves derived from the fully adjusted enhancer model was significantly larger in men than in women (0.78 vs. 0.75, p < 0.001), suggesting CAN score had higher discriminant capacity or predictive accuracy on the risk of 1-year heavy PCV utilizations for men than for women.
Discussion
This is the first study to examine the ability of an existing clinical complexity measures (CAN 2.0) to predict high PVC utilization among Veterans, and to evaluate differences in PCV prediction by sex. Overall, CAN quintiles are associated with PCV utilization. There is a significant interaction between sex and CAN quintiles and better predictive accuracy on the risk of 1-year heavy PCV utilization for men than for women. This finding is notable given that women evidenced slightly higher CAN scores, on average. Our results are important in showing that CAN scores can effectively be used to predict primary care utilization, but the predictive value may be lower for women Veterans, where there may be a particularly important imperative to identify those in need of higher levels of care coordination.
For both men and women, health care systems have a need and opportunity to identify complex patients to provide efficient health care that reduces risk and enhances quality of care. Complex patients of both sexes have multiple overlapping medical and/or mental health conditions, are often burdened with psychosocial issues, have frequent interactions with health care systems, and are at risk for poor health outcomes. 18 –20 Responsibility for management of complex patients falls mainly to primary care teams that may be under-resourced to meet their needs. These factors are particularly salient when applied to women Veteran VA patients and their primary care teams, where women patients have high rates of chronic pain, 21,22 trauma, 23,24 and mental health conditions. 25,26 In addition, in contrast to men, women require management through reproductive life stages including family planning, pregnancy, and menopause, screenings for breast and cervical cancer, and use non-VA care at higher rates than men Veterans, 2,27 requiring higher levels of care coordination. All these factors result in higher utilization of PCVs for women Veterans compared with men Veterans with ∼12% of women Veterans utilizing over six visits with primary care annually, 2 as well as gender disparities on some quality measures. 28
High utilization is an indicator of complexity, an indicator of unmet needs or unresolved health issues, and a burden on the health care system and providers themselves. An ability to identify patients who may become high utilizers is an upstream opportunity to improve outcomes and mitigate costly health care services that may follow. Our results show that, although not designed as such, the CAN score predicts high primary care utilization for both men and women. However, it has less predictive accuracy for women. The VA CAN score is largely based on chronic illness, medication use, and utilization, but fails to incorporate significant predictors of comorbidity that may be more common in women, including chronic pain, trauma, and reproductive health conditions. By adjusting for these factors, we found improved ability of the CAN score to predict utilization for women.
Other studies have examined diagnosis/utilization-based algorithms to predict future utilization, 29 –35 and their failure to capture nonclinical contributors to complexity. However, primary care physicians take into account patient's needs across medical and psychosocial aspects of care 36,37 and describe complex patients as those with multidimensional needs. 38 To incorporate physician-identified measures of complexity (including nonclinical contributors), Hong et al. 39 derived a Physician-Defined Complexity (PCD) Score by asking providers to identify complex patients. Electronic medical record data from those patients were used to identify candidate variables to create the PCD. Significantly, the PCD identified complex patients who were younger and less likely to be men, among other factors compared with complex patients identified by more traditional algorithms.
These potential deficiencies in traditional morbidity or complexity indexes are especially relevant to women Veterans using VA care who providers often find complex and challenging with a range of health concerns including comorbid mental health conditions and psychosocial issues. 40
VA has studied the effectiveness of utilizing the CAN score to identify high-risk patients and referring them to higher levels of care. In a randomized clinical trial, patients with high CAN scores were offered assignment to an intensive primary care management team that included interdisciplinary team meetings, medication management, home visits, and intensive social work case management. Results showed an actual increase in outpatient care utilization, but no increase in overall cost, 41 and a positive influence on high-risk patient experiences with care coordination and provider relationships. 42 These results, although promising, point to a need for a more targeted upstream approach, using an algorithm that more fully incorporates psychosocial dimensions of care, to identifying those at risk of high utilization so that appropriate resources can be developed and directed at mitigating burdens on existing patient care teams.
Our study has a number of strengths. We analyzed data from a large cohort of women and men. We also demonstrated the interactions between sex and CAN quintiles in the prediction of PCV utilization in the largest single-payer, national U.S. health care system.
We do note several important limitations. First, we relied on ICD-9-CM and ICD-10-CM, which is dependent on accurate provider coding, to identify comorbidities. Another limitation is that we excluded patients with <2 PCVs from our analysis, which limits our ability to examine patterns that may be important between CAN scores and low utilization. In addition to understanding these patterns in high utilizers, it is also critically important to probe factors that may be relevant for complex patients who engage only minimally with needed care. Of additional note, CAN scores disproportionately weight chronic conditions that are most common with advancing age. Accordingly, results from this young sample of post-9/11 Veterans should not be generalized to the larger population of Veterans utilizing VA, which is older and less diverse. Finally, several key “enhancers” relevant for women were excluded from the analysis because they are unavailable, inconsistently coded, or difficult to reliably extract from the medical record. These include suicidal ideation, and limited social support, among others, which are more common in women Veterans and may have important implications for complexity and utilization.
The CAN score was created and validated in the VA to predict hospitalization and mortality among a sample of older male population of Veterans. There remains a need for a complexity measure that accounts for the unique conditions common in women Veterans to predict high PCV utilization so that appropriate care coordination can be delivered.
Conclusion
Overall, our study found sex differences in VA clinical complexity measures used to predict high PCV utilization. The VA has an opportunity to develop or augment the existing complexity measures to predict high PCV utilization for both men and women Veterans to improve the delivery of care coordination services.
Footnotes
Author Disclosure Statement
No competing financial interests exist.
Funding Information
This study was funded by Veterans Affairs Health Service Research and Development. Project Number: RVR 19-485.
Supplementary Material
Supplementary Table S1
Supplementary Table S2
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
