Risk Prediction Tool for Medical Appointment Attendance Among HIV-Infected Persons with Unsuppressed Viremia

Abstract

Successful treatment of HIV infection requires regular clinical follow-up. A previously published risk-prediction tool (RPT) utilizing data from the electronic health record (EHR) including medication adherence, previous appointment attendance, substance abuse, recent CD4+ count, prior antiretroviral therapy (ART) exposure, prior treatment failure, and recent HIV-1 viral load (VL) has been shown to predict virologic failure at 1 year. If this same tool could be used to predict the more immediate event of appointment attendance, high-risk patients could be identified and interventions could be targeted to improve this outcome. We conducted an observational cohort study at the Vanderbilt Comprehensive Care Clinic from August 2013 through March 2014. Patients with routine medical appointments and most recent HIV-1 VL >200 copies/mL were included. Risk scores for a modified RPT were calculated based on data from the EHR. Odds ratios (OR) for missing the next appointment were estimated using multivariable logistic regression. Among 510 persons included, median age was 39 years, 74% were male, 55% were black, median CD4+ count was 327 cells/mm³ [Interquartile Range (IQR): 142–560], and median HIV-1 VL was 21,818 copies/mL (IQR: 2,030–69,597). Medium [OR 3.95, 95% confidence interval (CI) 2.08–7.50, p-value<0.01] and high (OR 9.55, 95% CI 4.31–21.16, p-value<0.01) vs. low RPT risk scores were independently associated with missing the next appointment. RPT scores, constructed using readily available data, allow for risk-stratification of HIV medical appointment non-attendance and could support targeting limited resources to improve appointment adherence in groups most at-risk of poor HIV outcomes.

Introduction

Each year in the United States approximately 50,000 persons are newly diagnosed with human immunodeficiency virus (HIV) infection.¹ While the overall incidence has decreased during the past decade, rates among certain subpopulations continue to rise.² For nearly 20 years, however, advancements in antiretroviral therapy (ART), combined with our rapidly evolving understanding of HIV pathogenesis, have led to dramatic decreases in HIV-related morbidity and mortality.^3,4 Those who are recently diagnosed can now have a life expectancy similar to HIV-uninfected individuals.⁵ However, like most chronic illnesses, successful treatment of HIV relies not only on the availability of effective treatments, but is dependent on the individual's ability to attend scheduled healthcare provider visits and adhere to daily medication.

HIV-infected patients who are unable to keep routine medical appointments have an increased risk of death,^6
–8 an association that remains after controlling for CD4+ lymphocyte count and treatment with ART.⁶ Additionally, patients who miss appointments are less likely to receive treatment with ART^9,10 and are more likely to develop AIDS-defining CD4+ counts, unsuppressed viremia, and higher cumulative viral burden.^11

–14 However, despite the benefit of engagement in care, national trends indicate that a substantial number of HIV-infected persons are poorly retained in care and rates of missed appointments remain high among the population.^6,15

–18

For these reasons, the importance of routinely monitoring and improving appointment adherence has been addressed by the Office of National AIDS Policy,¹⁹ the Department of Health and Human Services (DHHS),²⁰ the Institute of Medicine (IOM),²¹ as well as by expert panels.²² Various appointment adherence indicators and benchmarks have been established.^20,21,23 Yet, aside from recommending monitoring of appointment adherence in general, these guidelines lack tools to stratify large numbers of patients by risk in order to target those most in need in the setting of limited resources.

Although risk stratification tools have been developed to predict HIV disease progression and prognosis,^24
–26 a similar tool to aid in the identification of patients likely to miss appointments is not available. Beginning in July 2013, clinicians at the Vanderbilt Comprehensive Care Clinic (VCCC, Nashville, TN) implemented an evidence-based tool for assessing the risk of virologic failure at 1 year²⁷ among patients with uncontrolled viremia—defined as HIV-1 viral load (VL) >200 copies/milliliter (mL). Clinicians observed that patients with high risk scores based on this tool also had high rates of missing their next HIV healthcare provider appointment. Therefore, this study sought to determine whether a tool previously operationalized to stratify patients according to virologic failure risk at 1 year²⁷ could also stratify patients based on the risk of a related and potentially more immediate event: missing their next HIV primary care visit. Systematic risk assessment could support targeting of limited resources and interventions shown to improve engagement in care, such as enhanced case management,^28
–30 to those at highest risk.

Methods

Patient population

We conducted an observational cohort study among adult patients with HIV-1 infection at the VCCC from August 2013 through March 2014. Patients were included if they had a routine appointment scheduled with a physician or nurse practitioner during the study period and if their most recent HIV-1 VL was >200 copies/mL. Patients were included regardless of whether they were currently prescribed ART. Demographic data for the study population were abstracted from the electronic health record (EHR), and included age, race/ethnicity (white, black, Hispanic, other/unknown), sex, gender (including transgender status), year of entry into HIV care at the VCCC, and HIV transmission risk factor (heterosexual contact, male-to-male sexual contact, injection drug use, other/unknown). This study was approved by the Vanderbilt Institutional Review Board.

Risk prediction tool

We generated risk scores for missing the next medical appointment with a modified risk prediction tool (RPT) based on a previously published tool shown to predict virologic failure over the subsequent year among persons on ART.²⁷ The modified RPT included seven components: history of poor adherence to daily medications, history of non-attendance to healthcare provider appointments for HIV care, active substance abuse, most recent CD4+ lymphocyte count <100 cells/mm³, heavy prior exposure to ART, prior treatment failure, and most recent HIV-1 VL >200 copies/mL (Table 1). The most recent CD4+ lymphocyte count or HIV-1 VL was defined as the value closest to the date of study entry. As shown in Table 1, risk categories were defined as low (0–1 point), medium (2–3 points), or high (4 or more points).²⁷ The RPT components were obtained from the EHR by one investigator who utilized a standardized abstraction form. RPT scores were determined prior to each patient's upcoming scheduled appointment.

Table 1.

Risk Prediction Tool Component Definitions ^a

Component	Definition	Score
History of poor adherence to daily medications	Progress note(s) within the previous 12 months including documentation of patient self-report of regularly missing doses every week or healthcare provider concerns about adherence to daily medications.	1 point for yes, 0 for no
History of non-attendance to healthcare provider appointments for HIV care	Two or more no-shows for appointments with a medical physician, nurse practitioner, or adherence counselor during the previous 12 months OR most recent completed appointment≥12 months prior to enrollment.	1 point for yes, 0 for no
Substance abuse	Any of the following documented within the previous 12 months: -alcohol abuse or dependence, polysubstance abuse, substance abuse -use of cocaine, heroin, amphetamines OR Urine drug screen positive for methamphetamine, cocaine, non-prescribed opiates or non-prescribed benzodiazepines.	1 point for yes, 0 for no
CD4+ lymphocyte count <100 copies/mm³	Most recent available laboratory value.	1 point for yes, 0 for no
Heavy prior exposure to ART	Any prior exposure to NRTI, NNRTI, and PI classes OR a current regimen containing enfuvirtide.	1 point for yes, 0 for no
Prior treatment failure	Any prior documentation of viremia while on ART AND genotypic confirmation of resistance.	1 point for yes, 0 for no
HIV-1 VL >200 copies/mL	Most recent available laboratory value.	1 point for yes, 0 for no
		Interpretation of total score:0–1=low risk2–3=medium risk≥4=high risk

Adapted from Robbins GK, Johnson KL, Chang Y, et al. Predicting virologic failure in an HIV clinic. Clin Infect Dis 2010;50:779–786.

ART, antiretroviral therapy; EHR, electronic health record; HIV, human immunodeficiency virus infection; NNRTI, non-nucleoside reverse transcriptase inhibitor; NRTI, nucleot(s)ide reverse transcriptase inhibitor; PI, protease inhibitor; VL, viral load.

Appointment outcome

The primary outcome was appointment attendance, defined as “Completed,” “Cancelled by patient,” “Cancelled by clinic,” or “No show.” Only appointments scheduled with a physician or nurse practitioner were included for analysis. If an appointment was cancelled by the clinic due to inclement weather or a healthcare provider absence, attendance at the rescheduled appointment was assessed. Appointments that were cancelled by the patient or to which the patient no-showed were categorized as noncompleted appointments; all other appointment outcomes were categorized as completed. In sensitivity analyses, cancelled appointments were excluded from regression models to derive estimates more directly comparable with prior HIV clinical retention literature.¹⁰ Appointment outcome was abstracted from the EHR the week after the scheduled appointment.

Laboratory analysis

CD4+ lymphocyte counts were measured by flow cytometry. HIV-1 plasma VL were measured by polymerase chain reaction (Roche Cobas Ampliprep-Cobas Taqman HIV-1 version 2.0). The range of this assay is 20–10,000,000 copies/mL.

Statistical analysis

Fisher exact tests were used to compare categorical variables. Wilcoxon Rank-Sum and Kruskall-Wallis tests were used to compare continuous variables between two categories and three or more categories, respectively. Logistic regression was used to estimate odds ratios (OR) for missing the next appointment. All p-values were two-sided and considered statistically significant if <0.05. The adjusted model included the following demographic variables: age, race/ethnicity, sex, gender, year of entry into HIV care at the VCCC, and HIV risk factor. Year of entry into HIV care was modeled using restricted cubic splines with three knots.

Results

A total of 510 individuals were included; median age was 39 years, 74% were men, 55% were black, and 1% were male-to-female (MTF) transgender. The median CD4+ lymphocyte count was 327 cells/mm³ (IQR: 142–560) and the median HIV-1 VL was 21,818 copies/mL (IQR: 2,030–69,597). Self-reported HIV transmission risk factors included male-to-male sexual contact (53%), heterosexual contact (38%), injection drug use (7%), and other/unknown (3%) (Table 2).

Table 2.

Demographic Characteristics of the Study Population

Characteristic	Completed next appointment N=317	Did not complete next appointment N=193	p Value^a	All N=510
Age in years	38	40	0.26	39
Median (IQR)	(29–48)	(30–49)		(30–48)
Race
Number (%)
White	139 (44%)	51 (26%)	<0.001	190 (37%)
Black	148 (47%)	134 (69%)	<0.001	282 (55%)
Hispanic	23 (7%)	5 (3%)	0.03	28 (5%)
Other	7 (2%)	3 (2%)	0.75	10 (2%)
Male sex	245	132	0.03	377
Number (%)	(77%)	(68%)		(74%)
MTF transgender	4	2	1.00	6
Number (%)	(1%)	(1%)		(1%)
HIV risk factor
Number (%)
MSM	182 (57%)	87 (45%)	0.008	269 (53%)
Heterosexual contact	113 (36%)	79 (41%)	0.26	192 (38%)
IDU	15 (5%)	19 (10%)	0.03	34 (7%)
Other/unknown	7 (2%)	8 (4%)	0.28	15 (3%)
Year of entry into	2012	2009	<0.001	2011
care at the VCCC	(2006–2013)	(2002–2013)		(2005–2013)
Most recent CD4+	355	291	0.12	327
count (cells/mm³)	(152–583)	(129–489)		(142–560)
Most recent HIV-1	21,192	25,426	0.94	21,818
VL (copies/mL)	(2303–67,079)	(1311–72,252)		(2030–69,597)

For comparison between those who completed and did not complete their next HIV healthcare provider appointment.

HIV, human immunodeficiency virus infection; IDU, injection drug use; IQR, interquartile range; MTF, male-to-female; MSM, male-to-male sexual contact; VCCC, Vanderbilt Comprehensive Care Clinic; VL, viral load.

Among the included 510 patients, 193 (38%) did not complete their next appointment. Fifty-four appointments were not completed due to cancellation by the patient, and 139 were not completed due to an appointment no-show. Patients who did not complete their next appointment were more likely to be black, had received care at the VCCC for a greater number of years, and were more likely to report injection drug use (IDU) as an HIV risk factor at the time they began care. Those who did not complete their next appointment were less likely to be male or to report male-to-male sexual contact (MSM) as an HIV risk factor (Table 2).

Among the 510 patients, 126 (25%) met criteria for the low risk group, 244 (48%) met criteria for the medium risk group, and 140 (27%) met criteria for the high risk group. Compared to those in the low risk group, patients in the high risk group were more likely to be older in age and to report heterosexual contact or IDU as an HIV risk factor at the time they began receiving care at the VCCC. High risk patients were less likely to be male or to report MSM as their HIV risk factor when compared to low risk patients. Additionally, high risk patients were enrolled in care at the VCCC for a greater number of years compared to low risk patients. Individuals in the high risk group had higher median HIV-1 VL and lower median CD4+lymphocyte count compared to those in the low risk group, although these findings may be partially explained by the inclusion of these two variables into the RPT score calculation (Table 3).

Table 3.

Demographic Characteristics of Study Population by Risk Category

Characteristic	Low risk N=126	p Value ^a	Medium risk N=244	p Value^b	High risk N=140	p Value^c	All N=510
Age in years	32	<0.001	38	<0.001	44	<0.001	39
Median (IQR)	(26–42)		(29–49)		(37–50)		(30–48)
Race
Number (%)
White	56 (44%)	0.07	84 (34%)	0.82	50 (36%)	0.17	190 (37%)
Black	58 (46%)	0.03	142 (58%)	1.00	82 (59%)	0.05	282 (55%)
Hispanic	7 (6%)	1.00	14 (6%)	0.82	7 (5%)	1.00	28 (5%)
Other	5 (4%)	0.28	4 (2%)	0.66	1 (1%)	0.10	10 (2%)
Male sex	104	0.09	181	0.08	92	0.002	377
Number (%)	(83%)		(74%)		(66%)		(74%)
MTF transgender	2	1.00	3	1.00	1	0.60	6
Number (%)	(2%)		(1%)		(1%)		(1%)
HIV risk factor
Number (%)
MSM	87 (69%)	0.001	125 (51%)	0.06	57 (41%)	<0.001	269 (53%)
Heterosexual contact	35 (28%)	0.05	93 (38%)	0.16	64 (46%)	0.003	192 (38%)
IDU	0	<0.001	19 (8%)	0.35	15 (11%)	<0.001	34 (7%)
Other/unknown	4 (3%)	1.00	7 (3%)	1.00	4 (3%)	1.00	15 (3%)
Year of entry into care at the VCCC	2013 (2012–2013)	<0.001	2010 (2006–2013)	<0.001	2005 (2000–2010)	<0.001	2011 (2005–2013)
Most recent CD4+ count (cells/mm³)	487 (322–655)	<0.001	348 (156–579)	<0.001	182 (43–313)	<0.001	327 (142–560)
Most recent HIV-1 VL (copies/mL)	21,475 (2293–44,632)	0.35	12,949 (1296–67,085)	<0.001	46,234 (6610–106,002)	0.003	21,818 (2030–69,597)

For comparison of low and medium risk groups; ^bfor comparison of medium and high risk groups; ^cfor comparison of low and high risk groups.

HIV, human immunodeficiency virus; IDU, injection drug use; IQR, interquartile range; MSM, male-to-male sexual contact; MTF, male-to-female; VCCC, Vanderbilt Comprehensive Care Clinic; VL, viral load.

Medium risk patients differed from low risk patients in a manner similar to high risk patients, in that they were more likely to be older, less likely to report MSM as an HIV risk factor, more likely to report IDU as an HIV risk factor, had been enrolled as VCCC patients for a greater number of years, and had lower median CD4+ lymphocyte counts. There were no differences between medium and low risk patients with regards to sex or median HIV-1 VL. However, medium risk patients were more likely than low risk patients to be black (Table 3).

The distribution of the RPT components by appointment outcome is shown in Table 4. Of 510 persons, 210 (41%) had a history of poor adherence to daily medications, 259 (51%) had a history of non-attendance to healthcare provider appointments for HIV care, 139 (27%) had recent history of substance abuse, 101 (20%) had a CD4+ lymphocyte count <100 cells/mm³ on most recent available laboratory value, 82 (16%) were heavily exposed to ART, and 77 (15%) had prior virologic failure. Patients who did not complete their next appointments were more likely to score a point for the RPT components of poor adherence to medications, non-attendance to HIV healthcare provider appointments, substance abuse, and heavy prior exposure to ART.

Table 4.

Risk Prediction Tool Components By Next Appointment Outcome

Characteristic	Completed appointment N=317	Did not complete appointment N=193	p Value ^a	All N=510
Poor adherence to medications	99 (31%)	111 (58%)	<0.001	210 (41%)
Non-attendance to healthcare provider appointments for HIV care	110 (35%)	149 (77%)	<0.001	259 (51%)
Substance abuse	73 (23%)	66 (34%)	0.008	139 (27%)
CD4+ lymphocyte count <100 cells/mm³	63 (20%)	38 (20%)	1.00	101 (20%)
Heavy prior exposure to ART	38 (12%)	44 (23%)	0.002	82 (16%)
Prior treatment failure	41 (13%)	36 (19%)	0.10	77 (15%)
HIV-1 VL >200 copies/mL	317 (100%)	193 (100%)	1.00	510 (100%)

For comparison between those who completed and did not complete their next HIV healthcare provider appointment.

ART, antiretroviral therapy; HIV, human immunodeficiency virus; VL, viral load.

In unadjusted analyses, medium or high RPT scores, black race, and IDU as HIV risk factor were associated with increased odds of missing the next HIV healthcare provider appointment. Male sex was associated with decreased odds of missing the next appointment. In adjusted analyses, medium or high RPT scores and black race remained independently associated with missing the next HIV healthcare provider appointment. Compared to low risk RPT scores, medium risk RPT scores were associated with 3.95 times the odds of missing the next appointment [95% confidence interval (CI) 2.08–7.50, p<0.01] and high risk RPT scores were associated with 9.55 times the odds of missing the next appointment (95% CI 4.31–21.16, p<0.01). Black race was associated with 2.32 times the odds of missing the next appointment, compared to those who reported being white (95% CI 1.48–3.64, p<0.01) (Table 5). Results of the regression models were similar when appointments not completed due to cancellation by the patient (n=54) were excluded (data not shown).

Table 5.

Regression Model Results for Odds of Missed Healthcare Provider Appointment

Characteristic	Unadjusted OR (95% CI)	p Value	Adjusted OR^a (95% CI)	p Value
Risk category
Low	Reference		Reference
Medium	4.09 (2.31–7.24)	<0.01	3.95 (2.08–7.50)	<0.01
High	8.80 (4.78–16.22)	<0.01	9.55 (4.31–21.16)	<0.01
Age (per 10 years)	1.10 (0.95–1.30)	0.20	0.94 (0.77–1.15)	0.64
Race
White	Reference		Reference
Black	2.46 (1.66–3.67)	<0.01	2.32 (1.48–3.64)	<0.01
Hispanic	0.59 (0.21–1.64)	0.31	0.54 (0.17–1.76)	0.31
Other	1.17 (0.29–4.69)	0.83	1.61 (0.42–6.24)	0.49
Male sex	0.64 (0.43–0.95)	0.03	0.84 (0.46–1.52)	0.57
MTF transgendered	0.82 (0.15–4.52)	0.34	1.16 (0.22–6.08)	0.86
HIV risk factor
MSM	Reference		Reference
Heterosexual contact	1.46 (0.99–2.15)	0.05	0.90 (0.50–1.63)	0.73
IDU	2.65 (1.29–5.46)	<0.01	1.64 (0.76–3.56)	0.21
Other/unknown	2.39 (0.84–6.81)	0.10	1.71 (0.61–4.83)	0.31
Most recent CD4+ (per 100 cells/mm³)	0.95 (0.90–1.02)	0.15	1.03 (0.95–1.12)	0.50
Most recent HIV-1 VL (per log10 copies/mL)	0.97 (0.81–1.17)	0.77	0.96 (0.77–1.21)	0.75

Adjusted for all variables in the table as well as year of cohort entry using restricted cubic splines with three knots.

CI, confidence interval; IDU, injection drug use; MSM, male-to-male sexual contact; MTF, male-to-female; OR, odds ratio; VCCC, Vanderbilt Comprehensive Care Clinic; VL, viral load.

Discussion

We found that a previously published tool²⁷ shown to predict virologic failure over the next year among patients with HIV-infection on ART can also be used to help predict whether patients with unsuppressed HIV viremia will attend their next medical appointment for routine HIV care. In our adjusted model, the odds of missing the next appointment were 3.95 times greater for medium risk patients, compared to low risk patients (95% CI 2.08–7.50, p<0.01). Risk for missing the next appointment rose further as the RPT score increased: The odds of missing the next appointment were 9.55 times higher for patients with the highest RPT scores, compared to those with the lowest scores (95% CI 4.31–21.16, p<0.01). To our knowledge, this is the first study to evaluate the ability of a multi-component risk prediction tool utilizing data readily available in the EHR to predict future appointment attendance among patients with HIV-infection.

Effective interventions have been identified to improve appointment attendance.^28

–32 Given the increasing pressure to manage large panels of complex patients in a manner that produces optimal outcomes with minimal use of resources, identifying ways to conduct population-level triage remains critical. In addition to predicting the magnitude of risk for missing the next appointment, this tool stratified a large cohort of over 500 patients with unsuppressed viremia based on severity of risk. Of 510 patients, 140 (27%) were found to have the highest risk of missing their next routine HIV care appointment. Thus, applying the tool resulted in a smaller, more manageable number of high-risk patients for which limited resources and supportive services could be prioritized.

An additional strength of this tool is its practicality, as it utilizes data from the EHR that are routinely collected as part of HIV care.²⁷ The rapidity with which the RPT components can be abstracted from readily available data position it as a tool that could be incorporated into routine clinical care with existing funding and staff. Moreover, clinicians can utilize this tool prior to the outcome of interest—the patient's next routine HIV care appointment. This allows for the receipt of real-time information to guide resource-planning and service utilization prior to scheduled appointments.

In addition to associations between RPT scores and appointment attendance, we found an independent association between black race and appointment non-attendance (adjusted OR 2.32, 95% CI 1.48–3.64, p<0.01). This finding is consistent with existing evidence of racial disparities in appointment attendance and retention in care among HIV-infected persons.^10,14,16,33 The association between race and appointment attendance could be explained by potential unmeasured mediators such as socioeconomic status, lack of transportation, proximity to health care centers, challenges navigating the healthcare system, geographic mobility, point of HIV-infection identification, pregnancy, and aspects of the patient–provider relationship.^{10,34

–40} Additionally, factors more unique to patients with HIV-infection, including stigma and HIV status disclosure, are known to affect appointment attendance.⁴¹ These factors are not routinely measured as part of clinical care and would be difficult to operationalize.

Recent critiques of healthcare research stress the importance of clinicians partnering with physician-scientists in order to make discoveries that will have rapid, real-world impact.^42,43 Therefore, it is important to note that this study evolved from a broader quality improvement project that aimed to integrate a systematic method for assessing the risk of missing the next healthcare appointment among patients not meeting the goal of viral suppression. In order to assess risk for these patients, we applied an evidence-based tool²⁷ and evaluated its performance within the clinical practice environment. In turn, this allowed us to generate practice-based evidence, which will not only influence clinic processes but will have a lasting impact on patient care at our clinic.

One limitation of our study is the reliance on healthcare provider documentation and patient self-report for ascertainment of medication adherence and drug use. Assessment and documentation of medication adherence and substance use may vary among healthcare professionals. However, HIV-care guidelines recommend assessing these issues at each visit so misclassification of these two RPT components should be limited.²⁰ In addition, more objective assessments of adherence (drug levels, pill counts, pharmacy filling records) and substance use (urine and/or serum drug testing) are not routinely collected as part of clinical care.

This study is also limited in terms of the generalizability of its findings. The ability of the RPT to stratify patients by risk in order to predict appointment attendance may not apply to other populations. First, we restricted our analysis to patients with HIV-1 VL>200 copies/mL and the results may not apply to patients with HIV-1 VL<200 copies/mL. However, patients who are not meeting the goal of viral suppression represent a group that is arguably most in need of targeted interventions to improve appointment adherence. Second, quickly determining RPT scores may not be as feasible in settings in which the RPT components are not readily available or easily retrieved, including those settings that do not utilize an EHR. Third, this RPT may perform differently when applied to populations that differ based on demographic characteristics and HIV transmission risk factors. The RPT should be evaluated in these populations prior to integration into clinical care.

In conclusion, we have shown that a risk prediction tool composed of accessible and readily available data from the EHR can be used to predict appointment attendance among individuals with HIV-infection. The odds of missing the next appointments were almost four times greater for patients with medium RPT scores and almost 10 times greater for those with high RPT scores, compared to those with low scores. Furthermore, the tool can stratify large cohorts of patients into smaller groups based on risk, potentially allowing limited resources to be targeted to those who are most in need of medical care adherence and interventions to improve HIV outcomes. Future studies are needed to evaluate the performance of the tool among other populations, including those with suppressed viremia. Additionally, further investigation is needed to determine whether interventions lead to improved outcomes based on risk stratification.

Footnotes

Acknowledgments

We thank the anonymous reviewers whose thoughtful comments significantly strengthened the article. We gratefully acknowledge the Vanderbilt Comprehensive Care Clinic and its Clinical Pharmacy Services team. This work was supported in part by the Ryan White Care Act, which provides funding support to the VCCC. ACP was supported by grant number K08 AI104352 from the National Institute of Allergy and Infectious Diseases at the National Institutes of Health.

Author Disclosure Statement

The authors have no conflicting financial interests.

References

Centers for Disease Control and Prevention. Estimated HIV Incidence in the United States, 2007–2010. HIV Surveillance Supplimental Report. October 29, 2014 2012; 17(4):Available at: http://www.cdc.gov/hiv/topics/surveillance/resources/reports/ - supplemental (Last accessed October 29, 2014 ).

Johnson

, Hall

, Hu

, Lansky

, Holtgrave

, Mermin

. Trends in diagnoses of HIV infection in the United States, 2002–2011. JAMA, 2014; 312:432–434.

Porter

, Babiker

, Bhaskaran

, et al. Determinants of survivial following HIV-1 seroconversion after the introduction of HAART. Lancet, 2003; 362:1267–1274.

Palella

, Delaney

, Moorman

. Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. N Engl J Med, 1998; 228:853–860.

Nakagawa

, May

, Phillips

. Life expectancy living with HIV: Recent estimates and future implications. Curr Opin Infect Dis, 2013; 26:17–25.

Mugavero

, Lin

, Willig

, et al. Missed visits and mortality among patients establishing initial outpatient HIV treatment. Clin Infect Dis, 2009; 48:248–256.

Horberg

, Hurley

, Silverberg

, Klein

, Quesenberry

, Mugavero

. Missed office visits and mortality risk among HIV-infected subjects in a large healthcare system in the United States. AIDS Patient Care STDS, 2013; 27:442–449.

Mugavero

, Westfall

, Cole

, et al. Beyond core indicators of retention in HIV care: Missed clinic visits are independently associated with all-cause mortality. Clin Infect Dis, 2014; 59:1471–1479.

Rebolledo

, Kourbatova

, Rothenberg

, del Rio

. Factors associated with utilzation of HAART amongst hard-to-reach HIV-infected individuals in Atlanta, Georgia. J AIDS HIV Res, 2011; 3:63–70.

10.

Mugavero

, Lin

, Allison

, et al. Racial disparities in HIV virologic failure: Do missed visits matter?. J Acquir Immune Defic Syndr, 2009; 50:100–108.

11.

Berg

, Safren

, Mimiaga

, Grasso

, Boswell

, Mayer

. Nonadherence to medical appointments is associated with increased plasma HIV RNA and decreased CD4 cell counts in a community-based HIV primary care clinic. AIDS Care, 2005; 17:902–907.

12.

Crawford

, Sanderson

, Thornton

. A comparison study of methods for measuring retention in HIV medical care. AIDS Behav, 2013; 17:3145–3151.

13.

Mugavero

, Amico

, Westfall

, et al. Early retention in HIV care and viral load suppression: Implications for a test and treat approach to HIV prevention. J Acquir Immune Defic Syndr, 2012; 59:86–93.

14.

Howe

, Napravnik

, Cole

, et al. African American race and HIV virological suppression: Beyond disparities in clinic attendance. Am J Epidemiol, 2014; 179:1484–1492.

15.

Centers for Disease Control and Prevention. Vital signs: HIV prevention through care and treatment-United States. Morbid Mortal Weekly Rep, 2011; 60:1618–1623.

16.

Whiteside

, Cohen

, Bradley

, Skarbinsk

, Hall

, Lansky

. Progress along the continuum of HIV care among blacks with diagnosed HIV-United States, 2010. Morbid Mortal Weekly Rep, 2014; 63:85–89.

17.

Mugavero

, Amico

, Horn

, Thompson

. The state of engagement in HIV care in the United States: From cascade to continuum to control. Clin Infect Dis, 2013; 57:1164–1171.

18.

Westergaard

, Hess

, Astemborski

, Mehta

, Kirk

. Longitudinal changes in engagement in care and viral suppression for HIV-infected injection drug users. AIDS, 2013; 27:2559–2566.

19.

White House Office of National AIDS Policy. The National HIV/AIDS Strategy for the United States. Washington DC: The White House. 2010; Available at: http://www.whitehouse.gov/administration/eop/onap/nhas (Last accessed October 20, 2014 ).

20.

Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents. Department of Health and Human Services. Available at: http://aidsinfo.nih.gov/contentfiles/AdultandAdolescentGL003371.pdf (Last accessed October 20, 2014 ).

21.

Institute of Medicine. Monitoring HIV care in the United States: Indicators and data systems. http://www.iom.edu/Reports/2012/Monitoring-HIV-Care-in-the-United-States.aspx (Last accessed October 20, 2014 ).

22.

Thompson

, Mugavero

, Amico

, et al. Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an international association of physicians in AIDS care panel. Ann Intern Med, 2012; 156:817–833.

23.

Health Resources and Services Administration HIV/AIDS Bureau. HIV/AIDS Bureau Performance Measures. Department of Health and Human Services. 2013; Available at: http://www.hab.hrsa.gov/deliverhivaidscare/habperformmeasures.html (Last accessed October 31, 2014 ).

24.

Bebu

, Tate

, Rimland

, et al. The VACS index predicts mortality in a young, healthy HIV population starting highly active antiretroviral therapy. J Acquir Immune Defic Syndr, 2014; 65:226–230.

25.

May

, Porter

, Sterne

, Royston

, Egger

. Prognostic model for HIV-1 disease progression in patients starting antiretroviral therapy was validated using independent data. J Clin Epidemiol, 2005; 58:1033–1041.

26.

Mocroft

, Ledergerber

, Zilmer

, et al. Short-term clinical disease progression in HIV-1-positive patients taking combination antiretroviral therapy: The EuroSIDA risk-score. AIDS, 2007; 21:1867–1875.

27.

Robbins

, Johnson

, Chang

, et al. Predicting virologic failure in an HIV clinic. Clin Infect Dis, 2010; 50:779–786.

28.

, Lai

, Liu

, et al. Impact of nurse-led case management program with retention in care on mortality among people with HIV-1 infection: A prospective cohort study. Intl J Nurs Stud, 2012; 49:656–663.

29.

Higa

, Marks

, Crepaz

, Liau

, Lyles

. Interventions to improve retention in HIV primary care: A systematic review of US studies. Curr HIV/AIDS, 2012; 9:313–325.

30.

Gardner

, Giordano

, Marks

, et al. Enhanced personal contact with HIV patients improves retention in primary care: A randomized trial in 6 US HIV clinics. Clin Infect Dis, 2014; 59:725–734.

31.

Brennan

, Browne

, Horgan

. A systematic review of health science interventions to improve linkage with or retention in HIV care. AIDS Care, 2014; 26:804–812.

32.

Gardner

, Marks

, Craw

, et al. A low-effort, clinic-wide intervention improves attendance for HIV primary care. Clin Infect Dis, 2012; 55:1124–1134.

33.

Eberhart

, Yehia

, Hillier

, et al. Behind the cascade: Analyzing spatial patterns along the HIV care continuum. J Acquir Immune Defic Syndr, 2013; 64:S42–S51.

34.

Centers for Disease Control and Prevention. Health disparities and inequalities report-United States, 2013. Morbid Mortal Weekly Rep Supp, 2013; 62:1–187.

35.

Traeger

, O'Cleirigh

, Skeer

, Mayer

, Safren

. Risk factors for missed HIV primary care visits among men who have sex with men. J Behav Med, 2012; 35:548–556.

36.

Kempf

, McLeod

, Boehme

, et al. A qualitative study of barriers and facilitators to retention-in-care among HIV-positive women in the rural southeastern United States: Implications for targeted interventions. AIDS Patient Care STDS, 2010; 24:515–520.

37.

Magnus

, Herwehe

, Murtaza-Rossini

, et al. Linking and retaining HIV patients in care: The importance of provider attitudes and behaviors. AIDS Patient Care STDS, 2013; 27:297–303.

38.

Taylor

, Reyes

, Levine

, et al. Patterns of geographic mobility predict barriers to engagement in HIV care and antiretroviral treatment adherence. AIDS Patient Care STDs, 2014; 28:284–295.

39.

Siddiqui

, Bell

, Sangi-Haghpeykar

, Minard

, Levison

. Predictive factors for loss to postpartum follow-up among low income HIV-infected women in Texas. AIDS Patient Care STDs, 2014; 28:248–253.

40.

Richey

, Halperin

, Pathmanathan

, Sickels

, Seal

. From diagnosis to engagement in HIV care: Assessment and predictors of linkage and retention in care among patients diagnosed by emergency department based on testing in an urban public hospital. AIDS Patient Care STDs, 2014; 28:277–279.

41.

Wohl

, Galvan

, Myers

, et al. Do social support, stress, dsclosure, and stigma influence retention in HIV care for Latino and African American men who have sex with men and women?. AIDS Behav, 2011; 15:1098–1110.

42.

Ammerman

, Smith

, Calancie

. Practice-based evidence in public health: Improving reach, relevance, and results. Annu Rev Public Health, 2014; 35:47–63.

43.

Kessler

, Gasgow

. A proposal to speed translation of healthcare research into practice: Dramatic change is needed. Am J Prev Med, 2011; 40:637–644.