Development and Validation of a Risk Prediction Tool to Identify People with HIV Infection Likely Not to Achieve Viral Suppression

Abstract

Identifying people with HIV infection (PHIV), who are at risk of not achieving viral suppression, is important for designing targeted intervention. The aim of this study was to develop and test a risk prediction tool for PHIV who are at risk of not achieving viral suppression after a year of being in care. We used retrospective data to develop an integer-based scoring method using backward stepwise logistic regression. We also developed risk score categories based on the quartiles of the total risk score. The risk prediction tool was internally validated by bootstrapping. We found that nonviral suppression after a year of being in care among PHIV can be predicted using seven variables, namely, age group, race, federal poverty level, current AIDS status, current homelessness status, problematic alcohol/drug use, and current viral suppression status. Those in the high-risk category had about a 23 increase in the odds of nonviral suppression compared with the low-risk group. The risk prediction tool has good discriminative performance and calibration. Our findings suggest that nonviral suppression after a year of being in care can be predicted using easily available variables. In settings with similar demographics, the risk prediction tool can assist health care providers in identifying high-risk individuals to target for intervention. Follow-up studies are required to externally validate this risk prediction tool.

Introduction

In the United States, at the end of 2015, about 1.1 million people were living with HIV infection, and in 2015, there were an estimated 38,000 new HIV infections.¹ In 2017, Florida had the highest number of estimated new HIV infections (4800).¹ Within Florida, Miami-Dade County had the highest number of people with HIV infection (PHIV); ∼29,969 in 2017.²

Once diagnosed with HIV, PHIV need to be linked to and retained in care, take antiretroviral therapy (ART), and adhere to their medications to have successful HIV care outcomes. The HIV care continuum, defined as stages of HIV medical care that PHIV go through, from initial diagnosis to achieving viral suppression,³ includes five steps, namely, HIV diagnosis, linkage to HIV care, retention in HIV care, adherence to ART, and viral suppression.⁴ Viral suppression is the final step and ultimate goal of the HIV care continuum and is usually a reflection of success in HIV care. Viral suppression benefits the individual living with HIV and the community. Virally suppressed PHIV have slower disease progression and increased survival.^5
–7 At the community level, virally suppressed individuals are less likely to transmit the virus to others.^8
–10 Despite the availability of ART, a substantial number of PHIV are not virally suppressed.¹¹ For example, in 2015, in 39 US states and District of Colombia, 40.2% of the PHIV were not virally suppressed.¹²

Given the importance of viral suppression, there is a need to develop evidence-based strategies to monitor and predict this outcome. There are numerous cross-sectional and longitudinal studies that have identified factors associated with viral suppression among PHIV.^{13

–24} These factors include young age,^13,14 gender,^14
–16 black race,^{13,15,17
–19} unstable housing,²⁰ substance use, higher baseline viral load,²¹ long duration of HIV infection,¹⁸ poor general health status,^18,22 being US born,²³ and low educational attainment.²⁴ There is a need to translate these findings about individual factors into validated and easy-to-use risk tools for use in predicting an individual's risk of not achieving viral suppression.

Studies have been conducted to develop risk prediction tools for virologic failure,²⁵ and for predicting extended high viremia among newly diagnosed people.²⁶ Clinical and behavioral factors related to suboptimal adherence, recent cluster of differentiation 4 (CD4) count, drug and/or alcohol abuse, prior ART exposure, prior treatment failure, and recent HIV-1 viral load were used to predict virologic failure after 1 year among those who were virologically suppressed on ART at enrollment.²⁵ Researchers have also developed risk prediction tools for HIV disease progression, particularly for mortality.^27
–29 The aim of our study was to develop and test a risk prediction tool for PHIV who are in care but are at risk of not achieving viral suppression after a year of being in care, to use for triaging those in need of more assistance. If people likely not to achieve viral suppression are identified early, intervention strategies could be implemented to assist these individuals into achieving viral suppression and ultimately improve their quality of life.³⁰

Methods

Data source and study population

We developed and internally validated a risk prediction tool for nonviral suppression using retrospective data from the Miami-Dade County Ryan White Program (RWP) Part A/Minority AIDS Initiative (MAI) for the calendar years 2016–2017. The data set included 6492 PLWH who were in care between 2016 and 2017 in the RWP Part A/MAI. In care was defined as having at least one viral load or CD4 count test or encounter with healthcare professional in each year. All exposures were measured in 2016, and viral suppression was measured in 2017.

The RWP is a comprehensive system of care for PHIV. It provides primary medical care and other support for PHIV without health insurance. In the United States, more than half of PHIV receive services through the Ryan White HIV/AIDS Program each year.³¹ The RWP services include outpatient/ambulatory health services, oral health care, other professional services (legal services and permanency planning), food bank, medical transportation (in the form of vouchers), mental health services, medical case management (including treatment adherence), health insurance premium and cost sharing assistance, local AIDS pharmaceutical assistance, substance abuse care and services (both outpatient and residential), and outreach services.

Predictor variables

We selected sociodemographic, clinical, and behavioral variables based on evidence from the literature and completeness of information in the data set.^{13

–24} All characteristics were parameterized as categorical variables and refer to the year 2016. Age was categorized as 18–24, 25–39, and ≥40 years. All other variables were binary and included sex assigned at birth (male/female), homelessness (yes/no), race (black/white or other), transgender (yes/no), Hispanic ethnicity (yes/no), alcohol/drug use resulting in any problem in patients daily activity or legal issue or hazardous situation (referred as problematic alcohol/drug use in this article) (yes/no), self-reported feelings of depression or anxiety (yes/no), patient getting the food he/she needs (yes/no), patient had CDC (Centers for Diseases Control and Prevention)-defined AIDS as of 2016 (yes/no), patient virally suppressed in 2016 based on the last viral load laboratory test in 2016 (yes/no), patient had access to transportation for health care/dental/social service appointments (yes/no), patient had a history of injection drug use (IDU; yes/no), and federal poverty level (FPL) <100% (yes/no). FPL <100% in 2016 was defined as having a household income less than $11,880 for a single person.³² We defined problematic alcohol/drug use as having any of the following. (a) Has alcohol/drug use resulted in legal problems, (b) Has alcohol/drug use resulted in hazardous situation, and (c) Is your alcohol/drug use preventing you from carrying out your daily activities? All predictor variables were obtained from the patient intake assessment (information collected at the time of entry into the RWP Part A/MAI), comprehensive health assessment (biannual assessment of all RWP Part A/MAI patients), or laboratory data.

Outcome

The outcome, viral suppression, was a binary variable, and nonviral suppression was defined as having viral load ≥200 copies/mL in the last viral load measurement in 2017.³³

Inclusion and exclusion criteria

We included PHIV who were in care in 2016 and 2017 and who had a comprehensive health assessment in 2016. Comprehensive health assessment is a health and social needs assessment of RWP patients that is completed every 6 months to determine plan of care and needs for referrals to other services. Patients who were younger than 18 years in January 2016, died in 2016 or 2017, had no comprehensive health assessment in 2016, or were out-of-network referrals in 2016 or 2017 were excluded. Out-of-network referrals are people who were referred to the RWP from a non-RWP provider. Patients whose case was closed because of moving to another state/county, financial ineligibility, or incarceration for greater than 6 months during 2016 or 2017 were also excluded.

Analysis

First, we conducted bivariate analysis to assess the association between each predictor variable and the outcome and estimated crude odds ratio (OR). All variables associated with nonviral suppression at p < 0.1 in the bivariate analysis were included in the initial logistic regression model. With stepwise backward elimination, we retained only significant factors (p < 0.05) in the final model.³⁴

We assessed calibration using calibration plots by dividing subjects into deciles of risk according to their model predictions, and the observed nonviral suppression levels among the subjects. Each decile was plotted against the average predicted probability of nonviral suppression and compared with the 45° line (perfect calibration).³⁵ The ability of the prediction model to distinguish events versus nonevents (discrimination) was measured by the concordance statistic or C-statistic (which is equal to the area under the receiver operating characteristic curve).³⁵

We assessed internal validity with a bootstrapping procedure, extracting 1000 samples with replacement, of the same size as the original data set (n = 6492).^36,37 For each sample, we used the same procedure that was used on the original data set (stepwise backward logistic regression model). Then we calculated optimism by comparing the final model performance (C-statistic) between the bootstrap samples and the original data. The bootstrap-corrected C-statistic was computed by subtracting the optimism from the original C-statistic.³⁴

Risk score development

We aimed to develop a simple risk score tool that could be easily assessed in a variety of settings to identify PHIV who are at risk of not achieving viral suppression after a year of being in care. After obtaining the beta-coefficients from the final logistic regression model, the scores for each predictor were determined by multiplying each beta-coefficient by 10 and rounding to the nearest integer.^34,38

The total risk score was calculated by adding the scores for all existing risk factors. To develop an easily interpretable method to classify patients according to the risk of not achieving viral suppression, we divided the risk score into three strata (by placing cut-points at the 25th and 75th percentiles of the model's total risk score distribution). We also calculated the sensitivity, specificity, positive predictive value, and negative predictive value for a range of potential cutoff points. All statistical analyses were performed using SAS software version 9.4 (SAS Institute, Cary, NC). This study was approved by the Florida International University Institutional Review Board.

Results

Of the 8014 PHIV who were in care in the RWP Part A/MAI in 2016 in Miami-Dade County, 1522 (19.0%) were excluded for various reasons (Fig. 1); 571 of whom were not in care in 2017. Of the 6492 PHIV in the final data set, 606 (9.4%) were not virally suppressed in 2017.

FIG. 1.

Flow diagram of exclusion criteria in the present study.

The majority of the PHIV were >40 years old (65.8%), male (76.3%), and virally suppressed in 2016 (87.5%) (Table 1). In the bivariate analysis, age in 2016, race, Hispanic ethnicity, poverty level, homelessness, problematic alcohol/drug use, feeling depressed or anxious, viral suppression status in 2016, and AIDS status as of 2016 were associated with nonviral suppression in 2017 at p < 0.001; whereas food needs, history of IDU, and sex assigned at birth were associated with nonviral suppression in 2017 at p < 0.05. Transgender status and access to transportation to appointments were not associated with nonviral suppression in 2017.

Table 1.

Population Characteristics of People with HIV Infection (N = 6492) by Viral Suppression Status and Association of Factors with Nonviral Suppression

Characteristics	Total	Virally suppressed, n (%)	Not virally suppressed, n (%)	Unadjusted OR (95% CI)	p
Total	6492	5883 (90.6)	609 (9.4)
Age in 2016, years					<0.001
18–24	330	287 (87.0)	43 (13.0)	1.75 (1.25–2.46)
25–39	1905	1674 (87.9)	231 (12.1)	1.61 (1.35–1.93)
≥40	4257	3922 (92.1)	335 (7.9)	Ref.
Sex at birth					0.003
Male	4952	4516 (91.2)	436 (8.8)	Ref.
Female	1540	1367 (88.8)	173 (11.2)	1.31 (1.09–1.58)
Race					<0.001
White/other	3991	3742 (93.8)	249 (6.2)	Ref.
Black	2501	2141 (85.6)	360 (14.4)	2.53 (2.13–3.00)
Hispanic ethnicity					<0.001
Yes	3747	3506 (93.6)	241 (6.4)	Ref.
No	2745	2377 (86.6)	368 (13.4)	2.25 (1.90–2.67)
FPL <100%					<0.001
Yes	2729	2378 (87.1)	351 (12.9)	2.01 (1.69–2.37)
No	3763	3505 (93.1)	258 (6.9)	Ref.
Homeless					<0.001
Yes	339	265 (78.2)	74 (21.8)	2.93 (2.23–3.85)
No	6153	5618 (91.3)	535 (8.7)	Ref.
Alcohol/drug use resulted in any problem in daily activity or legal issue or hazardous situation					<0.001
Yes	130	89 (68.5)	41 (31.5)	4.70 (3.21–6.87)
No	6362	5794 (91.1)	568 (8.9)	Ref.
Are you feeling depressed or anxious?					<0.001
Yes	958	826 (86.2)	132 (13.8)	1.69 (1.38–2.08)
No	5534	5057 (91.4)	477 (8.6)	Ref.
Getting the food he/she needs					<0.019
Yes	6398	5805 (90.7)	593 (9.3)	Ref.
No	94	78 (83.0)	16 (17.0)	2.01 (1.17–3.46)
Had CDC-defined AIDS in 2016					<0.001
Yes	2634	2313 (87.8)	321 (12.2)	1.72 (1.46–2.03)
No	3858	3570 (92.5)	288 (7.5)	Ref.
Virally suppressed in 2016?					<0.001
Yes	5682	5344 (94.0)	338 (6.0)	Ref.
No	810	539 (66.5)	271 (33.5)	7.95 (6.62–9.54)
History of IDU					0.022
Yes	108	91 (84.3)	17 (15.7)	1.83 (1.08–3.09)
No	6384	5792 (90.7)	592 (9.3)	Ref.
Has access to transportation to appointments					0.137
Yes	5883	5342 (90.8)	541 (9.2)	Ref.
No	609	541 (88.8)	68 (11.2)	1.24 (0.95–1.62)
Transgender					0.385
Yes	37	32 (86.5)	5 (13.5)	1.52 (0.59–3.90)
No	6455	5851 (90.6)	604 (9.4)	Ref.

AIDS, acquired immunodeficiency syndrome; CDC, Centers for Diseases Control and Prevention; CI, confidence interval; FPL, federal poverty level; IDU, injection drug use; OR, odds ratio.

In the stepwise backward logistic regression model, 12 variables were entered in the initial model, and seven variables maintained statistical significance at p-value <0.05. These risk factors include being in the age group 25–39 (β = 0.27, p < 0.001) or age group 18–24 (β = 0.06, p < 0.05), black race (β = 0.32, p < 0.001), poverty level <100% (β = 0.17, p < 0.001), homeless (β = 0.27, p < 0.001), problematic alcohol/drug use (β = 0.37, p < 0.001), diagnosed with AIDS as of 2016 (β = 0.24, p < 0.001), and not virally suppressed in 2016 (β = 0.91, p < 0.001) (Table 2). The C-statistic for the derivation model was 0.767% (Fig. 2). The optimism-corrected C-statistic was 0.763% (optimism = 0.004). The calibration plot shows good calibration with a predicted and observed probability of viral suppression aligning with the 45° line.

FIG. 2.

ROC curves for the final logistic regression model. ROC, receiver operating characteristic.

Table 2.

Final Predictors of Nonviral Suppression and Associated Risk Scoring System

Characteristics	Beta estimate	Score^a
Age in 2016
18–24	0.06	1
25–39	0.27	3
≥40	Ref.	0
Race
White/other	Ref.	0
Black	0.32	3
FPL <100%
Yes	0.17	2
No	Ref.	0
Homeless
Yes	0.27	3
No	Ref.	0
Alcohol/drug use resulted in any problem in daily activity or legal issue or hazardous situation
Yes	0.37	4
No	Ref.	0
Had CDC-defined AIDS in 2016
Yes	0.24	3
No	Ref.	0
Virally suppressed in 2016
Yes	Ref.	0
No	0.91	9

Scores are formed by multiplying the beta-coefficients by 10 and then rounding to the nearest integer.

AIDS, acquired immunodeficiency syndrome; CDC, Centers for Diseases Control and Prevention; FPL, federal poverty level.

The risk score ranged from 0 to 26 (Table 3). A patient will have highest risk score (score = 26) if the patient is aged 25–39, black, homeless, poverty level <100%, had AIDS as of 2016, had problematic alcohol/drug use, and had unsuppressed viral load in 2016. Nonviral suppression in 2016 greatly predicted nonviral suppression in 2017, and more than one-third of the total risk score was contributed by this variable. The simplified integer-based risk score performed well in the derivation data set (C-statistic = 0.768%). The distribution of predicted and observed percentage by these risk scores is provided in Fig. 3. An increase of one point in the risk score was associated with 1.2 increase in the odds of nonviral suppression (OR 1.22; 95 CI 1.20 − 1.24).

FIG. 3.

Distribution of predicted and observed percentages by these risk scores.

Table 3.

Risk Score 4 Tool for Nonviral Suppression After a Year of Being in Care and Predicted Risks Associated with Total Scores

Risk factor	Score	Patient's score	Risk score	Predicted risk of nonviral suppression, %
Age in 2016, years		—
18–24	1		0	2.6
25–39	3		1	3.1
≥40	0		2	3.7
Race		—	3	4.6
Other	0		4	5.5
Black	3		5	6.7
FPL <100%		—	6	8.0
Yes	2		7	9.6
No	0		8	11.5
Homeless		—	9	13.7
Yes	3		10	16.3
No	0		11	19.1
Alcohol/drug use resulted in any problem in daily activity or legal issue or hazardous situation		—	12	22.5
Yes	4		13	26.2
No	0		14	30.2
Had CDC-defined AIDS as of 2016		—	15	34.6
Yes	3		16	39.2
No	0		17	44.1
Virally suppressed in 2016		—	18	49.0
Yes	0		19	54.0
No	9		20	58.9
Total score		—	21	63.7
			22	68.2
			23	72.3
			24	76.1
			25	79.2
			26	82.6

How to use this tool: A risk score for each patient can be calculated by adding the scores for each risk factor. For example, if a patient is a 27-year-old black, FPL >100%, not homeless, has no alcohol/drug use resulted in any problem in daily activity or legal issue or hazardous situation, presents with AIDS diagnosis, and unsuppressed viral load, the health care provider can put the following scores in the patient's score column: Age in 2016 = 3; Race = 3; FPL <100% = 0; Homeless = 0; Has no alcohol/drug use resulted in any problem in daily activity or legal issue or hazardous situation = 0; Has CDC-defined AIDS as of 2016 = 3; and Virally suppressed in 2016 = 9. By adding all these scores, (3 + 3 + 0 + 0 + 0 + 3 + 9), the total risk score for the patient is 18. Then looking at the risk score and predicted risk columns, a patient with 18 risk scores will have a probability of 49.0% to have nonviral suppression by the end of 1-year follow-up.

This risk score tool works if there is complete information on all the six factors. If there is missing information on any of these factors, a separate risk score tool is required.

AIDS, acquired immunodeficiency syndrome; CDC, Centers for Diseases Control and Prevention; FPL, federal poverty level.

Table 3 shows a one-page scoring and decision tool that can be used in health facilities. This scoring and decision tool includes the list of the seven variables and space to record the score for each variable and a total score. On the right side, it includes the risk of nonviral suppression associated with each total score computed from the risk prediction score. To illustrate the application of the risk score, consider a patient who is a 27-year-old black, poverty level >100%, has permanent housing, no problematic alcohol/drug use, presents with AIDS diagnosis, and unsuppressed viral load. Then, according to Table 3, the total risk score of the patient can be calculated as 3 + 3 + 0 + 0 + 0 + 3 + 9, which will add up to 18. Looking at the right side of Table 3, this person has a 49.0% probability of not being virally suppressed by the end of next year.

Based on the percentile distribution of the total risk score, we created three categories. These were low risk (score 0–1), medium risk (score 2–7), and high risk (score ≥8). About 15% (n = 969) of the study population were in the low-risk category, and 1.6% (n = 15) of these were not virally suppressed. About 65% (n = 4243) of the study population were in medium-risk category, of whom 6.0% (n = 255) were not virally suppressed. About 20% (n = 1280) of the study population were in the high-risk category, and 26.5% (n = 339) of these were not virally suppressed. Compared with those who are in the low-risk category, those who are in the medium-risk and high-risk category were more likely not to be virally suppressed [OR 4.06 (CI 2.40–6.87) and OR 22.89 (CI 13.54–38.68), respectively].

We estimated sensitivity, specificity, and predictive values for various cutoff points in the risk scoring tool. A risk score of ≥7 as a cutoff point has a sensitivity of 63%, specificity of 77%, positive predictive value of 21%, and negative predictive value of 95%, whereas risk of ≥8 as a cutoff point has a sensitivity of 57%, specificity of 85%, positive predictive value of 31%, and negative predictive value of 94%. A risk of ≥9 as a cutoff point has a sensitivity of 52%, specificity of 88%, positive predictive value of 31%, and negative predictive value of 94%.

Discussion

We found that nonviral suppression by the end of 1-year follow-up time can be predicted using seven variables, which are easily ascertained by patient history and medical record. These variables include current age group, race, poverty level, current AIDS diagnosis, current homelessness, problematic alcohol/drug use, and current viral suppression status. The risk prediction tool has a total risk score of 26, and the risk for nonviral suppression increases as the risk score increases. In addition to predicting the magnitude of risk of nonviral suppression associated with each risk score, we also stratified the cohort into risk groups. Those in the high-risk category had about 23 times the risk of having nonviral suppression compared with the low-risk group. The risk prediction tool has good discriminative performance and calibration.

Many studies have identified individual risk factors associated with poor attainment of viral suppression.^{13

–24} Previous studies have shown the predictive role of age group, race, poverty level, AIDS diagnosis, alcohol/drug use, and homelessness on viral suppression.^{13
–15,17–21,25,27} Poverty and homelessness may be predictive of viral suppression due to competing needs.³⁹ AIDS status could affect viral suppression due to advanced nature of the disease.⁴⁰ The predictive role of current viral load to future virologic failure and other HIV disease progression has been demonstrated in previous risk prediction tools.^25,27 There is no evidence of direct mechanism through which race can predict viral suppression. In populations without great disparities in socioeconomic status and access to care such as the military⁴¹ and populations who receive care from culturally competent health care providers,⁴² the black race has not been found to be predictive of HIV care outcomes. In the current study, the black race is likely serving as a proxy for unmeasured factors such as low educational level, stigma, discrimination, mistrust of the health system, and quality of provider relationship that may be differentially affecting the black PHIV.⁴³⁻⁴⁵

Health care providers can use different cutoff points depending on availability of resources. If we consider sensitivity and specificity equally important, the cutoff value of 7 in the risk score gave a maximized value of sensitivity and specificity (63% and 77%, respectively). The corresponding positive and negative predictive values were 21% and 95%, respectively. However, based on the importance of false-positives and false-negatives, health care providers may choose to use different cutoff points. The cutoff point 7 identified 62% of individuals who failed to achieve viral suppression in next year, whereas the cutoff point 9 identified 52% of individuals who failed to achieve viral suppression in the next year. Moreover, the cutoff point 7 would put 26.8% of our population for intervention, whereas the cutoff point 9 would put 15.7% of our population for intervention. A lower cutoff value in the risk score would put a large proportion of our population for intervention and would identify the majority of individuals who failed to achieve viral suppression in the next year.

Our study builds on the study by Robbins et al., using more recent data. However, our study is different from the study by Robbins et al. in the definition of the outcome. Robbins et al. included patients who were virologically suppressed on ART at enrollment and defined virologic failure as (a) two consecutive measurements of HIV RNA level of >400 copies/mL or (b) one measurement of HIV RNA level of >400 copies/mL and no confirmatory test in the subsequent 3 months.²⁵ In our study, we included all PHIV regardless of viral suppression status, and our outcome of interest, nonviral suppression, was defined as having viral load ≥200 copies/mL in the last viral load measurement of the subsequent year. Moreover, while Robbins et al. included factors such as adherence to ART and antiretroviral history, which are not available in our data set, our study considered additional socioeconomic factors such as poverty level, access to transportation to medical appointments, food needs, transgender status, and AIDS status to develop the risk prediction model.

We attempted to explore alternative models with a reduced number of predictors (data not shown). After we exclude homelessness from the model, the discriminative performance of the model was similar (C-statistic = 0.763). To assess the performance of the model in situations where viral load measurement is not available, we excluded viral suppression status in 2016 from the model, but the C-statistic greatly reduced to 0.70.

Risk prediction tools that are simple to use, accurate in predicting risk, are generalizable across contexts, and use routinely collected variables are needed to identify patients at high risk for poor outcomes and to provide individualized risk assessment.⁴⁶ The risk prediction tool developed in this study needs external validation to evaluate its performance in other populations. The risk score could be useful in settings similar to the RWP. At the time a person with HIV visits an HIV care provider, the provider can quickly and easily use this tool to predict the probability of the person being not virally suppressed by the end of next year. Thus, the scoring can be useful to stratify PHIV in risk categories so that resources are directed to those at greater risk. Accordingly, patients can be targeted for intervention. Depending on the available resources and infrastructure, multifaceted interventions can be implemented to improve the success of the HIV care continuum. Addressing service-related, medical, and psychosocial factors, designing community-based interventions, including management and/or patient navigation,⁴⁷ home-based health care, economic empowerment, and population-specific interventions such as youth-friendly clinics and services^48,49 could improve the success of patients in the HIV care continuum.

About 8% (571) of those receiving care in 2016 were not in care in 2017 (lost to follow-up). We compared the baseline characteristics of our study population (those in care in 2017) and those lost to follow-up in 2017. Compared with our study population, those lost to follow-up were more likely to be 25 − 39 years old (36.1% vs. 29.3%; p < 0.001), black (53.9% vs. 38.5%; p < 0.001), non-Hispanic (59.2% vs. 42.3%; p < 0.001), being at poverty level of <100% (58.1% vs. 42.0%; p < 0.001), homeless (13.3% vs. 5.2%; p < 0.001), not virally suppressed in 2016 (27.3% vs. 12.5%; p < 0.001), feeling depressed or anxious (19.1% vs. 14.8%; p < 0.001), and have problematic alcohol/drug (7.0% vs. 2.0%; p < 0.001). These differences in baseline characteristics indicate that those lost to follow-up could have worse viral suppression status compared with our study population.

Despite the strengths of this simple risk prediction score model, there are several limitations that need to be acknowledged. First, we had a large sample size, and the model had good discrimination and calibration in the bootstrapped samples, but the model should undergo external validation to see the performance of the risk prediction model/score in other populations. Our population are low-income PHIV and high number of immigrants especially Latinos; hence, the predictive performance of the model/score may differ if the underlying population has different sociodemographic and behavioral characteristics from ours. Second, our study included people with new and existing HIV diagnosis. People with new HIV diagnosis may have different challenges to achieve viral suppression. Therefore, they may need a different risk score. Third, we depended on self-report for feeling depressed or anxious, problematic alcohol/drug use; this may be underreported. Fourth, exposures are measured at any time point in 2016; therefore, there might be differential follow-up time. However, we took the first comprehensive health assessment measurement of 2016 to ensure adequate follow-up time. Last, we were not able to measure adherence to ART and the duration of time the patients had been on ART. Including information about adherence to and duration of patients under ART could have improved the discrimination of the predictive model.

In summary, we have identified a set of readily available variables that can be used to predict nonviral suppression after a year of being in care among PHIV. The predictors of nonviral suppression were age group, race, poverty level, AIDS diagnosis, homelessness, problematic alcohol/drug use, and viral suppression status. The tool has good discriminative ability. In addition, the tool can be used to stratify PHIV into risk groups that can be identified for targeted intervention. In settings with similar demographics, the risk prediction tool can assist clinicians and health care providers to identify high-risk individuals and target for interventions. Follow-up studies are required to externally validate this risk prediction tool.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This research was supported, in part, by the National Institute on Minority Health and Health Disparities (NIMHD) under award numbers 1R01MD012421, 1R01MD013563, 5S21MD010683, K01MD013770, and U54MD012393, and Florida International University graduate school dissertation year fellowship. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Minority Health and Health Disparities or National Institutes of Health.

References

Centers for Diseases Control and Prevention. HIV/AIDS. Statistics Center. 2018. Available at: https://www.cdc.gov/hiv/statistics/overview/index.html (Last accessed January 10, 2020 ).

Florida Department of Health. Bureau of Communicable Diseases. HIV/AIDS Section. Epidemiological Profile for Area 11, Miami-Dade County. 2018. Available at: http://miamidade.floridahealth.gov/programs-and-services/infectious-disease-services/hiv-aids-services/_documents/10-26-18-update-HIV-Surveillance/_documents/FS-2017-MIAMI-DADE.pdf (Last accessed January 10, 2020 ).

U.S. Department of Health & Human Services. HIV Care Continuum. 2016. Available at: https://www.hiv.gov/federal-response/policies-issues/hiv-aids-care-continuum (Last accessed January 10, 2020 ).

Kay

, Batey

, Mugavero

. The HIV treatment cascade and care continuum: Updates, goals, and recommendations for the future. AIDS Res Ther, 2016; 13:35.

Lima

, Hogg

, Harrigan

, et al. Continued improvement in survival among HIV-infected individuals with newer forms of highly active antiretroviral therapy. AIDS, 2007; 21:685–692.

Montaner

JS.

Treatment as prevention—A double hat-trick. Lancet, 2011; 378:208–209.

Samji

, Cescon

, Hogg

, et al. Closing the gap: Increases in life expectancy among treated HIV-positive individuals in the United States and Canada. PLoS One, 2013; 8:e81355.

Philbin

, Tanner

, Duval

, et al. Linking HIV-positive adolescents to care in 15 different clinics across the United States: Creating solutions to address structural barriers for linkage to care. AIDS Care, 2014; 26:12–19.

Cohen

, Chen

, McCauley

, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med, 2011; 365:493–505.

10.

Shah

, Risher

, Berry

, et al. The epidemiologic and economic impact of improving HIV testing, linkage, and retention in care in the United States. Clin Infect Dis, 2016; 62:220–229.

11.

Centers for Diseases Control. HIV Continuum of Care. 2017. Available at: https://www.cdc.gov/nchhstp/newsroom/2017/HIV-Continuum-of-Care.html (Last accessed January 10, 2020 ).

12.

Centers for Diseases Control and Prevention. Selected National HIV Prevention and Care Outcomes in the United States. 2018. Available at: https://www.cdc.gov/hiv/pdf/library/factsheets/cdc-hiv-national-hiv-care-outcomes.pdf (Last accessed January 10, 2020 ).

13.

Crepaz

, Dong

, Wang

, et al. Racial and ethnic disparities in sustained viral suppression and transmission risk potential among persons receiving HIV care—United States, 2014. MMWR Morb Mortal Wkly Rep, 2018; 67:113–118.

14.

Whiteside

, Cohen

, Bradley

, et al. Progress along the continuum of HIV care among blacks with diagnosed HIV—United States, 2010. MMWR Morb Mortal Wkly Rep, 2014; 63:85–89.

15.

Sheehan

, Fennie

, Mauck

, et al. Retention in HIV care and viral suppression: Individual- and neighborhood-level predictors of racial/ethnic differences, Florida, 2015. AIDS Patient Care STDS, 2017; 31:167–175.

16.

Joseph Davey

, Abrahams

, Feinberg

, et al. Factors associated with recent unsuppressed viral load in HIV-1-infected patients in care on first-line antiretroviral therapy in South Africa. Int J STD AIDS, 2018; 29:603–610.

17.

Geter

, Sutton

, Armon

, et al. Trends of racial and ethnic disparities in virologic suppression among women in the HIV Outpatient Study, USA, 2010–2015. PLoS One, 2018; 13:e0189973.

18.

Castel

, Kalmin

, Hart

, et al. Disparities in achieving and sustaining viral suppression among a large cohort of HIV-infected persons in care—Washington, DC. AIDS Care, 2016; 28:1355–1364.

19.

Colasanti

, Kelly

, Pennisi

, et al. Continuous retention and viral suppression provide further insights into the HIV care continuum compared to the cross-sectional HIV care cascade. Clin Infect Dis, 2016; 62:648–654.

20.

Muthulingam

, Chin

, Hsu

, et al. Disparities in engagement in care and viral suppression among persons with HIV. J Acquir Immune Defic Syndr, 2013; 63:112–119.

21.

Tanner

, Lachowsky

, Ding

, et al. Predictors of viral suppression and rebound among HIV-positive men who have sex with men in a large multi-site Canadian cohort. BMC Infect Dis, 2016; 16:590.

22.

Blank

, Fletcher

, Verdecias

, et al. Factors associated with retention and viral suppression among a cohort of HIV+ women of color. AIDS Patient Care STDS, 2015; 29(Suppl 1):S27–S35.

23.

Myers

, Lin X Skarbinski

. Antiretroviral therapy and viral suppression among foreign-born HIV-infected persons receiving medical care in the United States: A complex sample, cross-sectional survey. Medicine, 2016; 95:e3051.

24.

Beer

, Mattson

, Bradley

, et al. Understanding cross-sectional racial, ethnic, and gender disparities in antiretroviral use and viral suppression among HIV patients in the United States. Medicine, 2016; 95:e3171.

25.

Robbins

, Johnson

, Chang

, et al. Predicting virologic failure in an HIV clinic. Clin Infect Dis, 2010; 50:779–786.

26.

Powers

, Price

, Karita

, et al. Prediction of extended high viremia among newly HIV-1-infected persons in sub-Saharan Africa. PLoS One, 2018; 13:e0192785.

27.

Mocroft

, Ledergerber

, Zilmer

, et al. Short-term clinical disease progression in HIV-1-positive patients taking combination antiretroviral therapy: The EuroSIDA risk-score. AIDS, 2007; 21:1867–1875.

28.

Bebu

, Tate

, Rimland

, et al. The VACS index predicts mortality in a young, healthy HIV population starting highly active antiretroviral therapy. J Acquir Immune Defic Syndr, 2014; 65:226–230.

29.

Nugent

, Edmonds

, Lusiama

, et al. Predicting mortality in HIV-infected children initiating highly active antiretroviral therapy in a resource-deprived setting. Pediatr Infect Dis J, 2014; 33:1148–1155.

30.

National Institute of Health. NIH HIV/AIDS Research Priorities and Guidelines for Determining AIDS Funding. 2015. Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-137.html (Last accessed January 10, 2020 ).

31.

The Ryan White HIV/AIDS Program, HIV/AIDS Bureau. 2019. Available at: https://hab.hrsa.gov/about-ryan-white-hivaids-program/about-ryan-white-hivaids-program (Last accessed January 10, 2020 ).

32.

United States Department of Health and Human Services. Computations for the 2016 poverty guidelines. 2016. Available at: https://aspe.hhs.gov/computations-2016-poverty-guidelines (Last accessed January 10, 2020 ).

33.

Centers for Diseases Control and Prevention. Selected National HIV Prevention and Care Outcomes. 2018. Available at: https://www.cdc.gov/hiv/pdf/library/slidesets/cdc-hiv-prevention-and-care-outcomes.pdf (Last accessed January 10, 2020 ).

34.

Steyerberg

EW.

Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer, 2009.

35.

Steyerberg

, Vickers

, Cook

, et al. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology, 2010; 21:128–138.

36.

Steyerberg

, Harrell

Jr , Borsboom

, et al. Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis. J Clin Epidemiol, 2001; 54:774–781.

37.

Han

, Song

, Choi

. How to develop, validate, and compare clinical prediction models involving radiological parameters: Study design and statistical methods. Korean J Radiol, 2016; 17:339–350.

38.

Austin

, Lee

, D'Agostino

, et al. Developing points-based risk-scoring systems in the presence of competing risks. Stat Med, 2016; 35:4056–4072.

39.

Kalichman

, Hernandez

, Kegler

, et al. Dimensions of poverty and health outcomes among people living with HIV infection: Limited resources and competing needs. J Community Health, 2015; 40:702–708.

40.

Langebeek

, Gisolf

, Reiss

, et al. Predictors and correlates of adherence to combination antiretroviral therapy (ART) for chronic HIV infection: A meta-analysis. BMC Med, 2014; 12:142.

41.

Silverberg

, Leyden

, Quesenberry

Jr ., et al. Race/ethnicity and risk of AIDS and death among HIV-infected patients with access to care. J Gen Intern Med, 2009; 24:1065–1072.

42.

Saha

, Korthuis

, Cohn

, et al. Primary care provider cultural competence and racial disparities in HIV care and outcomes. J Gen Intern Med, 2013; 28:622–629.

43.

Centers for Diseases Control and Prevention. HIV and African Americans. 2019. Available at: https://www.cdc.gov/hiv/group/racialethnic/africanamericans/index.html (Last accessed June 06, 2019 ).

44.

Freeman

, Gwadz

, Silverman

, et al. Critical race theory as a tool for understanding poor engagement along the HIV care continuum among African American/black and Hispanic persons living with HIV in the United States: A qualitative exploration. Int J Equity Health, 2017; 16:3.

45.

Gaston

, Alleyne-Green

. The impact of African Americans' beliefs about HIV medical care on treatment adherence: A systematic review and recommendations for interventions. AIDS Behav, 2013; 17:31–40.

46.

McNairy

, Abrams

, Rabkin

, et al. Clinical decision tools are needed to identify HIV-positive patients at high risk for poor outcomes after initiation of antiretroviral therapy. PLoS Med, 2017; 14:e1002278.

47.

Raj

, Yore

, Urada

, et al. Multi-site evaluation of community-based efforts to improve engagement in HIV care among populations disproportionately affected by HIV in the United States. AIDS Patient Care STDS, 2018; 32:438–449.

48.

Casale

, Carlqvist

, Cluver

. Recent interventions to improve retention in HIV care and adherence to antiretroviral treatment among adolescents and youth: A systematic review. AIDS Patient Care STDS, 2019; 33:237–252.

49.

Bulsara

, Wainberg

, Audet

, et al. Retention in HIV care in Australia: The perspectives of clinicians and clients, and the impact of medical and psychosocial comorbidity. AIDS Patient Care STDS, 2019; 33:415–424.