Developing and validating a clinical risk score to predict losses in the PISCIS cohort of people with HIV

Abstract

Background

People lost to follow-up (LTFU) from HIV care have an increased risk of worse health. The objective of this study is to create and validate a risk score to predict LTFU among PLWH in Catalonia and the Balearic Islands.

Methods

6661 PLWH were included. LTFU were those without contact with HIV care for 12 months or more. Logistic regression models were used to assess the role of independent factors on LTFU. The validation included a 10-fold iteration to predict the performance of the regression model and the Area under the ROC Curve (AUC). Regression coefficients were rounded and summed to construct the score.

Results

Determinants of LTFU included being younger than 34 years (OR: 1.80, CI, 1.44–2.23), not having been born in Spain (OR: 1.32, 1.11–1.58), men who inject drugs (OR: 2.10, 1.38–3.19), having a detectable viral load (OR: 3.14, 2.47–3.99), and ≤2.5 years since HIV diagnosis (OR: 3.84, 3.10–4.75). The validation of determinants resulted in a mean AUC of 0.69 and the risk-score revealed that 28.8% had a medium and 3.4% a high risk of LTFU respectively.

Conclusions

Findings can be used to prevent LTFU in HIV care.

Keywords

Lost to follow-up HIV cohort studies

Introduction

According to UNAIDS, by the year 2030, 95% of the people living with HIV (PLWH) should be diagnosed, 95% of them should be on treatment and 95% should be virally suppressed.¹ Current estimations show that 37.7 million (30.2 million–45.1 million) people live with HIV globally, 84% know their HIV status, 87% of them are receiving treatment and 90% of the people in treatment are virally suppressed.² In Catalonia (Spain), the PISCIS Cohort³ has shown that an estimated 89% of PLWH were diagnosed, of these 78% were under treatment, and 73% were virally suppressed.⁴ Although data are close to the target, more efforts are needed to increase retention in treatment (second 95%) and avoid people becoming lost to follow-up (LTFU).

Findings from previous studies show that people LTFU have an increased risk of mortality^5–7 and worse HIV prognosis while consistent HIV care showed a slower disease progression and a shorter time to achieve viral suppression, which also implies a lower risk of HIV transmission.⁸ Regarding its determinants, the risk of LTFU is higher among people who inject drugs (PID), migrants, and those with psychiatric disease,⁹ younger patients¹⁰ those with lower CD4 cells¹¹ and those whose perceived their first visits as unsatisfactory.¹² In Spain, Teira et al. analysed the determinants of LTFU in PLWH and reported that younger age, being men, having an immigration background, and a higher number of years on antiretroviral treatment (ART) are associated with an increased risk of losing contact with HIV care.¹³

Developing and validating a score to predict LTFU might be useful in clinical practice to enhance retention especially among those PLWH at higher risk of losing contact with healthcare services. The objective of this study is to validate a risk score to be used in clinical settings to predict losses of follow-up of PLWH in Catalonia and the Balearic Islands.

Methods

Population

The PISCIS Cohort is a population-based, longitudinal, multicentre project following PLWH aged 16 years or more since 1998.³ As of December 2020, 28 017 PLWH had been enrolled into the 17 hospitals in the cohort, contributing to 273 488 person-years of follow-up; 80.9% are male, 28.8% migrants and 35.4 years median of age. The coverage of the PISCIS Cohort is estimated at 82% and 60% of the total PLWH in Catalonia and the Balearic Islands respectively.¹⁴ On average the median number of patient visits to HIV care in the 17 hospitals of the Cohort PISCIS is two per year (interquartile range 1–3). Patients included in this study were those who entered the cohort in January of 2016 or later and according to the definition of care retention established for the PISCIS Cohort,¹⁵ the Consensus Document for the HIV Treatment in Spain¹⁶ and the Quality Care Indicators for the care of PLWH in Spain,¹⁷ LTFU was defined as not having any visits to any of the participant hospitals in the cohort for 12 months or more before December 2021. In addition, patients suspected to be LTFU were searched in the Public Data Analysis Program for Health Research and Innovation (PADRIS),¹⁸ an administrative electronic repository which accesses all health records of any individual seen through the Catalan Health System to check if they didn’t have other visits or contacts with healthcare services outside the centres in the Cohort, were transferred to other hospitals in Catalonia, or were in care and treatment in other healthcare services (i.e. in case of incarceration); and in any of these cases they were not considered as LTFU. According to this definition, some patients could have been lost for more than 12 months while others could have returned to the Cohort. In addition, some of these patients could be considered as medical care interrupters, however, this study is focused on the analysis of predictors of an event of no visit in a 12-months period of time. As reengagements were not analysed, the term LTFU is used to encompass both LTFU and medical care interruptions, in order to simplify the understanding of the study.

Variables

Variables included in the development of the score included last record of age split in a tertile (16–34, 35–44 and 45 years or older), country of birth (Spain or other), transmission group (men who have sex with men [MSM], heterosexual men, heterosexual women, men who inject drugs [MID], women who inject drugs [WID] or other), latest CD4 cell count (<200, 200–350, 350–500, >500 cells per millimetre), latest viral load (VL) available (undetectable ≤50 copies/millilitre and detectable >50 copies/millilitre), years since HIV diagnosis split in quartiles (0 to 2.5 years, 2.6 to 5.5, and 5.6 years or more) and size of the hospital based on number of PLWH of the cohort registered since 2016 at each of the 19 hospitals classified as: “large”, hospitals attending between 700 and 1761 patients (n = 3); “medium” hospitals attending between 413 and 617 patients (n = 4); and “small” hospitals attending less than 237 patients (n = 12).

Development and validation of the risk score

The risk score was derived from the 6661 PLWH enrolled in the PISCIS Cohort since January 2016. The first step in the development of the score was to construct a multivariable logistic regression model adjusted by those factors yielding a p-value < 0.1 in a previous univariate analysis. In order to be considered as independently related with the risk of LTFU, factors should yield a p-value < 0.05 in the adjusted multivariable model. Secondly, the cross-validation step consisted in estimating the area under the curve (AUC) iteratively for independent sub-samples extracted from the 6661 eligible PLWH. Derivation and evaluation groups were compared by means of the X² test with respect to clinical and demographic characteristics.

The ability of the variables to predict LTFU or medical care interruptions in the PISCIS Cohort was determined by a ten-fold cross-validated area under the receiver operating characteristic (AUROC) curve. Two random groups of PLWH were created, 90% of them (n = 5995) were assigned to the derivation group and 10% were assigned to the validation group (n = 666). The cross validation is one of the most common resampling techniques for evaluating predictive models and consists of splitting a sample into derivation and validation sets and consists to estimate the true positive rate and false positive rate and their ROC curve. In the 10-fold cross validation, the dataset is randomly divided into 10 approximately equally sized subsamples (or folds).¹⁹ We chose this method because it works for binary outcomes (in our case LTFU vs in Follow-up) and logit and probit regression models can be used with a set of independent variables providing fitted probabilities for the risk of LTFU. In addition, this method is used to avoid overly optimistic performance estimates that can be induced by evaluating a predictive model in the same sample in which it was developed, and the 10 times repetition performed by the cross-validation is one of two methods of predictive model evaluation with the least bias and variance in simulations.²⁰ The ‘cvauroc’ command in Stata was employed to derive and average the AUCs corresponding to each fold (here k = 10) and bootstrapping the cross-validated AUC was used to obtain statistical inference and bias corrected 95% confidence intervals (CIs).

The predictive scoring model of LTFU was constructed by incorporating the values of the Odds Ratios (OR) for each independent associated variable in the final adjusted multivariable analyses. All OR were rounded to the nearest whole number and converted into scores (for instance; if the OR was 1.73, the score of that variable was rounded to 2). The total risk score was generated by summing individual score values for each variable and splitting the total in tertiles; the highest hypothetical score is equal to 11 and cut-offs were categorized as “low” if the sum was 3 or less, “middle” if the score was between 4 and 7 and “high” if the score was 8 or more. To avoid collinearity between gender and transmission group, we combine these two variables into one: heterosexual men (HTSM), men who have sex with men (MSM), women infected through sex (WITS), men who inject drugs and women who inject drugs. Differences were considered statistically significant if p-values were equal or below 0.05.

Analyses were performed using STATA v15 (StataCorp, College Station, TX, USA).

Results

Sample characteristics

The Table 1 shows the main socio-demographic and clinical characteristics of PLWH on follow-up, LTFU, and PLWH who died during the study period (deaths). The median of age among patients LTFU was younger than those in follow-up or dead groups (35.2, 36.0 and 46.4 years respectively), and by age group, 31.3% of the younger patients were LTFU, while 19.7% of the older patients were LTFU. There were also differences between groups of PLWH on follow-up, LTFU, and deaths in terms of the country of birth: 26.4% of the patients born out of Spain were LTFU, while this proportion was 23.0% among people born in Spain. By transmission group, 35.5% of the men who used injecting drugs were LTFU and 20.6% of the heterosexual men were LTFU. CD4 cell count varied across LTFU and PLWH in follow-up, 29.8% of the patients with CD4 <200 where LTFU and among those with CD4 >500 the proportion of LTFU was 26.1%. According to the last available record of VL, 42.6% of the patients whose VL were detectable were LTFU, and among patients whose VL was detectable, 16.7% were LTFU. According to time since HIV diagnosis, 44.3% of the patients with less than 2.5 years since diagnosis were LTFU and 22.7% diagnosed more than 5.6 years ago were LTFU. Finally, the size of the hospital in the cohort was also associated with the risk of LTFU, with 28.7% of the patients registered in small hospitals considered as LTFU, while in large hospitals that proportion was 23.3%.

Table 1.

Characteristics of PLWH registered the PISCIS Cohort in 2016 or after according to their status, socio-demographic and clinical factors.

	In follow-up	LTFU	Deaths	Total
Population, n (%)	4932 (72.4)	1729 (25.4)	150 (2.2)	6811 (100)
Mean age* (SD)	36.0 (10.1)	35.2 (9.6)	46.4 (13.2)	35.9 (10.2)
Age group at last visit, n (%)*
16–34 years	1597 (67.0)	736 (31.3)	17 (0.7)	2350 (100)
35–44 years	1711 (74.1)	570 (24.6)	29 (1.3)	2310 (100)
45 years or more	1624 (75.5)	423 (19.7)	104 (4.8)	2151 (100)
Country of birth*
Spain	1601 (73.2)	494 (23.0)	78 (3.8)	2179 (100)
Other	2494 (72.1)	919 (26.4)	37 (1.5)	3450 (100)
Missing	831 (70.3)	316 (26.7)	35 (3.0)	1182 (100)
Transmission group, n (%)*
HTSM	581 (75.3)	159 (20.6)	32 (3.9)	772 (100)
MSM	3188 (74.9)	1041 (24.5)	27 (0.6)	4256 (100)
WITS	475 (74.6)	151 (23.7)	11 (1.7)	637 (100)
Men WID	180 (52.8)	121 (35.5)	40 (11.7)	341 (100)
Women WID	46 (63.9)	17 (23.6)	9 (12.5)	72 (100)
Missing	462 (63.0)	240 (32.7)	31 (4.3)	733 (100)
Median CD4 cell count, last record (SD)	737.2 (339.8)	683.4 (387.4)	628.9 (1321.4)	723.9 (397.6)
Median CD4 count (SD)*
>500	3735 (72.5)	1345 (26.1)	69 (1.4)	5149 (100)
350–500	706 (77.4)	193 (21.2)	13 (8.7)	912 (100)
200–350	356 (73.4)	112 (23.1)	17 (3.5)	485 (100)
<200	135 (50.9)	79 (29.8)	51 (19.3)	265 (100)
Viral load, last record*
Undetectable	4367 (81.8)	891 (16.7)	81 (1.5)	5339 (100)
Detectable	295 (49.5)	254 (42.6)	47 (7.9)	596 (100)
Missing	270 (30.8)	584 (66.7)	22 (2.5)	876 (100)
Years since diagnosis of HIV*
0–2.5 years	614 (50.5)	539 (44.3)	63 (5.2)	1216 (100)
2.6–5.5 years	2062 (79.5)	509 (19.6)	22 (0.8)	2593 (100)
5.6 or more	2256 (75.5)	681 (22.7)	65 (2.3)	3002 (100)
Size of the hospital*
Small	956 (68.0)	403 (28.7)	47 (3.3)	1406 (100)
Medium	1484 (72.6)	509 (24.9)	50 (2.4)	2043 (100)
Large	2492 (74.5)	817 (23.3)	53 (1.6)	3362 (100)

SD: Standard deviation, HTSM: Heterosexual men, MSM: men who have sex with men; WITS: women infected through sex; Men WID: Men who inject drugs; Ref.: Reference category. *Differences between groups are statistically significant, p-value <0.05.

After excluding dead patients, the eligible sample for developing the score included 6661 PLWH, 5995 (90%) were assigned to the derivation cohort and 666 (10%) to the validation cohort (Figure 1). As expected, validation and derivation samples had similar socio-demographic and clinical characteristics (p-values >0.05; Table 2).

Figure 1.

Selection of patients for the study.

Table 2.

Demographic and clinical characteristics of patients in both derivation and validation samples.

	Derivation sample, n (%)	Validation sample, n (%)	Total, n (%)
Patients	5995 (90)	666 (10)	6661 (100)
On follow-up	4428 (89.8)	504 (10.2)	4932 (100)
LTFU	1567 (90.6)	162 (9.4)	1729 (100)
Age group at last visit
16–34 years	2102 (90.1)	231 (9.9)	2333 (100)
35–44 years	2051 (89.9)	230 (10.1)	2281 (100)
45 years or more	1842 (90.0)	205 (10.0)	2047 (100)
Country of birth
Spain	1885 (89.4)	216 (10.6)	2101 (100)
Other	3085 (90.3)	328 (9.7)	3413 (100)
Missing	1025 (89.4)	122 (10.6)	1147 (100)
Transmission group
HTSM	633 (89.6)	77 (10.4)	740 (100)
MSM	3808 (90.0)	421 (10.0)	4229 (100)
WITS	565 (90.3)	61 (9.7)	626 (100)
Men WID	263 (87.4)	38 (12.6)	301 (100)
Women WID	57 (90.5)	6 (9.5)	63 (100)
Missing	639 (91.1)	63 (8.9)	702 (100)
CD4 cell, last record
>500	4575 (90.1)	505 (9.9)	5080 (100)
350–500	812 (90.3)	87 (9.7)	899 (100)
200–350	418 (89.3)	50 (10.7)	468 (100)
<200	190 (88.8)	24 (11.2)	214 (100)
Viral load, last record
Undetectable	4726 (89.9)	532 (10.1)	5258 (100)
Detectable	504 (91.8)	45 (8.2)	549 (100)
Missing	765 (89.6)	89 (10.4)	854 (100)
Years since HIV diagnosis
0–2.5	1046 (90.7)	107 (9.3)	1153 (100)
2.6–5.5	2299 (89.4)	272 (10.6)	2571 (100)
5.6 or more	2650 (90.2)	287 (9.8)	2937 (100)
Size of the hospital
Small	1239 (91.2)	120 (8.8)	1359 (100)
Medium	1775 (89.1)	218 (10.9)	1993 (100)
Large	2981 (90.1)	328 (9.9)	3309 (100)

HTSM: Heterosexual men, MSM: men who have sex with men; WITS: women infected through sex; Men WID: Men who inject drugs.

Predictors of LTFU and scoring

Table 3 shows unadjusted and adjusted predictors of LTFU. Adjusted OR showed that as age decreases the risk of LTFU increases; patients aged 34 years or less showed the highest risk of LTFU (OR: 1.80, 95% CI 1.44–2.23) followed by those aged 35–44 years (OR: 1.32, 95% CI 1.07–1.71). People not born in Spain showed a higher risk of LTFU (OR: 1.32, 95% CI 1.11–1.58). Among the categories of mode of transmission, men who injected drugs showed the highest risk of LTFU (OR: 2.10, 95% CI 1.38–3.19). Regarding the last record available of VL, PLWH considered as detectable showed a higher risk of LTFU (OR: 3.14, 95% CI 2.47–3.99). The risk of LTFU was higher among those with a diagnosis of HIV within the last 2.5 years (OR: 3.84, 95% CI 3.10–4.75) followed by those with 5.6 years or more (OR: 1.26, 95% CI 1.04–1.53) since the diagnosis of HIV. Finally, patients attending small hospitals were at higher risk of LTFU (OR: 1.40, 95% CI 1.12–1.76). The last record of CD4 cell count had no statistical relevance in the risk of becoming LTFU.

Table 3.

Unadjusted and adjusted logistic regression analysis of factors associated with LTFU risk in the derivation cohort and values for calculating the risk of LTFU score.

	Unadjusted Odds ratio, 95% Confidence Interval	Adjusted Odds ratio, 95% Confidence Interval	Score for calculating the risk of LTFU
Age group at last visit
16–34 years	1.77 (1.54–2.03)	1.80 (1.44–2.23)	2
35–44 years	1.28 (1.11–1.47)	1.32 (1.07–1.71)	1
45 years or more	Ref.
Country of birth
Spain	Ref.	Ref.
Other	1.20 (1.06–1.36)	1.32 (1.11–1.58)	1
Transmission group
HTSM	Ref.	Ref.
MSM	1.19 (0.99–1.44)	1.24 (0.95–1.61)	-
WITS	1.16 (0.90–1.49)	0.99 (0.70–1.40)	-
Men WID	2.46 (1.83–3.28)	2.10 (1.38–3.19)	2
Women WID	1.35 (0.75–2.41)	1.14 (0.50–2.61)	-
CD4 cell, last record
>500	Ref.	Ref.
350–500	0.76 (0.64–0.90)	0.89 (0.71–1.12)	-
200–350	0.87 (0.70–1.09)	0.92 (0.68–1.24)	-
<200	1.62 (1.22–2.16)	1.21 (0.80–1.84)	-
Viral load, last record
Undetectable	Ref.	Ref.
Detectable	4.22 (5.51–5.06)	3.14 (2.47–3.99)	3
Years since HIV diagnosis
0–2.5 years	3.55 (3.06–4.13)	3.84 (3.10–4.75)	4
2.6–5.5 years	Ref.
5.6 or more	1.22 (1.07–1.39)	1.26 (1.04–1.53)	1
Size of the hospital
Large	Ref.	Ref
Medium	1.05 (0.92–1.19)	1.05 (0.87–1.26)	-
Small	1.28 (1.12–1.48)	1.40 (1.12–1.76)	1

HTSM: Heterosexual men, MSM: men who have sex with men; WITS: women infected through sex; Men WID: Men who inject drugs; Ref.: Reference category. Associations statistically significant are shown in bold.

Validation of the predictive score

The value of the ROC area was 0.69 (95% CI: 0.67–0.71). Figure 2 shows the 10 ROC curves for the 10-fold models which on average had an AUC value ranging between 0.635 and 0.738 (see Table 4). In order to represent the use of the score, Figure 3 show the patient’s distribution according to each value in the risk score. A total of 67.8% of the sample used in this study showed a low risk score of LTFU (score ≤3), 28.8% of the sample showed medium risk values of LTFU (scores 4–7) and 3.4% showed a high risk of LTFU (scores ≥8).

Figure 2.

Cross-valitaed and k-fold ROC curves.

Table 4.

Area under de curve of the 10 validation fold.

Fold	N	AUC
1	526	0.674
2	526	0.719
3	526	0.675
4	526	0.696
5	526	0.693
6	526	0.702
7	526	0.635
8	526	0.680
9	526	0.738
10	526	0.700

Model: probit; Mean AUC = 0.69; Bootstrap bias corrected 95%CI = 0.67–0.71; cvSD AUD: 0.028.

Figure 3.

Classification of patients according to the risk score.

Discussion

In this study we developed and validated a tool to predict the risk of losing contact with HIV care in the hospitals of the PISCIS Cohort, one of the largest HIV cohorts in Europe. Men who inject drugs, those younger than 34 years of age, those with a detectable VL, having a diagnosis of HIV for less than 2.5 years, those born out of Spain and those attending hospitals with less than the 4% of the patients enrolled in the PISCIS showed the higher risk of LTFU. The area under the ROC curve derived from the validation process reached 0.69, which is considered acceptable²¹ for predicting the risk of becoming LTFU in this HIV Cohort, considering that the perfect classification is 1.0 and 0.5 is a random classifier. The development of and validation of this score is based on actual data and its interpretation reflects the situation of PLWH in PISCIS, one of the largest HIV cohorts in southern Europe. This tool might be useful for identifying PLWH at high-risk of LTFU in our cohort.

Factors associated with a higher risk score of LTFU

The risk score presented in this study could be a handy tool to predict the risk of losing contact with HIV in specialized care and is based on the own patient’s clinical and socio-demographic factors. Value scores assigned to each risk factor were derived from a well-fitted logistic regression model adjusted by variables that have shown consistent association with LTFU in previous studies. For instance, the fact that young PLWH show an increased risk of losing contact with HIV is concordant with previous studies published that show low adherence and more frequent interruptions in care among this population.^{11,13,22–26} Explanations for this include the existance of structural barriers that affect especially young PLWH, for instance, lack of transportation, job obligations, and problems dealing with family duties.²⁷ In addition, other possible explanations of making young PLWH lose contact with HIV care can be the presence of mental health disorders, such as, substance use disorders and depression²⁸, as well as other factors like an increased perception of HIV-related stigma, which is more prevalent among younger PLWH.²⁹

The fact that people born out of Spain have a higher risk-score is also comparable to findings from previous studies that showed an association between migration and an increased risk of LTFU. A study carried out in France found that PLWH born in Sub-saharan Africa are more often LTFU than people born in France or in other European countries²⁶ and studies from Spain and Italy showed similar results, not only for Sub-saharan African PLWH, but also for foreingners in general.^13,22,30 In Europe, the prevalence of HIV has been reported as disproportionally high among some sub-populations of migrants. Even though most western European countries, including Spain, provide ART to migrants regardless of their legal status (documented or undocumented),³¹ a number of barriers have been described to explain the higher risk of LTFU among migrants, including poor access to HIV services, lack of cross-border collaboration, high mobility/dispersal, lack of community support, stigma, lower social status, knowledge and attitudes of health professionals, and time constraints of services.³² In addition, in the case of Spain, some experiences of migrants in accessing healthcare services might also explain their excess of LTFU risk. For instance, previous studies pointed out at difficulties with accepting becoming HIV-positive, with managing the health effects of having HIV or of its treatment, and with sharing the diagnosis with loved ones³³ and other studies also for Spain described additional barriers like waiting time for an appointment, lacking a health card and waiting times in clinic.³⁴

The data of our study show that men living with HIV, especially those who inject drugs have an increased risk-score of LTFU and they score higher than those from other transmission groups, like heterosexual males. This result confirm previous studies,^13,22,35 but we found that women who inject drugs do not share the same risk of losing contact with HIV care than men who inject drugs. While previous studies show that women are at lower risk of LTFU^36,37 and have better adherence to ART than males in both middle-high income³⁸ and low-income countries,³⁹ our data suggested that this fact is true even among PID in which women have a lower risk of LTFU than PID men.

We also observed that detectable VL is associated with an increased risk of LTFU. People with VL load at their last visit to HIV care (just before becoming LTFU) have a higher risk-score of LTFU, as it has been also observed in previous studies. A record of detectable VL among patients in ART has been previously associated with LTFU^36,40 and those PLWH that reach viral supression and then have a virological breakthrough (reaching a viral load of >1000 copies/mL after previous suppression) also have an increased risk of LTFU. Authors suggested that those who struggle to adhere to therapy, initially noted by an increase in viral load, signaling less than ideal adherence to taking ART, may also not adhere to health care follow-up.⁴¹

Results of this study show that PLWH who have lived with HIV for less than 2.5 years have an increased risk-score of LTFU, and among all factors used to build this score, the value of the risk of those diagnosed with HIV less than 2.5 years is the highest by far (value of 4). These results are similar to those reported by Teira et al. in Spain¹³ showing that a year of increase in the years lived with HIV reduces the risk of LTFU, and with another study carried out in Nigeria showing that during the first year of treatment nearly a half of patients are LTFU.⁴² Although socioeconomic contexts might play an important role on the data interpretation, explanations for this include the fact that people with a recent HIV diagnosis are more difficult to retain in care and some previous evidence showed that once patients passed the first years of diagnosis their rates of retantion are higher than 90%³⁰ and LTFU is more common among patients early enrolled in ART than those who at least pased 6 months since diagnosis.^43,44

Patients attending the smaller hospitals of the PISCIS Cohort have an increased risk-score of LTFU. Three of the 19 hospitals of the cohort attended all together up to 51.0% of the PLWH, four hospitals had 28.7% patients, and the rest (20.3%) were attended in small hospitals. This finding is concordant with a previous study carried out in Spain.¹³ In terms of the cascade of care, a Dutch study showed that retention in care is higher in middle and large hospitals (compared to small) and authors suggested that complex and large centres are more likely to have HIV and infectious diseases specialized units, with a multidisciplinary working-team, like nurses, social workers and psychologists and this might contribute to increase retention.⁴⁵

Finally, a previous study carried out in the Spanish CoRIS Cohort assessed medical care interruptions, defined as those happened during an interval of 15 months between two visits.⁴⁶ These authors revealed that approximately one-third of the analyzed patients experienced at least one interruption in care between 2004 and 2019. More importantly, some of the factors described by the authors align with those found in this study as potentially predictors of LTFU: younger age, injecting drug use, and having been born out Spain. Although concepts are different, this suggests that medical care interruptions could serve as a proxy for LTFU, and vice versa, and the clinical application of this scoring system could prove beneficial in preventing both phenomena.

Comparison with similar risk-scores to predict LTFU in HIV

A previous study carried out in the United States developed and validated a model to predict the likelihood of missing upcoming HIV care visits and found that it is most strongly associated with having missed previous visits, and those patients with a high risk of missing upcoming visits are also those who missed three or more visits in the past.⁴⁷ As in our study, these authors showed that the risk of missing upcoming visits is higher among younger PLWH, those less likely to be virally suppressed, and added that other factors not included in our study such as lack of health insurance, having moderate to severe symptoms of depression and anxiety, and reporting current illicit drug use increases the risk of missing HIV visits.

Limitations, strengths and implications

This study has some limitations worth commenting. First, we have considered as LTFU those PLWH not visiting any of the 19 hospitals of Catalonia and the Balearic Islands, but no further searches were made about the possible transfer of these patients to other parts of Spain or other countries. Some of the patients considered as LTFU were born out of Spain they might be on treatment elsewhere. Secondly, although for the development of this risk-score we used data of a seven-year period (2016–2021), the characteristics of PLWH in Catalonia and followed in the PISCIS Cohort are very dynamic and some of the factors considered as predictors of LTFU might have changed, for instance, different groups of MSM, born in and out Spain, naïve and not naïve, in ART and out of ART, as a previous study using data of PISCIS has shown.⁴⁸ Thirdly, the development and validation of this risk-score was made using only data from the PISCIS Cohort and no external data was used to compare the validity of these factors and it cannot be generalized out of the PISCIS Cohort. Nevertheless, the results of this study can be used by other researchers in other HIV cohorts to assess how the risk-scores of this study apply to other contexts and cohorts. Finally, the definition of LTFU was made based on the clinical recommendations for Catalonia and Spain, establishing that patients should visit HIV medical care at least once in a 12-months period. We have not assessed other outcomes like care interruptions, shorter times of absence, patients that leave treatment and then reengaged, or patients that have been out of HIV care for 12-months more than once during the study period. All these could have led to an overstimation of retention in care. Moreover, possible transfers out of the Catalan and Baleraic HIV specialized care were not assessed, as the data to verify this in the PISCIS Cohort are not available. However, predicting LTFU using this score could be an useful tool to be used in the PISCIS and can be extrapolable to similar cohorts in Europe.

Strengths of this study include the large size, geographical diversity, multiple sites, and demographic and clinical diversity of the study population. No previous tools have been developed in large HIV Cohorts in Spain to predict the risk of LTFU using data collected systematically for more than 20 years.

In summary, this score revealed that 28.8% and 3.4% of the PISCIS Cohort (analyzed since 2016) showed a medium or high risk of LTFU respectively, and their risk is associated with socio-demographic like younger age, foreign place of birth and use of injecting drugs (only among men); and clinical variables like detectable viral load, less than 2.5 years since HIV diagnosis and attending a small hospital. Since the data required to construct this score is routinely gathered, its utilization will prove both practical and beneficial in everyday clinical practice. Clinic-based approaches for evaluating patients based on their socio-demographic and clinical attributes can be implemented in PISCIS and other large HIV cohorts as part of a concerted effort to effectively bridge gaps in the HIV care continuum. This initiative will enhance adherence and facilitate progress toward achieving the UNAIDS’ 95-95-95 targets.

Footnotes

PISCIS Study Group

J. Casabona, E. Muntada, A. Bruguera, G. Escaramís (CEEISCAT); JM. Miró, J. Ambrosioni (Hospital Clínic-Idibaps, Universitat de Barcelona); A. Imaz (Unitat de VIH i ITS, Servei de Malalties Infeccioses, Hospital Universitari de Bellvitge, IDIBELL); P. Domingo (Unitat de VIH/SIDA Hospital de la Santa Creu i Sant Pau); JM. Llibre (Fundació Lluita contra les Infeccions -Hospital Universitari Germans Trias i Pujol); F. Fanjul (Servei Medicina Interna, Hospital Universitari Son Espases); G. Navarro (Unitat de VIH/SIDA, Parc Tauli Hospital Universitari-Universitat Autònoma de Barcelona); V. Falcó Ferrer (Servei de Malalties Infeccioses, Hospital Universitari Vall d'Hebron, Vall d'Hebron Research Institute (VHIR)); H. Knobel (Servei de Malalties Infeccioses, Hospital del Mar); L. Arbonés (Hospital de Mataró-Consorci Sanitari del Maresme); R. Martín, S. Moreno, J. Aceiton (CEEISCAT); J. Mallolas (Hospital Clínic-IDIBAPS-Universitat de Barcelona); J. Tiraboschi (Hospital Universitari de Bellvitge-IDIBELL); A. Curran, J. Burgos (Hospital Universitari Vall d’Hebron); B. Revollo (Fundació Lluita contra les Infeccions -Hospital Universitari Germans Trias i Pujol); P. Prieto, MM. Gutierrez, (Hospital de la Santa Creu i Sant Pau); J. Murillas (Hospital Son Espases de Mallorca); F. Homar, A. Villoslada (Hospital Son Llàtzer); E. González (Hospital de Santa Maria); J. Peraire (Hospital Joan XXIII); L. Albiach (Hospital de Mataró-Consorci Sanitari del Maresme); E. Leon (Consorci Sanitari Integral); A. Mera (Hospital de Palamós); I. Vilaró (Hospital General de Vic); A. Orti (Hospital Verge de la Cinta de Tortosa), David Dalmau (Hospital Universitari Mútua Terrassa); A. Jaen (Fundació Docència i Recerca MútuaTerrassa); E. Deig (Hospital General de Granollers); E. De Lazzari, L. Berrocal (Hospital Clínic-IDIBAPS-Universitat de Barcelona); L. Rodríguez Vázquez (Hospital Universitari Vall d’Hebron); F. Gargoulas, T. Vanrell (Hospital Son Espases i Hospital Son Llàtzer); JC. Rubia (Consorci Sanitari Integral); J. Vilà (Serveis de Salut Integrats Baix Empordà); M. Martínez (Fundació Docència i Recerca MútuaTerrassa); B. Morell (Hospital de Santa Maria); M. Tamayo (Hospital General de Granollers); D. Nomah, J. Palacio-Vieira (CEEISCAT); M. Laguno, M. Martínez-Rebollar, JL. Blanco, E. Martínez, A. Torres, L. de la Mora, A Inciarte, I. Chivite, A. González-Cordon, A. Foncillas (Hospital Clínic-IDIBAPS-Universitat de Barcelona); A. Jou, E. Negredo (Fundació Lluita contra les Infeccions -Hospital Universitari Germans Trias i Pujol); M. Saumoy, A. Silva, S. Scévola (Hospital Universitari de Bellvitge-IDIBELL); J. Navarro, P. Suanzes, P. Alvarez (Hospital Universitari Vall d’Hebron); M. Gracia Mateo (Hospital de la Santa Creu i Sant Pau); M. Riera Jaume, M. García-Gasalla, MA. Ribas, A. Campins, M. Peñaranda, ML. Martin, F. Artigues, A. Ferre (Hospital Son Espases de Mallorca); S. Calzado, M. Cervantes, M. Navarro (Unitat de VIH/SIDA, Parc Tauli Hospital Universitari), A. Payeras, C. Cifuentes, P. Sorní, M. Molero (Hospital Son Llàtzer); N. Abdulghani, T. Comella, R. Sola (Hospital de Santa Maria); C. Viladés, B. Villar, A. Martí, A. Rull (Hospital Joan XXIII); MF. Solano (Hospital de Mataró-Consorci Sanitari del Maresme); C. Escrig (Hospital Verge de la Cinta de Tortosa), M. Cairó, X. Martinez-Lacasa, R. Font, L. Gisbert (Hospital Universitari Mútua Terrassa); L. Macorigh (Hospital General de Granollers). J. Hernández (GTT), en representació del Comitè 1er de Desembre.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project leading to these results (PISCIS Cohort) has received funding from “la Caixa” Banking Foundation under the project code LCF/PR/PR17/51120008. This work is supported by a grant from the La Marató de TV3 Foundation (project code 239/C/2018) aimed at the analysis of the LTFU patients of the PISCIS Cohort. The funding bodies played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript. JMM received a personal 80:20 research grant from Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain, during 2017–24.

ORCID iD

Jorge Palacio-Vieira

References

UNAIDS . Fast-track: ending the AIDS epidemic by 2030, https://www.unaids.org/sites/default/files/media_asset/JC2686_WAD2014report_en.pdf (2014).

UNAIDS . Global HIV statistics, Fact sheet, https://www.unaids.org/sites/default/files/media_asset/UNAIDS_FactSheet_en.pdf (2021).

Jaen

Casabona

Esteve

, et al. Clinical-epidemiological characteristics and antiretroviral treatment trends in a cohort of HIV infected patients. The PISCIS Project. Med Clin 2005; 124: 525–531.

Benito

Reyes-Urueña

Fuster

, et al. Monitoring and evaluation of HIV in Catalonia, getting closer to the 90-90-90 objectives. Rev Esp Salud Publica. 2019; 93: e1-16.

Mugavero

Lin

H-Y

Willig

, et al. Missed visits and mortality among patients establishing initial outpatient HIV treatment. Clin Infect Dis Off Publ Infect Dis Soc Am 2009; 48: 248–256.

Edwards

Cole

Westreich

, et al. Loss to clinic and five-year mortality among HIV-infected antiretroviral therapy initiators. PLoS One. 2014; 9(7): e102305. doi:10.1371/journal.pone.0102305.

Giordano

Gifford

White

, et al. Retention in care: a challenge to survival with HIV infection. Clin Infect Dis 2007; 44: 1493–1499.

Crawford

Sanderson

Thornton

. Impact of poor retention in HIV medical care on time to viral load suppression. J Int Assoc Phys AIDS Care 2014; 13: 242–249.

Lebouché

Yazdanpanah

Gérard

, et al. Incidence rate and risk factors for loss to follow-up in a French clinical cohort of HIV-infected patients from January 1985 to January 1998. HIV Med 2006; 7: 140–145.

10.

Cuzin

Dellamonica

Yazdanpanah

, et al. Characteristics and consequences of medical care interruptions in HIV-infected patients in France. Epidemiol Infect 2016; 144: 2363–2370.

11.

Mocroft

Kirk

Aldins

, et al. Loss to follow-up in an international, multicentre observational study. HIV Med 2008; 9: 261–269.

12.

Dang

Westbrook

Hartman

, et al. Retaining HIV patients in care: The role of initial patient care experiences. AIDS Behav 2016; 20: 2477–2487.

13.

Teira

Espinosa

Gutiérrez

, et al. Losses to follow-up of HIV-infected people in the Spanish VACH cohort over the period between 2013 and 2014: The importance of sociodemographic factors. Enferm Infecc Microbiol Clín 2019; 37: 361–366.

14.

Bruguera

Nomah

Moreno-Fornés

, et al. Cohort profile: PISCIS, a population-based cohort of people living with HIV in Catalonia and Balearic Islands. Int J Epidemiol 2023; 52: e241–e252.

15.

Campbell

CNJ

Ambrosioni

Miro

, et al. The continuum of HIV care in Catalonia. AIDS Care - Psychol Socio-Med Asp AIDSHIV 2015; 27: 1449–1454.

16.

Grupo de Estudio del SIDA-SEIMIC (GESIDA) . Gobierno de España, Ministerio de Sanidad. Documento de consenso de GeSIDA/División de Control de VIH, ITS, Hepatitis virales y Tuberculosis del Ministerio de Sanidad respecto al tratamiento antirretroviral en adultos infectados por el virus de la inmunodeficiencia humana (Actualización enero 2023). https://gesida-seimc.org/documento-de-consenso-de-gesidaplan-nacional-sobre-el-sida-respecto-al-tratamiento-antirretroviral-en-adultos-infectados-por-el-virus-de-la-inmunodeficiencia-humana-actualizacion-enero-2023/ (2023).

17.

GeSIDA . Indicadores de calidad asistencial de GeSIDA para la atención de personas que viven con el VIH, https://guiasclinicas.gesida-seimc.org/version/?do=show&pk=8 (2021).

18.

Generalitat de Catalunya . Agència de Qualitat i Avaluació Sanitàries de Catalunya. Barcelona, Spain: Programa Públic d’Analítica de Dades per a la Recerca i la Innovació en Salut a Catalunya – PADRIS, https://aquas.gencat.cat/ca/detall/article/padris (2017).

19.

Luque-Fernandez

Redondo-Sánchez

Maringe

. cvauroc: Command to compute cross-validated area under the curve for ROC analysis after predictive modeling for binary outcomes. STATA J 2019; 19: 615–625.

20.

Steyerberg

Harrell

Borsboom

, et al. Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001; 54: 774–781.

21.

Mandrekar

. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol Off Publ Int Assoc Study Lung Cancer 2010; 5: 1315–1316.

22.

Sobrino-Vegas

Gutiérrez

Berenguer

, et al. The Cohort of the Spanish HIV Research Network (CoRIS) and its associated biobank; organizational issues, main findings and losses to follow-up. Enferm Infecc Microbiol Clín 2011; 29: 645–653.

23.

Giordano

Hartman

Gifford

, et al. Predictors of retention in HIV care among a national cohort of US veterans. HIV Clin Trials 2009; 10: 299–305.

24.

Lourenço

Colley

Nosyk

, et al. High levels of heterogeneity in the HIV cascade of care across different population subgroups in British Columbia, Canada. PLoS One 2014; 9: e115277.

25.

Martinez-Guerra

Valdez-Ventura

Caro-Vega

, et al. Gaps in the continuum of care in HIV-positive adults and the need for caution in those returning to care after loss to follow-up. AIDS Care. 2023; 35(10): 1604–1611.

26.

Fournier

Parienti

J-J

Champenois

, et al. Incidence and risk factors for medical care interruption in people living with HIV in a French provincial city. PLoS One 2020; 15: e0240417.

27.

Rudy

Murphy

Harris

, et al. Patient-related risks for nonadherence to antiretroviral therapy among HIV-infected youth in the United States: A study of prevalence and interactions. AIDS Patient Care STDS 2009; 23: 185–194.

28.

Haas

Lienhard

Didden

, et al. Mental health, ART adherence, and viral suppression among adolescents and adults living with HIV in South Africa: A cohort study. AIDS Behav. 2023; 27(6): 1849–1861. doi:10.1007/s10461-022-03916-x.

29.

Mugo

Kohler

Kumar

, et al. Effect of HIV stigma on depressive symptoms, treatment adherence, and viral suppression among youth with HIV. AIDS. 2022; 37(5): 813–821. doi:10.1097/QAD.0000000000003473.

30.

Comelli

Izzo

Donato

, et al. Disengagement and reengagement of HIV continuum of care in a single center cohort in northern Italy. HIV Res Clin Pract 2019; 20: 1–11.

31.

Deblonde

Sasse

Del Amo

, et al. Restricted access to antiretroviral treatment for undocumented migrants: a bottle neck to control the HIV epidemic in the EU/EEA. BMC Publ Health 2015; 15: 1228.

32.

Baggaley

Nazareth

Divall

, et al. National policies for delivering tuberculosis, HIV and hepatitis B and C virus infection services for refugees and migrants among Member States of the WHO European Region. J Trav Med 2023; 30: taac136.

33.

Gogishvili

Flórez

Costa

, et al. A qualitative study on mixed experiences of discrimination and healthcare access among HIV-positive immigrants in Spain. BMC Publ Health 2021; 21: 385.

34.

Ndumbi

Del Romero

Pulido

, et al. Barriers to health care services for migrants living with HIV in Spain. Eur J Publ Health 2018; 28: 451–457.

35.

Ndiaye

Ould-Kaci

Salleron

, et al. Incidence rate and risk factors for loss to follow-up in HIV-infected patients from five French clinical centres in Northern France - January 1997 to December 2006. Antivir Ther 2009; 14: 567–575.

36.

Seong

Choi

Kim

, et al. Rate of and risk factors for loss to follow up in HIV-infected patients in Korea: The Korea HIV/AIDS cohort study. Infect Chemother. 2023; 55(1): 69–79. doi:10.3947/ic.2022.0059.

37.

Mushy

Mtisi

Mboggo

, et al. Predictors of the observed high prevalence of loss to follow-up in ART-experienced adult PLHIV: A retrospective longitudinal cohort study in the Tanga Region, Tanzania. BMC Infect Dis 2023; 23: 92.

38.

Gari

Doig-Acuña

Smail

, et al. Access to HIV/AIDS care: A systematic review of socio-cultural determinants in low and high income countries. BMC Health Serv Res 2013; 13: 198.

39.

Frijters

Hermans

Wensing

AMJ

, et al. Risk factors for loss to follow-up from antiretroviral therapy programmes in low-income and middle-income countries. AIDS Lond Engl 2020; 34: 1261–1288.

40.

Mberi

Kuonza

Dube

, et al. Determinants of loss to follow-up in patients on antiretroviral treatment, South Africa, 2004–012: A cohort study. BMC Health Serv Res 2015; 15: 259.

41.

Orrell

Kaplan

Wood

, et al. Virological breakthrough: A risk factor for loss to follow-up in a large community-based cohort on antiretroviral therapy. AIDS Res Treat 2011; 2011: 469127.

42.

Agolory

Auld

Odafe

, et al. High rates of loss to follow-up during the first year of pre-antiretroviral therapy for HIV patients at sites providing pre-ART care in Nigeria, 2004–2012. PLoS One 2017; 12: e0183823.

43.

Jamieson

Rosen

Phiri

, et al. How soon should patients be eligible for differentiated service delivery models for antiretroviral treatment? Evidence from a retrospective cohort study in Zambia. BMJ Open 2022; 12: e064070.

44.

Arici

Ripamonti

Maggiolo

, et al. Factors associated with the failure of HIV-positive persons to return for scheduled medical visits. HIV Clin Trials 2002; 3: 52–57.

45.

Engelhard

EAN

Smit

Van Sighem

, et al. Impact of HIV care facility characteristics on the cascade of care in HIV-infected patients in The Netherlands. AIDS Lond Engl 2016; 30: 301–310.

46.

Izquierdo

Rava

Moreno-García

, et al. HIV medical care interruption among people living with HIV in Spain, 2004–2020. AIDS Lond Engl 2023; 37: 1277–1284.

47.

Pence

Bengtson

Boswell

, et al. Who will show? Predicting missed visits among patients in routine HIV primary care in the United States. AIDS Behav 2019; 23: 418–426.

48.

Palacio-Vieira

Moreno-Fornés

Díaz

, et al. Who is lost to follow-up in HIV care? Assessment of care retention over time and the impact of COVID-19. Longitudinal analysis of the PISCIS cohort. HIV Med. 2023; 24(9): 965–978. doi:10.1111/hiv.13486.