Early Warning Indicators for First-Line Virologic Failure Independent of Adherence Measures in a South African Urban Clinic

Abstract

We sought to develop individual-level Early Warning Indicators (EWI) of virologic failure (VF) for clinicians to use during routine care complementing WHO population-level EWI. A case-control study was conducted at a Durban clinic. Patients after≥5 months of first-line antiretroviral therapy (ART) were defined as cases if they had VF [HIV-1 viral load (VL)>1000 copies/mL] and controls (2:1) if they had VL≤1000 copies/mL. Pharmacy refills and pill counts were used as adherence measures. Participants responded to a questionnaire including validated psychosocial and symptom scales. Data were also collected from the medical record. Multivariable logistic regression models of VF included factors associated with VF (p<0.05) in univariable analyses. We enrolled 158 cases and 300 controls. In the final multivariable model, male gender, not having an active religious faith, practicing unsafe sex, having a family member with HIV, not being pleased with the clinic experience, symptoms of depression, fatigue, or rash, low CD4 counts, family recommending HIV care, and using a TV/radio as ART reminders (compared to mobile phones) were associated with VF independent of adherence measures. In this setting, we identified several key individual-level EWI associated with VF including novel psychosocial factors independent of adherence measures.

Introduction

Virologic response is usually the earliest indication of antiretroviral therapy (ART) uptake, adherence, and overall health for individuals living with HIV in well-resourced settings. Virologic failure (VF) may indicate complete nonadherence to ART or can represent suboptimal adherence, eventually resulting in HIV drug resistance (HIVDR), which can compromise future ART options. South Africa has made ART available for a large number of individuals with HIV. Many urban clinical programs within South Africa have reported low rates of VF despite a high volume of patients.¹ However, as clinics in high-prevalence areas continue to scale-up ART delivery, a growing proportion of individuals with HIV will either be asymptomatic or harder to retain in care. These populations will inevitably challenge even the most effective programs to maintain low levels of VF.

The factors that contribute to VF are complex and interact at multiple levels.^2,3 The World Health Organization (WHO) developed a set of early warning indicators (EWI) designed to identify programs and regions where HIVDR may be of great concern.⁴ These EWI include retention on first-line ART, on-time drug pickup at the pharmacy and clinic appointment keeping, and viral load (VL) suppression 12 months after ART initiation. Several countries are using EWI in order to focus efforts towards improving healthcare delivery in those settings with suboptimal EWI scores.^4
–6 Although these “system-level” factors are useful at a programmatic and regional level, clinicians need “individual-level” factors that could help identify and predict which patients may be at risk of VF while on ART or prior to initiation. While many studies have explored psychosocial, structural, or clinical factors associated with clinic retention, ART adherence or VF, none have attempted a comprehensive assessment in this setting.^7
–9

The Risk Factors for Virological Failure (RFVF) study was undertaken to ascertain individual-level factors associated with VF on ART in order to develop EWI for clinicians. For programs that lack resources for systematic VL monitoring, these risk factors can also identify those individuals with potential VF for more targeted use of VL monitoring.

Methods

Clinical setting

The RFVF study was conducted at McCord Hospital (MCH) in Durban, South Africa, which is a regional referral center that has been treating patients with ART since 2002. MCH received partial support from the President's Emergency Plan for AIDS Relief and South African government funding for ART which began in February 2004. Patients pay a monthly fee ($15 USD) for clinic services. Routine viral load (VL) monitoring occurs 5 months after starting ART. All patients receive initial ART education. Pharmacy refills and pill counts are recorded for each patient in the clinic. If the VL was≤1000 copies/mL (cpm), patients are maintained on this regimen and followed with annual VL monitoring thereafter. If the VL was>1000 cpm, a repeat VL is done 1–3 months later with concurrent adherence counseling. If the VL remains>1000 cpm, regimen changes are considered based upon the level of adherence and resistance testing which is performed on all cases.

Study participants

From October 2010 through June 2012, all individuals with HIV attending the MCH HIV clinic≥18 years who were receiving≥5 months of their first ART regimen (substitutions allowed for toxicity) were offered participation in this study if they met the criteria for a case or control.

Study design

An unmatched case-control design was chosen for this study because the rate of VF was too low to justify a prospective cohort study and the intention was to allow for investigation of all potential risk factors. Cases were defined as patients having an initial VL>1000 cpm after≥5 months of their first ART regimen. Controls (2:1) were defined as patients with virologic suppression (VL≤1000 cpm) on≥5 months of their first ART regimen. In general, cases were identified as having VF within 1–2 weeks of a visit to the clinic (which corresponded to a pharmacy refill). These patients were notified and enrolled into the study if they agreed to participate within 1–2 weeks from that date. Their enrollment date was therefore 2–3 weeks from the most recent refill. Controls were randomly selected patients in the clinic who met the eligibility criteria and agreed to participate. Their date of enrollment corresponded to a pharmacy refill date.

Data collection

All participants who provided consent and were enrolled into the study underwent a single, semi-structured interview in their preferred language with the research coordinator who was blinded to the participant's case-control status. This interview consisted of a questionnaire, validated neurocognitive assessment,¹⁰ the Kessler 10 (K-10) depression scale,¹¹ and an unannounced pill count. The questionnaire contained demographic, socioeconomic (including a wealth index,^12,13 employment, education, and cohabitants), psychosocial (including substance abuse,¹⁴ food insecurity,^15
–17 traditional African medicine use, safe sex practices, faith, stigma,¹⁸ and intimate partner violence¹⁹) and clinic satisfaction indices.²⁰ There were also specific questions about ART adherence and clinic attendance based upon a modified AIDS Clinical Trials Group (ACTG) adherence questionnaire.²¹ The study physician met with each patient to review their medical history, as well as to administer the symptom screen and Karnofsky score.²² Clinical information,¹ pharmacy refill dates/quantities, and laboratory data were abstracted from medical records and entered onto a case report form (CRF). Further details of the data collected can be found in Table 1. Study data were managed using REDCap electronic data capture tools hosted at Emory University.²³

Table 1.

Quantitative Measures That Were Adapted or Modified to Accommodate Local Cultural and Language Context and Needs of the Study

Domain	Measure	References
Demographic	Age, gender, race/ethnicity, education level, sensory impairment assessment	ACTG Adherence²¹
Socioeconomic	Income, employment status, occupation, housing, transportation to clinic, payer source for healthcare Assets Food insecurity^15,16	ACTG Adherence²¹ Wealth Index^12,13 Modified HFIAS index¹⁷
Psychosocial	Marital status, family and partner information, HIV disclosure, safe sex assessment, religious faith assessment, traditional and alternative health practice assessment Substance abuse assessment Intimate partner violence assessment, Depression scale Clinic satisfaction survey Stigma assessment¹⁸	ACTG Adherence²¹ and CAGE¹⁴ DHS Domestic Violence Module¹⁹ Kessler 10¹¹ Dahab et al.²⁰
Symptoms and exam	Symptom assessment Functional status Neurocognitive testing	ACTG Adherence²¹ Karnofsky Performance Status²² Digit Span Forward/Backward, Trail-Making Test A/B¹⁰
Medical history	AIDS-defining conditions¹, serious non-AIDS-defining conditions, HIV-1 RNA viral loads, CD4 T cell counts	Not applicable
Medications	ARV regimens (current and previous), ART initiation site and referral, ART education, Adherence sessions, self-reported Adherence questionnaire, ART refill and ARV reminders, concomitant medications	ACTG Adherence²¹
Access	Pharmacy refill dates and dispensed amounts over the preceding 180 days	MPR²⁴
Adherence	Pill counts at enrolment visit	Pill Count Adherence^25,26

ACTG, AIDS Clinical Trials Group; ART, Antiretroviral Therapy; ARV, Antiretroviral; DHS, Demographic and Health Survey; HFIAS, Household Food Insecurity Assessment Scale; MPR, Medication Possession Ratio.

Statistical analysis

Two explanatory covariates were derived from the primary data collected. The “Access” covariate incorporated antiretroviral (ARV) refill dates and quantity dispensed. This formula was based on the medication possession ratio (MPR)²⁴ accounting for all refills occurring in the 180 days following the earliest refill date until enrollment. The “Adherence” covariate utilized the enrollment date pill count and incorporated the dispensed pills over the previous 180 days.^25,26 The primary outcome assessed in this study was the dichotomized participant assignment as a case or control. Separate sensitivity analyses used a VL threshold of >50 cpm. All variables from the questionnaire and CRF, as well as explanatory covariates, were independently analyzed for their association with the primary outcome in univariable analyses. Although all variables were examined, only significant (p<0.05) and epidemiologically important factors were further analyzed. Individual analyses by domain were undertaken to identify appropriate variable categories, correlations, and interactions between variables and ascertain which variables have the highest likelihood of success in multivariable models. Several logistic regression multivariable models were constructed using model selection to arrive at each final model. Model 1 (baseline factors) attempted to identify the factors present at the initiation of ART most associated with the primary outcome. Model 2 included all time-updated variables except for the Access or Adherence measures. Model 3 included those socioeconomic and psychosocial variables from Model 2 that were likely to be correlated with the Access measure. Model 4 included those psychosocial, symptom, and clinical variables from Model 2 that were likely to be correlated with the Adherence measure. Model 5 was considered the full model including all time-updated variables and the Access and Adherence measures. Receiver operator characteristic (ROC) curves were constructed for each model. Subgroup analyses were performed to assess variables associated with VF among those individuals having only 12 months of first-line ART. All statistical analyses were performed in SAS (SAS Institute, Version 9.3, Cary, NC).

Ethics

The RFVF study was approved by the ethics committee at McCord Hospital and by the institutional review board at Emory University in Atlanta, Georgia.

Results

Cohort description

Only the significant and otherwise important characteristics of study participants are shown in Table 2. Overall, the mean age was 39.6 years old and 64.6% were women. Over 78% received an income and 19.0% were unemployed. Sixty-five percent reported having a current partner, and 41.5% reported disclosing their status to a partner. The mean (SD) K-10 score was 12.8 (3.4) with 55.0% scoring >12. The most commonly reported symptoms were depression (34.5%), a rash (32.5%), and fatigue (32.1%). Nearly half (43.9%) felt their symptoms were related to ARVs. Only 34.7% of participants had no obvious neurocognitive impairment, while 35.4% had evidence of HIV-associated dementia. Lipodystrophy was documented in 29.5% of participants. The median (IQR) CD4 count was 300.5 cell/μL (183.5–448.0), and tuberculosis was the most frequent opportunistic infection (54.8%). The mean (SD) duration of ART was 30.2 months (24.3). The vast majority of participants (88.9%) used their mobile phone to remind them to take ARVs or visit the clinic. The median (IQR) MPR was 1.03 (0.96–1.07), and median (IQR) pill count adherence ratio was 1.12 (1.05–1.17). Cases and controls differed substantially across all of the domains examined using univariable comparisons.

Table 2.

Selected Cohort Characteristics for Cases and Controls

Domain/characteristic	Control N=300	Case N=158	p Value
Demographic
Age at enrollment (mean)	40.9	37.1	<0.0001
Gender (%female)	71.0	52.5	0.0001
Ethnicity (%black)	98.7	99.4	0.66
Socioeconomic
Education (mean years)	10.2	11.0	0.0093
Income (%yes)	82.0	72.2	0.017
Employment (%UE)	16.0	24.7	0.033
Housing (%rent/own)	52.7	45.6	0.17
Transportation (%personal vehicle)	9.7	19.6	0.0035
Wealth Index 1 (mean)	−0.1	0.3	0.83
Wealth Index 2 (mean)	0.1	−0.2	0.053
Payer source for ARVs (%family/spouse)	15.7	25.3	0.017
Psychosocial
Religious faith (%yes)	91.3	82.9	0.009
Religious activity (%no religion/not active)	41.7	60.8	0.0001
TM (%ever took)	58.3	56.3	0.69
Have a current partner (%yes)	60.7	73.4	0.0073
Practiced safe sex in past 6 months (%always)	95.0	84.2	0.0002
Safe sex practice (%used condoms)	56.0	69.4	0.0064
Family member HIV status (%positive)	38.0	50.0	0.017
Have an ART supporter (%yes)	11.3	22.2	0.0036
Clinic experience (%pleased)	88.0	65.2	<0.0001
Traditional K-10 score (mean)	12.3	13.7	<0.0001
Traditional K-10 score (%12+)	48.3	67.7	<0.0001
Tired question from K-10 (%yes)	39.7	53.2	0.0075
Symptoms and exam
Fever, chills or sweats (%no)	83.7	74.7	0.025
Fatigue (%no)	76.0	52.5	<0.0001
Memory difficulty (%no)	84.3	72.2	0.0029
Nausea or vomiting (%no)	94.3	86.1	0.0043
Diarrhea (%no)	91.7	82.3	0.0052
Felt sad or depressed (%no)	71.3	54.4	0.0004
Felt nervous or anxious (%no)	82.0	68.4	0.0014
Rash (%no)	74.7	53.8	<0.0001
Headache (%no)	75.7	64.6	0.016
Sexual dysfunction (%no)	83.7	69.0	0.0005
Weight loss or wasting (%no)	86.7	78.5	0.032
Hair loss (%no)	95.3	89.9	0.029
Any symptom felt related to ARVs (%yes)	39.0	52.6	0.0078
Karnofsky score (mean)	97.7	95.7	0.026
Neurocognitive assessment
None	35.3	33.5	0.89
ANI/MND	30.0	29.7
HAD	34.7	36.7
Medical history and laboratory values
Tuberculosis (%yes)	54.7	55.1	1.00
Lipodystrophy (%yes)	37.0	15.2	<0.0001
Recent CD4 count in cells/μL (median)	359.0	206.0	<0.0001
Recent CD4 count (%≥350 cells/μL)	52.0	22.8	<0.0001
Recent HIV RNA VL copies/mL for cases (median)	—	95,221
Medications
Mean ART duration (months)	33.0	24.7	<0.0001
Initiating ARV clinic (%Sinikithemba)	92.7	84.8	0.013
Recommended ART
Doctor or nurse	42.7	44.9	0.0068
Family	19.3	31.0
Friend	16.7	10.1
Other	21.3	13.9
Current ART regimen contains
Stavudine (d4T)	17.3	27.8	0.0077
Zidovudine (ZDV)	24.7	15.2
Other (tenofovir, didanosine, abacavir)	58.0	57.0
HIV education and training sessions (%3+)	97.3	91.1	0.0050
Adherence counseling sessions
0 or 1 session	6.0	17.8	<0.0001
2, 3, or 4 sessions	78.0	57.3
5+ sessions	16.0	24.8
Mechanism to remember to take ARVs
Mobile phone (%yes)	91.0	84.8	0.060
TV or radio (%yes)	5.3	13.9	0.0022
Clock or watch alarm (%yes)	10.3	12.7	0.44
Other (%yes)	4.7	9.5	0.067
Fluconazole use in the past 6 months (%yes)	1.0	8.9	<0.0001
TS use in the past 6 months (%yes)	44.7	63.9	0.0001
INH or RIF use in the past 6 months (%yes)	9.3	21.5	0.0005
ETB use in the past 6 months (%yes)	1.3	5.7	0.014
Access MPR
Median	1.03	1.00	0.83
≥Overall median (%)	49.3	42.6	0.17
≥90% (%)	89.3	83.5	0.10
≥80% (%)	96.7	94.3	0.23
≥70% (%)	99.0	94.9	0.010
Highest overall quartile (%)	11.0	20.7	0.2665
Upper middle overall quartile (%)	38.3	21.9
Lower middle overall quartile (%)	25.3	34.2
Lowest overall quartile (%)	25.3	23.2
Adherence pill count ratio
Median	1.13	1.08	<0.0001
≥Overall median (%)	56.3	41.1	0.0033
≥90% (%)	92.7	85.4	0.020
≥80% (%)	94.7	88.0	0.015
≥70% (%)	96.0	88.6	0.0045
Highest quartile (%)	29.9	11.4	0.0008
Upper middle quartile (%)	26.4	29.8
Lower middle quartile (%)	24.0	23.4
Lowest quartile (%)	19.8	35.5

ANI, asymptomatic neurocognitive impairment; ART, ARV treatment; ARVs, antiretrovirals; ETB, ethambutol; HAD, HIV-associated dementia; INH, isoniazid; K-10, Kessler 10 depression scale; MND, mild neurocognitive disorder; MPR, medication possession ratio; RIF, rifampicin; TM, any form of traditional (African, Chinese, Indian) or alternative/complimentary medicine; TS, trimethoprim-sulfamethoxazole; UE, unemployed (seeking work or not seeking work); VL, viral load.

Baseline risk factors (Model 1)

Younger age and male gender were associated with VF in nearly all MV models (Table 3). Additional risk factors for VF that were likely present at ART initiation included having a treatment supporter (OR 1.991), not having or not being active with a religious faith (1.634), having at least one family member living with HIV (1.613), having had a family member recommend the individual to seek ART, use of stavudine (d4T) in the current ART regimen, and use of fluconazole (4.973). The ROC area under the curve (AUC) for baseline risk factors was 0.7824 (Fig. 1A).

FIG. 1.

Receiver operator characteristic curves for multivariable models 1–5 (A–E).

Table 3.

Multivariable Analyses of Risk Factors for Virologic Failure Using Logistic Regression

Domain/Risk Factor	Model 1	Model 2	Model 3^e	Model 4	Model 5
Demographic
Age (per 5 year increase)	0.956^c	0.837^a	0.865^a	0.807^b	0.860
Gender (male)	1.995^c	2.262^c	2.524^d	2.682^c	2.416^c
Socioeconomic				—
Education (per 1 year)		1.112	1.111		1.108
Transportation (personal)	1.771^a	1.789	1.979^a		2.034
Pay for care (family/spouse)	1.517		1.631
Psychosocial
Faith activity (none)	1.634^b	1.722^a	1.533^a	1.854^b	1.802^b
Practice safe sex (<always)	—	5.500^d	3.108^c	5.905^d	5.023^c
Family HIV+ (none)	0.620^b	0.593^a	0.665^a	0.588^a	0.500^b
Treatment supporter (yes)	1.991^b	1.910	1.807^a	1.699	1.783
Clinic feel pleased (yes)	—	0.448^b	0.387^d	0.517^b	0.509^b
Depression (12+)	—	3.136^d	3.064^d	2.689^c	3.021^c
Symptoms and exam	—		—
Fatigue		2.532^c		2.471^c	2.470^c
Diarrhea		2.555^b		2.026	2.079
Sadness				1.401	1.409
Skin lesions		1.720^a		2.011^b	1.992^b
Medical history			—
Lipodystrophy (yes)	—	0.428^b		0.611	0.608
Log CD4 (per 1.0 increase)	—	0.079^d		0.078^d	0.078^d
Medications			—
ARV duration (per 1 month)	0.995	1.001		1.007	1.008
Recommend HIV clinic
Friend vs family	0.424^b	0.311^b		0.279^b	0.266^b
Other vs family	0.446^b	0.376^b		0.350^b	0.397^b
Provider vs family	0.879^b	0.760^b		0.806^b	0.855^b
First clinic (Sinikithemba)	0.503^a	0.440^a		0.487
ARV training sessions (3+)	0.350^a
Adherence counseling	—
2–4 vs 0–1		0.370		0.383	0.378
5+ vs 0–1		0.416		0.470	0.419
Current regimen
ZDV vs d4T	0.619^b	0.649^a		0.699	0.691^a
Other vs d4T	0.489^b	0.455^a		0.484	0.435^a
Recall ARVs (TV/radio)	—	3.519^c		3.363^b	3.681^c
Trimethoprim/sulfa (yes)	1.625^a	0.624
Fluconazole (yes)	4.973^b	2.636			3.006
Ethambutol (yes)	2.729	2.800		3.606	3.025
Access (0.1)	—	—	0.962	—
Adherence (0.1)	—	—	—	0.753^b	0.763^b

Odds ratios presented; ^a p<0.10, ^b p<0.05, ^c p<0.01, ^d p<0.001; ^eAdjusted for ARV duration which was significant; ARV, antiretroviral; Model 1, baseline variables (excluding any time updated variables); Model 2, full model without access or adherence variables; Model 3, socioeconomic and psychosocial adjusted for access (forced); Model 4, psychosocial, symptoms, clinical events and meds adjusted for adherence; Model 5, full model with Access and Adherence forced; — variable or domain was excluded a priori.

Overall risk factors excluding Access or Adherence measures (Model 2)

In a full MV model that included all domains except Access and Adherence, the factors that were associated with VF at the time of study enrollment included less than always practicing safe sex (5.500), not being pleased with their clinic experience (2.232), K-10 score >12 (3.136), symptoms of fatigue (2.532) and diarrhea (2.555), the absence of lipodystrophy (2.366), having lower recent CD4 counts (12.658), having a family member recommend the individual to seek ART, and having used a TV or radio as a reminder to take their ARVs (3.519) instead of a mobile phone. The ROC AUC was 0.8867 (Fig. 1B).

Access-related risk factors (Model 3)

In a MV model that included the Access measure, the following remained significant after adjusting for the Access variable: practiced unsafe sex (3.108), K-10 >12 (3.064), and not being pleased with their clinic experience (2.584). The ROC AUC was 0.7952 (Fig. 1C).

Adherence-related risk factors (Model 4)

Factors known to influence Adherence that remained significant after adjusting for the Adherence variable included not having or not being active with a religious faith (1.854), not being pleased with their clinic experience (1.934), practiced unsafe sex (5.905), K-10 >12 (2.689), symptoms of fatigue (2.471) or rash (2.011), low recent CD4 count (12.821), family member recommended the patient to seek ART, the use of d4T in the current regimen, and using a TV/radio as a reminder for ARVs (3.363). The Adherence measure was also significant (1.328). The ROC AUC was 0.8851 (Fig. 1D).

Overall risk factors including Access and Adherence measures (Model 5)

After adjusting for both the Access and Adherence variables, the following remained associated with VF: practiced unsafe sex (5.023), having at least one family member living with HIV (2.000), not being pleased with their clinic experience (1.965), K-10 >12 (3.021), symptoms of fatigue (2.470) or rash (1.992), low recent CD4 count (12.821), family member recommended the patient to seek ART, and used a TV/radio as a reminder for ARVs (3.681). Again, Adherence was significant (1.311), while Access had to be removed due to high collinearity with Adherence. The ROC AUC was 0.8881 (Fig. 1E).

Sensitivity and subgroup analyses

The following sensitivity and subgroup analyses (data not shown) are described in terms of how they differed from the whole cohort analyses. When the outcome was changed to a VL threshold of>50 cpm, there were 265 controls and 193 cases (35 participants, 7.64%, were reclassified). Only six participants had a VL between 200 and 1000 cpm. For model 1, ART duration and ethambutol (ETB) use were significantly associated with VF. Models 2 and 3 did not differ from the whole cohort models respectively. In model 4, diarrhea, the current ART regimen, and fluconazole use were significantly associated with VF, whereas Adherence was not significant. In model 5, age, diarrhea, and fluconazole use were significant, whereas Adherence was not. When only participants with 12 months of ART were included in the analyses, model 1 was largely unchanged. For model 2, being recommended by a family member to receive HIV treatment, the absence of lipodystrophy, and symptoms of fatigue and diarrhea were no longer significantly associated with VF. In model 3, being employed was associated with VF. In model 4, the first clinic ART the participant received ART and the current regimen were associated with VF whereas fatigue and family recommendation for ART were not. Finally in model 5, fatigue and family recommendation for ART were not associated with VF.

Discussion

The RFVF study sought to define the individual-level determinants for VF which could be used in this setting to identify patients at risk and what their specific barriers are prior to ART initiation and while on treatment (Table 4). This would enable targeted approaches for interventions and could serve as surrogate measures for VF in settings where VL monitoring is not available. Most of the determinants found were consistent across a large number of models and subgroup analyses. Key demographic, socioeconomic, psychosocial, and clinical elements were associated with VF independent of adherence measures.

Table 4.

Key Risk Factors

Baseline (while initiating or suppressed on ART)	On ART without Access/Adherence measures ^a	On ART with Access/Adherence measures ^a
Age	Depression	Depression
Gender	Unsafe sex practices	Unsafe sex practices
Faith	Clinic experience	Clinic experience
Family member HIV+	Fatigue	Fatigue
Treatment supporter	Diarrhea	Rash
Clinic recommendation	Lipodystrophy	Current CD4 count
Current regimen	Current CD4 count	ARV reminders
Fluconazole use	ARV reminders	Adherence

These factors do not include those that were identified as baseline risk factors.

Within the Demographic domain, younger age was a predictor of VF at baseline and had a trend towards being associated with VF when examining all variables but was not independent of Access or Adherence. This indicated that age impacts VF largely through obtaining and taking ARVs. Poor adherence and higher rates of VF for younger individuals have been described in many different settings.^27

–33 Additionally, male gender was associated with VF at baseline and in all other models confirming findings from previous studies. Not always dependent on adherence, these studies have shown men have poor health-seeking behaviors, higher baseline VL, lower ARV concentrations, late presentation with advanced disease, and inadequate retention in care, although women tend to have more ARV-related adverse events.^34

–39 From the Socioeconomic domain, relying on one's own vehicle for transportation to clinic had a trend towards an association with VF and appeared to be independent of the Access variable, which was unexpected based on previous reports.^9,40 It is likely that this variable is highly correlated with another significant factor such as male gender.

From the Psychosocial domain there were several novel variables identified. Symptoms consistent with depression and fatigue, as well as practicing unsafe sex, were markedly associated with VF in all models where those variables were examined independent of Access and Adherence. Depression has been linked to poor adherence but is also associated with other factors that could be independently associated with VF such as alcohol abuse, which was infrequent in this study.^41
–43 Unsafe sex may be a marker for behaviors leading to VF such as neglecting other health-related advice given during ART training and adherence counseling. It could also be more directly linked with VF through HIV superinfection or sexually transmitted infections that could cause an increase in VL.⁴⁴ Not being active with a religious faith and having at least one family member with HIV were associated with VF in all models but was not highly significant. Religious faith has been shown to improve adherence, but some studies have shown that certain practices could encourage prayer in lieu of ART.^45
–47 In contrast to other studies, stigma and traditional African medicine use was not specifically identified as a risk factor for VF.^9,48,49 Although having a family member with HIV could promote mutual support, if the family member is ill, time, attention and ARVs may be diverted away from the participant and to that family member.⁵⁰ Individuals who were pleased with their clinic experience were more likely to be controls in all models as has been shown in other studies that described the influence of the healthcare environment on clinic attendance and adherence.^{9,40,51

–55}.

Several clinical factors were associated with VF. Low CD4 count at start of ART was highly associated with VF in all models and independent of Access or Adherence, confirming findings from previous studies.⁵⁶ It is important to note that, although a low CD4 count is highly associated with VF, it may be a consequence rather than a cause of VF. The absence of lipodystrophy was found to be a risk factor for VF but was not significant after controlling for adherence. This likely represents a strong correlation between ART adherence and fat redistribution syndrome, contrary to a previous report from Rwanda.⁵⁷ The use of d4T was also associated with VF at baseline when compared to TDF (most commonly used), ZDV, ABC, and ddI. The ART regimen only had a trend towards significance when more variables were included in the model. Side effects and toxicity-related treatment discontinuation may be more frequent for d4T compared to ZDV-containing ART.^58,59 Despite this finding, rates of overall virologic suppression were not statistically inferior when comparing d4T+3TC and ZDV+ddI with non-nucleoside reverse transcriptase inhibitor (NNRTI)⁵⁹ or indinavir-containing ART⁶⁰ in previous studies. TDF-containing regimens have been shown to be better tolerated and have fewer side effects than d4T-containing ART but comparably effective.^61,62 However, amongst participants failing a TDF-containing regimen, we and others reported a high rate of the K65R mutation (>65%⁶³ and 46%,⁶⁴ respectively). This finding is under surveillance to assess if this is a unique association in subtype C.

If a family member recommended the individual for ART, they were more likely to be a case in all models and independent of Access or Adherence. It is not entirely clear how family member referral could impact treatment outcomes. It is possible that family-driven stigma could be playing a role, but it is more likely that this represents a lack of connection to primary care services or access to ART such as is common for men in this setting.⁶⁵ This may alternatively reflect the fact that this is a proxy for more symptomatic disease at baseline prompting immediate family referral and subsequent association with poor outcomes. Finally, use of a television or radio to remind individuals to take their ARVs was suboptimal compared to using a mobile phone. Mobile phone reminders have been shown in clinical trials to promote adherence when weekly text messaging, daily calls, or alarms are used.^66
–68

Access and Adherence measures were highly correlated, ρ=0.68 (p<0.0001) (Fig. 2). Although poor Access (measured by the MPR) was significant in a UV comparison (≥70%), it was not significant in any of the MV models as a continuous measure. The MPR does identify major interruptions in ART, which is ideal for NNRTI-based therapy and was effective in this setting for identifying second-line VF,⁶⁹ but with the limitation in mind that it does not necessarily represent the proportion of days covered, which is a more accurate measure of adherence. Suboptimal Adherence (measured by pill count), however, was significantly associated with VF in all models where it was examined.

FIG. 2.

Correlation between Access and Adherence variables for cases and controls. (Color for this image can be found at www.liebertpub.com/apc.)

When a VL >50 cpm was used to define cases, only 35 participants were reclassified, indicating that the lower threshold identified only a small percentage of additional cases. A recent modeling study showed greater cost-effectiveness when using the higher VL threshold in this setting.⁷⁰ Using the lower cutoff, diarrhea and fluconazole use were associated with VF. While diarrhea could lead to diminished absorption, fluconazole increases ARV concentrations. Although this could lead to increased side effects and reduced ARV adherence, this association seems less plausible. When restricting the analyses to individuals with only 12 months of ART, fatigue and family members recommending ART were no longer associated with VF.

Ultimately these factors interact in highly complex ways in determining an individual's virologic response to ART (Fig. 3). Pill adherence is a necessary final step required for VL suppression (assuming optimal pharmacokinetics) and can be a reasonable surrogate for actual pills ingested, although pill dumping has been described in this setting,⁷¹ as well as poor counting performance by clinical staff. Access to clinic refills as measured by the MPR is not always a reliable indicator of adherence, as it is not necessarily indicating ingested pills and individuals can obtain ARVs through other unmeasured sources. Although socioeconomic barriers have been well-described as leading to treatment interruptions, missed visits, and poor adherence,^8,72,73 these factors appear to be overshadowed by psychosocial and clinical factors in our study, reinforcing the importance of a comprehensive approach to assessing determinants of health. Socioeconomic and institutional barriers can often be more readily addressed with discrete interventions (transportation and food assistance or improving the clinic environment),⁹ whereas psychosocial barriers can be more challenging to address.

FIG. 3.

Schema of social, behavioral and clinical factors related to virologic response for individuals receiving antiretroviral therapy (ART). Pill adherence is necessary for viral load (VL) suppression; however, access to ART is neither necessary nor sufficient. Individuals can obtain ART from family or friends and may not swallow pills despite obtaining them. Socioeconomic factors (i.e., transportation to clinic) and co-morbid illnesses have a more direct effect on ART access. Likewise, concomitant medications and psychosocial factors (i.e., stigma, disclosure, and depression) are more directly associated with ART adherence but also impact ART access. Institutional (healthcare, religious, governmental), community (neighborhood, dyadic) and societal (cultural, infrastructure, policy) factors have more global impact at all lower levels in this paradigm. (Color for this image can be found at www.liebertpub.com/apc.)

There were limitations to the current study. The RFVF study used a case-control design and as such, recall bias is a possibility since many of the variables obtained from the questionnaire relied upon a participant's recollection of events that had occurred over the past 6 months. Moreover, causality has not been established through this analysis, and in fact, reverse causation may even exist. Despite this suggestion, we have identified surrogate markers that can be used by large programs to focus their counseling and monitoring efforts on those individuals at risk before VF or HIVDR occurs. Although most cases were told of their virologic status within a week or two prior to their interview, this could have impacted some of the responses about overall well-being. While we did not assess for HIVDR prior to ART initiation, which could be a risk factor for VF within the first 6 months of ART initiation, the rates of transmitted HIVDR have been reportedly low to non-existent in KwaZulu-Natal.⁷⁴ Furthermore, baseline HIV genotyping is not routinely available in South Africa and could therefore not be integrated into clinical algorithms at this time. Finally, only one clinical site was examined, which does not permit comparisons across diverse programs or geographic settings (rural or peri-urban).

In summary, the RFVF study provided real-world indicators which could be used to identify patients—either at the time of starting ART and or while receiving ART—for VF, an event with important long-term consequences. This would enable programs to tailor specific interventions for individuals with the intention to reduce the likelihood of VF and HIVDR. Furthermore, these determinants could be used in settings that do not have resources for routine VL monitoring to assist clinicians in making decisions about more targeted use of VL testing. It is important to validate this questionnaire in other urban as well as rural settings. Eventually a more refined questionnaire could be tested prospectively in order to develop a risk score.

Footnotes

Acknowledgments

We would like to express our deepest admiration and appreciation for the patients who participated in the study and the work of the Sinikithemba Clinic at McCord Hospital in Durban, South Africa for their commitment to improve patient care and support research. The tremendous contributions on the part of the counselors, medical records staff, nurses, and medical officers have been essential to the success of this study. Sabelo Dladla, Roma Maharaj, Kristy Nixon, Melisha Pertab, Sifiso Shange, and John Klopfer provided vital assistance for the data collection and analysis. A special thanks to Daniel and Alessandra Marconi for forbearance.

Author Disclosure Statement

D.R.K. is a consultant to, or has received research funding from Abbott, Boehringer-Ingelheim, Bristol-Myers Squibb, Gilead, GlaxoSmithKline, Merck, and Roche. For the remaining authors, none were declared.

Funding: Grant support from Emory University Center for AIDS Research (CFAR) (P30 AI050409) and the Emory School of Medicine Division of Infectious Diseases, NIH (P30 AI60354 to Harvard University CFAR and K24 RR16482 to D.R.K.), Harvard University Program on AIDS, CDC Cooperative Agreement (U62/CCU123541-01), Elizabeth Glaser Pediatric AIDS Foundation as part of Project HEART, Research and Health Sciences IT Division (UL1RR025008), and the Gilead Foundation.

References

Marconi

, Sunpath

, Lu

, et al. Prevalence of HIV-1 drug resistance after failure of a first highly active antiretroviral therapy regimen in KwaZulu Natal, South Africa. Clin Infect Dis, 2008; 46:1589–1597.

Hong

, Nachega

, Kelley

, Bertagnolio

, Marconi

, Jordan

. The global status of HIV drug resistance: Clinical and public-health approaches for detection, treatment and prevention. Infect Disord Drug Targets, 2011; 11:124–133.

Nachega

, Marconi

, van Zyl

, et al. HIV treatment adherence, drug resistance, virologic failure: Evolving concepts. Infect Disord Drug Targets, 2011; 11:167–174.

Bennett

, Jordan

, Bertagnolio

, et al. HIV drug resistance early warning indicators in cohorts of individuals starting antiretroviral therapy between 2004 and 2009: World Health Organization global report from 50 countries. Clin Infect Dis, 2012; 54:S280–S289.

Dzangare

, Gonese

, Mugurungi

, et al. Monitoring of early warning indicators for HIV drug resistance in antiretroviral therapy clinics in Zimbabwe. Clin Infect Dis, 2012; 54:S313–316.

Hong

, Jonas

, Dumeni

, et al. Population-based monitoring of HIV drug resistance in Namibia with early warning indicators. J Acquir Immune Defic Syndr, 2010; 55:27–31.

Fox

, Cutsem

, Giddy

, et al. Rates and predictors of failure of first-line antiretroviral therapy and switch to second-line ART in South Africa. J Acquir Immune Defic Syndr, 2012; 60:428–437.

Coetzee

, Kagee

. The development of an inventory to assess the structural barriers to clinic attendance and pill-taking amongst users of antiretroviral therapy. AIDS Behav, 2013; 17:319–328.

Dahab

, Kielmann

, Charalambous

, et al. Contrasting reasons for discontinuation of antiretroviral therapy in workplace and public-sector HIV programs in South Africa. AIDS Patient Care STDS, 2011; 25:53–59.

10.

Singh

, Joska

, Goodkin

, et al. Normative scores for a brief neuropsychological battery for the detection of HIV-associated neurocognitive disorder (HAND) among South Africans. BMC Res Notes, 2010; 3:28.

11.

Kessler

, Andrews

, Colpe

, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med, 2002; 32:959–976.

12.

Shin

, Munoz

, Espiritu

, et al. Psychosocial impact of poverty on antiretroviral nonadherence among HIV-TB coinfected patients in Lima, Peru. J Int Assoc Physicians AIDS Care (Chic Ill), 2008; 7:74–81.

13.

Nachega

, Chaisson

, Goliath

, et al. Randomized controlled trial of trained patient-nominated treatment supporters providing partial directly observed antiretroviral therapy. AIDS, 2010; 24:1273–1280.

14.

Ewing

. Detecting alcoholism. The CAGE questionnaire. JAMA, 1984; 252:1905–1907.

15.

Weiser

, Frongillo

, Ragland

, Hogg

, Riley

, Bangsberg

. Food insecurity is associated with incomplete HIV RNA suppression among homeless and marginally housed HIV-infected individuals in San Francisco. J Gen Intern Med, 2009; 24:14–20.

16.

Weiser

, Tuller

, Frongillo

, Senkungu

, Mukiibi

, Bangsberg

. Food insecurity as a barrier to sustained antiretroviral therapy adherence in Uganda. PLoS One, 2010; 5:e10340.

17.

Coates

, Swindale

, Bilinsky

. Household Food Insecurity Access Scale (HFIAS) for Measurement of Household Food Access: Indicator Guide (v. 3). Washington, D.C.: Food and Nutrition Technical Assistance Project, Academy for Educational Development; 2007.

18.

Steward

, Herek

, Ramakrishna

, et al. HIV-related stigma: Adapting a theoretical framework for use in India. Soc Sci Med, 2008; 67:1225–1235.

19.

Kishor

, Johnson

. Profiling Domestic Violence – A Multi-Country Study. Calverton, Maryland: ORC Macro; 2004.

20.

Dahab

, Charalambous

, Karstaedt

, et al. Contrasting predictors of poor antiretroviral therapy outcomes in two South African HIV programmes: A cohort study. BMC Public Health, 2010; 10:430.

21.

Chesney

, Ickovics

, Chambers

, et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: The AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG). AIDS Care, 2000; 12:255–266.

22.

Karnofsky

. Criteria of Performance Status (P. S.). New York: McGraw-Hill; 1954.

23.

Harris

, Taylor

, Thielke

, Payne

, Gonzalez

, Conde

. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform, 2009; 42:377–381.

24.

Leslie

, et al. Calculating medication compliance, adherence, and persistence in administrative pharmacy claims databases. Pharmaceut Program, 2008; 1:13–19.

25.

Lee

, Grace

, Foster

, et al. How should we measure medication adherence in clinical trials and practice?. Ther Clin Risk Manag, 2007; 3:685–690.

26.

Ndubuka

, Ehlers

. Adult patients' adherence to anti-retroviral treatment: A survey correlating pharmacy refill records and pill counts with immunological and virological indices. Int J Nurs Stud, 2011; 48:1323–1329.

27.

Weintrob

, Fieberg

, Agan

, et al. Increasing age at HIV seroconversion from 18 to 40 years is associated with favorable virologic and immunologic responses to HAART. J Acquir Immune Defic Syndr, 2008; 49:40–47.

28.

Paterson

, Swindells

, Mohr

, et al. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Intern Med, 2000; 133:21–30.

29.

Le Moing

, Chene

, Carrieri

, et al. Predictors of virological rebound in HIV-1-infected patients initiating a protease inhibitor-containing regimen. AIDS, 2002; 16:21–29.

30.

Diazgranados

, Silva

, Bermudez

, Roncancio

, Diruggiero

, Mantilla

. Rate and predictors of optimal virologic response to antiretroviral therapy in Colombia. Int J Infect Dis, 2007; 11:531–535.

31.

Greenbaum

, Wilson

, Keruly

, Moore

, Gebo

. Effect of age and HAART regimen on clinical response in an urban cohort of HIV-infected individuals. AIDS, 2008; 22:2331–2339.

32.

Parienti

, Massari

, Descamps

, et al. Predictors of virologic failure and resistance in HIV-infected patients treated with nevirapine- or efavirenz-based antiretroviral therapy. Clin Infect Dis, 2004; 38:1311–1316.

33.

Silverberg

, Leyden

, Horberg

, DeLorenze

, Klein

, Quesenberry

Jr.

Older age and the response to and tolerability of antiretroviral therapy. Arch Intern Med, 2007; 167:684–691.

34.

Anude

, Eze

, Onyegbutulem

, et al. Immuno-virologic outcomes and immuno-virologic discordance among adults alive and on anti-retroviral therapy at 12 months in Nigeria. BMC Infect Dis, 2013; 13:113.

35.

Currier

, Spino

, Grimes

, et al. Differences between women and men in adverse events and CD4+ responses to nucleoside analogue therapy for HIV infection. The Aids Clinical Trials Group 175 Team. J Acquir Immune Defic Syndr, 2000; 24:316–324.

36.

Clark

. Sex differences in antiretroviral therapy-associated intolerance and adverse events. Drug Saf, 2005; 28:1075–1083.

37.

Cornell

, Schomaker

, Garone

, et al. Gender differences in survival among adult patients starting antiretroviral therapy in South Africa: A multicentre cohort study. PLoS Med, 2012; 9:e1001304.

38.

Drain

, Losina

, Parker

, et al. Risk factors for late-stage HIV disease presentation at initial HIV diagnosis in Durban, South Africa. PLoS One, 2013; 8:e55305.

39.

Mills

, Bakanda

, Birungi

, et al. Life expectancy of persons receiving combination antiretroviral therapy in low-income countries: A cohort analysis from Uganda. Ann Intern Med, 2011; 155:209–216.

40.

Ware

, Wyatt

, Geng

, et al. Toward an understanding of disengagement from HIV treatment and care in sub-Saharan Africa: A qualitative study. PLoS Med, 2013; 10:e1001369; discussion e1001369.

41.

Peltzer

, Friend-du Preez

, Ramlagan

, Anderson

. Antiretroviral treatment adherence among HIV patients in KwaZulu-Natal, South Africa. BMC Public Health, 2010; 10:111.

42.

, Phiri

, Bussmann

, Gaolathe

, Marlink

, Wester

. Psychosocial factors affecting medication adherence among HIV-1 infected adults receiving combination antiretroviral therapy (cART) in Botswana. AIDS Res Hum Retroviruses, 2010; 26:685–691.

43.

Wagner

, Goggin

, Remien

, et al. A closer look at depression and its relationship to HIV antiretroviral adherence. Ann Behav Med, 2011; 42:352–360.

44.

Holstad

, Diiorio

, McCarty

. Adherence, sexual risk, and viral load in HIV-infected women prescribed antiretroviral therapy. AIDS Patient Care STDS, 2011; 25:431–438.

45.

Watt

, Maman

, Jacobson

, Laiser

, John

. Missed opportunities for religious organizations to support people living with HIV/AIDS: Findings from Tanzania. AIDS Patient Care STDS, 2009; 23:389–394.

46.

Kisenyi

, Muliira

, Ayebare

. Religiosity and adherence to antiretroviral therapy among patients attending a public hospital-based HIV/AIDS clinic in Uganda. J Relig Health, 2013; 52:307–317.

47.

Zou

, Yamanaka

, John

, Watt

, Ostermann

, Thielman

. Religion and HIV in Tanzania: Influence of religious beliefs on HIV stigma, disclosure, and treatment attitudes. BMC Public Health, 2009; 9:75.

48.

McGuire

, Munyenyembe

, Szumilin

, et al. Vital status of pre-ART and ART patients defaulting from care in rural Malawi. Trop Med Int Health, 2010; 151:55–62.

49.

Nozaki

, Kuriyama

, Manyepa

, Zyambo

, Kakimoto

, Barnighausen

. False beliefs about ART effectiveness, side effects and the consequences of non-retention and non-adherence among ART patients in Livingstone, Zambia. AIDS Behav, 2013; 17:122-1–26.

50.

Mellins

, Brackis-Cott

, Dolezal

, Abrams

. The role of psychosocial and family factors in adherence to antiretroviral treatment in human immunodeficiency virus-infected children. Pediatr Infect Dis J, 2004; 23:1035–1041.

51.

Coetzee

, Kagee

, Vermeulen

. Structural barriers to adherence to antiretroviral therapy in a resource-constrained setting: the perspectives of health care providers. AIDS Care, 2011; 23:146–151.

52.

Hardon

, Akurut

, Comoro

, et al. Hunger, waiting time and transport costs: Time to confront challenges to ART adherence in Africa. AIDS Care, 2007; 19:658–665.

53.

Dahab

, Charalambous

, Hamilton

, et al. “That is why I stopped the ART”: Patients' & providers' perspectives on barriers to and enablers of HIV treatment adherence in a South African workplace programme. BMC Public Health, 2008; 8:63.

54.

Roura

, Busza

, Wringe

, Mbata

, Urassa

, Zaba

. Barriers to sustaining antiretroviral treatment in Kisesa, Tanzania: A follow-up study to understand attrition from the antiretroviral program. AIDS Patient Care STDS, 2009; 23:203–210.

55.

Cole

, Abel

. Climate of care and nurses' attitudes towards AIDS in the emergency department. Emerg Nurse, 2000; 8:18–24.

56.

Badri

, Lawn

, Wood

. Utility of CD4 cell counts for early prediction of virological failure during antiretroviral therapy in a resource-limited setting. BMC Infect Dis, 2008; 8:89.

57.

van Griensven

, Zachariah

, Mugabo

, Reid

. Weight loss after the first year of stavudine-containing antiretroviral therapy and its association with lipoatrophy, virological failure, adherence and CD4 counts at primary health care level in Kigali, Rwanda. Trans R Soc Trop Med Hyg, 2010; 104:751–757.

58.

McGrath

, Njoroge

, John-Stewart

, et al. Increased incidence of symptomatic peripheral neuropathy among adults receiving stavudine- versus zidovudine-based antiretroviral regimens in Kenya. J Neurovirol, 2012; 18:200–204.

59.

Phidisa

IIWTfPP

, Ratsela

, Polis

, et al. A randomized factorial trial comparing 4 treatment regimens in treatment-naive HIV-infected persons with AIDS and/or a CD4 cell count <200 cells/muL in South Africa. J Infect Dis, 2010; 202:1529–1537.

60.

Squires

, Gulick

, Tebas

, et al. A comparison of stavudine plus lamivudine versus zidovudine plus lamivudine in combination with indinavir in antiretroviral naive individuals with HIV infection: Selection of thymidine analog regimen therapy (START I). AIDS, 2000; 14:1591–1600.

61.

Phanuphak

, Ananworanich

, Teeratakulpisarn

, et al. A 72-week randomized study of the safety and efficacy of a stavudine to zidovudine switch at 24 weeks compared to zidovudine or tenofovir disoproxil fumarate when given with lamivudine and nevirapine. Antivir Ther, 2012; 17:1521–1531.

62.

Gallant

, Staszewski

, Pozniak

, et al. Efficacy and safety of tenofovir DF vs stavudine in combination therapy in antiretroviral-naive patients: A 3-year randomized trial. JAMA, 2004; 292:191–201.

63.

Sunpath

, Wu

, Gordon

, et al. High rate of K65R for antiretroviral therapy-naive patients with subtype C HIV infection failing a tenofovir-containing first-line regimen. AIDS, 2012; 26:1679–1684.

64.

Van Zyl

, Liu

, Claassen

, et al. Trends in genotypic HIV-1 antiretroviral resistance between 2006 and 2012 in South African patients receiving first- and second-line antiretroviral treatment regimens. PLoS One, 2013; 8:e67188.

65.

Cornell

, McIntyre

, Myer

. Men and antiretroviral therapy in Africa: Our blind spot. Trop Med Int Health, 2011; 16:828–829.

66.

Pop-Eleches

, Thirumurthy

, Habyarimana

, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: A randomized controlled trial of text message reminders. AIDS, 2011; 25:825–834.

67.

Puccio

, Belzer

, Olson

, et al. The use of cell phone reminder calls for assisting HIV-infected adolescents and young adults to adhere to highly active antiretroviral therapy: A pilot study. AIDS Patient Care STDS, 2006; 20:438–444.

68.

Tran

, Nguyen

, Hoang

, Hwang

. Determinants of antiretroviral treatment adherence among HIV/AIDS patients: A multisite study. Glob Health Action, 2013; 6:19570.

69.

Murphy

, Sunpath

, Castilla

, et al. Second-line antiretroviral therapy: Long-term outcomes in South Africa. J Acquir Immune Defic Syndr, 2012; 61:158–163.

70.

Estill

, Egger

, Blaser

, et al. Cost-effectiveness of point-of-care viral load monitoring of ART in resource-limited settings: Mathematical modelling study. AIDS, 2013.

71.

Steel

, Nwokike

, Joshi

Development of a Multi-method Tool to Measure ART Adherence in Resource-Constrained Settings: The South Africa Experience. In: USAID, ed. Vol Submitted to the U.S. Agency for International Development by the Rational Pharmaceutical Management Plus Program. Arlington, VA: Management Sciences for Health; 2007.

72.

Kagee

, Remien

, Berkman

, Hoffman

, Campos

, Swartz

. Structural barriers to ART adherence in Southern Africa: Challenges and potential ways forward. Glob Public Health., 2011; 6:83–97.

73.

Shriver

, Everett

, Morin

. Structural interventions to encourage primary HIV prevention among people living with HIV. AIDS, 2000; 14:S57–62.

74.

Manasa

, Katzenstein

, Cassol

, et al. Primary drug resistance in South Africa: Data from 10 years of surveys. AIDS Res Hum Retroviruses, 2012; 28:558–565.