Multiple self-report measures of antiretroviral adherence correlated in Sierra Leone,but did they agree?

Abstract

In resource-poor settings, studies validating multiple self-report measures of adherence are limited and do not include data from West Africa. We prospectively assessed the associations between multiple self-report measures of adherence in 58 patients receiving antiretroviral therapy. Self-report measures included a 30-day visual analog scale, 30-day qualitative single-item measure, Adult AIDS Clinical Trial Group 4-day recall, and 3-level categorical 7-day qualitative measure. Unannounced pill count was the objective measure. Spearman's rho correlation coefficients, Bland-Altman plots, and receiver operator curve analyses were performed. Median and mean adherence by pill count were 81.8% and 78.6%, respectively. All self-report measures had either intermediate or high correlation with the pill count, and the 7-day measure had the highest level of correlation with pill count (r = 0.72). All self-report measures demonstrated good agreement when mean pill count adherence was greater than 90%. All but the 7-day measure posed challenges to patient understanding and administration of the measure. In this sample of participants that displayed largely suboptimal adherence, the 7-day measure was preferable, but all self-report measures demonstrated relatively good agreement with the objective criterion pill count measure and are adequate for clinical use in settings such as Sierra Leone.

Keywords

HIV/AIDS treatment antiretroviral therapy adherence self-report validation sub-Saharan Africa Sierra Leone

Introduction

Access to antiretroviral therapy (ART) has outpaced the capacity of health systems in sub-Saharan Africa to monitor the population for virologic failure and drug resistance.¹ Clinicians rely on effective adherence monitoring and interventions because high levels of adherence to ART lead to viral suppression and treatment success.^2–5 Self-report measures are practical tools for monitoring adherence in clinical settings, especially in resource-poor settings.

Studies validating self-report measures of adherence in resource-poor settings are limited.^6,7 Because of educational, cultural, and socioeconomic differences, self-report measures may have different validity in different populations. An example is the correlation of adherence measured by electronic monitoring systems to that measured by a 30-day visual analog scale (VAS). For patients in Houston, the correlation was 0.37 (p < 0.01), while for patients in the capital of Uganda, Kampala, it was 0.77 (p < 0.01).^6,8 Although not systemically studied, educational, cultural, and socioeconomic (ecological) differences may have significantly contributed to these differences.

Just as the validation of multiple self-report measures may yield a wide range of results in resource-rich and resource-poor settings,⁹ populations in African regions have cultural and other ecological differences, so the same self-report measures may demonstrate different validity. To the best of our knowledge, no study in West Africa has compared the validity of multiple self-report measures. We sought to validate four self-report measures with an objective measure of adherence in Koidu Town, Sierra Leone.

Methods

Participants

We sequentially recruited people receiving care from the HIV clinic at Koidu Government Hospital in Koidu Town, Sierra Leone, into a prospective cohort study. Sierra Leone has 14 districts and multiple tribes, but the national language, English Krio, is widely spoken. Koidu Town is the district capital of Kono District, and patients attending this HIV clinic were living in semi-urban and rural settings. This prospective cohort was a convenience sample designed specifically to study adherence. We targeted 50 participants based on time and resources available. We included ART-naïve and -experienced, Krio-speaking adult patients (over age 18) who were receiving generic ART. We included both ART-naïve and -experienced because there is no evidence to support that duration on ART biases the validity of self-report measures. We excluded patients that were pregnant on ART prophylaxis, people not taking ART, or people whose antiretroviral regimen changed during the course of the study.

Informed consent was obtained from all patients enrolled in the study. The consent form was read to illiterate participants, and the participants made a thumbprint ink mark to indicate their agreement to enroll in the study. There was no compensation for participation. This study was approved by the Institutional Review Board of Baylor College of Medicine and Sierra Leone Ethics and Scientific Committee.

Participants were followed for 3 months. An enumerator visited the participants on a monthly basis. The exact date of visit was not provided to preserve the “unannounced” nature of the pill count. Participants were called on the day of visit and met at a mutually agreed location, which tended to be the place of residence. During each visit, the patient was asked to self-report their adherence with the four measures and allowed their pills to be counted.

Instruments and data collection

We used unannounced pill counts as our objective measure because studies have demonstrated its high correlation with electronic monitoring systems (r = 0.91).¹⁰ We used the following self-report measures: 30-day VAS,¹¹ a 30-day qualitative single-item measure,¹² the Adult AIDS Clinical Trial Group (AACTG) 4-day recall,¹³ and a 3-level categorical 7-day qualitative measure (Appendix 1). The 7-day measure is used in government health facilities of Sierra Leone. Instruments were translated into Krio and back-translated into English. There was not a precise translation for the word “very good” used in the 30-day qualitative single-item measure. In Krio, a word can be repeated to create emphasis, so the enumerators repeated the word “good” to indicate “very good.”

Patients were visited at least four times during the course of the study (at baseline and at three monthly follow-up visits). On occasions, participants did not have access to their pills, and the enumerator had to repeat the visit. The goal was four successful visits, or four time-points per person. Demographic data were collected, including age, gender, marital status, educational attainment, work status, income, and literacy level.

Data analysis

Data collected from the self-report measures were standardized into adherence scores on a scale from 0 to 1. Adherence from VAS was calculated from a percent of pills reported taken over the last 30 days. The qualitative single-item measure had six possible responses ranging from “very poor” to “excellent”, and was analyzed as 0%, 20%, 40%, 60%, 80%, and 100%. Adherence from AACTG was calculated with the equation, [1 – (number of doses reported missed / doses prescribed over the prior 4 days)]. Adherence from the 7-day measure was the number of doses reported taken / doses prescribed over the prior 7 days. Adherence was measured from unannounced pill counts as follows: 1 – (number pills remaining between counts / number pills prescribed between counts).

Since pill count adherence could only be calculated after two visits, self-reported adherence during the first visit could not be compared to pill count. Pill count adherence was calculated for the latter three visits, and the analysis was conducted with three time-points per person.

Adherence was analyzed as a continuous variable. Adherence determinations of >100% were truncated at 100. Mean adherence per measure for each person was calculated across the multiple assessments, and these averages were used to calculate mean and median adherence per measure of the study population.

Descriptive and univariate statistics were calculated for the population. Correspondence between the five measures was compared. Correlation coefficients were calculated using Spearman’s rho. Spearman’s rho was chosen over Pearson’s correlation coefficient because the non-parametric analysis technique of Spearman’s rho can closely fit one-tailed data such as that of adherence. Correlation coefficients were defined as low, intermediate, and high with the following values: <0.4, low; 0.4 to <0.7, intermediate; and 0.7 and more, high. In addition to assessing correlation, agreement was assessed with receiver operator curve (ROC) curves and Bland-Altman plots.¹⁴ ROC curves measured agreement through a comparison of area under the curve (AUC). Bland-Altman measured agreement through a parametric approach based on analysis of variance and graphical methods. In the graph, points close to 0 indicated more agreement while points far from 0 indicated less agreement. Acceptable levels of agreement were considered “good” while unacceptable levels of agreement were considered “poor.” Analysis was performed in STATA 12.0 and R statistical package. We considered two-sided p values <0.05 statistically significant and <0.10 indicative of a trend.

Results

Sample characteristics

Seventy patients were screened between July 2010 and November 2011 at the HIV clinic. Of sixty-seven participants enrolled, nine participants were excluded because they were HIV-infected but not taking ART (4), were lost to follow-up (3), were HIV-negative (1), or died (1). Fifty-eight participants completed the adherence measures at each assessment.

Mean age was 34.6 years (SD = 11.1). Most participants (74%) were women, and most participants (69%) lived on less than 2 dollars (USD) per day. One participant reached a college-level education, and half were illiterate. Most were employed (59%), but common forms of employment were low-income jobs such as petty trades, police work, and military work (Table 1). All participants were on a twice-daily antiretroviral regimen, and most took the three-in-one pill of co-formulated zidovudine, lamivudine, and nevirapine (90%). No one took a protease inhibitor.

Table 1.

Demographic data of study population.

Category	Characteristic	Frequency	Percent
Gender (n = 58)	Male	15	26
Gender (n = 58)	Female	43	74
Age (n = 58)	18–35	30	52
Age (n = 58)	35 and more	28	48
ART (n = 58)	Naïve	20	34
ART (n = 58)	Experienced	38	66
Marital status (n = 58)	Single	12	21
	Married	31	53
	Widowed	11	19
	Discordant	4	7
Employment status (n = 58)	Employed	34	59
Employment status (n = 58)	Not employed	24	41
Income in USD per day (n = 58)	<1	21	36
	1–2	19	33
	>2	18	31
Education (n = 58)	None	18	31
	Any primary school	12	21
	Any secondary school	27	47
	Any college	1	2
Literacy (n = 58)	Can read	29	50
Literacy (n = 58)	Can write	27	47

ART, antiretroviral therapy.

Levels and demographic correlates of adherence

For each measure, median adherence was as follows: 81.8% for pill count, 80.0% for VAS, 73.3% for the qualitative single item measure, 86.7% for AACTG, and 86.0% for the 7-day measure. Mean adherence (SD) for each measure was as follows: 78.6% (18.2) for pill count, 78.7% (15.8) for VAS, 72.5% (16.2) for the qualitative single item measure, 82.5% (19.4) for AACTG, and 87.7% (10.9) for the 7-day measure.

According to the pill count, only 31.0% of participants had over 90% mean adherence, and the range of adherence was between 6.8% and 100%. There was a tendency for participants with the ability to write to have higher levels of adherence (82.1% mean adherence) compared to those who could not write (75.6% mean adherence; p = 0.09; Table 2). Adherence by pill count did not differ by other demographic characteristics (Table 2). There was no difference in adherence between the assessments.

Table 2.

Comparison of demographic characteristics with adherence by unannounced pill count.

Category	Characteristic	n	Adherence (%)	t-Test
Gender (n = 58)	Male	15	82.1	0.20
Gender (n = 58)	Female	43	77.4	0.20
Age (n = 58)	18–35	30	75.7	0.11
Age (n = 58)	35 and more	28	81.7	0.11
ART (n = 58)	Naïve	20	78.3	0.47
ART (n = 58)	Experienced	38	78.7	0.47
Income in USD per day (n = 58)	<1	21	75.3	0.11
	>2	18	82.7	0.11
	<1	21	75.3	0.12
	>1	37	80.5	0.12
	<2	40	76.8	0.12
	>2	18	82.7	0.12
Literacy (n = 58)	Cannot read	29	75.9	0.14
	Can read	29	81.2	0.14
	Cannot write	31	75.6	0.09
	Can write	27	82.1	0.09

ART: antiretroviral therapy.

Correlations and levels of agreement between adherence measures

All self-report measures of adherence had either intermediate or high levels of correlation with pill count. Of the self-report measures, the 7-day measure had the highest correlation with pill count (r = 0.72). VAS, AACTG, and 7-day measure were all associated with each other at high levels of correlation (r > 0.7). The qualitative single-item measure demonstrated intermediate levels of correlation with VAS, AACTG, and 7-day measure (r = 0.56–0.68). All correlations were significant (p < 0.01) (Table 3).

Table 3.

Correlations coefficients between measures using Spearman’s rho.

Measure	Pill count	VAS	Single item	AACTG	7-Day
Pill count	1	0.57	0.52	0.61	0.72
VAS	1	0.68	0.78	0.72
Single item	1	0.56	0.62
AACTG	1	0.88
7-Day	1

AACTG: Adult AIDS Clinical Trial Group; VAS: visual analog scale.

All correlations were significant at p < 0.01.

ROC curves demonstrated good agreement between self-report measures. Values of the AUC for the ROC curves of pill count adherence greater than 90% were as follows: AACTG, 0.68; 7-day measure, 0.71; the qualitative single-item measure, 0.72; and VAS, 0.73 (Figure 1). In concordance with the ROC curves, the Bland-Altman method was used to plot all combinations of adherence measures. When comparing pill count adherence with self-report measures, good agreement was demonstrated at high levels (>90%) of mean adherence. Lower levels of mean adherence suggested poor agreement (Figure 2).

Figure 1.

Receiver operator curve (ROC) curves and area under the curve (AUC) for self-report measures when adherence to antiretroviral therapy (ART) was classified as greater than 90% by pill count. (a) Adult AIDS Clinical Trial Group (AACTG). (b) 7-Day. (c) Single item. (d) visual analog scale (VAS).

Figure 2.

Bland-Altman plots of unannounced pill counts and the self-report measures. (a) Agreement between pill count and Adult AIDS Clinical Trial Group (AACTG). (b) Agreement between pill count and 7-Day. (c) Agreement between pill count and single item. (d) Agreement between pill count and VAS.

Discussions

To our knowledge, this is the first published study to assess validity of multiple self-reported adherence measures in West Africa and to report levels of adherence in Sierra Leone. We were able to demonstrate intermediate or high levels of correlation in self-reported adherence measures with pill count. A study in Uganda validating a 3-day measure and VAS found even higher levels of correlation.⁶ Ecological differences in the self-reported patterns of adherence may partially explain the variation of findings between Uganda and Sierra Leone. Nonetheless, our findings support the validity of these measures in Sierra Leone and other settings with low-literacy populations of sub-Saharan Africa.

Clinicians in Sierra Leone use a 7-day qualitative measure with 3 response options (excellent, good, and poor) to monitor adherence. Although 7-day self-reports have been used in other studies, we were not aware of a study validating this 3-level categorical 7-day qualitative measure. The 7-day measure performed well in our study population and demonstrated high levels of correlation compared to the pill count. The 7-day measure also performed well as compared to the other self-report measures.

Although correlation coefficients broadly supported the validity of the self-report measures, we analyzed the agreement of measures to discern changes at different adherence levels. ROC curves confirmed agreement at greater than 90% adherence levels among measures. The Bland-Altman plots also demonstrated good agreement when participants reported greater than 90% adherence to ART, but inspection of the plots revealed that agreement deteriorated as levels of adherence dropped. Other studies have demonstrated variations of agreement at different adherence levels,¹⁵ and these findings emphasize the limitations of self-report measures compared to pill count.

Median pill count adherence (81.8%) reflected rates similar to rates observed in resource-rich countries.^16,17 Although studies suggest that virologic suppression can be achieved with 80% adherence to non-nucleoside reverse transcriptase inhibitors,¹⁸ nearly half of our participants had less than 80% adherence to ART, placing them at risk for drug resistance and virologic failure. In the setting of this study, health professionals do not routinely ask the patient to self-report adherence, yet these results highlight the need for adherence monitoring.

The enumerators and participants reported several limitations related to the administration of the instruments. All participants received some assistance when completing the VAS, suggesting that the VAS might have performed better in a higher literacy population.⁸ Some participants confused days with doses when answering AACTG, making the measure difficult to administer in these instances. Even though enumerators put emphasis on the word “good” to indicate “very good” when administering the qualitative single item, participants demonstrated difficulty distinguishing between “good” and “very good.” In contrast, the enumerators and participants reported that the 7-day qualitative measure was simple to administer and answer, respectively.

There are several limitations with this study. Even though the three adherence assessments per person improved reliability in measurement, the sample size was small, so small differences in the validity of the instruments may have been undetected due to type II error. It is unlikely that a larger sample size would have changed validity because we found significant associations, but a larger sample size would likely have demonstrated significant demographic correlates of adherence. Second, the different recall periods could have confounded results.¹² Although pill count has a high correlation to electronic monitoring of adherence, the measurement of viral load would have strengthened the outcomes. Sierra Leone, however, did not have the capacity to perform viral load assays.

Self-report measures are practical screening tools for adherence, and clinicians are more likely to use simple, fast, and valid measures. The 3-level categorical 7-day qualitative measure was the most accurate measure in our clinical setting, and compared to the administration of other self-report measures, this qualitative measure revealed its practical strengths in our population. If clinicians choose to use VAS or the qualitative single item, we suggest that they provide patients with assistance on the VAS and that they consider a more meaningful way of describing “very good” on the qualitative single item. Nonetheless, the correspondence and agreement of all of the self-report measures support their utility when compared to the objective measure of unannounced pill counts. Clinics throughout Sierra Leone can select the measure they wish to use based on patient-provider preference and be reasonably confident that the measure will be accurate.

All measures revealed that a large proportion of the participants had subpar levels of adherence, placing them at risk for virologic failure and drug resistance. In Koidu Town and other settings where adherence is generally subpar, the use of self-report measures should be encouraged for identifying those in need of adherence support.

Meetings

Poster presented at 6th International Conference on HIV Treatment Adherence; Miami, FL, May 2011.

Footnotes

Acknowledgments

We thank the following people for their contributions to this work: HIV counselors for recruiting the participants, Fodei Dabo and Alex Joe for enumerating the adherence measures, Katie Hsih, Fatu Conteh, and Kulani Jalata for monitoring the enumerators, and all of the HIV-infected people that participated. Non-governmental organization Wellbody Alliance funded this study.

Conflict of interest

The authors declare no conflict of interest.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

Kahn

Marseille

Moore

. CD4 cell count and viral load monitoring in patients undergoing antiretroviral therapy in Uganda: cost effectiveness study. BMJ 2011; 343: d6884–d6884.

Paterson

Swindells

Mohr

. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Int Med 2000; 133: 21–30.

Bangsberg

Hecht

Charlebois

. Adherence to protease inhibitors, HIV-1 viral load, and development of drug resistance in an indigent population. AIDS 2000; 14: 357–366.

Simoni

Pearson

Pantalone

. Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV-1 RNA viral load. A meta-analytic review of randomized controlled trials. J Acquir Immune Defic Syndr 2006; 43: S23–S35.

de Bruin

Viechtbauer

Schaalma

. Standard care impact on effects of highly active antiretroviral therapy adherence interventions: a meta-analysis of randomized controlled trials. Arch Intern Med 2010; 170: 240–250.

Oyugi

Byakika-Tusiime

Charlesbois

. Multiple validated measures of adherence indicate high levels of adherence to generic HIV antiretroviral therapy in a resource-limited setting. J Acquir Immune Defic Syndr 2004; 36: 1100–1102.

Minzi

Naazneen

. Validation of self-report and hospital pill count using unannounced home pill count as methods for determination of adherence to antiretroviral therapy. Tanzan J Health Res 2008; 10: 84–88.

Buscher

Hartman

Kallen

. Validity of self-report measures in assessing antiretroviral adherence of newly diagnosed, HAART-naïve, HIV patients. HIV Clin Trials 2011; 12: 244–254.

Simoni

Kurth

Pearson

. Self-report measures of antiretroviral therapy adherence: a review with recommendations for HIV research and clinical management. AIDS Behav 2006; 10: 227–245.

10.

Bangsberg

Hecht

Charlebois

. Comparing objective measures of adherence to HIV antiretroviral therapy: electronic medication monitors and unannounced pill counts. AIDS Behav 2001; 5: 275–281.

11.

Giordano

Guzman

Clark

. Measuring adherence to antiretroviral therapy in a diverse population using a visual analogue scale. HIV Clin Trials 2004; 5: 74–79.

12.

Safren

Skolnik

. Optimal recall period and response task for self-reported HIV medication adherence. AIDS Behav 2008; 12: 86–94.

13.

Chesney

. The challenge of adherence. Bull Exper Treat AIDS 1999; 12: 10–13.

14.

Bland

Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–310.

15.

Munoz-Moreno

Fumaz

Ferrer

. Assessing self-reported adherence to HIV therapy by questionnaire: the SERAD (Self-Reported Adherence) Study. AIDS Res Hum Retroviruses 2007; 23: 1166–1175.

16.

Bangsberg

Deeks

. Is average adherence to HIV antiretroviral therapy enough? J Gen Intern Med 2002; 17: 812–813.

17.

Mills

Nachega

Buchan

. Adherence to antiretroviral therapy in sub-Saharan Africa and North America: a meta-analysis. JAMA 2006; 296: 679–690.

18.

Maggiolo

Ravasio

Ripamonti

. Similar adherence rates favor different virologic outcomes for patients treated with nonnucleoside analogues or protease inhibitors. Clin Infect Dis 2005; 40: 158–163.