Abstract
In resource-poor settings, studies validating multiple self-report measures of adherence are limited and do not include data from West Africa. We prospectively assessed the associations between multiple self-report measures of adherence in 58 patients receiving antiretroviral therapy. Self-report measures included a 30-day visual analog scale, 30-day qualitative single-item measure, Adult AIDS Clinical Trial Group 4-day recall, and 3-level categorical 7-day qualitative measure. Unannounced pill count was the objective measure. Spearman's rho correlation coefficients, Bland-Altman plots, and receiver operator curve analyses were performed. Median and mean adherence by pill count were 81.8% and 78.6%, respectively. All self-report measures had either intermediate or high correlation with the pill count, and the 7-day measure had the highest level of correlation with pill count (r = 0.72). All self-report measures demonstrated good agreement when mean pill count adherence was greater than 90%. All but the 7-day measure posed challenges to patient understanding and administration of the measure. In this sample of participants that displayed largely suboptimal adherence, the 7-day measure was preferable, but all self-report measures demonstrated relatively good agreement with the objective criterion pill count measure and are adequate for clinical use in settings such as Sierra Leone.
Keywords
Introduction
Access to antiretroviral therapy (ART) has outpaced the capacity of health systems in sub-Saharan Africa to monitor the population for virologic failure and drug resistance. 1 Clinicians rely on effective adherence monitoring and interventions because high levels of adherence to ART lead to viral suppression and treatment success.2–5 Self-report measures are practical tools for monitoring adherence in clinical settings, especially in resource-poor settings.
Studies validating self-report measures of adherence in resource-poor settings are limited.6,7 Because of educational, cultural, and socioeconomic differences, self-report measures may have different validity in different populations. An example is the correlation of adherence measured by electronic monitoring systems to that measured by a 30-day visual analog scale (VAS). For patients in Houston, the correlation was 0.37 (p < 0.01), while for patients in the capital of Uganda, Kampala, it was 0.77 (p < 0.01).6,8 Although not systemically studied, educational, cultural, and socioeconomic (ecological) differences may have significantly contributed to these differences.
Just as the validation of multiple self-report measures may yield a wide range of results in resource-rich and resource-poor settings, 9 populations in African regions have cultural and other ecological differences, so the same self-report measures may demonstrate different validity. To the best of our knowledge, no study in West Africa has compared the validity of multiple self-report measures. We sought to validate four self-report measures with an objective measure of adherence in Koidu Town, Sierra Leone.
Methods
Participants
We sequentially recruited people receiving care from the HIV clinic at Koidu Government Hospital in Koidu Town, Sierra Leone, into a prospective cohort study. Sierra Leone has 14 districts and multiple tribes, but the national language, English Krio, is widely spoken. Koidu Town is the district capital of Kono District, and patients attending this HIV clinic were living in semi-urban and rural settings. This prospective cohort was a convenience sample designed specifically to study adherence. We targeted 50 participants based on time and resources available. We included ART-naïve and -experienced, Krio-speaking adult patients (over age 18) who were receiving generic ART. We included both ART-naïve and -experienced because there is no evidence to support that duration on ART biases the validity of self-report measures. We excluded patients that were pregnant on ART prophylaxis, people not taking ART, or people whose antiretroviral regimen changed during the course of the study.
Informed consent was obtained from all patients enrolled in the study. The consent form was read to illiterate participants, and the participants made a thumbprint ink mark to indicate their agreement to enroll in the study. There was no compensation for participation. This study was approved by the Institutional Review Board of Baylor College of Medicine and Sierra Leone Ethics and Scientific Committee.
Participants were followed for 3 months. An enumerator visited the participants on a monthly basis. The exact date of visit was not provided to preserve the “unannounced” nature of the pill count. Participants were called on the day of visit and met at a mutually agreed location, which tended to be the place of residence. During each visit, the patient was asked to self-report their adherence with the four measures and allowed their pills to be counted.
Instruments and data collection
We used unannounced pill counts as our objective measure because studies have demonstrated its high correlation with electronic monitoring systems (r = 0.91). 10 We used the following self-report measures: 30-day VAS, 11 a 30-day qualitative single-item measure, 12 the Adult AIDS Clinical Trial Group (AACTG) 4-day recall, 13 and a 3-level categorical 7-day qualitative measure (Appendix 1). The 7-day measure is used in government health facilities of Sierra Leone. Instruments were translated into Krio and back-translated into English. There was not a precise translation for the word “very good” used in the 30-day qualitative single-item measure. In Krio, a word can be repeated to create emphasis, so the enumerators repeated the word “good” to indicate “very good.”
Patients were visited at least four times during the course of the study (at baseline and at three monthly follow-up visits). On occasions, participants did not have access to their pills, and the enumerator had to repeat the visit. The goal was four successful visits, or four time-points per person. Demographic data were collected, including age, gender, marital status, educational attainment, work status, income, and literacy level.
Data analysis
Data collected from the self-report measures were standardized into adherence scores on a scale from 0 to 1. Adherence from VAS was calculated from a percent of pills reported taken over the last 30 days. The qualitative single-item measure had six possible responses ranging from “very poor” to “excellent”, and was analyzed as 0%, 20%, 40%, 60%, 80%, and 100%. Adherence from AACTG was calculated with the equation, [1 – (number of doses reported missed / doses prescribed over the prior 4 days)]. Adherence from the 7-day measure was the number of doses reported taken / doses prescribed over the prior 7 days. Adherence was measured from unannounced pill counts as follows: 1 – (number pills remaining between counts / number pills prescribed between counts).
Since pill count adherence could only be calculated after two visits, self-reported adherence during the first visit could not be compared to pill count. Pill count adherence was calculated for the latter three visits, and the analysis was conducted with three time-points per person.
Adherence was analyzed as a continuous variable. Adherence determinations of >100% were truncated at 100. Mean adherence per measure for each person was calculated across the multiple assessments, and these averages were used to calculate mean and median adherence per measure of the study population.
Descriptive and univariate statistics were calculated for the population. Correspondence between the five measures was compared. Correlation coefficients were calculated using Spearman’s rho. Spearman’s rho was chosen over Pearson’s correlation coefficient because the non-parametric analysis technique of Spearman’s rho can closely fit one-tailed data such as that of adherence. Correlation coefficients were defined as low, intermediate, and high with the following values: <0.4, low; 0.4 to <0.7, intermediate; and 0.7 and more, high. In addition to assessing correlation, agreement was assessed with receiver operator curve (ROC) curves and Bland-Altman plots. 14 ROC curves measured agreement through a comparison of area under the curve (AUC). Bland-Altman measured agreement through a parametric approach based on analysis of variance and graphical methods. In the graph, points close to 0 indicated more agreement while points far from 0 indicated less agreement. Acceptable levels of agreement were considered “good” while unacceptable levels of agreement were considered “poor.” Analysis was performed in STATA 12.0 and R statistical package. We considered two-sided p values <0.05 statistically significant and <0.10 indicative of a trend.
Results
Sample characteristics
Seventy patients were screened between July 2010 and November 2011 at the HIV clinic. Of sixty-seven participants enrolled, nine participants were excluded because they were HIV-infected but not taking ART (4), were lost to follow-up (3), were HIV-negative (1), or died (1). Fifty-eight participants completed the adherence measures at each assessment.
Demographic data of study population.
ART, antiretroviral therapy.
Levels and demographic correlates of adherence
For each measure, median adherence was as follows: 81.8% for pill count, 80.0% for VAS, 73.3% for the qualitative single item measure, 86.7% for AACTG, and 86.0% for the 7-day measure. Mean adherence (SD) for each measure was as follows: 78.6% (18.2) for pill count, 78.7% (15.8) for VAS, 72.5% (16.2) for the qualitative single item measure, 82.5% (19.4) for AACTG, and 87.7% (10.9) for the 7-day measure.
Comparison of demographic characteristics with adherence by unannounced pill count.
ART: antiretroviral therapy.
Correlations and levels of agreement between adherence measures
Correlations coefficients between measures using Spearman’s rho.
AACTG: Adult AIDS Clinical Trial Group; VAS: visual analog scale.
All correlations were significant at p < 0.01.
ROC curves demonstrated good agreement between self-report measures. Values of the AUC for the ROC curves of pill count adherence greater than 90% were as follows: AACTG, 0.68; 7-day measure, 0.71; the qualitative single-item measure, 0.72; and VAS, 0.73 (Figure 1). In concordance with the ROC curves, the Bland-Altman method was used to plot all combinations of adherence measures. When comparing pill count adherence with self-report measures, good agreement was demonstrated at high levels (>90%) of mean adherence. Lower levels of mean adherence suggested poor agreement (Figure 2).
Receiver operator curve (ROC) curves and area under the curve (AUC) for self-report measures when adherence to antiretroviral therapy (ART) was classified as greater than 90% by pill count. (a) Adult AIDS Clinical Trial Group (AACTG). (b) 7-Day. (c) Single item. (d) visual analog scale (VAS). Bland-Altman plots of unannounced pill counts and the self-report measures. (a) Agreement between pill count and Adult AIDS Clinical Trial Group (AACTG). (b) Agreement between pill count and 7-Day. (c) Agreement between pill count and single item. (d) Agreement between pill count and VAS.

Discussions
To our knowledge, this is the first published study to assess validity of multiple self-reported adherence measures in West Africa and to report levels of adherence in Sierra Leone. We were able to demonstrate intermediate or high levels of correlation in self-reported adherence measures with pill count. A study in Uganda validating a 3-day measure and VAS found even higher levels of correlation. 6 Ecological differences in the self-reported patterns of adherence may partially explain the variation of findings between Uganda and Sierra Leone. Nonetheless, our findings support the validity of these measures in Sierra Leone and other settings with low-literacy populations of sub-Saharan Africa.
Clinicians in Sierra Leone use a 7-day qualitative measure with 3 response options (excellent, good, and poor) to monitor adherence. Although 7-day self-reports have been used in other studies, we were not aware of a study validating this 3-level categorical 7-day qualitative measure. The 7-day measure performed well in our study population and demonstrated high levels of correlation compared to the pill count. The 7-day measure also performed well as compared to the other self-report measures.
Although correlation coefficients broadly supported the validity of the self-report measures, we analyzed the agreement of measures to discern changes at different adherence levels. ROC curves confirmed agreement at greater than 90% adherence levels among measures. The Bland-Altman plots also demonstrated good agreement when participants reported greater than 90% adherence to ART, but inspection of the plots revealed that agreement deteriorated as levels of adherence dropped. Other studies have demonstrated variations of agreement at different adherence levels, 15 and these findings emphasize the limitations of self-report measures compared to pill count.
Median pill count adherence (81.8%) reflected rates similar to rates observed in resource-rich countries.16,17 Although studies suggest that virologic suppression can be achieved with 80% adherence to non-nucleoside reverse transcriptase inhibitors, 18 nearly half of our participants had less than 80% adherence to ART, placing them at risk for drug resistance and virologic failure. In the setting of this study, health professionals do not routinely ask the patient to self-report adherence, yet these results highlight the need for adherence monitoring.
The enumerators and participants reported several limitations related to the administration of the instruments. All participants received some assistance when completing the VAS, suggesting that the VAS might have performed better in a higher literacy population. 8 Some participants confused days with doses when answering AACTG, making the measure difficult to administer in these instances. Even though enumerators put emphasis on the word “good” to indicate “very good” when administering the qualitative single item, participants demonstrated difficulty distinguishing between “good” and “very good.” In contrast, the enumerators and participants reported that the 7-day qualitative measure was simple to administer and answer, respectively.
There are several limitations with this study. Even though the three adherence assessments per person improved reliability in measurement, the sample size was small, so small differences in the validity of the instruments may have been undetected due to type II error. It is unlikely that a larger sample size would have changed validity because we found significant associations, but a larger sample size would likely have demonstrated significant demographic correlates of adherence. Second, the different recall periods could have confounded results. 12 Although pill count has a high correlation to electronic monitoring of adherence, the measurement of viral load would have strengthened the outcomes. Sierra Leone, however, did not have the capacity to perform viral load assays.
Self-report measures are practical screening tools for adherence, and clinicians are more likely to use simple, fast, and valid measures. The 3-level categorical 7-day qualitative measure was the most accurate measure in our clinical setting, and compared to the administration of other self-report measures, this qualitative measure revealed its practical strengths in our population. If clinicians choose to use VAS or the qualitative single item, we suggest that they provide patients with assistance on the VAS and that they consider a more meaningful way of describing “very good” on the qualitative single item. Nonetheless, the correspondence and agreement of all of the self-report measures support their utility when compared to the objective measure of unannounced pill counts. Clinics throughout Sierra Leone can select the measure they wish to use based on patient-provider preference and be reasonably confident that the measure will be accurate.
All measures revealed that a large proportion of the participants had subpar levels of adherence, placing them at risk for virologic failure and drug resistance. In Koidu Town and other settings where adherence is generally subpar, the use of self-report measures should be encouraged for identifying those in need of adherence support.
Meetings
Poster presented at 6th International Conference on HIV Treatment Adherence; Miami, FL, May 2011.
Footnotes
Acknowledgments
We thank the following people for their contributions to this work: HIV counselors for recruiting the participants, Fodei Dabo and Alex Joe for enumerating the adherence measures, Katie Hsih, Fatu Conteh, and Kulani Jalata for monitoring the enumerators, and all of the HIV-infected people that participated. Non-governmental organization Wellbody Alliance funded this study.
Conflict of interest
The authors declare no conflict of interest.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
