Abstract
The USA and international recommendations no longer emphasize using risk factors to target groups for HIV-testing. Using a Guatemalan database of HIV tests, we developed a clinical prediction rule to guide decisions on HIV-testing. Prior to HIV-testing, data were collected on demographics, risk factors and prior testing. Based on a theoretical construct incorporating demographics, known HIV risk factors and symptoms, we developed a logistic regression model to predict HIV seropositivity. Between 2000 and 2005, 16,471 tests were performed, of which 19.8% were positive. The algorithm successfully predicted 1883 of 2489 HIV-positive tests (sensitivity 76%, likelihood ratio [LR]-positive 2.45) and 6282 of 9086 HIV-negative tests (specificity 69%, LR-negative 0.35). Although the model indices are robust, applying the model in a clinical setting would have little impact on improving selective testing practices. Our findings support current recommendations for universal HIV-testing, not selective testing based on risk factors. Before these recommendations can be adopted widely in Guatemala, treatment access needs to be assured and protections put in place for people diagnosed with HIV infection.
INTRODUCTION
In September of 2006, the Centers for Disease Control and Prevention (CDC) announced revised recommendations for HIV-testing, which called for universal screening in all health-care settings, essentially promoting ‘routine voluntary HIV-screening as a normal part of medical practice.’ 1 The call for universal screening continued an evolution in CDC guidelines away from policies that had suggested ‘targeted’ screening of high-risk populations identified on the basis of behavioural, clinical or demographic characteristics. The call for universal screening was based on an evidence that one-fourth of persons with HIV in the USA were unaware of their infection, 2 that risk-based testing did not identify many patients with HIV infection 3–5 and that universal HIV screening had been extremely effective in preventing transfusion and perinatal HIV transmission. 6,7 Knowledge of HIV infection appears to reduce risky sexual behaviours 8,9 and can lead to earlier treatment, which may also have an impact on the transmissibility of the virus. 10
The 2004 WHO/UNAIDS policy statement on HIV testing is somewhat more restrictive than that of the CDC. 11 WHO/UNAIDS recommends that HIV-testing should be offered to asymptomatic patients in settings where HIV is prevalent and therapy is available. These guidelines reflect very serious practical and ethical concerns associated with HIV-testing. Carrying an HIV diagnosis can be psychologically devastating, have important legal ramifications and lead to stigmatization. Testing positive is only clearly beneficial in contexts where persons living with HIV do not suffer discrimination and have access to effective treatments. 12 These concerns are not new; they were apparent in the gay community from the earliest days of the epidemic in the USA. 13
Guatemala, Central America's most populous country, is thought to have an HIV prevalence of 1%, although good epidemiological data are lacking. 14 Access to antiretroviral therapy (ART) has expanded in the past several years due to efforts of Doctors without Borders, the Guatemalan National AIDS programme and the Global Fund; there are currently 10 sites providing ART in Guatemala. It is estimated that 61,000 Guatemalans are living with HIV infection, of whom approximately 7852 received ART in 2006. 15 There are no reliable estimates of how many people have AIDS and thus need ART.
The Clinica Familiar Luis Angel Garcia (Luis Angel Garcia Family Clinic) is one of Guatemala's largest HIV-testing sites and currently performs over 3000 HIV tests annually on both adults and children; approximately 17% are positive. The National AIDS office in Guatemala has made no formal recommendations regarding who should be tested for HIV. In order to inform public health policy in Guatemala, we examined six years' of HIV-testing data to determine if a risk algorithm could predict HIV infection in adults tested between 2000 and 2006. Our aim was to develop a clinical prediction rule that would aid both clinicians and public health authorities in targeting HIV-testing. Applying a prediction rule could facilitate selective testing by reducing the number of low-risk patients tested.
SETTING
The LAGFC is an HIV specialty clinic established in 1988 in the Hospital General San Juan de Dios (General Hospital), one of Guatemala's two national hospitals. General Hospital is located in the centre of Guatemala city, a city of 2.5 million. Whereas the majority of patients at the clinic come from the capital, the clinic accepts patients from throughout the country. It provides both inpatient and outpatient consultation services as well as postexposure prophylaxis to hospital employees and students who suffer needlestick injuries. 16 Through its parent organization, the Asociación de Salud Integral, the Clinic is associated with a variety of HIV-prevention activities. 17
METHODS
This study reports on analysis of data collected as part of the routine HIV-testing activities of the clinic.
Subjects
We examined data on all HIV tests conducted at the clinic from January 2000 through December 2005. Subjects included individuals who presented to the clinic requesting an HIV test, those for whom an HIV test had been requested by other units of the hospital (typically medicine, surgery and maternity) and blood donors who had a positive HIV serology. Conscious patients provided informed consent for testing; in cases where the patient was unconscious, consent was given by two physicians in accordance with Guatemalan law. 18 Only individuals over 13 (considered as adults by the clinic) were included in this report.
Survey instrument
A structured survey was administered face-to-face by trained HIV counsellors at the time of obtaining consent for HIV-testing and prior to receiving their test results. The structured questionnaire included sociodemographic characteristics (e.g. marital status, ethnicity, education and employment status, income), reasons for testing, HIV risk factors and prior testing history.
Laboratory testing
Serum samples were collected from all patients who presented for testing at the clinic during this period. HIV status was confirmed by the clinic laboratory using two HIV-1 antibody enzyme-linked immunosorbent assays. A western blot was used in indeterminate cases.
Data management and statistical analysis
The database was maintained in Guatemala using a Microsoft Access database and analysed in the USA with SPSS version 15.0 (SPSS, Inc, Chicago, IL, USA).
Choice of variables for the clinical prediction rule
We developed a clinical prediction rule based on the assumption that HIV infection was associated with three broad domains:
Demographics were measured using four indicators: income (divided into quartiles), education (none, primary, secondary, university), marital status (married, cohabiting, single, separated/divorced/widowed) and self-reported ethnicity (Mayan and Ladino, the two major ethnic groups in Guatemala).
Risk behaviours/exposures were combined into a risk index identified by six factors: (a) a sexual partner who was known to be HIV-positive; (b) men who reported having sex with other men; (c) a history of any sexually transmitted disease; (d) involvement in the sex trade (employment as police/military, commercial sex workers and clients of commercial sex workers); (e) less than consistent condom usage; (f) history of intravenous drug use. One point was assigned to report of each risk, resulting in a scale that ranged from 0 (no risks reported) to 6 (all risks reported). In addition to the risk index, number of sexual partners in the past five years was included in the model, categorized as 1, 2–20 or 21+ partners.
Symptoms: We collected data on six symptoms most closely associated with HIV: weight loss, diarrhoea, fevers/sweats, chronic cough, recurrent ulcers and oral fungus. These six symptoms were included in the model as a dichotomy reflecting either absence of all symptoms, or presence of any.
Development of the clinical prediction rule
The clinical prediction rule was created using a sequential logistic regression model to predict HIV status. In the sequential model, demographic measures (income, education, marital status and ethnicity) were entered in a single initial block, followed by a block containing variables measuring risk and presence of symptoms. This method allowed the researchers to control statistically the effects of demographics and to isolate the unique effects of symptoms and risks in predicting serostatus. Preliminary bivariate and descriptive univariate analyses were conducted prior to the regression to evaluate the relationship between selected predictors and HIV infection, and verify that the data met the requirements of the model.
Ethics
This study was approved by the Research Committee at the Hospital General, the Zugueme Independent IRB and the Institutional Review Board at Montefiore Medical Center.
FINDINGS
Description of the sample
Our sample contained data on 16,471 tests of adults: 32.9% of these were repeat tests (22.5% negative/inconclusive prior tests, 10.4% had previous positive results). The overall HIV prevalence in this sample was 19.8%. Dividing the sample by gender and location within the hospital of testing, we tested 4418 male outpatients, 6764 female outpatients, 2959 male inpatients and 2330 female inpatients.
Univariate analysis
Univariate analysis of the associations between the individual components of prediction rule and HIV infection are presented in Table 1. Notably, all traditional HIV risk factors (except for inconsistent condom usage) were associated with HIV infection as were the six physical symptoms. Among the sociodemographic variables, HIV infection was associated with lower income, lower education and Mayan ethnicity. The highest income group had a lower risk of HIV infection than the no-income group, and income in the first three quintiles was associated with a slightly greater risk than the no-income group. Married persons were at lowest risk of HIV infection; the highest risk was associated with being widowed/separated/divorced. Being single or cohabitating was associated with an intermediate risk.
Association of risk variables with HIV infection (n = 16,471 adult tests)
CSW = commercial sex worker; OR = odds ratio; CI = confidence interval; MSM = male having sex with men; STD = sexually transmitted disease; i.v. = intravenous; LL = lower limit; UL = upper limit
REF indicates the category that is reference group for the logistic model
Failure of risk algorithm
The risk algorithm was generated based on all patient records that had complete data for all elements of the model. This sample included 11,575 tests (70.3% of the 16,471 adult tests). The logistic model yielded the estimated probability of a positive HIV test result for each patient based on their individual values for the predictors in the model, which was used as the basis for the prediction rule.
Determining the empirically optimal cutoff for the prediction rule involved examining factors such as sensitivity, specificity, false-positive/negative rates and positive/negative predictive values. Youden's Index for this sample (an index that seeks to maximize both sensitivity and specificity) was highest at a cut-off value of 0.2 (Table 2). At that cut-off, the clinical prediction rule correctly identified only 1883 of 2489 infected patients (sensitivity of 76%, likelihood ratio [LR]-positive of 2.45) at this level. The algorithm correctly identified 6282 of 9086 negative patients (specificity 69%, LR-positive, LR-negative 0.35). Table 3 provides details of the functioning of the algorithm in the four major subsections of the sample.
Classification indices for selected cut-off points
Accuracy of the clinical prediction rule with a cut-off of P = 0.2 for major subgroups of the study population [JF1]
DISCUSSION
In our sample of over 11,000 HIV tests, a clinical prediction rule showed poor ability to detect who was infected; many patients with HIV infection were incorrectly judged to be at low risk. The algorithm failed despite the fact that the traditional risk factors we measured were highly correlated with HIV infection. This finding supports the existing literature that risk-based testing fails to identify many HIV-infected individuals. 3–5
Why might so many cases of HIV infection occur in people who were judged low-risk by our algorithm? There are several plausible explanations. Patients might not want to report risk behaviours or may be unaware of situations of risk (such as an HIV-infected partner). It is possible that our testing and counselling procedure was inadequate to measure behaviour risks or that a different set of questions might have better captured risks and exposures. However, counsellors in the clinic were well-trained and highly motivated to adhere to study guidelines for data collection. Finally, there might be routes of contagion (such as informal injections) about which we did not inquire.
There are inherent limitations in attempts to classify patients based on low-frequency events. When the distribution of outcomes is skewed, as seropositivity was in this study, classification based on predictive models favours the more frequent event, here negative test results. While the sensitivity of positive classifications suffers, this does not reflect a misfit of the data with the model. Examination of the individual predictors indicates that they are highly associated with clinically relevant differences in odds of positive serostatus, although this did not translate to a successful classification algorithm. Despite the fact that our regression model failed in this regard, we did validate the utility of the individual risk factors in signalling HIV infection in our population, e.g. 57% of patients with diarrhoea were HIV-positive. We also demonstrated that many of the traditional HIV risk factors and certain demographic factors were associated with HIV infection.
The failure of our clinical prediction rule would support the UNAIDS/WHO and CDC recommendations that HIV-testing should be universal and not targeted. But the UNAIDS/WHO guidelines also take social context (HIV prevalence, availability of treatment) into account. 11 We would argue that several preconditions must be met in Guatemala before universal testing can be adopted. There should exist a national supply of reliable HIV-test kits. All testing should be done voluntarily following national protocols, with consent and counselling. Counselling may be particularly important for those testing negative, who will be the majority of those tested. Testing positive should lead to treatment. Measures should be taken to assure non-discrimination against those testing positive. In our opinion, HIV-testing should only occur in those settings where these conditions are met.
A logical place to start developing universal testing protocols would be among pregnant women, although nationally only about 30% of women receive prenatal care. Nonetheless, many women deliver in hospitals where rapid testing and administration of antiretroviral medication can be carried out. With the provision of prophylaxis against maternal-to-child transmission, knowledge of HIV status has benefits to individual patients (both pregnant women and their families). Such universal testing will also provide valuable epidemiological data about HIV in Guatemala. However, care will need to be taken to protect pregnant women from abandonment or abuse by their partners. Again, routine testing may not be appropriate for all patients.
In summary, when considering public policy for HIV-testing, the risk profile of the individual seems less important than the social and medical context in which testing takes place.
Footnotes
ACKNOWLEDGEMENTS
Dr Anderson's participation in this project was supported by a Fulbright Senior Scholars Fellowship. This work was supported in part by the Center for AIDS Research at the Albert Einstein College of Medicine and Montefiore Medical Center funded by the National Institutes of Health (NIH AR-51519). We gratefully acknowledge the help of the staff at the Clinica Familiar Luis Angel Garcia who conducted these surveys. We also thank Drs Peter Selwyn and Clyde Schechter for their review of a draft of this paper and helpful suggestions.