A randomized trial of computer-based reminders and audit and feedback to improve HIV screening in a primary care setting

INTRODUCTION

Early diagnosis of HIV infection is essential because it permits providers to implement interventions aimed at reducing the morbidity and mortality of HIV disease. Early identification of HIV infection enables providers to start appropriate antiretroviral therapy, ¹ and to begin prophylaxis for opportunistic infections if appropriate. ² Early identification also allows for counselling to reduce risky behaviours that can provide a substantial public health benefit. ^3–7

Despite the strong rationale for early identification of HIV, diagnosis of HIV infection is often made late. Surveillance data from the Centers for Disease Control and Prevention (CDC) indicate that 43% of HIV-positive patients develop AIDS within one year of initial diagnosis, suggesting that they have been infected for many years before diagnosis. ⁸ An ongoing cohort study in the Department of Veterans Affairs (VA) found that 20% of patients had an AIDS-defining illness when they presented for HIV care, and that about 40% had CD4 count of 200 cells per mm³ or less. ⁹ Further evidence indicates that risk-based screening misses a substantial proportion of HIV. ¹⁰

We conducted a randomized trial of computer-based reminders and audit and feedback to enhance rates of HIV screening. The purpose of our evaluation was to assess the feasibility of implementing an HIV screening guideline in the VA and to assess the obstacles to successful implementation. Our intervention included an educational intervention for all providers, and computer-based clinical reminders and feedback to the intervention providers. We evaluated whether the intervention increased rates of testing for HIV in primary care settings, and assessed the proportion of primary care patients who had risk behaviours or medical conditions that would make them candidates for screening based on risk-based testing.

METHODS

We conducted a randomized controlled trial at the VA Palo Alto Health Care System, from January 2001 to September 2001, to implement an intervention to improve HIV screening and testing rates. All attending physicians and registered nurse practitioners (RNPs) from VA general medicine clinics at five sites were eligible to participate in the trial (n = 39). Providers were randomized on the basis of their clinic location and their position (attending physician or RNP) to protect against any site or position biases and to equalize the number of patients seen by each provider within each group. Participation in the trial was voluntary and providers signed an informed consent form if they agreed to participate. Providers were randomized to an intervention or control group after they agreed to participate. The randomization sequence was generated by the statistician. Intervention group providers received an education session, computer-based reminders, and feedback about HIV screening for a six-month period, as described in detail below. Control group providers received only the education session. The study was approved by the Stanford University Institutional Review Board.

Because these providers see a large group of outpatients at the VA, our study was designed to have 92% power to detect an absolute increase of 10% in HIV testing; we felt that a less than 10% difference was not clinically important. To verify that we had sufficient power, we calculated a 95% confidence interval for the difference in our screening rates to determine how well the difference could be estimated for a test at the 5% significance level.

Intervention

All providers received an educational session, at their clinic locations, which discussed the importance of HIV screening and testing in the VA, the policies and processes in place for obtaining informed consent, and documenting pre- and post-test counselling. Providers were also given a demonstration of the computer-based clinical reminders.

Intervention providers received one of two types of computer-based reminders for each patient. An HIV risk assessment reminder appeared (Figure 1a) if a patient did not have a documented risk behaviour for HIV and had not been HIV tested at a VA facility from January 1999 to December 2000. A risk behaviour was defined as a documented ICD-9 code ¹¹ for substance abuse, alcohol abuse, hepatitis, or sexually transmitted diseases. This reminder requested that providers assess HIV risk behaviours (Figure 1a). An HIV test reminder appeared if a patient had a documented HIV risk behaviour and had not been tested at a VA facility from January 1999 to December 2000. This reminder noted that a documented risk behaviour was present and suggested that the provider offer HIV testing (Figure 1b). Providers were required to complete an interactive dialog box to resolve the reminders (Figure 2a and b). The reminders were not mandatory and could be ignored by the provider; however, they continued to appear on the medical record each time it was accessed until they were resolved. The reminders also provided a link to a website through which providers could access the CDC guideline for HIV testing and counselling. Providers also received a detailed guide on how to use the reminders.

OPEN IN VIEWER

Figure 1

(a) HIV risk assessment reminder. This figure shows the computer-based reminder that providers receive if a patients has not been tested at the VA and there is no documentation of HIV risk behaviours in the electronic medical record. The reminder shows as due under the list of clinical reminders. (b) HIV testing reminder. This figure shows the computer-based reminder that providers receive if a patients has not been tested at the VA and there is documentation of HIV risk behaviours in the electronic medical record. The reminder shows as due under the list of clinical reminders

OPEN IN VIEWER

Figure 2

(a) Interactive dialog box of the computer-based clinical reminder for the risk assessment reminder. This figure shows the interactive dialog box that providers need to complete in order for the electronic medical record to document that the provider has seen and responded to the reminder for risk assessment. The provider can check any of the options listed and this is recorded as part of the electronic medical record. (b) Interactive dialog box of the computer-based clinical reminder for the testing reminder. This figure shows the interactive dialog box that providers need to complete in order for the electronic medical record to document that the provider has seen and responded to the reminder for HIV testing. The provider needs to check off the appropriate box and this is recorded as part of the electronic medical record. If the provider offers testing and the patient accepts, the provider needs to document that informed consent was obtained and has the option of either conducting pretest counselling or requesting that counselling be conducted by an HIV care provider

Providers received feedback every two months on their actions to resolve the reminders. The feedback was provided through both electronic mail messages and a paper report. It provided a summary of the number of reminders received for each type of reminder and the proportion of patients assessed based on the reminders.

Trial outcomes

We assessed the following outcomes: change in HIV screening rates between the control and intervention providers, guideline-concordant testing decisions among providers, effect of feedback, and provider attitudes and knowledge about HIV screening, clinical practice guidelines and clinical reminders.

Primary outcomes

Our prespecified primary outcome was the between-group difference in the change in HIV screening rates in the six-month period prior to the intervention and the six-month period of the intervention. To determine this outcome, we monitored testing rates in both groups for six months prior to the initiation of the intervention and for six months during the intervention to assess changes in testing rates within the control group and the intervention group. We then calculated the difference between the groups as

where P _RI1 is the proportion of patients tested in the intervention group in the preintervention period, P _RI2 is the proportion of patients tested in the intervention group in the intervention period, P _RC1 is the proportion of patients tested in the control group in the preintervention period and P _RC2 is the proportion of patients tested in the control group in the intervention period.

We used Student's t-test to assess differences in outcomes. The difference in the proportion of patients tested prior to and during the intervention was calculated for each provider and the mean difference between intervention and control providers was compared using Student's t-test. We also compared the difference in testing rates between the two time periods for intervention and control providers independently.

Secondary outcomes

We assessed three secondary outcomes: the degree to which testing was guideline concordant, adherence to reminders, and provider attitude and knowledge. To assess guideline concordance of testing, we reviewed medical records of all tested patients and a random sample of medical records of untested patients. At the time of our study, CDC guidelines recommended risk-based testing. The medical record abstractor was blinded to the assignment of the provider to control or intervention group. Testing was considered guideline concordant if there was documentation of a risk behaviour or a reason for the test, including patient request. A risk behaviour was defined as injection drug use, other substance use (including alcohol use), multiple sexual partners, men who have sex with men, unprotected sex with a high-risk partner, a history of sexually transmitted diseases, a history of hepatitis B or hepatitis C, a history of a blood transfusion prior to 1985, or symptoms for a suspected or diagnosed opportunistic infection. We also considered that use of alcohol only as the sole risk behaviour might not be an appropriate risk behaviour and so examined guideline concordance testing rates after excluding it from our definition of risk behaviour. Among untested patients, provider action to not test was considered guideline concordant if a patient had a recent HIV test, the provider performed risk assessment and did not consider the patient at risk or if the patient refused to take an HIV test. We calculated the rates of guideline-concordant testing among the control and intervention providers for the preintervention and intervention periods.

We also assessed provider adherence to reminders by calculating the proportion of risk assessment and testing reminders that were resolved for each provider. We used Student's t-test to compare the differences in the proportion of reminders resolved between each feedback report and also to compare the differences between the proportion of risk assessment reminders and testing reminders that were resolved.

Our final secondary outcome was to determine providers' attitudes toward and knowledge of HIV screening, clinical practice guidelines and clinical reminders. To assess this outcome, providers were asked to complete a survey prior to and after the intervention; furthermore, intervention providers were asked their opinions on the clinical reminders and the influence of feedback on their actions.

Data were managed and analysed using EpiInfo ¹² and SAS (Statistical Analysis Software, SAS Institute, Cary, NC, USA).

The funder had no role in the design, interpretation, or reporting of this trial.

RESULTS

Of 39 providers who were eligible to participate in the trial, 32 (82%) agreed to participate. Fifteen (47%) providers were randomized to the intervention group and 17 (53%) to the control group; 13 (41%) participants were men and 26 (81%) were attending physicians. There was no difference between intervention and control providers with respect to gender or position (RNP versus attending physician).

DISCUSSION

In this multisite, randomized controlled trial, we assessed whether implementation of computer-based reminders with audit and feedback would improve rates of risk behaviour assessment and HIV screening in a primary care setting. Importantly, we found that one-third of our primary care patients had documented risk behaviours for HIV based on a detailed review of over a thousand charts. We also found that 75% of providers indicated they did not have sufficient time to perform testing and counselling, and that approximately half of our providers did not believe risk assessment was appropriate in their patient population of older veterans. This finding is particularly notable given the high rates of risk behaviour we identified through chart review of untested patients. Consistent with the providers' stated concerns about barriers to testing, fewer than 2% of patients were tested in this primary care population despite the high prevalence of risk behaviours.

Our intervention with reminders and feedback failed to increase rates of testing or risk assessment, which were low at baseline and remained so. Our intervention was of modest intensity, by design. Although providers received the reminder, they could choose to ignore it. They also received regular feedback, but even with this feedback, chose to resolve reminders only 5–11% of the time. We used a modest intensity intervention because in our practice setting, providers receive a series of reminders, and more aggressive reminders (for example, reminders that must be resolved) have met with resistance from providers. At the time of our trial, a new interface to the clinical information system was being implemented. Because of lags in the deployment of this interface to the clinical information system, some providers received only the reminders on paper summaries of the chart, which may have limited the effectiveness of the reminders.

In contrast to our findings, a study by Goetz and colleagues found that a large-scale implementation of HIV testing reminders and feedback, as part of a multimodal intervention, increased testing rates from about 5% to approximately 10%, a finding that was statistically and clinically significant. ¹³ Their intervention included, in addition to reminders and feedback, provider activation through social marketing using local opinion leaders, and organizational changes to facilitate testing. The organizational changes included streamlined pretest counselling, encouragement of nurse-based rather than physician-based testing and use of telephone notification for negative test results. Given that 75% of our providers cited lack of time as a barrier to testing, an important element of the success of this multimodal intervention by Goetz and colleagues may have been the intensive effort to reduce logistic and organizational barriers to testing, and the use of social marketing to motivate providers to test. Our intervention did not include these elements. A plausible interpretation of the contrasting findings of the two studies is that clinical reminders and passive feedback alone were not sufficient to change HIV testing behaviour in our study given the barriers to testing that providers noted.

The CDC now recommends routine opt-out screening for HIV for patients aged 13–64 in all health-care settings in which the prevalence of HIV is greater than 0.1%. ¹⁰ This policy is a major change from the prior recommendation of targeted risk-based screening that was in effect when our study was done. Several analyses have demonstrated that HIV screening is cost-effective at this prevalence threshold of 0.1%. ^14–16 In a related study, we found that the prevalence of undocumented HIV infection met or exceeded this threshold in each of six geographically diverse VA outpatient settings that we assessed using a blinded serological survey (HIV prevalence 0.1–2.8%). ¹⁷

How best to integrate routine HIV screening into primary care is unresolved. Our study suggests that relatively low-intensity interventions may not change testing rates substantially. The broader-based intervention by Goetz and colleagues suggests that a multimodal approach will be more successful, although testing rates were still relatively low. In a recent randomized trial, Anaya and colleagues found that nurse-initiated screening with streamlined counselling and rapid HIV testing increased both the rate of testing and the receipt of test results substantially. ¹⁸ Additional studies have demonstrated that use of rapid HIV tests increases the receipt of test results. ^19,20 Together, these studies suggest that integration of routine screening in primary care may require approaches that simplify and streamline testing.

Our study has several limitations. Our setting was primary care clinics in our VA health-care system; these patient populations may differ substantially from other primary care settings. Thus, whether our results are applicable to other primary care settings is uncertain. In addition, VA has an electronic medical record system that provides powerful tools for alerting clinicians that are unavailable in many settings. However, at the time our trial was performed in 2001, the system was transitioning to a new interface, which limited the access that some clinicians had to electronic version of the reminders, and may have limited the effectiveness of the reminders. Although our trial was performed in 2001, screening practices have not changed in our primary care clinics, and thus we believe our findings of low screening rates, despite high levels of risk behaviour are highly relevant to current practice. Finally, as described above, our intervention was primarily focused on clinical reminders with feedback, and did not include organizational changes or social marketing approaches that are likely to enhance the effectiveness of an intervention.

In summary, we found in primary care clinics in our VA health-care system that HIV testing rates were low, that providers were unaware of the high rates of risk behaviour in their patient population, that providers perceived important barriers to testing and that low-intensity clinical reminders and feedback did not increase rates of screening. Our findings, along with those of related studies, suggest that integration of routine opt-out HIV screening into primary care will likely be facilitated by multifaceted interventions that include careful attention to reducing barriers to testing.