HIV Quality Performance Measures in a Large Integrated Health Care System

Abstract

HIV quality performance measurements are critical to evaluating a care program's success in areas of testing, access to and retention in care, care processes and outcomes. Kaiser Permanente (KP) provides care to over 8 million Americans and over 19,000 HIV-infected adults. We undertook a quality performance measurement program to assess the care and outcomes for our HIV-positive patient population. We also examined HIV testing practices among our HIV-uninfected patients presenting with a sexually transmitted infection. Our metrics were extracted electronically (encompassing two time periods: July 1, 2005 through June 30, 2006 and the entire calendar year 2007) and did not require any manual data extraction, which was a primary objective of our strategy. For most individual care measures, improvement over time was noted, with 85% or more performance seen on some measures (accessing care and initiating antiretroviral therapy). Opportunities for improvement were identified on other measures, such as diagnosing HIV at an earlier stage of infection, and more consistent Pneumocystis jiroveci pneumonia prophylaxis. Over 90% of our patients on antiretroviral therapy had maximal viral control, along with high median antiretroviral medication adherence. Our results compare favorably to those of other organizations, with a KP HIV mortality rate less than 50% of the overall U.S. rate. These results have implications for improving our care process going forward, as well as for the new U.S. domestic HIV/AIDS Strategy.

Introduction

A s it enters its fourth decade, HIV/AIDS, once considered a terminal illness, has progressed to a complex, chronic manageable disease.^1,2 HIV-positive patients are now living significantly longer lives through the use of appropriate potent antiretroviral therapy (ART), in conjunction with opportunistic infection prophylaxis/treatment and appropriate medical health screenings, along with greater retention in medical care.³ Earlier diagnosis and entry into care has been associated with improved outcomes. However, implementing these care processes often requires multifaceted, costly, and coordinated efforts of many health professionals.^4

–7

Quality health care requires quality performance assessment to ensure appropriate results.⁸ A standardized set of measures is essential for measuring performance across a variety of health care systems and clinics. Other organizations have attempted to assess HIV-related quality performance, but these efforts have not been coordinated, nor produced comparable measurements.^9

–12 The HIV Medicine Association (HIVMA), National Committee on Quality Assurance (NCQA), American Medical Association (AMA), and US Department of Health and Human Services' Health Resources and Services Administration (HRSA) convened an expert panel to create a coordinated panel of HIV performance measures in 2007–2008 and subsequently endorsed by the National Quality Forum, which is being adopted on a continued basis by these organizations and others.¹³ The measures, however, did not include assessments of HIV diagnosis or entry into care.

Kaiser Permanente (KP) is the largest civilian provider of HIV care in the United States and is the second largest provider of HIV care in the United States (the largest provider being the Veterans Administration). There were nearly 17,000 active HIV-infected KP members in 2007. Our HIV mortality rate is 1.6%, compared to 3.4% nationwide in the United States.¹⁴ We initiated an HIV quality measures program in 2007 to measure the spectrum of care from HIV diagnosis to achieving maximal viral control. A key element of the program was the exclusive use of electronic health records/databases for derivation of quality measures. Later, we sought to reconcile our measures with the HIVMA/NCQA/AMA/HRSA (“expert panel”) HIV quality measures. Our goal is to track these measures over time and address potential care gaps. We report our measures' derivation and the results of our first two measurement cycles.

Methods

Subjects

KP is an integrated health maintenance organization (HMO) serving various geographic populations in the United States. Patients in our system receive multidisciplinary health care, including HIV specialty care. The HIV-positive population in KP is representative of the states they serve; data indicate that members overall are very similar to the general population with regard to age, gender, and race/ ethnicity, with only slight underrepresentation of those in lower and higher income and education categories.¹⁵ Moreover, HIV-positive patients in KP also reflect the general populations because patients with low incomes are eligible for state Medicaid HMO. For example, current KP Northern California HIV-infected patients are largely male (89.3%), men having sex with men (MSM; 77.9%), and Caucasian (62.7%). These statistics are remarkably similar to demographics of reported AIDS cases in California, which are also largely male (91.1%), MSM (76.9%), and Caucasian (56.8%).¹⁶

Appropriate databases and disease registries were queried to identify eligible patients. Eligible patients were older than 17 years of age from our California, Georgia, Mid-Atlantic (District of Columbia, Maryland, Virginia) Northwest (Oregon and Washington state), and Ohio regions. Depending on the measure, patients had to meet membership duration criteria to be included in the denominator. For example, for outcomes consistent with the expert panel developed measures, patients had to be a KP member for at least 8 months of the 12-month measurement period (Table 1).

Table 1.

Measures Specifications

Measure	Numerator	Denominator	Denominator membership requirements	NQF specified (Y/N)
Testing for HIV with diagnosed STI (incident viral hepatitis diagnoses only)	Patients tested for HIV during 10 days before through 90 days after 1st STI positivity	Patients not known HIV-infected who test positive for chlamydia, gonorrhea, syphilis, incident Hepatitis B, incident Hepatitis C during measurement period	Member in month of STI test and each of 3 subsequent months (to detect HIV test) and 12 prior months (to rule out HIV)	N
Testing for HIV with diagnosed STI (all new hepatitis diagnoses)	Patients tested for HIV during 10 days before through 90 days after 1st STI positivity	Patients not known HIV-infected who test positive for chlamydia, gonorrhea, syphilis, new hepatitis B diagnosis, new hepatitis C diagnosis during measurement period	Member in month of STI test and each of 3 subsequent months (to detect HIV test) and 12 prior months (to rule out HIV)	N
CD4 <200 at HIV diagnosis	Patients whose lowest CD4 test in 1st 90 days after initial HIV diagnosis is <200 cells	Patients with initial CD4 cell count and with initial HIV diagnosis and care in measurement period in KP	Member in month of HIV detection and in each of 3 subsequent months (for CD4 assessment)	N
In care within 90 days of new HIV diagnosis	Patients with CD4 test done within 90 days of first HIV care in KP (initial diagnosis or transfer into KP)	Patients getting first care for HIV in measurement period, including transfers	member in month of HIV detection and in each of 3 subsequent months (for CD4 assessment)	N
Seen by HIV specialist or primary care at least twice Annually	Patients “in care” defined as having ≥2 medical visits (HIV specialist or primary care) in measurement period with ≥60 days between visits	All known HIV-infected patients known in KP at the mid-point of the measurement period	Member for ≥8 months in measurement period	Y
CD4⁺ count performed at least every 6 months	Patients with CD4 count measurement (absolute or patients) at least every 6 months (≥120 days apart)	Patients “in care” (see above)	Member for ≥8 months in measurement period	Y
PJP Prophylaxis if CD4 <200^a	Patients prescribed Pneumocystis jirovecii prophylaxis within 3 months of 1st CD4 count <200/μL	Patients “in care” (see above) with ≥1 CD4 cell count <200/μL in measurement period	Member for ≥8 months in measurement period	Y
ART prescribed if CD4 <200^a	Patients on potent antiretroviral regimen (following DHHS definition of combination antiretroviral therapy) during measurement period	Patients “in care” (see above) with lowest CD4 cell count in measurement period is <200/μL	Member for ≥8 months in measurement period	Y
ART Prescribed if CD4 <350^a	Patients on potent antiretroviral regimen (following DHHS definition of combination antiretroviral therapy) during measurement period	Patients “in care” (see above) with lowest CD4 cell count in measurement period is <350/μL	Member for ≥8 months in measurement period	Y
ART adherence ≥90%^a	Patients with ≥90% adherence to all medications in antiretroviral regimen over first to last fill in measurement period	Patients who received combination antiretroviral therapy for ≥3 months and in care ≥8 months in KP during measurement period	Member for ≥9 months in the measurement period	N
HIV RNA below limits of quantification on ART	Patients with HIV RNA below limits of quantification ever during measurement period (excluded from numerator if no HIV RNA measured during measurement period)	Patients “in care” (see above) and received combination antiretroviral therapy for ≥3 months and in care ≥8 months in KP during measurement period	Member for ≥8 months in measurement period	Y

Patients must receive their medications through KP pharmacy.

STI, sexually transmitted infection; PJP, Pneumocystis jirovecii prophylaxis; KP, Kaiser Permanente; NQF, National Quality Forum; ART, antiretroviral therapy; DHHS, Department of Health and Human Services.

Quality measure development

The KP HIV quality measures were developed in 2006 through discussions of the KP HIV Interregional Initiative Steering Committee, which includes clinical and programmatic leadership (including researchers and data consultants) from all KP regions and Group Health Cooperative. The steering committee considered relevant information pertaining to known HIV gaps in care and quality outcomes, including: the need for earlier diagnosis, the association of HIV with other sexually transmitted infections, improved outcomes associated with earlier access and greater retention to care, the association of appropriate screening and prophylaxis with improved outcomes, the need for antiretroviral therapy (ART) at certain CD4 T-cell count levels, a need for high adherence to ART to ensure maximal viral control, and the importance of maximal HIV viral control to promote improved health outcomes. From these considerations, a set of HIV quality performance measures suitable for KP were established:

Diagnosing HIV

Testing for HIV among HIV-uninfected members diagnosed with sexually transmitted infection (new diagnosis of syphilis, gonorrhea, chlamydia, incident or prevalent hepatitis B or C).

Determining percent of new HIV diagnoses who met AIDS criteria (CD4 less than 200 per microliter) at diagnosis (late diagnosis).

Getting Patients into Care and Retention in Care

Time until newly diagnosed KP HIV-infected members first tested for CD4 count.

Percent of patients seen by HIV specialist at least twice annually (at least 60 days apart; Endorsed by National Quality Forum; 2007 only).

Care Processes

Percent of HIV-infected members with CD4⁺ cell count performed at least every 6 months.

Percent of HIV-infected members with CD4 less than 200 per microliter taking Pneumocystis jiroveci (PJP) prophylaxis.

Percent of HIV-infected members with CD4 less than 350 per microliter taking antiretroviral therapy.

Care Results

Percent of HIV-infected members on antiretroviral therapy who have appropriate antiretroviral medication adherence.

Percent of HIV-infected members on antiretroviral therapy who have maximal viral control.

Measures 4, 5, 6, 7, and 9 are included in the expert panel measures¹³; however, data regarding measures 4 and 7 were only collected in the 2007 dataset. See Table 1 for the numerator and denominator specifications for our measures.

Data collection

Electronic databases in all KP regions capture a variety of data including patient demographics, hospitalization and outpatient visit diagnoses, laboratory results, and medications dispensed. For each measure, an operational definition was first developed. We then determined which data would be required for the measure and in which database (pharmacy, laboratory, outpatient or inpatient diagnosis, and utilization) the required data resided. Data specifications were sent to the appropriate programmers with instructions for data transmission. For the California and Northwest regions, the data was derived from local databases and HIV registries and then sent to Northern California region for data synthesis. The other regions' data were derived centrally through a centralized data collection system. All data were subsequently combined into one dataset and performance on each quality measure was calculated.

Analysis

We analyzed data for the nine quality measures after finalizing the composite dataset. For the first time period, only the California, Georgia, and Northwest regions contributed data. Thus, for these regions we were able to compare the two time periods. Comparisons were made by either z test for proportional outcomes, or t test (parametric) or Wilcoxon signed-rank sum test (nonparametric) for continuous outcomes.

We did not perform risk adjusted analysis because we were assessing quality for the whole population. We felt the performance measures were applicable to the whole population for each measure's denominator, and the organization's performance should not be impacted by gender, age, race, or ethnicity. Prior KP data has shown that we do not differentiate by race/ethnicity with similar outcomes for white, black, and Latino populations.¹⁷

Funding was internal from Kaiser Permanente. However, the authors alone analyzed the data and take responsibility for all conclusions.

Results

In the 2005–2006 period, 13,064 HIV-infected patients were available for analysis from the four regions. From these same four regions, 13,861 patients were available for analysis in the 2007 time period. Incorporating the other two regions, we had 16,037 HIV-infected patients in total for evaluation in the 2007 time period (Table 2). Comparing the two time periods, there was not a statistically significant difference in terms of gender when the four regions only are considered (p = 0.59), but there is a statistically significant difference when all six regions are compared to the four regions only (p < 0.001), reflecting the higher prevalence of HIV infection in females in our East Coast regions, compared to our West Coast regions. We did not see a statistically significant difference in age (either mean or percent over 50 years of age) between time periods or regions.

Table 2.

Demographics

	Time period (and regions covered)
Variable	2005/2006 (4 regions)	2007 (4 regions)	2007 (6 regions)
Number	13,064	13,861	16,037
Gender:
Female (n, %)	1391 (10.6%)	1440 (10.4%)	2113 (13.2%)
Age:
Median (IQR)	46.0 (40, 53)	46.0 (40, 53)	46.0 (40, 53)
Percent 50 years or older	36.3%	37.7%	37.3%

IQR, interquartile range.

The results for the different quality measures are shown in Table 3. Looking first at testing for HIV, we see significant improvement (2.1–4.9% increases) in HIV testing rates in the four regions between the two time periods. These increases hold whether we are considering only hepatitis B or C incident cases or any newly diagnosed prevalent cases. We saw a statistically significant improvement (+3.5%; p < 0.001) in testing over the two time periods. In addition, we are diagnosing patients earlier as the percent of patients with CD4 less than 200 per microliter at time of HIV diagnosis decreased 6.6% for the four regions in the second time period (p = 0.04).

Table 3.

Metric Results

	Time period (and regions covered)
Variable	2005/2006 (4 regions)	2007 (4 regions)	2007 (6 regions)	p Value (all regions comparing both time periods)	p Value (4 regions comparing both time periods)
Diagnosing HIV
Testing for HIV with diagnosed STI (incident viral hepatitis diagnoses only)	55.8%	57.9%	55.2%	0.32	<0.001
Testing for HIV with diagnosed STI (all new hepatitis diagnoses)	39.9%	44.8%	43.4%	<0.001	<0.001
CD4 <200 at HIV diagnosis	28.8%	22.2%	27.1%	0.19	0.04
Getting Patients into Care and Retention in Care
In care within 90 days of new HIV diagnosis	92.2%	89.7%	88.6%	<0.001	0.009
Seen by HIV specialist or primary care at least twice annually	Not Measured	78.4%	76.8%	N/A	N/A
Care Processes
CD4⁺ count performed at least every 6 Months	76.2%	87.4%	86.3%	<0.001	<0.001
PJP prophylaxis if CD4 <200	71.0%	68.7%	68.9%	0.16	0.14
ART prescribed if CD4 <200	87.3%	86.5%	85.1%	0.13	0.18
ART prescribed if CD4 <350	Not consistent with guidelines	85.0%	83.5%	N/A	N/A
Outcome Measures
Median ART adherence	92.8%	94.2%	93.8%	<0.001^a	<0.001^a
ART adherence ≥90%	58.2%	63.5%	61.8%	<0.001	<0.001
HIV RNA below limits of quantification on ART	86.1%	89.8%	88.8%	<0.001	<0.001

Wilcoxon signed-rank test.

STI, sexually transmitted infection; PJP, Pneumocystis jiroveci prophylaxis; ART, antiretroviral therapy.

While we found a high percentage of newly identified HIV-infected patients (either newly diagnosed or new transfer into KP for care) had HIV-specific care established within 90 days, we did see a statistically significant, albeit small, decrease (2.5% for the four regions and 3.6% for all patients) in the two time periods. Over 75% of the patients were retained in care, as defined by the expert panel, during the 2007 time period.

Other care process measures were consistent between both time periods, either among the four regions only or in all patients. We found more than 80% of our patients did receive regular CD4⁺ count testing, with a statistically significant improvement over time. We also found more than 85% received an antiretroviral therapy prescription if their CD4⁺ count was less than 200 per microliter. Similar results were seen for the later time period if a 350 per microliter cutoff was employed. However, nearly 30% of our patients eligible for PJP prophylaxis were not given such medications, and the percent is even lower in the later time period (although not statistically significantly different).

Median antiretroviral therapy adherence (all drugs combined for the calendar period) was high (>90%) for both time periods and all comparisons (Fig. 1). Median adherence, though, rose significantly in the later time period compared to the earlier period (p < 0.001). The percent with more than 90% adherence was more than half for all time periods, and also significantly increased in the later time period. We found that the percent of patients with maximally controlled virus on antiretroviral therapy (<75 copies per milliliter at any time in the measurement period) was over 85% also for both time periods, and significantly increased in the later time period. Looking at the 2007 data only, we looked to see what the predictors of achieving maximal viral control were. While gender was not significant (p = 0.25), more than 90% antiretroviral adherence was associated with greater likelihood of maximal viral control (odds ratio [OR] = 2.66 [95% confidence interval: 2.34–3.02], p < 0.001), as well as older age to a lesser extent (OR = 1.38 [1.21–1.58], p < 0.001).

FIG. 1.

Adherence to antiretroviral therapy by group.

Discussion

Our results indicate that HIV quality measures can be derived and measured over time in a large integrated health care system. We ascertained our measurements electronically using common case definitions and specifications, which we synthesized for interpretation and action. Our results indicate more than 85% performance in connecting newly identified HIV-positive patients with care and more than 75% retention of those patients in care. We found greater than 75% twice-yearly CD4⁺ count testing and greater than 90% median adherence to HIV medications among our patients. These processes and outcomes led to more than 85% of our patients on antiretroviral therapy having HIV RNA levels below limits of quantification. Our outcome measures of antiretroviral adherence and maximal viral control appear to be significantly improving with time. There are potential areas for improvement, especially in HIV testing among patients at high risk for HIV infection and Pneumocystis jiroveci prophylaxis.

Quality performance measurement is of value internally if potential gaps in care are identified and rectified. Based on these results, we have instituted new HIV testing guidelines for our providers, specifically calling for HIV testing within 90 days of a new sexually transmitted infection or hepatitis diagnosis. Furthermore, at the local level, we are generating lists of patients who are not receiving at least two visits annually or not receiving appropriate testing and medications, as dictated by these measures and guidelines. These lists are provider specific and intended for use by our HIV care providers in order to improve the quality of clinical care. Tracking these measures over time will determine our success with these efforts. Of note, for our California regions, we have created Web-based interfaces with our HIV registries so that providers and care managers can determine in a more real-time basis which of their patients are in need of care.

Many of our measures were based on previously identified gaps in care in KP. As published previously, in the 1990s KP regions diagnosed patients later in their disease course than desirable; 43% met AIDS criteria at time of HIV diagnosis.¹⁸ The current data shows a 37% improvement by 2007 in earlier HIV diagnosis, compared to the previously published results. Current results also indicate that we are testing more patients for HIV when they have a sexually transmitted infection, although not yet at desirable rates.

As in other diseases, HIV practice guidelines change over time. One example of this change in guidelines is when patients should initiate antiretroviral therapy. When we first created our HIV performance quality measures, there was general agreement that patients with a CD4 less than 200 per microliter required antiretroviral therapy.¹⁹ However, between 2006 and 2007, this CD4⁺ criterion was raised to 350 cells per microliter.²⁰ Hence, we needed to change our quality performance measures. This performance measure may change as consideration is given to higher CD4⁺ cell counts for ART initiation,^3,21 given improved survival data recently published (which includes KP data).²² Where performance levels have remained greater than 85% but have decreased in the later time period (as in accessing care), we are placing a renewed emphasis on those measures.

Comparisons between KP's results and other health care systems can be difficult, as there is often no uniformity of measure specifications nor measurement periods. The measures developed for National Quality Forum, with endorsement by HRSA, AMA-PCPI, and NCQA, go a long way toward enabling useful comparisons.¹³ However, only the Veterans' Administration (VA) has published data utilizing these measure specifications.²³ Looking at those results, we demonstrate similar performance for PJP prophylaxis, CD4⁺ count monitoring, antiretroviral therapy prescription, and achieving maximal viral control. Even here, however, discrepancy in measure specification exists as the VA's viral control specifications are different than ours.²⁴

Other groups have published data that can be used for comparisons, despite lack of identical specifications. The Center for Disease Control and Prevention (CDC) notes that 32% of all HIV diagnoses in the United States meet AIDS criteria within 1 year of diagnosis²⁵ and the VA noted 31% met AIDS criteria at time of diagnosis²⁶; our results on these measures are lower. We have 89–92% (depending on time period) patients accessing care within 90 days; by comparison, Kaiser Family Foundation reports 50% of newly identified HIV-infected patients are not in care in one year and the CDC reports 28% are not in care in 1 year.^27,28 Extrapolating from data published from San Francisco municipal sexually transmitted disease (STD) clinics, 73.6% of their newly diagnosed identified HIV-infected patients were in care within 3 months, which is lower than our performance.²⁹ We postulate that our ability to get patients into care efficiently is due to our electronic health records (permitting tracking and quicker identification of patients with unique diagnoses) and identified HIV care teams in each medical center or region.

Retention in care is an important quality measure, because retention is necessary for delivering and monitoring HIV care and enabling consistent antiretroviral therapy administration. The recent emphasis on “test and treat” strategies for diagnosing and initiating therapy for HIV also require patients to be retained in care settings.³⁰ Our retention rate is comparable to the VA. The Kaiser Family Foundation and CDC report only 50%–72% being in care nationally, which is lower than our rate.^27,28 Again, this metric will benefit greatly from a standard definition, which will enable greater comparability across treatment platforms. We had 84%–87% rates of antiretroviral therapy initiation; by contrast, the CDC notes only 55% of HIV-infected person 15–49 years of age are on treatment, and the Institutes of Medicine approximate only 50% of HIV-positive Americans overall are on treatment.⁸ It is possible that much of the difference in antiretroviral utilization is related to our caring for an insured population, including public assistance HMO.

There are some quality measures for which finding comparison measurements is more difficult. HIV testing rates among Americans diagnosed with a sexually transmitted infection are not readily available. Similar to our results, only 40% of patients diagnosed with hepatitis B or C received HIV antibody testing in a general medicine clinic in East Harlem, New York, and only 40% had HIV cotesting among sexually transmitted infection (STI) diagnoses at Jackson Memorial Hospital in Miami.^31,32 We compared favorably to a study of commercial health plans that indicated only 9.5% of STI diagnosed patients had concurrent HIV antibody testing, although the infections included were not identical to ours.³³ Looking specifically at men having sex with men, CDC surveillance from a national sample of STD clinics found only 56% of those men visiting the clinic were tested for HIV—results similar to ours.³⁴ However, in general the CDC reports overall testing rates,³⁵ and not more targeted testing for populations at higher risk of HIV infection.

Maximal viral control is a clear goal of antiretroviral therapy.²¹ Adherence to antiretroviral therapy regimen is often the major predictor of achieving maximal viral control.^36,37 Our logistic regression analysis confirmed this finding. We also found that older age predicted maximal viral control, which we have reported previously.³⁸ However this is in contrast to other groups' data that failed to find a difference.^39
–41 For KP, these data are important because they indicate we should target adherence efforts at our younger HIV-positive patients when initiating antiretroviral therapy.

Our results have implications for the new U.S. domestic HIV/AIDS Strategy and the accompanying implementation plan, released in July 2010 by President Obama.^42,43 Ongoing assessment of achieving the goals of the strategy, including engagement in care and quality outcomes for HIV-positive Americans, is an essential component of the plan. To that end, the use of HIV quality metrics and the harmonization of those metrics across health care institutions are necessary. We demonstrate that electronic health records can be used to obtain such data with common definitions, and the results can be used to create meaningful analyses of such care programs. Furthermore, benchmarks could be created by which HIV care programs could be monitored. We do not establish benchmarks here, but we feel that such benchmarks should reflect the federal emphasis on meaningful use electronic health records and how they can be used to improve care and derive quality performance data.

We feel strongly, however, that our results are applicable to the greater U.S. HIV care population. Our patient demographics reflect the states we serve.^15,16 The HIV epidemic is different in various localities with regards to race/ethnicity, risk behavior, and other factors.²⁵ However, the standards for delivery of care, and the measurements of those standards, should not be influenced by these differences in patient demographics. Furthermore, although KP has high utilization of electronic health records that can support many of these measures, most medical practices in the United States are being required to include electronic health records in the near future. We do acknowledge a lack of cost data to further demonstrate a positive impact of our quality measures program, which may be disconcerting to some clinicians eager to initiate a similar program. However, previously described KP data shows over a $10,000 per year savings in total care costs for HIV-positive patients on their first or second antiretroviral regimen compared to patients on their third or later regimen.⁴⁴ It is reasonable to assume that greater adherence to these quality metrics would lead to fewer regimen changes and lower care costs in the longer term.

As with most quality measure analyses, we acknowledge some limitations. Using our HIV case definition, our registries have reported previously greater than 95% sensitivity and specificity to identifying cases.^45
–47 Still, there may be some HIV-positive patients not captured by our case finding. Furthermore, it is possible our antiretroviral prescription and adherence rates could be underreported if patients do not receive their HIV medications from KP pharmacies. Also, it is possible that some patients diagnosed with an STI received an HIV antibody test within the prescribed time period but not within KP, which would not be captured by our data systems.

The expert panel includes other HIV quality metrics that we did not include.¹³ Specifically, we did not include in these reported measures annual STI screening, hepatitis B screening/immunization, and hepatitis C screening. These are worthy measures, which we hope to include them in the future as we continue to monitor our performance on the present measures.

In conclusion, we demonstrated that a large integrated health care system can systematically assess quality performance for HIV testing, access and retention in care, and HIV process and outcome measures. Striving for continued improved performance should lead to improved patient outcomes, including survival. Further, with harmonization of HIV quality performance measures across platforms (public and private institutions), comparisons between institutions and health insurance plans can be performed and changes evaluated. These measures have implications for quality improvement and national HIV/AIDS strategies and care processes.

Footnotes

Acknowledgments

The authors wish to thank Amanda Charbonneau and Courtney Ellis for their assistance with manuscript preparation.

Author Disclosure Statement

No competing financial interests exist.

References

Press

, Tyndall

, Wood

, Hogg

, Montaner

. Virologic and immunologic response, clinical progression, and highly active antiretroviral therapy adherence. J Acquir Immune Defic Syndr, 2002; 31,Suppl 3:S112–117.

Simoni

, Frick

, Pantalone

, Turner

. Antiretroviral adherence interventions: A review of current literature and ongoing studies. Top HIV Med, 2003; 11:185–198.

Aberg

, Kaplan

, Libman

et al. Primary care guidelines for the management of persons infected with human immunodeficiency virus: 2009 update by the HIV medicine Association of the Infectious Diseases Society of America. Clin Infect Dis, 2009; 49:651–681.

Guidelines for Prevention and Treatment of Opportunistic Infections in HIV-Infected Adults and Adolescents. MMWR, 2008; 58:1–207.

Ashman

, Conviser

, Pounds

. Associations between HIV-positive individuals' receipt of ancillary services and medical care receipt and retention. AIDS Care, 2002; 14,Suppl 1:S109–118.

Chesney

. Factors affecting adherence to antiretroviral therapy. Clin Infect Dis, 2000; 30,Suppl 2:S171–176.

Messeri

, Abramson

, Aidala

, Lee

. The impact of ancillary HIV services on engagement in medical care in New York City. AIDS Care, 2002; 14,Suppl 1:S15–29.

Institute of Medicine of the National Academies. Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act. Washington, D.C.: Institute of Medicine, 2004.

The HIV/AIDS Program: HAB Performance Measures. 2009. http://hab.hrsa.gov/special/habmeasures.htm. 2009 December 21.

10.

Agins

. HIVQUAL-US: Mission and History, 2010. www.hivqual.org/index.cfm/8018. 2010 February 23.

11.

Bozzette

, Phillips

, Asch

et al. Quality Enhancement Research Initiative for human immunodeficiency virus/;acquired immunodeficiency syndrome: Framework and plan. HIV-QUERI Executive Committee. Med Care, 2000; 38,6 Suppl 1:I60–69.

12.

Office of Medical Director New York State. New York State HIV Quality of Care Program, 2009. www.hivguidelines.org/Content.aspx?pageID = 9. 2009 December 21.

13.

Horberg

, Aberg

, Cheever

, Renner

, O'Brien

Kaleba E

, Asch

. Development of national and multiagency HIV care quality measures. Clin Infect Dis, 2010; 51:732–738.

14.

Horberg

. HIV Quality Measures in Kaiser Permanente. Paper presented at NCQA 4th Annual National Policy Conference, 2009 Washington, D.C..

15.

Krieger

. Overcoming the absence of socioeconomic data in medical records: Validation and application of a census-based methodology. Am J Public Health, 1992; 82:703–710.

16.

California Department of Health Services OoA. California AIDS Surveillance Report, Cumulative Cases as of December 31, 2005. 2006.

17.

Silverberg

, Leyden

, Quesenberry

Jr. , Horberg

. Race/ethnicity and risk of AIDS and death among HIV-infected patients with access to care. J Gen Intern Med, 2009; 24:1065–1072.

18.

Klein

, Hurley

, Merrill

, Quesenberry

Jr.

Review of medical encounters in the 5 years before a diagnosis of HIV-1 infection: Implications for early detection. J Acquir Immune Defic Syndr, 2003; 32:143–152.

19.

Guidelines for the Use of Antiretroviral Agents in HIV-1-Infected Adults. Adolescents. Washington, D.C.: Department of Health and Human Services, 2005.

20.

Panel on Antiretroviral Guidelines for Adult and Adolescents. Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents. Department of Health and Human Services. ed2008:1–128.

21.

Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the Use of Antiretroviral Agents in HIV-1-Infected Adults and Adolescents. Department of Health and Human ServicesDepartment of Health and Human Services2009; 1–161.

22.

Kitahata

, Gange

, Abraham

et al. Effect of early versus deferred antiretroviral therapy for HIV on survival. N Engl J Med, 2009; 360:1815–1826.

23.

Backus

, Boothroyd

, Phillips

et al. National quality forum performance measures for HIV/AIDS care: The Department of Veterans Affairs' experience. Arch Intern Med, 2010; 170:1239–1246.

24.

Backus

. HIV quality measures: VHA experience. American Medical Association Physician Consortium for Performance Improvement Quality in Action Webinar Series 2010. www.ama-assn.org/ama1/pub/upload/mm/370/lisa-backus.pdf. 2010 September 23.

25.

Center for Disease Control and Prevention. HIV/AIDS Surveillance Report. Center for Disease Control and Prevention, Vol 202008. www.cdc.gov/hiv/surveillance/resources/reports/2008report/index.htm.

26.

Valdiserri

. 2009. State of Care for Veterans with HIV/AIDS. US Department of Veterans Affairs, ed2010. www.hiv.va.gov/vahiv?page=prin-care-00.

27.

Rudy

, Mahoney-Anderson

, Loughlin

et al. Perceptions of human immunodeficiency virus (HIV) testing services among HIV-positive persons not in medical care. Sex Transm Dis, 2005; 32:207–213.

28.

Reed

, Hanson

, McNaghten

et al. HIV testing factors associated with delayed entry into HIV medical care among HIV-infected persons from eighteen states, United States, 2000–2004. AIDS Patient Care STDs, 2009; 23:765–773.

29.

Zetola

, Bernstein

, Ahrens

et al. Using surveillance data to monitor entry into care of newly diagnosed HIV-infected persons: San Francisco, 2006–2007. BMC Public Health, 2009; 9:17.

30.

Mugavero

, Davila

, Nevin

, Giordano

. From access to engagement: Measuring retention in outpatient HIV clinical care. AIDS Patient Care STDs, 2010; 24:607–613.

31.

Krain

, Wisnivesky

, Garland

, McGinn

. Prevalence of human immunodeficiency virus testing in patients with hepatitis B and C infection. Mayo Clin Proc, 2004; 79:51–56.

32.

Russell

. Social Determinants of Health (SDH) in Relation to Healthcare Reform and the National HIV/AIDS Strategy. Paper presented at: 40th Full Council Meeting of the Presidential Advisory Council on HIV/AIDS. Washington, D.C. 2010 October 1.

33.

Tao

, Zhang

. HIV testing of commercially insured patients diagnosed with sexually transmitted diseases. Sex Transm Dis, 2008; 35:43–46.

34.

Helms

, Weinstock

, Mahle

et al. HIV testing frequency among men who have sex with men attending sexually transmitted disease clinics: Implications for HIV prevention and surveillance. J Acquir Immune Defic Syndr, 2009; 50:320–326.

35.

Branson

, Handsfield

, Lampe

et al. Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Recomm Rep, 2006; 55:1–17quiz CE11–14.

36.

Chesney

. Adherence to HAART regimens. AIDS Patient Care STDs, 2003; 17:169–177.

37.

Horberg

, Silverberg

, Hurley

, Delorenze

, Quesenberry

. Influence of prior antiretroviral experience on adherence and responses to new highly active antiretroviral therapy regimens. AIDS Patient Care STDs, 2008; 22:301–312.

38.

Silverberg

, Leyden

, Horberg

, DeLorenze

, Klein

, Quesenberry

Jr.

Older age and the response to and tolerability of antiretroviral therapy. Arch Intern Med, 2007; 167:684–691.

39.

Grimes

, Otiniano

, Rodriguez-Barradas

, Lai

. Clinical experience with human immunodeficiency virus-infected older patients in the era of effective antiretroviral therapy. Clin Infect Dis, 2002; 34:1530–1533.

40.

Knobel

, Guelar

, Carmona

et al. Virologic outcome and predictors of virologic failure of highly active antiretroviral therapy containing protease inhibitors. AIDS Patient Care STDs, 2001; 15:193–199.

41.

Knobel

, Guelar

, Valldecillo

et al. Response to highly active antiretroviral therapy in HIV-infected patients aged 60 years or older after 24 months follow-up. AIDS, 2001; 15:1591–1593.

42.

The White House Office of National AIDS Policy. National HIV/AIDS Strategy for the United States. The White House Office of National AIDS Policy, ed2010. www.aids.gov/federal-resources/policies/national-hiv-aids-strategy.

43.

The White House Office of National AIDS Policy. National HIV/AIDS Strategy Federal Implementation Plan. The White House Office of National AIDS Policy, ed2010. www.aids.gov/federal-resources/policies/national-hiv-aids-strategy.

44.

Meenan

, O'Keeffe

Rosetti M

, Kimes

et al.

Excess Prevalence-Based Costs of Multiple HAART Switches among HIV+ Patients in a US Health Maintenance Organization [Abstract TUPE0236]

XVII International AIDS Conference. Mexico City, Mexico, 2006.

45.

Horberg

, Klein

, Hurley

et al. Efficacy and safety of ritonavir-boosted and unboosted atazanavir among antiretroviral-naive patients. HIV Clin Trials, 2008; 9:367–374.

46.

Horberg

, Tang

, Towner

et al. Impact of tenofovir on renal function in HIV-infected, antiretroviral-naive patients. J Acquir Immune Defic Syndr, 1999; 53:62–69.

47.

Horberg

, Silverberg

, Hurley

et al. Effects of depression and selective serotonin reuptake inhibitor use on adherence to highly active antiretroviral therapy and on clinical outcomes in HIV-infected patients. J Acquir Immune Defic Syndr, 2008; 47:384–390.