Comparison of Genotypic and Phenotypic HIV Type 1 Tropism Assay: Results from the Screening Samples of Cenicriviroc Study 202,a Randomized Phase II Trial in Treatment-Naive Subjects

Abstract

Cenicriviroc is a once-daily oral CCR5/CCR2 antagonist in development for treatment of HIV infection. CVC Study 202 (652-2-202; NCT01338883) excluded treatment-naive subjects demonstrated to harbor non-R5 (CXCR4-tropic or dual-mixed) tropic HIV-1 by either genotypic or phenotypic tropism testing. Here we compare the results of genotypic and phenotypic tropism testing in Study 202. A total of 304 subjects screened had paired genotypic and phenotypic results. Genotypic tropism testing (GTT) incorporated triplicate population sequencing using the geno2pheno algorithm and the PSSM algorithm, followed by ultradeep sequencing (UDS) for samples with R5 results. All samples were further evaluated with a phenotypic test, the enhanced-sensitivity Trofile assay (ESTA). Concordance between GTT and ESTA was 80% and increased to 84% when only geno2pheno was used for triplicate population sequencing. GTT (geno2pheno) classified 18% of the samples as non-R5 compared to 16% by ESTA. Only one-third of samples with non-R5 results by either test were classified as non-R5 by both tests. Median CD4(⁺) cell counts were lower in patients with concordant non-R5 results by UDS and ESTA than in subjects with an R5 result by either assay (p=0.0004). UDS detected non-R5 virus in an additional 27/304 subjects (median 15% non-R5, interquartile range: 3.7–62%) with R5 results by ESTA. In conclusion, the geno2pheno algorithm improves concordance of GTT with a clinically validated phenotypic tropism assay as does the use of UDS. These findings provide support for recent guidelines indicating that genotypic tropism testing may be considered as an alternative to phenotypic testing.

Introduction

Human immunodeficiency virus type 1 (HIV-1) infects cells through interaction with the CD4 receptor and one of two coreceptors, CCR5 or CXCR4.^1
–3 CCR5 coreceptor-using virus (R5 virus) predominates in 80–90% of recently infected and treatment-naive HIV-1 patients, while dual-tropic or mixed (D/M) populations of R5 virus and CXCR4 coreceptor-using virus (X4 virus) are found in up to 50% of late-stage and treatment-experienced patients.^4

–10 The presence of D/M or X4 virus is associated with lower CD4⁺ T cell counts, a higher viral load, and a more rapid progression to AIDS.^4,7,11,12 Small-molecule CCR5 antagonists such as maraviroc and cenicriviroc can effectively inhibit the interaction of R5 HIV-1 with the CCR5 coreceptor.^3,13 In clinical trials, maraviroc has been shown to be an effective antiretroviral agent in patients harboring R5 virus exclusively but did not sufficiently control viral load in patients harboring D/M or X4 virus.^14

–17 An HIV-1 tropism test is thus required prior to maraviroc administration to exclude patients harboring non-R5 virus from treatment with this drug (maraviroc package insert: www.viivhealthcare.com/∼/media/Files/G/GlaxoSmithKline-Plc/pdfs/us_selzentry.pdf).

A recombinant phenotypic tropism assay, the original Trofile assay (Monogram Biosciences, South San Francisco, CA),¹⁸ was used to screen patients for CCR5 antagonist eligibility in a number of clinical trials.^{14

–17,19
–21} This assay can reliably detect non-R5 virus accounting for at least 10% of the viral population in mixtures of R5 and non-R5 DNA clones.¹⁸ The second-generation, enhanced-sensitivity Trofile assay (ESTA) is more sensitive, demonstrating a lower limit of sensitivity of 0.3% for non-R5 virus in mixed DNA clones.²² In retrospective reanalyses of several clinical trials, ESTA demonstrated a better ability to detect minority X4 viral variants in D/M samples and an improved ability to predict clinical outcome as a function of tropism status when compared to the original Trofile assay.^14,23,24

Tropism screening was also integral to patient selection in clinical trials of vicriviroc, a CCR5 antagonist whose development was halted in 2010.^20,21 The original Trofile assay was used to screen patients for enrollment in clinical trials of vicriviroc. When the ACTG A5211 trial was retrospectively rescreened with the more sensitive ESTA, subjects who were reclassified as harboring D/M virus demonstrated a poorer virologic response,²⁴ again indicating that a more sensitive tropism assay is a better tool to screen patients for CCR5 eligibility.

Although sensitive tropism analysis seems crucial for evaluating eligibility for therapy with CCR5 antagonists, phenotypic testing is expensive and labor intensive and usually requires several weeks to report results. Thus, there is increased interest in alternative genotypic tropism testing (GTT) methods.²⁵ GTT methods that utilize Sanger DNA population sequencing and bioinformatic analysis of the HIV-1 envelope third variable loop (V3)²⁶ have a demonstrated ability comparable to that of the original Trofile assay for predicting clinical outcome when used to screen treatment-experienced patients for therapy with CCR5 antagonists.^27

–30 The sensitivity of population sequencing for detecting non-R5 variants may be improved by performing reverse transcription, polymerase chain reaction (PCR) amplification, and sequencing in triplicate, thereby increasing the sampling efficiency for minority non-R5 variants.^31,32 McGovern and colleagues used a triplicate population sequencing (TPS) tropism test to retrospectively rescreen the samples from treatment-naive patients in the Maraviroc versus Efavirenz Regimens as Initial Therapy (MERIT) study. Virologic outcomes for R5 subjects in the twice-daily maraviroc arm were noninferior to those in the efavirenz arm of the study, regardless of whether ESTA or TPS was used for screening.²⁸

The clinical performance of genotypic methods that provide even greater sensitivity for minority non-R5 variants in D/M samples has also been evaluated. Massively parallel ultradeep sequencing (UDS) methods³³ utilize clonal analysis of thousands of variants per sample and have a demonstrated ability to sensitively detect minority variants in mixed viral populations.^34

–37 UDS-based tropism assays have been used to retrospectively rescreen samples from several clinical trials of maraviroc, including the Maraviroc versus Optimized Therapy in Viremic Antiretroviral Treatment-Experienced Patients (MOTIVATE) and A4001029 trials for treatment-experienced patients.^36,38,39 Compared to ESTA, the UDS tropism assay yielded positive and negative predictive values for virologic response similar to maraviroc as a function of tropism status.³⁸ Similar results were obtained when the MERIT trial samples were rescreened with UDS: the proportion of subjects predicted to respond for maraviroc was similar with the UDS and ESTA methods.⁴⁰ Taken together, these findings suggest that GTT may be a viable option for assessing eligibility for CCR5 antagonist therapy.

Cenicriviroc (CVC) is a dual CCR5/CCR2 antagonist with potent in vitro anti-HIV activity that is currently being developed as a novel once-daily, orally administered therapy for HIV-1.^41
–43 In early clinical trials, treatment-experienced patients naive to CCR5 antagonists who received CVC monotherapy for 10 days experienced significant reductions in plasma viral load (pVL), and the drug was generally well-tolerated.⁴² CVC was also evaluated in a phase 2b study (CVC Study 202) for treatment-naive subjects: patients were screened using both genotypic and phenotypic tropism tests, and those with non-R5 results on either were excluded (see http://clinicaltrials.gov/show/NCT01338883). In this work, we assessed the concordance of GTT utilizing TPS combined with UDS, with the phenotypic ESTA in the screening samples for CVC Study 202.

Materials and Methods

Study design

Study 202 (652-2-202; NCT01338883) is a randomized, double-blind, double-dummy, 48-week comparative study evaluating efficacy, safety, and tolerability of 100 or 200 mg doses of CVC in combination with emtricitabine/tenofovir disoproxil fumarate (FTC/TDF) or efavirenz (EFV) plus FTC/TDF in HIV-1-infected, antiretroviral treatment-naive, adult patients with only CCR5-tropic virus. The study enrolled a total of 143 subjects, including 115 subjects randomized to either of the two CVC arms and 28 subjects to the EFV arm.

Subject selection criteria

Antiretroviral treatment-naive men and women (age ≥18 years) with an HIV-1 viral load of ≥1,000 copies/ml and a CD4(⁺) T cell count of ≥200 cells/μl were included in the study. Subjects were excluded if they had any evidence of genotypic primary HIV-1 antiretroviral drug resistance mutations or phenotypic resistance to TDF, FTC, or EFV. Both genotypic and phenotypic tropism screening were conducted for all subjects; those with evidence of X4 or D/M-tropic virus using either method were excluded from the study. Subjects with evidence of HIV-2, hepatitis B, or hepatitis C infection were also excluded. See http://clinicaltrials.gov/ct2/show/NCT01338883 for a more extensive list of subject inclusion and exclusion criteria. Study 202 is an ongoing, multicenter, multiinvestigator study conducted in the United States and Puerto Rico and approved by central or local institutional review boards or ethics committees at each study site. Written informed consent was obtained from all study participants.

Tropism screening methods

GTT using a commercial genotypic tropism assay was performed by Quest Diagnostics Nichols Institute, San Juan Capistrano, CA. This test uses a reflex strategy whereby TPS is followed by UDS if TPS results indicate only R5 virus, as described previously (Fig. 1).³⁸ Subjects with non-R5 results by TPS or UDS were excluded from the study. Phenotypic tropism testing was also performed for all study subjects with ESTA, a commercial tropism assay performed by Monogram Biosciences (South San Francisco, CA).²² If the ESTA result was nonreportable, but subsequently successful with a second sample, then the result from the later time point was used.

FIG. 1.

Cenicriviroc (CVC) study 202 screening tropism determination. Subjects with an R5 tropism result by genotypic tropism testing (GTT) and by the enhanced-sensitivity Trofile assay (ESTA) were eligible for inclusion in study 202 (see Materials and Methods).

Tropism determination by triplicate population sequencing

TPS was performed as described previously.³⁸ Viral RNA was extracted from 0.5 ml plasma on a MagNAPure LC automated extraction system (Roche Diagnostics Corp., Indianapolis, IN). Reverse transcription, primary and nested PCR amplification of the envelope V3 loop and DNA sequencing using an ABI 3730XL DNA Analyzer (Applied Biosystems, Foster City, CA) were performed in triplicate as described.^36,38,44 PCR primers contained Roche/454 sequencing adaptors and 10-nucleotide molecular identifiers to allow for sample pooling prior to UDS. Base calling and sequence assembly were performed automatically with ReCALL software as described.^38,44,45 Tropism assignment for the triplicate V3 loop sequences was performed with the geno2pheno algorithm (http://coreceptor.bioinf.mpi-inf.mpg.de)⁴⁶ and the PSSM algorithm (http://fortinbras.us/cgi-bin/fssm/fssm.pl).⁴⁷ A geno2pheno false-positive rate (FPR) cutoff of 5.75% was used, and sequences with FPR ≤5.75% were classified as non-R5. For PSSM interpretation, the PSSM_x4r5 scoring matrix was used, and sequences scoring ≥−4.75 were classified as non-R5. Subjects with a non-R5 result by either geno2pheno or PSSM were classified as non-R5 for the purpose of excluding subjects from Study 202. Comparative analyses of the individual geno2pheno and PSSM tropism results were also performed.

UDS tropism determination

PCR amplicons were barcoded and up to 16 samples and controls were pooled in each run. Emulsion PCR, bead enrichment, and sequencing on the Roche/454 GS Junior analyzer using Titanium sequencing chemistry (Roche/454 Life Sciences, Branford, CT) was performed according to the manufacturer's instructions and as described.³⁸ UDS reads were analyzed bioinformatically as described.³⁸ Tropism was assigned using the Geno2pheno 454 tool (http://g2p-454.bioinf.mpi-inf.mpg.de/index.php), with FPR scores of ≤3.5% indicating a non-R5 result. If ≥2% of the reads were scored as non-R5 then the final result was reported as non-R5.^38,39 Although we used a strategy of reflexing samples only from TPS R5-tropic subjects to UDS to determine subject eligibility, UDS was subsequently performed for all subjects for the purpose of correlation analysis.

Statistical analysis

Descriptive and univariate statistical analyses were performed with Analyze-it for Microsoft Excel, version 2.30 (Analyse-it Software, Ltd., www.analyse-it.com). Multivariate logistic regression analysis incorporating the CD4⁺ and viral load covariates that were significant in the univariate analysis as shown in Table 1 was performed using SAS version 9.2 (Cary, NC).

Table 1.

Baseline Characteristics of Study Subjects

	Groups (UDS/ESTA) ^a
	Group 1	Group 2	Group 3	Group 4
Characteristic	R5/R5	R5/non-R5	non-R5/R5	non-R5/non-R5	All subjects	p-value ^b
N	229	22	27	26	304
Median age (min, max)	35 (18,65)	32 (22,50)	32 (19,51)	37.5 (21,63)	34 (18,65)	NS
Gender
Male (%)	212 (93)	19 (86)	27 (100)	23 (88)	281 (92)	NS
Race
White (%)	131 (57)	13 (59)	15 (56)	13 (50)	172 (57)	NS
African-American (%)	76 (33)	9 (41)	9 (33)	11 (42)	105 (35)
Other (%)	22 (10)	0 (0)	3 (11)	2 (8)	27 (9)
HIV-1 subtype^c
Subtype B (%)	219 (96)	21 (95)	27 (100)	25 (96)	292 (96)	NS
V3 loop length (%)
105 bases	187 (82)	15 (68)	24 (89)	22 (85)	248 (82)	NS
≠ 105 bases	42 (18)	7 (32)	3 (11)	4 (15)	56 (18)
Immunologic and virologic values
Median (IQR) baseline CD4⁺ cells/μl^d	377 (289,491)	384 (294,378)	390 (292,538)	255 (178,347)	367 (275,489)	0.0004/.03^e
Median (IQR) log₁₀ baseline viral load (copies/ml)^d	4.5 (4.0,4.9)	4.3 (4.1,4.7)	4.2 (3.7,4.6)	4.9 (4.4,5.1)	4.5 (4.0,4.9)	0.0031/.13^e

Subject characteristics were grouped according to R5 or non-R5 concordance between the UDS tropism test and ESTA.

Differences in categorical variables (gender, race, subtype, V3 loop length) among groups 1–4 were assessed with the Chi square test. Differences in rank for continuous variables (age, median CD4⁺ cells, and log₁₀ viral load at baseline) among groups 1–4 were assessed with the Kruskal–Wallis test; differences were significant after Bonferroni correction for the latter three for each of the R5-containing groups contrasted to the non-R5 concordant group. NS, not significant (p>0.05).

The HIV-1 subtype was determined with geno2pheno software from the envelope V3 loop sequence.

Baseline values for CD4 and HIV-1 RNA for subjects in Groups 2, 3, and 4 are screening values carried forward as no baseline values are available for subjects found to be ineligible to participate in the study due to the presence of CXCR4-using variants.

Univariate/multivariate p-values. The differences in median log₁₀ baseline viral load between groups 1 and 4 were not significant after correcting for baseline CD4⁺ counts in a logistic regression model.

UDS, ultradeep sequencing; ESTA, enhanced-sensitivity Trofile assay.

Results

Baseline characteristics of screened subjects

A total of 392 subjects were screened. Tropism results were nonreportable for 21 (5.4%) of the genotypically tested samples and 55 (14%) of the samples tested by ESTA. Paired genotypic and phenotypic tropism results were available for 304 of the 392 subjects and are included in this comparison. The baseline characteristics of the 304 subjects, stratified by UDS and phenotypic tropism assay concordance status, are shown in Table 1. There was no significant association between age, gender, race, or HIV-1 subtype and the degree of concordance between the genotypic and phenotypic tropism test results (Table 1). Subjects with concordant non-R5 virus results (Table 1, group 4) had lower baseline CD4(⁺) T cell counts, and higher baseline viral load, than subjects with R5 results for one or both assays but only baseline CD4⁺ was significantly associated with concordant R5 results in a multivariate analysis (Table 1).

GTT (TPS with reflex UDS) compared to ESTA

Most (255/304; 84%) of the clinical samples yielded three amplified replicates; 27 (9%) samples yielded two amplified replicates, and the remaining 22 samples yielded only one. Of the samples with two or more replicates, 5% of the samples (14/282) had a discordant tropism classification by the geno2pheno algorithm for at least one replicate.

Most subjects were found to have exclusively R5 virus by either the GTT or ESTA (Table 2). The reflex GTT classified a higher proportion of subjects as non-R5 (25%) than did ESTA (16%). Almost a third (31%) of subjects were classified as non-R5 by at least one of the assays, excluding them from Study 202 (Table 2). The overall concordance between the GTT with the combined geno2pheno and PSSM_x4r5 algorithms and ESTA was 80% with 62/304 samples yielding discordant results (Table 2, GTT g+p reflex test). Among the 304 subjects, 45/62 discordants (73%) were classified as non-R5 by the reflex GTT but R5 by ESTA, versus 17/62 (27%) as non-R5 by ESTA but R5 by genotype (OR: 2.6; McNemar: p=0.0005).

Table 2.

Concordance of Genotypic Tropism Testing (GTT) with Enhanced-Sensitivity Trofile Assay (ESTA)

	Reference method: ESTA
	Non-R5 (%)
Test method (N=304)	Single assay	Test method+ESTA	Concordance (%)	Sensitivity^a (%)	Specificity^b (%)	Kappa^c	McNemar (OR, p)^d
ESTA	16	—	—	—	—	—	—	—
GTT g+p reflex^e	25	31	80	65	82	0.38	2.6	0.0005
GTT 5.75 reflex^f	18	25	84	56	89	0.42	1.4	0.3
UDS	17	25	84	54	90	0.42	1.3	0.6
TPS^g	13	22	86	46	93	0.42	0.7	0.3

Sensitivity is defined as concordant non-R5s between the test and reference methods/reference method non-R5s.

Specificity is defined as concordant R5s between the test and reference methods/reference method R5s.

Kappa: a measure of interrater agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement.

McNemar odds ratio: ratio of discordant non-R5_test R5_reference/R5_test non-R5_reference. p values<0.05 are shown in bold.

GTT with combined g2p_5.75 and PSSM_x4r5 −4.75 analysis. If an R5 result was obtained by g2p_5.75 or PSSM_x4r5 −4.75 then the UDS result was used to determine tropism. This reflex strategy was the actual one used to exclude subjects from the 202 Study.

If an R5 result was obtained by g2p_5.75 then the UDS result was used to determine tropism.

TPS used the geno2pheno_5.75 cutoff.

UDS, ultradeep sequencing; TPS, triplicate population sequencing.

We next examined whether the TPS algorithm choice would influence concordance of GTT with ESTA. In a recent study comparing GTT to ESTA for predicting clinical response to maraviroc in treatment-experienced clinical trial subjects, the PSSM algorithm accounted for a significant number of discordant predictions in which PSSM yielded a non-R5 result in samples with an R5 result by ESTA. However, the virologic responses in these subjects were comparable to those of other R5 subjects.³⁸ We thus performed a second comparison using only the geno2pheno algorithm to predict tropism for the TPS component (Table 2). Using this approach, concordance with ESTA increased to 84%; the reflex GTT classified 18% (56/304) of subjects as non-R5 compared to 16% (48/304) for ESTA. However, only 27 (9%) of all 304 samples, and 27 (35%) of the 77 samples that were non-R5 samples by either method, were classified as non-R5 by both methods. Slightly more subjects were classified as non-R5 by GTT and R5 by ESTA (29/50 discordants) than as R5 by GTT and non-R5 by ESTA (21/50 discordants). However, in contrast to the results obtained from the original GTT that included the PSSM classification, this difference was not statistically significant (McNemar odds ratio=1.4; p=0.32).

TPS compared to ESTA and UDS

Although UDS and ESTA have greater sensitivity than TPS for detecting minority non-R5 virus,³⁸ the concordance of TPS tropism predictions with ESTA (Table 2) and with UDS (Table 3) was above 80%. ROC curve analysis of the geno2pheno FPR cutoff relative to ESTA gave an AUC of 0.78, representing a fair level of accuracy (Supplementary Fig. S1; Supplementary Data are available online at www.liebertpub.com/aid). The geno2pheno algorithm with a 10% cutoff recommended by the European tropism testing guidelines⁴⁸ classified additional subjects as non-R5 relative to the 5.75% cutoff, increasing the sensitivity of TPS relative to ESTA but at the expense of reduced specificity and concordance (Supplementary Table S1). The geno2pheno algorithm using the 5.75% cutoff was 94% concordant with UDS (Table 3) whereas the 10% cutoff was 89% concordant with UDS but classified more subjects as non-R5 relative to UDS (Supplementary Table S1). V3 sequences with geno2pheno scores less than the TPS FPR cutoff (5.75%) but greater than the UDS FPR cutoff (3.5%) resulted in a small number of TPS/UDS discordant tropism assignments. Three subjects were classified as non-R5 by TPS but R5 by UDS with a geno2pheno TPS score of 5.3% in two of these three cases (Table 4).

Table 3.

Concordance of Enhanced-Sensitivity Trofile Assay (ESTA) and Triplicate Population Sequencing (TPS) with Ultradeep Sequencing

	Reference method: UDS
Test method	Concordance (%)	Sensitivity^a (%)	Specificity^b (%)	Kappa^c	McNemar (OR, p)^d
UDS	—	—	—	—	—	—
ESTA	84	49	91	0.42	0.8	0.6
TPS^e	94	70	99	0.75	0.19	0.004

Sensitivity is defined as concordant non-R5s between the test and reference methods/reference method non-R5s.

Specificity is defined as concordant R5 results between the test and reference methods/reference method R5s.

Kappa: a measure of interrater agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement.

McNemar odds ratio: ratio of discordant non-R5_test R5_reference/R5_test non-R5_reference.

TPS used the geno2pheno_5.75 cutoff.

Table 4.

Proportion of Sequence Reads Classified as Non-R5 by Ultradeep Sequencing (UDS) as a Function of Concordance with Enhanced-Sensitivity Trofile Assay (ESTA) and Triplicate Population Sequencing (TPS)

	ESTA/UDS ^a			TPS ^b /UDS ^a
Tropism result	N	Median (IQR) % non-R5 reads on UDS	Median (IQR) non-R5 pVL ^c	N	Median (IQR) % non-R5 reads on UDS	Median (IQR) non-R5 pVL ^c
R5/R5	229	0.0% (0.0%, 0.2%)	NA	248	0.0% (0.0%, 0.2%)	NA
R5/non-R5	27^d	15% (3.7%, 62%)	3.1 (2.7, 4.0)	16^e	13% (3.2%, 23%)	3.3 (2.9, 4.2)
Non-R5/R5	22	0.02% (0.0%, 0.4%)	NA	3	0.7% (—, —)	NA
Non-R5/non-R5	26	39% (14%, 92%)	4.4 (3.9, 4.9)	37	40% (14%, 93%)	4.1 (3.1, 4.7)

A median of 3,563 reads (IQR: 2,362, 4,670) per sample was obtained by UDS.

TPS scored with the geno2pheno_5.75 cutoff.

pVL is the log₁₀ plasma viral load multiplied by the proportion of non-R5 virus determined by UDS.

11/27 samples had ≥20% non-R5 variants by UDS.

7/16 sample had <10% non-R5.

UDS, ultradeep sequencing; ESTA, enhanced-sensitivity Trofile assay; TPS, triplicate population sequencing; R5, IQR, interquartile range; CCR5 tropic virus, non-R5, CXCR4, or dual/mixed tropic virus.

UDS compared to ESTA

UDS alone was 84% concordant with ESTA, and the methods both classified 16–17% of subjects as non-R5 (Table 2). ROC curve analysis of the UDS percent non-R5 cutoff relative to ESTA gave an AUC of 0.75, representing a fair level of accuracy (Supplementary Fig. S1). Overall, 25% (75/304) of the subjects were classified as non-R5 by either UDS or ESTA. Of these samples, 35% (26/75) were classified as non-R5 by both assays, representing 9% (26/304) of all samples. The sensitivity of UDS relative to ESTA for detecting non-R5 variants was 54% (Table 2), and the sensitivity of ESTA relative to UDS was 49% (Table 3). Samples with concordant non-R5 UDS and ESTA results had a significantly higher proportion of non-R5 UDS reads (39% vs. 15%) and a higher non-R5 pVL than did discordant samples with non-R5 results by UDS and R5 results by ESTA (Table 4; Mann-Whitney p<0.0005).

We also examined the differences in the proportion of non-R5 variants detected in TPS/UDS concordant and discordant samples. UDS had a median of 40% non-R5 UDS reads (median non-R5 pVL: 4.1 log copies) for TPS/UDS concordant samples (Table 4). For 27 discordant samples with non-R5 results by UDS but R5 results by TPS, the non-R5 UDS reads accounted for a median of only 13% of total reads (median non-R5 pVL: 3.3 log copies) (Table 4).

ESTA classified an additional 22/304 subjects as non-R5 who were classified as R5 by UDS, representing 7.2% of the study samples (Table 4). The median proportion of non-R5 UDS reads in this subgroup was 0.02%.

V3 loop length variation

V3 loop length variation has been identified as a potential source of disagreement between genotypic tropism prediction algorithms and between genotypic tropism prediction and phenotypic assays.⁴⁹ We found that 18% (56/304) of samples had a V3 loop length that was not the standard 35 codons ( ≠105 nucleotides; Table 1) and 53/56 (95%) V3 loops had a single-codon deletion and a length of 102 nucleotides. Of V3 loop length variants 7/56 (12%) and 11/56 (20%) were classified as non-R5 by ESTA and UDS, respectively, compared to 37/248 (15%) and 52/248 (21%) of the standard length V3 loops. Although a higher proportion (7/22; 32%) of the UDS R5 and ESTA non-R5 discordant samples (Table 1, Group 2) had a V3 loop with a deletion relative to the concordant non-R5 samples (Table 1, Group 4; 4/26, 15%), this difference did not reach statistical significance (Fisher exact test, p=0.31).

Discussion

The CVC phase 2b Study 202 used a rigorous tropism screening strategy that employed both GTT (TPS with a reflex to UDS) and a sensitive phenotypic tropism test to exclude patients with non-R5 virus. This screening strategy allowed us to compare the relative ability of commercially available genotypic and phenotypic tropism assays to identify treatment-naive subjects who are candidates for CCR5 antagonist-based therapy. In this study, we utilized a genotypic reflex method whereby triplicate amplification and population sequencing of the V3 loop are performed initially to rapidly identify many subjects harboring non-R5 virus, followed by UDS for more sensitive detection of low-abundance non-R5 variants. The inclusion of UDS analysis of all samples for the purposes of this comparison, including those classified as non-R5 by TPS, also allowed us to separately assess the performance of TPS and UDS relative to ESTA.

In general, replicate V3 loop population sequencing has been found to improve the sensitivity of GTTs for detecting minority non-R5 variants.^32,36,50 In the present study triplicate amplification likely increased the number of samples that could be successfully tested, as some samples yielded only one or two of the three replicates. In some samples, a non-R5 replicate was evident for only one or two of the three replicates and might not have been detected had the assay been performed only in singlicate.

The genotypic screening method used for Study 202, which combined the geno2pheno and PSSM algorithms for TPS, followed by UDS, had an overall concordance of 80% with ESTA. Compared to ESTA this approach identified significantly more subjects with non-R5 virus. When the GTT reflex test was combined with ESTA, approximately one-third of the treatment-naive subjects screened were excluded from the study (Table 2). In a previous retrospective analysis of treatment-experienced subjects, the PSSM algorithm was found to misclassify a significant number of ESTA and UDS R5 subjects as non-R5, reducing the specificity of the test.³⁸ In the current study, exclusion of the PSSM algorithm from the reflex GTT improved the concordance with ESTA and classified a comparable number of samples as non-R5. When combined with phenotypic screening, GTT using the geno2pheno algorithm alone classified approximately one-quarter of subjects from Study 202 as non-R5, compared to one-third of subjects who were actually excluded from the study using the combined geno2pheno and PSSM algorithm followed by ESTA. It is probable, therefore, that inclusion of the geno2pheno algorithm alone in a TPS+UDS reflex testing method is sufficient and may improve the specificity of tropism determinations. Indeed, the current commercially available reflex GTT no longer includes the PSSM algorithm and employs the geno2pheno algorithm with a 5.75% FPR in the TPS portion of the assay; R5 results are confirmed with UDS.

Interestingly, only one-third of the samples classified as non-R5 by either the reflex GTT or ESTA were classified as non-R5 by both methods, indicating that the sensitivity of each assay relative to the other was low. Similar results were obtained in the comparison of UDS to ESTA in the MERIT study, where only 15% of the 146 samples classified as non-R5 by either method were classified as such by both methods, and subjects with discordant genotype/ESTA tropism predictions achieved an intermediate virologic response between the concordant R5s and concordant non-R5s.⁴⁰ The direction of the discordance did not matter, suggesting that neither assay was superior to the other for predicting virologic outcome in patients treated with maraviroc. Similarly, when TPS alone was compared to ESTA for MERIT subjects receiving maraviroc twice daily, discordant subjects achieved an intermediate virologic response regardless of the direction of the discordance.²⁸ The global concordance between UDS and ESTA in this study was similar to that obtained in the much larger MERIT study (82% vs. 84%).⁴⁰

A similar comparison between ESTA and population sequencing or UDS was conducted for the A4001078 study of treatment-naive patients receiving maraviroc coadministered with either atazanavir/ritonavir or tenofovir/emtricitibine.⁵¹ Concordance with ESTA was high for both population sequencing and UDS.

In a retrospective reanalysis of a subset of samples from the MOTIVATE and A4001029 trials using the same assays described here, UDS and the reflex test had the same ability to predict virologic response to maraviroc as ESTA, in spite of differences in sensitivity of each assay relative to the other.³⁸ The similarity between the tropism screening results in these independent studies is encouraging, and suggests that genotypic tropism screening is reproducible in different settings.

Although tropism determination is important to determine eligibility for CCR5 antagonists and has been shown to impact clinical response to CCR5 antagonists in both treatment-naive and treatment-experienced patients, it is important to note that CCR5 antagonists are only one component of modern antiretroviral therapy. In the MOTIVATE trials, for example, the impact of tropism assignments diminished as the number of active agents in the background regimen increased.¹⁵ Therefore no tropism assay will be entirely predictive of virologic response to CCR5 antagonists.

Population sequencing alone may not be sufficiently sensitive to consistently detect non-R5 virus, particularly in D/M samples from treatment-experienced patients. While the TPS assay used in this study has a limit of detection of approximately 20% non-R5 virus,³⁸ a reanalysis of over 1,800 samples in the MOTIVATE-1, MOTIVATE-2, and A4001029 trials determined that a 2% cutoff for non-R5 virus was optimal for discriminating responders from nonresponders by UDS.³⁹ TPS alone using the geno2pheno algorithm had lower sensitivity for non-R5 variants relative to ESTA or UDS. In Study 202, the proportion of non-R5 virus determined by UDS was typically below the limit of detection for TPS when TPS and UDS were discordant, confirming that UDS increases the overall sensitivity for non-R5 variants compared to TPS.

We also examined a number of factors that may have influenced tropism status or concordance between GTT and ESTA. We found that lower baseline CD4⁺ T cell levels were significantly associated with concordant non-R5 tropism status. The association between non-R5 virus and low CD4⁺ cell counts has been reported elsewhere for both treatment-naive and treatment-experienced cohorts.^8,9,23 Interestingly, CD4⁺ cell counts were significantly lower only for the UDS/ESTA concordant non-R5 group, whereas the CD4⁺ cell counts were not significantly lower for subjects classified as non-R5 by only one assay (Table 1). No other baseline factors were found to be associated with tropism status. In some studies, V3 loop length variation was found to contribute to discordance between genotypic and phenotypic tropism predictions.⁴⁹ Although more UDS R5/ESTA non-R5 discordant samples had a single codon deletion in the present study, the difference did not reach statistical significance.

This study design had certain limitations. First, due to the mandatory exclusion of subjects with either a genotypic or a phenotypic non-R5 result, it was not possible to evaluate the relative ability of each assay to predict virologic response to CVC as a function of tropism status. Therefore, this analysis was limited to evaluating the performance of the genotypic tropism determination methods used relative to ESTA. Study 202 is ongoing and further analyses will be conducted to evaluate the tropism status and emergence of viral resistance variants in protocol-defined virologic failures. Future studies may also allow the evaluation of screening using only GTT as a tool to select appropriate candidates for CVC-based therapy.

Second, the tropism FPR cutoff of 5.75% for geno2pheno used in TPS and the FPR of 3.5% for 2% or more of the UDS reads were optimized in the treatment-experienced populations of the MOTIVATE and A4001029 studies in relation to the CCR5 antagonist maraviroc.^27,36 It is possible that alternative cutoffs could improve the predictive values of GTTs for treatment-naive subjects or that these cutoffs are suboptimal for cenicriviroc, a different CCR5 antagonist. We evaluated a geno2pheno FPR of 10%, which is also commonly used for tropism assays.⁴⁸ This higher FPR improved the sensitivity of the TPS relative to either ESTA or UDS, but reduced the specificity (Supplementary Table S1). ROC analysis (Supplementary Figs. S1–S3) of the FPR cutoffs used in TPS and of the 2% non-R5 proportion cutoff for UDS illustrated the relative tradeoffs of assay sensitivity and specificity relative to ESTA. However, given the absence of virologic outcome data for Study 202, it is not feasible to assess the clinical significance of alternative cutoffs. Nevertheless, the cutoffs employed in the current study have performed well in the retrospective reanalysis of the larger MERIT study treatment-naive population^28,40 with respect to predicting clinical outcome, suggesting that they are reasonable choices for the tropism classification of treatment-naive subjects.

A third limitation is that almost all (96%) of the subjects in this study harbored subtype B virus, potentially limiting the applicability of our conclusions for other HIV-1 subtypes. Although the vast majority of HIV-1 infections in the United States are subtype B, subtype C accounts for nearly 50% of HIV infections worldwide.⁵² A study of tropism prediction for HIV-1 subtype C found a high level of concordance between genotypic methods and a phenotypic tropism assay.⁵³ Additionally, 40% of the subjects in the MERIT study harbored non-B virus, and UDS and ESTA both had a similar ability to predict virologic outcome for these non-B subjects.⁴⁰ A recent study designed to assess both population sequencing and UDS tropism determination methods for a large number of non-B subtypes found similar levels of concordance to subtype B determinations with the original Trofile assay.⁵⁴ Subtype D had a lower specificity for non-R5 predictions, whereas subtype A had lower sensitivity for non-R5 predictions by population sequencing but not by UDS. Future prediction algorithm improvements may improve the performance of genotypic tropism predictions for other subtypes including subtype D and CRF02_AG.^55,56 GTT is likely to be appropriate for most patients harboring non-B virus, although additional data may be needed for rare subtypes.

GTT is widely used in Europe and is currently recommended by the European tropism testing guidelines.⁴⁸ In the United States, GTT (TPS with reflex to UDS) is commercially available and may be performed more rapidly than phenotypic testing. The cost of UDS is generally greater than that of population sequencing. However, the availability of benchtop instruments such as the GS Junior from Roche/454 used in this study, as well as instruments from other providers and the ability to pool samples in a single run through the use of DNA barcodes, has greatly reduced the cost per sample.³⁸ Consequently, GTT that includes TPS and reflex to UDS may be offered at a significantly lower cost than phenotypic testing. The updated U.S. Department of Health and Human Services Panel on Antiretroviral Guidelines for Adults and Adolescents has indicated that a genotypic tropism assay can be used as an alternative to a phenotypic tropism assay before initiation of a CCR5 antagonist-containing regimen.⁵⁷ Future clinical studies may further evaluate the role of GTT to determine cenicriviroc eligibility.

Sequence Data

UDS sequence data (SFF files) and population sequence data (fasta files) for this study will be available from the LANL HIV-DB Next Generation Sequence Archive (www.hiv.lanl.gov/content/sequence/HIV/NextGenArchive).

Footnotes

Acknowledgments

The authors would like to thank the CVC Study 202 participants, investigators, site personnel, and the Tobira study team.

These data were presented in part at the International Workshop on HIV and Hepatitis Virus Drug Resistance and Curative Strategies, Sitges, Spain, June 5–9 2012 (abstract 131).

Author Disclosure Statement

R.M.K., E.P.J., M.F.S., J.L.P., and R.L.P. are employed by Quest Diagnostics, a diagnostic testing company that offers diagnostic tests for HIV. E.L. and R.O. are employed by Tobira Therapeutics, a drug development company that is developing CVC for HIV therapy. B.VB. is an independent consultant employed by Tobira Therapeutics, Inc.

References

Berger

, Murphy

, and Farber

: Chemokine receptors as HIV-1 coreceptors: roles in viral entry, tropism, and disease. Annu Rev Immunol, 1999; 17:657–700.

Goodenow

and Collman

: HIV-1 coreceptor preference is distinct from target cell tropism: A dual-parameter nomenclature to define viral phenotypes. J Leukoc Biol, 2006; 80(5):965–972.

Ray

and Doms

: HIV-1 coreceptors and their inhibitors. Curr Top Microbiol Immunol, 2006; 303:97–120.

Connor

, Sheridan

, Ceradini

, Choe

, and Landau

: Change in coreceptor use correlates with disease progression in HIV-1-infected individuals. J Exp Med, 1997; 185(4):621–628.

Irlbeck

, Amrine-Madsen

, Kitrinos

, et al.: Chemokine (C-C motif) receptor 5-using envelopes predominate in dual/mixed-tropic HIV from the plasma of drug-naive individuals. AIDS, 2008; 22(12):1425–1431.

Shankarappa

, Margolick

, Gange

, et al.: Consistent viral evolutionary changes associated with the progression of human immunodeficiency virus type 1 infection. J Virol, 1999; 73(12):10489–10502.

Poveda

, Briz

, de Mendoza

, et al.: Prevalence of X4 tropic HIV-1 variants in patients with differences in disease stage and exposure to antiretroviral therapy. J Med Virol, 2007; 79(8):1040–1046.

Brumme

, Goodrich

, Mayer

, et al.: Molecular and clinical epidemiology of CXCR4-using HIV-1 in a large population of antiretroviral-naive individuals. J Infect Dis, 2005; 192(3):466–474.

Chalmet

, Dauwe

, Foquet

, et al.: Presence of CXCR4-using HIV-1 in patients with recently diagnosed infection: Correlates and evidence for transmission. J Infect Dis, 2012; 205(2):174–184.

10.

Moyle

, Wildfire

, Mandalia

, et al.: Epidemiology and predictive factors for chemokine receptor use in HIV-1 infection. J Infect Dis, 2005; 191(6):866–872.

11.

Daar

, Kesler

, Petropoulos

, et al.: Baseline HIV type 1 coreceptor tropism predicts disease progression. Clin Infect Dis, 2007; 45(5):643–649.

12.

Weiser

, Philpott

, Klimkait

, et al.: HIV-1 coreceptor usage and CXCR4-specific viral load predict clinical disease progression during combination antiretroviral therapy. AIDS, 2008; 22(4):469–479.

13.

Dorr

, Westby

, Dobbs

, et al.: Maraviroc (UK-427,857), a potent, orally bioavailable, and selective small-molecule inhibitor of chemokine receptor CCR5 with broad-spectrum anti-human immunodeficiency virus type 1 activity. Antimicrob Agents Chemother, 2005; 49(11):4721–4732.

14.

Cooper

, Heera

, Goodrich

, et al.: Maraviroc versus efavirenz, both in combination with zidovudine-lamivudine, for the treatment of antiretroviral-naive subjects with CCR5-tropic HIV-1 infection. J Infect Dis, 2010; 201(6):803–813.

15.

Fatkenheuer

, Nelson

, Lazzarin

, et al.: Subgroup analyses of maraviroc in previously treated R5 HIV-1 infection. N Engl J Med, 2008; 359(14):1442–1455.

16.

Gulick

, Lalezari

, Goodrich

, et al.: Maraviroc for previously treated patients with R5 HIV-1 infection. N Engl J Med, 2008; 359(14):1429–1441.

17.

Saag

, Goodrich

, Fatkenheuer

, et al.: A double-blind, placebo-controlled trial of maraviroc in treatment-experienced patients infected with non-R5 HIV-1. J Infect Dis, 2009; 199(11):1638–1647.

18.

Whitcomb

, Huang

, Fransen

, et al.: Development and characterization of a novel single-cycle recombinant-virus assay to determine human immunodeficiency virus type 1 coreceptor tropism. Antimicrob Agents Chemother, 2007; 51(2):566–575.

19.

Gulick

, Su

, Flexner

, et al.: Phase 2 study of the safety and efficacy of vicriviroc, a CCR5 inhibitor, in HIV-1-infected, treatment-experienced patients: AIDS clinical trials group 5211. J Infect Dis, 2007; 196(2):304–312.

20.

Suleiman

, Zingman

, Diaz

, et al.: Vicriviroc in combination therapy with an optimized regimen for treatment-experienced subjects: 48-week results of the VICTOR-E1 phase 2 trial. J Infect Dis, 2010; 201(4):590–599.

21.

Wilkin

, Su

, Krambrink

, et al.: Three-year safety and efficacy of vicriviroc, a CCR5 antagonist, in HIV-1-infected treatment-experienced patients. J Acquir Immune Defic Syndr, 2010; 54(5):470–476.

22.

Reeves

, Coakley

, Petropoulos

, and Whitcomb

: An enhanced-sensitivity Trofile™ HIV coreceptor tropism assay for selecting patients for therapy with entry inhibitors targeting CCR5: A review of analytical and clinical studies. J Viral Entry, 2009; 3(3):94–102.

23.

Wilkin

, Goetz

, Leduc

, et al.: Reanalysis of coreceptor tropism in HIV-1-infected adults using a phenotypic assay with enhanced sensitivity. Clin Infect Dis, 2011; 52(7):925–928.

24.

, Gulick

, Krambrink

, et al.: Response to vicriviroc in treatment-experienced subjects, as determined by an enhanced-sensitivity coreceptor tropism assay: Reanalysis of AIDS clinical trials group A5211. J Infect Dis, 2009; 200(11):1724–1728.

25.

Harrigan

and Geretti

: Genotypic tropism testing: Evidence-based or leap of faith?. AIDS, 2011; 25(2):257–264.

26.

Hwang

, Boyle

, Lyerly

, and Cullen

: Identification of the envelope V3 loop as the primary determinant of cell tropism in HIV-1. Science, 1991; 253(5015):71–74.

27.

McGovern

, Thielen

, Mo

, et al.: Population-based V3 genotypic tropism assay: A retrospective analysis using screening samples from the A4001029 and MOTIVATE studies. AIDS, 2010; 24(16):2517–2525.

28.

McGovern

, Thielen

, Portsmouth

, et al.: Population-based sequencing of the V3-loop can predict the virological response to maraviroc in treatment-naive patients of the MERIT trial. J Acquir Immune Defic Syndr, 2012; 61(3):279–286.

29.

Prosperi

, Bracciale

, Fabbiani

, et al.: Comparative determination of HIV-1 co-receptor tropism by enhanced sensitivity Trofile, gp120 V3-loop RNA and DNA genotyping. Retrovirology, 2010; 7:56.

30.

Recordon-Pinson

, Soulie

, Flandre

, et al.: Evaluation of the genotypic prediction of HIV-1 coreceptor use versus a phenotypic assay and correlation with the virological response to maraviroc: The ANRS GenoTropism study. Antimicrob Agents Chemother, 2010; 54(8):3335–3340.

31.

Symons

, Vandekerckhove

, Paredes

, et al.: Impact of triplicate testing on HIV genotypic tropism prediction in routine clinical practice. Clin Microbiol Infect, 2012; 18(6):606–612.

32.

Knapp

, McGovern

, Dong

, et al.: Factors influencing the sensitivity and specificity of conventional sequencing in human immunodeficiency virus type 1 tropism testing. J Clin Microbiol, 2013; 51(2):444–451.

33.

Margulies

, Egholm

, Altman

, et al.: Genome sequencing in microfabricated high-density picolitre reactors. Nature, 2005; 437(7057):376–380.

34.

Archer

, Braverman

, Taillon

, et al.: Detection of low-frequency pretherapy chemokine (CXC motif) receptor 4 (CXCR4)-using HIV-1 with ultra-deep pyrosequencing. AIDS, 2009; 23(10):1209–1218.

35.

Daumer

, Kaiser

, Klein

, Lengauer

, Thiele

, and Thielen

: Genotypic tropism testing by massively parallel sequencing: Qualitative and quantitative analysis. BMC Med Inform Decis Mak, 2011; 11:30.

36.

Swenson

, Moores

, Low

, et al.: Improved detection of CXCR4-using HIV by V3 genotyping: Application of population-based and “deep” sequencing to plasma RNA and proviral DNA. J Acquir Immune Defic Syndr, 2010; 54(5):506–510.

37.

Wang

, Mitsuya

, Gharizadeh

, Ronaghi

, and Shafer

: Characterization of mutation spectra with ultra-deep pyrosequencing: application to HIV-1 drug resistance. Genome Res, 2007; 17(8):1195–1201.

38.

Kagan

, Johnson

, Siaw

, et al.: A genotypic test for HIV-1 tropism combining Sanger sequencing with ultradeep sequencing predicts virologic response in treatment-experienced patients. PLoS One, 2012; 7(9):e46334.

39.

Swenson

, Mo

, Dong

, et al.: Deep sequencing to infer HIV-1 co-receptor usage: application to three clinical trials of maraviroc in treatment-experienced patients. J Infect Dis, 2011; 203(2):237–245.

40.

Swenson

, Mo

, Dong

, et al.: Deep V3 sequencing for HIV type 1 tropism in treatment-naive patients: A reanalysis of the MERIT trial of maraviroc. Clin Infect Dis, 2011; 53(7):732–742.

41.

Klibanov

, Williams

, and Iler

: Cenicriviroc, an orally active CCR5 antagonist for the potential treatment of HIV infection. Curr Opin Investig Drugs, 2010; 11(8):940–950.

42.

Lalezari

, Gathe

, Brinson

, et al.: Safety, efficacy, and pharmacokinetics of TBR-652, a CCR5/CCR2 antagonist, in HIV-1-infected, treatment-experienced, CCR5 antagonist-naive subjects. J Acquir Immune Defic Syndr, 2011; 57(2):118–125.

43.

Marier

, Trinh

, Pheng

, Palleja

, and Martin

: Pharmacokinetics and pharmacodynamics of TBR-652, a novel CCR5 antagonist, in HIV-1-infected, antiretroviral treatment-experienced, CCR5 antagonist-naive patients. Antimicrob Agents Chemother, 2011; 55(6):2768–2774.

44.

McGovern

, Harrigan

, and Swenson

: Genotypic inference of HIV-1 tropism using population-based sequencing of V3. J Visual Exp: JoVE, 2010(46).

45.

Woods

, Brumme

, Liu

, et al.: Automating HIV drug resistance genotyping with RECall, a freely accessible sequence analysis tool. J Clin Microbiol, 2012; 50(6):1936–1942.

46.

Sing

, Low

, Beerenwinkel

, et al.: Predicting HIV coreceptor usage on the basis of genetic and clinical covariates. Antivir Ther, 2007; 12(7):1097–1106.

47.

Jensen

, Li

, van 't Wout

, et al.: Improved coreceptor usage prediction and genotypic monitoring of R5-to-X4 transition by motif analysis of human immunodeficiency virus type 1 env V3 loop sequences. J Virol, 2003; 77(24):13376–13388.

48.

Vandekerckhove

, Wensing

, Kaiser

, et al.: European guidelines on the clinical management of HIV-1 tropism testing. Lancet Infect Dis, 2011; 11(5):394–407.

49.

Seclen

, Soriano

, Gonzalez

, Gomez

, Thielen

, and Poveda

: High concordance between the position-specific scoring matrix and geno2pheno algorithms for genotypic interpretation of HIV-1 tropism: V3 length as the major cause of disagreement. J Clin Microbiol, 2011; 49(9):3380–3382.

50.

Raymond

, Recordon-Pinson

, Saliou

, et al.: Improved V3 genotyping with duplicate PCR amplification for determining HIV-1 tropism. J Antimicrob Chemother, 2011; 66(9):1972–1975.

51.

Portsmouth

, Valluri

, Daumer

, et al.: Correlation between genotypic (V3 population sequencing) and phenotypic (Trofile ES) methods of characterizing co-receptor usage of HIV-1 from 200 treatment-naive HIV patients screened for Study A4001078. Antiviral Res, 2013; 97(1):60–65.

52.

Buonaguro

, Tornesello

, and Buonaguro

: Human immunodeficiency virus type 1 subtype distribution in the worldwide epidemic: Pathogenetic and therapeutic implications. J Virol, 2007; 81(19):10209–10219.

53.

Raymond

, Delobel

, Mavigner

, et al.: Prediction of HIV type 1 subtype C tropism by genotypic algorithms built from subtype B viruses. J Acquir Immune Defic Syndr, 2010; 53(2):167–175.

54.

Lee

, Harrigan

, Dong

, et al.: Comparison of population and 454 “deep” sequence analysis for HIV-1 tropism versus the original Trofile assay in non-B subtypes. AIDS Res Hum Retroviruses, 2013; 29(6):979–984.

55.

Raymond

, Delobel

, Chaix

, et al.: Genotypic prediction of HIV-1 subtype D tropism. Retrovirology, 2011; 8:56.

56.

Raymond

, Delobel

, Mavigner

, et al.: Genotypic prediction of human immunodeficiency virus type 1 CRF02-AG tropism. J Clin Microbiol, 2009; 47(7):2292–2294.

57.

Panel on Antiretroviral Guidelines for Adults and Adolescents: Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents. Department of Health and Human Services. Available at http://aidsinfo.nih.gov/ContentFiles/AdultandAdolescentGL.pdf. Accessed February 12, 2013 .

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB

0.15 MB

0.03 MB