In Vitro Restoration of Th17 Response During HIV Infection with an Antiretroviral Drug and Th17 Differentiation Cytokines

Abstract

The Th17 subset is preferentially depleted as compared to the Th1 subset in chronically HIV-infected patients, even after successful antiretroviral therapy. In this study, we have established an in vitro system utilizing primary human CD4 T cell cultures that recapitulates the dramatic loss of Th17 response upon HIV-1 infection that is accompanied with a less profound Th1 decrease. With this experimental system, we showed that blocking viral entry with CCR5 ligands or TAK779 reduced the infection and enhanced Th17 response but not Th1 response. Antiretroviral drug 3TC (lamivudine), given at the time of infection, completely prevented the loss of Th17 and Th1 responses but was ineffective when given after infection was already established. Only when Th17 differentiation cytokines were given along with 3TC to the cultures with established HIV infection was Th17 response fully restored and virus replication kept suppressed. Finally, a significant increase of Th17 response was achieved in peripheral lymphocytes of HIV-infected patients on antiretroviral therapy after treatment with Th17 differentiation cytokines. These data demonstrate the presence of CD4 T cells remaining capable of mounting Th17 response during HIV infection and indicate the potential use of immunotherapeutic modalities to supplement antiretroviral drugs for restoring Th17 response in chronically HIV-infected patients.

Introduction

O ne of the key features of HIV infection is the depletion of CD4 T cells and a dramatic immune suppression that ultimately leads to AIDS. During the early stages of HIV infection, CD4 T cells are first affected in the gut, where the percentages of CD4 T cells decrease a few weeks following infection.^1,2 Since the mucosal barrier is severely compromised, there is an increased translocation of bacterial products such as lipopolysaccharide (LPS), which may play a role in the activation and expansion of CD4 T cells that become potential targets for the virus.³ However, recent studies indicate that not all CD4 T cell subsets are equally affected by HIV,^4,5 although the molecular basis for the differences is not fully understood.

CD4 T cells differentiate into distinct functional subsets. For many years, only the two main lineages, Th1 and Th2, were much studied.⁶ Recently, several other functional T cells subsets have been reported including Treg, Th17, Th9, and Th22.^7

–10 Among these subsets, Th17 cells have taken center stage as the main players in Crohn's disease, psoriasis, and multiple sclerosis.¹¹ The main transcription factor that distinguishes the Th17 subset is the RORγt and, albeit not exclusively, these CD4 T cells express the surface markers CCR6, CD161, and interleukin (IL)-23R.^7,12
–14 Several cytokines have been found to be important in the differentiation of naive CD4 T cells into mature Th17 cells, including tumor growth factor (TGF)-β, IL-6, IL-1β, IL-23, and to some extent IL-21, which also functions in an autocrine fashion.^15,16 As the predominant subset in the gut-associated lymphoid tissues, Th17 cells produce IL-17, IL-21, and IL-22.^17,18 IL-17A stimulates the chemotaxis of immune cells such as neutrophils and, together with IL-22, plays a key role in the maintenance of tight junctions as well as the survival and regeneration of the gut epithelial cells, whereas IL-21 promotes more IL-17A synthesis and secretion.¹⁹

During HIV infection, Th17 cells have been shown to be preferentially depleted in the gut and to a lesser extent in the periphery. Brenchley et al. show a dramatic decrease of Th17 response along with a decrease of IL-23R⁺ cells in the gut in chronically infected patients.⁵ According to Gosselin et al., a higher frequency of viral DNA is found among the Th17 subset bearing both CCR6 and CCR4 indicating that these cells are more susceptible to HIV infection.⁴ However, this issue remains controversial and not all functional Th17 populations in different anatomical compartments may be preferentially infected.^5,20 Even following long-term antiretroviral therapy (ART), the CD4 T lymphocytes are not fully restored in the gut mucosa in all the patients.^21,22 According to Macal et al., the percentage of IL-17A-producing CD4 T cells increased in the gut of some HIV-infected patients after long-term ART.²² However, it is not clear why ART can restore the percentage of Th17 cells in the intestinal mucosa of some patients and not in others. In the pathogenic SIV/macaque models, Th17 cells are also depleted from the gut of infected animals and these cells disappear at the very early stages of infection.^23,24 Since α₄β₇ has been described as an HIV receptor, one potential explanation that has been offered for the increased susceptibility of Th17 cells to SIV infection is their increased α₄β₇ expression.^25,26 Parallel to the human findings, ART cannot completely restore Th17 cells in the intestinal mucosa of chronically SIV-infected macaques.²⁷ Given the key role that Th17 cells play in immunity, a therapy that fully restores the Th17 cells in all patients should offer significant benefits.

A recent study indicates that loss of Th17 cells may not be simply due to direct infection by the virus but also a result of bystander cell death induced by elevated activity of indoleamine 2,3-dioxygenase (IDO) during HIV infection.²⁸ IDO, which is found in antigen-presenting cells and activated after Toll-like receptor signaling, is the rate-limiting enzyme in the tryptophan catabolism. The accumulation of tryptophan catabolites favors the differentiation of Treg cells over Th17 cells.²⁸ In addition, viral proteins such as gp120, Tat, Nef, or Vpr have also been shown to induce CD4 T cell apoptosis independent of any viral infection.^29

–32 Irrespective of the mechanisms, due to the importance of Th17 cells for maintaining the integrity of gut mucosal barrier, Th17 depletion during HIV or SIV infection likely contributes to increased translocation of bacterial products across the gut mucosa. In support with this notion, Brenchley et al. showed an increased level of LPS in plasma of HIV-infected patients.³ In the SIV/macaque model, a concurrent Salmonella infection, which in healthy animals stimulates many Th17-related genes, was able to spread from the gut to the mesenteric lymph nodes; and this effect correlated with Th17 loss in the gut.³³ Nevertheless, very little has been studied to explore strategies that may be employed to prevent Th17 loss or to restore the response when Th17 suppression has occurred during HIV infection.

Given that Th17 cells play such an important role in immunity against pathogens and maintenance of the mucosal barrier, and are significantly affected during HIV infection, we established an in vitro primary cell culture system to investigate systematically the differential effects of HIV infection on the Th1 and Th17 subsets. With this experimental system, we were able to reproduce the greater loss of the Th17 response as compared to Th1 response in HIV-infected CD4 T cell cultures. This culture system was subsequently utilized for testing HIV inhibitors and Th17 differentiation cytokines for the capacity to block or restore the Th17 depletion induced by active HIV replication. Finally, peripheral lymphocytes from HIV-infected patients on ART were tested for their responsiveness to Th17 differentiation cytokines. These studies reveal novel findings indicating the potential usage of immunotherapeutic strategies to expand and reconstitute the Th17 response in HIV-infected patients with virus-suppressive ART.

Materials and Methods

Cells and study subjects

Peripheral blood mononuclear cells (PBMCs) from healthy HIV-seronegative donors were isolated from leukopacks (New York Blood Center, Long Island City, NY), while PBMCs of HIV-seropositive patients were obtained from whole blood of volunteers referred to us by an ACT2 clinical trial screener affiliated with the NYU Center for AIDS Research (CFAR). The study was reviewed and approved by the VA NY Harbor Healthcare System Institutional Review Board and all human volunteers gave written informed consent. Viral load and CD4 T cell counts of all HIV-seropositive donors were assessed at each visit.

Cells isolation and infection

PBMCs were isolated from leukopacks or whole blood by Histopaque centrifugation. Subsequently, CD4 T cells were enriched with the untouched CD4 T cell isolation kit (Invitrogen Dynal AS, Oslo, Norway). CD4 T cells were stimulated on 24-well plates precoated with anti-CD3 and anti-CD28 antibodies (eBiosciences, San Diego, CA) in complete medium (RPMI, 10% FBS, l-glutamine 2 nM, and penicillin 50 U/ml) and incubated at 37°C, 5% CO₂. After 48 h, the cells were washed, resuspended in complete medium (4×10⁶/ml), and infected with R5-tropic HIV-1 primary isolate 97ZA009. For some experiments, other HIV-1 isolates [DJ263, BX08 (R5 tropic), BZ167 (dual tropic), MN, or IIIB (X4 tropic)] were also tested. Virus stock (50–500 ng/ml) was pelleted by centrifugation at 15,500 rpm, 60 min, 4°C, and then resuspended in 100 μl of complete medium prior to addition to cells. After 3–4 h, cells were washed and cultured in complete medium plus 20 U/ml of IL-2 (Roche Diagnostics, Indianapolis, IN). For controls, uninfected cells of the same donors were maintained in parallel under the same conditions.

For some experiments, cultures were also treated with the Th17 differentiation cytokine cocktail made of IL-1β, IL-6, IL-23 (10 ng/ml each, eBiosciences), and TGF-β (2 ng/ml, R and D Systems, Minneapolis, MN) and/or 3TC (lamivudine) (5 μg/ml), while virus infection was done in the presence of CCR5 ligands [MIP-1β, MIP-1α, RANTES (R & D Systems) at 100 ng/ml each] or TAK779 (5 μg/ml). To inactivate virus, we used the protocol of Gorny et al.³⁴ in which virus was incubated at 37°C, with 220 μg/ml aldrithiol-2 (AT-2) (Sigma-Adrich, St. Louis, MO) for 1 h and pelleted by centrifugation as indicated above. Virus inactivation was confirmed by negative viral-DNA PCR in cultures incubated with the AT-2-treated virus.

Intracellular staining assay and flow cytometry

CD4 T cells from the different cultures were collected at the designated days and tested for cytokine production and virus infection in the intracellular staining assay as previously performed.³⁵ Surface and intracellular staining was done using fluorescent antibodies against the following antigens: CD3 (APC-Cy7), CD8 (PERCp-Cy5.5 or APC), IFN-γ (PE-Cy7), CCR6 (PE-Cy7), CD161 (FITC) (BD Biosciences), IL-17A (APC or PERCP-Cy5.5) (eBiosciences), p24 (FITC; clone KC57) (Beckman Coulter, Hialeah, FL), and IL-23R (PE) (R and D Systems). Sample acquisition (250,000 events/sample) was done in a FACSCanto and data were analyzed with FlowJo (version 7.1). Intracellular staining of ex vivo PBMCs from HIV-seropositive subjects was done using the same protocol as above.

To measure cell proliferation in the cultures, cells were stained with CFSE (Invitrogen) on day 0 according to the manufacturer's protocol, and analyzed by flow cytometry at the designated day. Cells were also analyzed for apoptosis by PE-conjugated Annexin V (BD Biosciences) binding.

Statistical analysis

Statistical analyses were done with Graph Pad Prism or by programming in the statistical language R (Free Software Foundation). Two-tailed paired t-test or nonparametric Exact Wilcoxon Rank Sum test was used.

Results

Detrimental effects of HIV-1 infection on Th17 and Th1 responses

To better understand helper CD4 T cell suppression commonly observed in HIV-infected subjects, we established a culture system in which a mixed population of naive and memory CD4 T cells purified from PBMCs of HIV-seronegative donors was activated with anti-CD3 and anti-CD28 antibodies and then infected with HIV primary isolates. After 5 days of infection with the R5-tropic isolate 97ZA009, the capacity of the CD4 T cells to produce Th1 and Th17 cytokines was assessed in a standard intracellular cytokine staining assay, whereas virus infection was monitored by intracellular p24 expression. Utilizing this in vitro system, we were able to capture the suppressive effects of HIV infection on both Th1 and Th17 responses. As shown by data in Fig. 1A from cells of one representative donor, a reduction from 45% to 18% of IFN-γ⁺ CD4 T cells was detected upon HIV infection, which was evident from the presence of 2.6% p24⁺ CD4 T cells. The effect on the Th17 response was even greater, as the percentage of IL-17A⁺ CD4 T cells decreased from 3.6% in the uninfected culture to an almost undetectable level of 0.4% in the infected culture. When six different R5-tropic or X4-tropic HIV isolates were compared, a similar pattern of virus-induced suppression was observed (Fig. 1B). The percentages of IFN-γ-producing CD4 T cells in the HIV-infected cultures were reduced down to 52–85% of the uninfected control except for one case in which the percentage was 128% of the uninfected control. By contrast, the reduction of the IL-17A⁺ CD4 T cells was more dramatic to reach 5% to 47% of the response in the uninfected control. The reduction levels of IFN-γ⁺ and IL17A⁺ CD4 T cells varied greatly among individual donors for reasons that are still unknown, but statistical analysis of data from 21 subjects corroborated that the Th17 response was significantly lower than Th1 response after HIV infection (p<0.0001; Fig. 1C).

FIG. 1.

Effects of HIV-1 infection on interferon (IFN)-γ and interleukin (IL)-17A production by CD4 T cells. CD4 T cells isolated from HIV-seronegative donors were stimulated with anti-CD3 and anti-CD28 antibodies, infected with HIV-1 primary isolates for 5 days, and assessed for the capacity to produce IFN-γ and IL-17A after PMA/ionomycin stimulation for 6 h. IFN-γ and IL-17A production was measured by intracellular cytokine staining (ICS). The cut-off for IFN-γ⁺ and IL-17A⁺ cells was determined based on unstimulated cells from the same cultures (not shown). HIV-1 infection in the CD4 T cells was also monitored by intracellular p24 staining. (A) Dot plots of uninfected and 97ZA009-infected CD4 T cells after intracellular staining with antibodies to p24, IFN-γ, or IL-17A. (B) Relative percentages of CD4 T cells capable of producing IFN-γ (Th1) or IL-17A (Th17) after 5 days of infection with six different R5-tropic or X4-tropic HIV-1 isolates. The frequencies of IFN-γ⁺ and IL-17A⁺ CD4 T cells in uninfected cultures are normalized to 100%. Three or more repeat experiments were done with selected isolates. Values from representative experiments are shown. (C) Relative frequencies of IFN-γ⁺ CD4 T cells (Th1) and IL-17A⁺ CD4 T cells (Th17) in cultures from 21 donors after 7 days of HIV infection. The frequencies of Th1 or Th17 cells in infected cultures were calculated relative to those in uninfected cultures from the corresponding donors (normalized to 100%). Statistical significance was determined with paired two-tailed t-test. (D) Representative dot plots of uninfected and infected CD4 T cells stained with CCR6 and CD161 at day 7 (left panel) and the relative percentages of CD4 T cells expressing both CCR6 and CD161 in uninfected vs. infected cultures from two donors (right panel). (E) Expression levels of IL-23R on total CD4 T cells in uninfected, BX08-infected, and 97ZA009-infected cultures as indicated by mean fluorescence intensity (MFI). Average and standard error values from three different donors are shown. Paired Student's t-test was used to determine statistical significance.

Furthermore, staining of the cells for CCR6, CD161, and IL-23R, which are commonly though not exclusively expressed on the surface of Th17 cells,^7,14 revealed that the loss of Th17 responses was accompanied by a 75–90% decline in the percentages of CCR6⁺CD161⁺ CD4 T cells in the HIV-infected cultures from two subjects (Fig. 1D). The level of IL-23R also decreased in the infected cultures, but did reach statistical significance (Fig. 1E), possibly because of IL-23R expression on CD4 T cells other than Th17 cells. Moreover, increased apoptosis (by 9%) and decreased cell proliferation (by 18%) were observed among total CD4 T cells in the infected culture (Supplementary Fig. S1; Supplementary Data are available online at www.liebertonline.com/aid). Comparable Th17 and Th1 suppressive effects were observed when this experiment was set up with unfractionated PBMCs infected with HIV (Supplementary Fig. S2). These data demonstrate the relevance of our in vitro experimental system in recapitulating HIV suppressive effects on Th1 and Th17 responses that were previously shown in HIV-infected subjects and SIV-infected primate models.^5,23,24,33 Importantly, consistent with previous ex vivo observations, this system demonstrates the greater effects of HIV infection on Th17 response as compared to Th1 response, allowing us to further investigate approaches to preserve or recover these Th responses.

Maintenance of Th17 response but not Th1 response by CCR5 ligands or a CCR5 antagonist

A number of antiviral agents that block HIV infection effectively have been identified, including the chemokine ligands and antagonists for the virus coreceptors. CCR5 ligands (MIP-1α, MIP-1β, and RANTES) are produced by various types of immune cells including the CD4 T cells themselves, and they protect CD4 T cells from R5-tropic HIV in paracrine and autocrine fashions.^35
–37 Previously, it has been shown that, unlike Th1 cells, Th17 cells do not synthesize CCR5 ligands,³⁸ and the lack of these HIV-blocking chemokines may be a factor that contributes to the increased susceptibility of this particular subset to HIV infection. To determine the capacity of these natural antiviral chemokines to preserve the Th1 versus Th17 responses during HIV infection, CD4 T cell cultures were incubated with R5-tropic HIV in the presence of a cocktail of the three CCR5 ligands. For comparison, we also tested the RANTES antagonist TAK779, which induces an allosteric change in the CCR5 receptor and efficiently prevents HIV infection.³⁹ Figure 2A shows that after 5 days of infection in the presence of CCR5 ligands, there was a 70% decline (1.53–0.46%) in the percentage of p24⁺ CD4 T cells as compared to the untreated HIV-infected culture. In contrast, with TAK779 the numbers of p24⁺ cells decreased by >90% to 0.10%, near the background level seen in the uninfected culture. In spite of the significantly lower percentages of infected cells, no improvement in Th1 response was detected upon treatment with CCR5 ligands or TAK779, although the Th1 response was not severely suppressed in the infected culture (only ∼20% decline). The percentages of IFN-γ-producing cells in the infected culture treated with CCR5 ligands (19.36%) or TAK779 (21.75%) were lower than that of the uninfected culture (33.99%) and comparable to the untreated infected culture (25.38%). On the other hand, a significant increase in the percentages of IL17A-producing CD4 T cells was seen upon treatment with CCR5 ligands (0.86%) or TAK779 (0.91%) as opposed to that of the untreated infected culture (0.57%), although the increase did not reach the level of the uninfected culture (1.22%). Analysis of CD4 T cell cultures from three different donors revealed similar results (Fig. 2B). Of note, the CCR5 ligands and TAK779 themselves did not increase the percentages of IFN-γ⁺ or IL-17A⁺ cells in the uninfected cultures (Supplementary Fig. S3). These data demonstrate that although the Th17 response was more drastically suppressed than the Th1 response during HIV infection, the Th17 response could be partially protected by treatment with the natural CCR5 ligands or the synthetic antagonist TAK779. By contrast, these CCR5 blockers had no significant effects on Th1 response.

FIG. 2.

Th1 and Th17 responses in HIV-infected cultures treated with CCR5 ligands or TAK779. CD4 T cells were stimulated with anti-CD3 and anti-CD28 for 48 h and then infected with HIV-1 97ZA009 in the presence of CCR5 ligands: MIP-1α, MIP-1β, and RANTES for 5 days. The RANTES antagonist TAK779 was also tested for comparison. Cells incubated with AT-2-inactivated HIV serve as controls. Cells were stimulated with PMA/ionomycin for 6 h, and stained with antibodies to p24, IFN-γ, and IL-17A. (A) Representative dot plots of CD4 T cells in uninfected cultures or infected cultures that were left untreated or treated with CCR5 ligands or TAK779. The percentages of CD4 T cells positive for IFN-γ, IL-17A, and p24 are shown. (B) Effects of CCR5 ligands or TAK779 on the frequencies of IFN-γ⁺ or IL-17A⁺ CD4 T cells in HIV-infected cultures. Data from each of the three different donors tested are depicted with different symbols. Frequencies of the respective CD4 T cells in the uninfected cultures from the same donors are normalized to 100%. p24% indicates the average and range of the percentages of CD4 T cells expressing p24 under the designated experimental conditions. Statistical significance was determined with the exact Wilcoxon rank sum test. (C) Representative dot plots from uninfected, HIV-infected, and AT-2-inactivated HIV-treated cultures showing the percentages of CD4 T cells expressing p24 and IFN-γ or IL-17A. (D) Relative percentages of CD4 T cells producing IFN-γ (Th1) and IL-17A (Th17) after exposure with infectious or inactivated HIV. Two R5-tropic HIV-1 isolates (97ZA009 and BX08) were tested. Frequencies of Th1 and Th17 cells in uninfected cultures are normalized to 100%.

Given that reduction of HIV infection by inhibitors targeting the CCR5 receptor did not fully protect the Th17 and Th1 responses, we tested complete virus inactivation by aldrithiol-2 (AT-2) for comparison. Figure 2C shows that unlike the culture treated with infectious virus, the culture exposed to AT-2-inactivated virus yielded no p24⁺ CD4 T cells and showed no decrease in the numbers of IFN-γ⁺ and IL-17A⁺ cells. The same results were seen with two different HIV isolates inactivated with AT-2 (Fig. 2D). These data indicate that Th1 and Th17 responses may be preserved if HIV infection can be blocked completely.

Preservation of Th17 and Th1 responses by the antiretroviral drug 3TC added at the time of infection

The standard ART administered to HIV-infected individuals potently suppresses virus replication and lowers the viral load to undetectable levels in the majority of patients. Here, we tested one of the reverse transcriptase inhibitors, 3TC, to determine its capacity to preserve Th1 and Th17 responses in HIV-infected CD4 T cell cultures. We treated the cells with 3TC from day 0 of infection and assessed production of IFN-γ and IL-17A after 7 days of infection. Representative dot plots from one donor are shown in Fig. 3A, while data from all three donors tested are summarized in Fig. 3B. As expected, the percentages of IFN-γ and IL-17A-producing cells were reduced in the infected cultures. The decrease levels of IL-17A⁺ cells were significant, while those of IFN-γ⁺ cells were not. When virus infection is inhibited by addition of 3TC on day 0, the percentages of IFN-γ⁺ cells and IL-17A⁺ cells were comparable to those in the uninfected control, and this same pattern was observed with cells from all donors tested. Overall, these data show that the suppressive effects of HIV infection on the Th1 and Th17 subset can be prevented by efficient blocking of HIV infection with the administration of ART at the time of HIV exposure.

FIG. 3.

Effects of 3TC treatment that blocks the initial HIV infection on Th1 and Th17 responses. CD4 T cell cultures were infected with HIV-1 97ZA009 and treated with 3TC starting from day 0. IFN-γ and IL-17A production in response to PMA/ionomycin stimulation was measured by intracellular cytokine staining. Infection was detected by p24 intracellular staining. (A) Representative dot plots of IFN-γ or IL-17A production in uninfected and infected CD4 T cells cultured in the presence or absence of 3TC. (B) Relative frequencies of CD4 T cells synthesizing IFN-γ or IL-17A upon HIV-1 infection in the presence or absence of 3TC in cultures from three different donors (depicted with different symbols). Frequencies of IFN-γ⁺ or IL-17A⁺ CD4 T cells in the uninfected cultures from the same donors are normalized to 100%. p24% indicates the average and range of the percentages of CD4 T cells expressing p24 under the designated experimental conditions. Statistical significance was determined with the exact Wilcoxon rank sum test.

Expansion of Th17 response by Th17 differentiating cytokines plus 3TC after establishment of HIV infection

While Th17 cells can be detected in acutely infected patients, this population is severely depleted in chronically infected patients and is not reconstituted fully after successful ART, which indicates that ART, once HIV infection is established, is not sufficient to restore the Th17 response.²² Using our in vitro system we emulated the therapeutic usage of 3TC, which is typically administered after HIV infection is detected. We infected CD4 T cells with HIV for 4 days, washed the cells, and treated them with 3TC. Similar to that seen in ex vivo PBMCs from HIV⁺ patients on ART, the delay in treatment with 3TC did not restore the percentages of IL-17A⁺ cells to the levels observed in the uninfected control. The numbers of Th17 cells in 3TC-treated infected cultures were not significantly different from the levels in untreated virus-infected cultures in all three donors tested, even though virus replication was uniformly suppressed to <0.5% (Fig. 4A, B). Comparable to data in Fig. 3, the Th1 response was not significantly affected by HIV infection and the 3TC treatment had minimal impact. These data show that 3TC treatment after HIV infection is established is not effective for restoring the Th17 response.

FIG. 4.

Expansion of the Th17 response after the establishment of HIV infection by 3TC and Th17 differentiation cytokines. Isolated CD4 T cells were infected with HIV-1 97ZA009 for 4 days, then washed and treated with 3TC in the presence or absence of Th17 differentiation cytokines (IL-1β, TGF-β, IL-6, and IL-23) for 6 more days. Cells were restimulated with PMA/ionomycin for 6 h, and IFN-γ and IL-17A production was measured by intracellular cytokine staining. (A) Dot plots of CD4 T cells positive for p24 and IFN-γ or IL-17A from one representative donor are shown. (B) Summary of the relative changes in IFN-γ⁺ or IL-17A⁺ CD4 T cell frequencies in the designated cultures from each of three donors tested. (C, D) Effects of Th17 differentiation cytokines on Th1 and Th17 responses in uninfected and infected cultures in the absence of 3TC. CD4 T cell cultures treated with IL-2 alone or IL-2 and the Th17 differentiation cytokines (IL-1β, TGF-β, IL-6, and IL-23) were left uninfected or infected with HIV for 8 days. Cells were then stimulated with PMA/ionomycin for 6 h, and stained for intracellular p24, IFN-γ, and IL-17A. Dot plots of IFN-γ⁺ and IL-17A⁺ CD4 T cells from one donor under the different culture conditions are shown (C). (D) Graphs depict changes of IFN-γ⁺ and IL-17A⁺ CD4 T cell responses in cultures from each of four donors tested. The percentages of CD4 T cells producing IFN-γ or IL-17A in uninfected cultures with no cytokine cocktail are normalized to 100%. p24% indicates the average and range of the percentages of p24⁺ CD4 T cells under the different experimental conditions. Statistical significance was determined with the exact Wilcoxon rank sum test.

We subsequently evaluated whether immune-modulatory cytokines might be employed along with ART to augment the Th17 response after HIV infection is established. Naive CD4 T cells differentiate into Th17 cells in the presence of cytokines IL-1β, IL-6, TGF-β, and IL-23, which also play an important role in the proliferation and maintenance of the memory Th17 subset.¹⁶ Using our system, which includes the naive and memory T cells populations, we tested the effects of these cytokines on the Th17 and Th1 responses in uninfected and HIV-infected cultures in the absence of ART. CD4 T cells were cultured with only IL-2 as done for all experiments above or with IL-2 and a cocktail of IL-1β, IL-6, TGF-β, and IL-23. Increased percentages of IFN-γ⁺ cells and IL-17A⁺ cells were observed in the uninfected cultures treated with the cytokine cocktail (Fig. 4C). Importantly, the levels of both Th1 and Th17 responses were also enhanced in the HIV-infected culture. The Th1 response increased by over 2-fold, while the increase of Th17 response was almost 5-fold, although the percentage of IL-17A⁺ cells did not reach that seen in the uninfected culture treated with the same cocktail. Figure 4D summarizes the data from all four different donors tested and confirms that addition of the cytokine cocktail significantly improved the level of Th17 responses in both infected and uninfected cultures (p=0.014 and p=0.011, respectively). The effect on Th1 response was highly variable, which was not surprising in view of the fact that this cocktail favors differentiation of Th17 cells and not Th1 cells. However, for the majority of donors, the cytokine cocktail also increased the percentages of p24⁺ CD4 T cells.

We then tested whether the Th17 differentiation cytokine cocktail augments the Th17 response in cultures where HIV infection was already established but successfully suppressed by 3TC (Fig. 4A, B). The data show that only when the Th17 differentiation cytokine cocktail was added to 3TC-treated HIV-infected cultures, were we able to detect robust Th17 responses that were comparable or higher than the levels of the uninfected controls and were significantly higher than the infected cells in all three donors (p=0.05) (Fig. 4B). The Th17 differentiation cocktail along with 3TC also increased the percentage of IFN-γ⁺ cells (p=0.05), by over 2-fold to an impressive 54.8% in one culture but only by 1.1- to 1.2-fold for the other two donors. Importantly, viral proliferation remained suppressed (<0.5% of p24⁺) in the presence of 3TC and Th17 differentiation cytokines, while at the same time the Th17 and Th1 responses expanded. Overall, these results show that once HIV infection is established, 3TC treatment alone has minimal impact on the Th17 response, but its combination with the Th17 differentiation cytokines allows expansion of the Th17 response without enhancing virus replication.

Enhancing the Th17 response in peripheral lymphocytes from ART recipients by Th17 differentiation cytokines

To determine whether the Th17 differentiation cytokine cocktail of IL-1β, IL-6, TGF-β, and IL-23 can improve the Th17 response of CD4 T cells from ART recipients, we isolated PBMCs from ART patients with undetectable plasma viremia and cultured the cells with IL-2 only or with IL-2 and the Th17 differentiation cytokines for 8 days. The reverse transcriptase inhibitor 3TC was also added to prevent reactivation of virus replication in the cultures. The data show that the percentages of both the IFN-γ⁺ and IL-17A⁺ CD4 T cells increased after treatment with the Th17 differentiation cocktail (Fig. 5). The dot plots of CD4 T cells from cultures of one representative ART patient are shown in Fig. 5A. The percentage of IFN-γ-producing CD4 T cells increased from 13.9% to 19.4% while the percentage of IL-17A-producing cells increased over 2-fold from 0.11% to 0.27%. The percentage of p24⁺ cells was maintained below 0.1% under both conditions. The data from all seven ART recipients tested are summarized in Fig. 5B. For all seven subjects, the percentages of IL-17A-producing T cells increased by an average of 9.77-fold (range 1.14–21.67; p=0.016). Except for one donor, the IFN-γ⁺ CD4 T cell frequencies also increased, albeit only by an average of 1.39-fold (range: 0.92–2.17; p=0.0079). These results demonstrate that CD4 T cells capable of differentiating and producing IL-17A are present in chronically HIV-infected patients on ART and that these cells can potentially be expanded by cytokines or other immune therapeutic strategies.

FIG. 5.

Th1 and Th17 responses in PBMC cultures from ART patients upon treatment with 3TC and Th17 differentiating cytokines. PBMCs from ART patients were cultured with 3TC plus IL-2 or IL-2 and the Th17 differentiation cytokine cocktail (IL-1β, TGF-β, IL-6, and IL-23) for 8 days. IFN-γ and IL-17A production was measured by intracellular cytokine staining in response to PMA/ionomycin stimulation. (A) Dot plots of IFN-γ⁺ and IL-17A⁺ CD4 T cells from cultures of a representative ART recipient where the cells were either treated with IL-2 or IL-2 and the cytokine cocktail. (B) Changes in the percentages of IFN-γ⁺ and IL-17A⁺ CD4 T cells from seven different HIV⁺ donors on ART upon treatment with IL-2 vs. IL-2 plus the cytokine cocktail. Statistical significance was determined by paired two-tailed t-test.

Discussion

In this study, an in vitro culture system was established to recapitulate the dramatic loss of the Th17 response and the more modest decrease of the Th1 response observed in HIV-infected individuals. With this culture system, we demonstrate the following main findings: (1) Depletion of Th1 and Th17 responses was associated with active HIV replication and could be spared by complete virus inactivation. (2) Incomplete blocking of HIV infection by the CCR5 natural ligands (MIP-1α, MIP-1β, and RANTES) or the CCR5 antagonist TAK779 only partially protected the Th17 response, while treatment with the antiretroviral drug 3TC at the time of virus exposure resulted in full protection of the Th17 and Th1 responses. (3) After the infection was established, 3TC alone suppressed virus replication without fully restoring the Th17 response; however, the combination of 3TC and the Th17 differentiation cytokines IL-6, IL-1β, TGF-β, and IL-23 allowed expansion of the Th17 response to the levels observed in the uninfected control. Undoubtedly, this in vitro system is an invaluable tool that can be employed to further understand HIV pathogenesis and its detrimental effects on Th17 cells.

Although our in vitro system affirms the greater suppressive effects of active HIV replication on the Th17 response as compared to the Th1 response, the reasons for this difference are not fully understood and require further investigation. Notably, Th1 cell population has been shown to produce anti-HIV chemokines, particularly MIP-1β, while Th17 cells express very little or no MIP-1β.³⁸ Our earlier study demonstrates that antigenic stimulation of HIV-specific IFN-γ⁺ Th1 cell lines renders the cells more resistant to R5-tropic HIV infection and this resistance is associated with MIP-1β production by the Th1 cells themselves.³⁵ Comparable results were reported by other laboratories with Th1 clones responding to influenza antigens⁴⁰ and other polyclonal stimuli.⁴¹ Lack of autocrine production of MIP-1β and other CCR5 ligands by Th17 cells may enhance their susceptibility to infection at least with the R5-tropic isolates, and studies to evaluate this possibility are in progress. Consistent with this idea, addition of exogenous CCR5 ligands provided some levels of protection to the Th17 cells, but had minimal effects on the Th1 cells (Fig. 2). Nevertheless, it is also still unclear whether the higher loss of the Th17 response as compared to the Th1 response results from an increased rate of direct virus infection of the Th17 cells. Brenchley et al. reported that sorted IL-17⁺ cells and IFN-γ⁺ cells from PBMCs of HIV-infected subjects showed no significant difference in their viral DNA burdens,⁵ while another study by Gosselin et al. detected an increased frequency of viral DNA among the Th17 subset bearing CCR6 and CCR4, indicating preferential HIV infection among distinct Th17 subpopulations.⁴

The importance of active virus replication in suppressing Th17 responses is supported by our data showing that Th17 responses were fully preserved when complete blocking of virus infection was achieved by 3TC treatment at the time of virus exposure. In contrast, only partial Th17 restoration was attained when low levels of virus replication remained. However, quantitative correlates between the level of residual virus replication and the extent of Th17 suppression to determine if a minimal threshold exists have not been established. In the primate SIV model, Cecchinato et al. also show that the level of SIV viral load negatively correlates with the frequency of Th17 cells in the blood, lymph nodes, and gut mucosal tissues.²⁴ In ART-naive chronically infected patients, subjects whose viral load reached >2000 displayed dramatic reduction of Th17 response in the gut mucosa, but with successful ART of >5 years, Th17 responses in the gut mucosa could be augmented when the gut-associated total CD4 T cell counts were restored by >50%.²² Nevertheless, data from our study and others²² show that full restoration of Th responses was not always accomplished by ART after the establishment of chronic infection (Fig. 4). Rather, prevention of Th dysfunctions was most effective by blockage of HIV infection by ART at the time of exposure, providing support for prophylactic ART and its clinical application among high-risk populations. Indeed, a recent study in which prophylactic ART was given to men with a high risk of HIV exposure demonstrates a 44% reduction of HIV incidence as compared to placebo.⁴²

For the majority of HIV-infected patients, ART is not initiated until months or years after infection, at which point the Th17 cells are already depleted. In the macaques, the Th17 cells are preferentially depleted from the gut in less than 2 weeks following natural pathogenic SIV infection.^23,24 In our hands, we could not fully recover the Th17 response when ART was given 4 days after HIV infection. This result agrees with the study of Kader et al. in which ART given to SIV-infected macaques at week 13 did not recover Th17 response.²⁷ In HIV-infected humans, even a complete suppression of the viral loads with long-term ART does not fully restore Th17 response in all of the patients.²²

Given that ART alone is not sufficient to restore the Th17 response, we evaluated whether the Th17 response might be stimulated in the presence of HIV infection by the addition of Th17 differentiation cytokines: IL-1β, IL-6, TGF-β, and IL-23.¹⁶ A substantial increase of the frequency of IL-17A⁺ cells was observed with the cytokine cocktail, but not surprisingly the percentage of virus-infected p24⁺ cells also increased. In view of the failure of ART alone to completely restore the Th17 response and the capacity of the Th17 differentiation cytokines to boost the Th17 response at the cost of enhancing viral replication, we decided to test a combination of these two modalities. When we allowed HIV infection to establish first in the cultures for several days and then introduced both 3TC and the Th17 differentiation cytokine cocktail, we successfully restored and enhanced the Th17 response while keeping the virus infection in check. Furthermore, when we stimulated PBMCs of ART patients with the cytokine cocktail in the presence of 3TC, expansion of the Th17 response was observed in each of the seven subjects tested, albeit to different degrees. This key finding points to the presence of CD4 T cell populations that remain responsive and can be expanded to boost the Th17 responses in ART patients. Previous studies showed that the cocktail of IL-1β, IL-6, TGF-β, and IL-23 drives differentiation of naive CD4 T cells to become Th17 cells as well as expands Th17 memory cells.¹⁶ The proportions of naive and memory populations that respond to this cytokine cocktail are not known and need further investigation, but the frequencies may vary greatly among different ART patients to result in different levels of Th17 expansion attainable by the cytokine cocktail treatment.

The capacity of a cytokine cocktail to expand the Th17 response in PBMCs from ART patients indicates the potential benefit of a cytokine therapy in conjunction with ART for chronically HIV-infected patients. The use of cytokines as immunotherapy for HIV is not without precedent.⁴³ In the past decade, several groups have tested IL-2 to stimulate T cells in HIV-infected patients,⁴⁴ and more recently others have explored the use of IL-15, IL-7, or IL-21.^45
–47 Since cytokine alone is not likely to be effective, if virus replication is not controlled, a combination of ART and IL-2 was tested and found to yield a greater expansion of CD4 T cells than ART alone.⁴⁸ However, two recent clinical trials, ESPRIT and SILCAAT, to evaluate the clinical benefits of the ART plus IL-2 therapy showed no significant difference in risks for opportunistic disease, serious clinical events, and death as compared to ART alone.⁴⁹ A more recent study showed that treatment of patients during early stages of HIV infection with ART and IL-2 expanded total CD4 T cells and Treg cells, but might decrease Th17 frequency in the periphery.⁵⁰ Whether this reflects changes in the gut mucosal tissues is not yet known. Our in vitro studies demonstrate that Th17 differentiation cytokines, rather than IL-2, expanded Th17 response in cultures with established HIV infection. However, it is unknown if all or only a subset of the four cytokines (IL-1β, IL-6, TGF-β, and IL-23) are required and if they can be administered to ART patients without exacerbating hyperimmune activation and other immune dysfunctions associated with chronic HIV infection. Further studies are also needed to determine the capacity of these cytokines to expand Th17 cells at the gut-associated lymphoid tissues, where these cells play a crucial role in maintaining mucosal integrity. Considering the critical role Th17 cells play in maintaining the integrity of the gut mucosal tissues and in controlling opportunistic infections commonly associated with HIV disease, immunotherapy that reconstitutes these cells in the peripheral blood and in the gut of HIV-infected patients has the potential to produce additional clinical benefits beyond those achievable by ART alone.

Footnotes

Acknowledgments

The authors thank Dr. Pablo Colon for patient referral, Sandra Cohen for providing virus stocks, Diana Virland for assistance with flow cytometers, Dr. Sagarika Banerjee for assistance with PCR, and the subject volunteers. The following reagents were obtained through the NIH AIDS Research and Reference Reagent Program, Division of AIDS, NIAID, NIH: 3TC and TAK779. This work was supported by research funds from NIH grants (AI-084807 and AI-085958), the Research Enhancement Award Program of the U.S. Department of Veterans Affairs, and the New York University Center for AIDS Research Immunology Core (Grant AI-27742).

Author Disclosure Statement

No competing financial interests exist.

References

Brenchley

, Schacker

, Ruff

et al. CD4+ T cell depletion during all stages of HIV disease occurs predominantly in the gastrointestinal tract. J Exp Med, 2004; 200,6:749–759.

Mattapallil

, Douek

, Hill

, Nishimura

, Martin

, Roederer

. Massive infection and loss of memory CD4+ T cells in multiple tissues during acute SIV infection. Nature, 2005; 434,7037:1093–1097.

Brenchley

, Price

, Schacker

et al. Microbial translocation is a cause of systemic immune activation in chronic HIV infection. Nat Med, 2006; 12,12:1365–1371.

Gosselin

, Monteiro

, Chomont

et al. Peripheral blood CCR4+CCR6+ and CXCR3+CCR6+CD4+ T cells are highly permissive to HIV-1 infection. J Immunol, 2010; 184,3:1604–1616.

Brenchley

, Paiardini

, Knox

et al. Differential Th17 CD4 T-cell depletion in pathogenic and nonpathogenic lentiviral infections. Blood, 2008; 112,7:2826–2835.

Romagnani

. The Th1/Th2 paradigm. Immunol Today, 1997; 18,6:263–266.

Annunziato

, Cosmi

, Santarlasci

et al. Phenotypic and functional features of human Th17 cells. J Exp Med, 2007; 204,8:1849–1861.

Soroosh

, Doherty

. Th9 and allergic disease. Immunology, 2009; 127,4:450–458.

Duhen

, Geiger

, Jarrossay

, Lanzavecchia

, Sallusto

. Production of interleukin 22 but not interleukin 17 by a subset of human skin-homing memory T cells. Nat Immunol, 2009; 10,8:857–863.

10.

Fehervari

, Sakaguchi

. CD4+ Tregs and immune control. J Clin Invest, 2004; 114,9:1209–1217.

11.

Miossec

. IL-17 and Th17 cells in human inflammatory diseases. Microbes Infect, 2009; 11,5:625–630.

12.

Acosta-Rodriguez

, Rivino

, Geginat

et al. Surface phenotype and antigenic specificity of human interleukin 17-producing T helper memory cells. Nat Immunol, 2007; 8,6:639–646.

13.

Ivanov

, McKenzie

, Zhou

et al. The orphan nuclear receptor RORgammat directs the differentiation program of proinflammatory IL-17+ T helper cells. Cell, 2006; 126,6:1121–1133.

14.

Kleinschek

, Boniface

, Sadekova

et al. Circulating and gut-resident human Th17 cells express CD161 and promote intestinal inflammation. J Exp Med, 2009; 206,3:525–534.

15.

Acosta-Rodriguez

, Napolitani

, Lanzavecchia

, Sallusto

. Interleukins 1beta and 6 but not transforming growth factor-beta are essential for the differentiation of interleukin 17-producing human T helper cells. Nat Immunol, 2007; 8,9:942–949.

16.

Manel

, Unutmaz

, Littman

. The differentiation of human T(H)-17 cells requires transforming growth factor-beta and induction of the nuclear receptor RORgammat. Nat Immunol, 2008; 9,6:641–649.

17.

Wei

, Laurence

, Elias

, O'Shea

. IL-21 is produced by Th17 cells and drives IL-17 production in a STAT3-dependent manner. J Biol Chem, 2007; 282,48:34605–34610.

18.

Liang

, Tan

, Luxenberg

et al. Interleukin (IL)-22 and IL-17 are coexpressed by Th17 cells and cooperatively enhance expression of antimicrobial peptides. J Exp Med, 2006; 203,10:2271–2279.

19.

Ouyang

, Kolls

, Zheng

. The biological functions of T helper 17 cell effector cytokines in inflammation. Immunity, 2008; 28,4:454–467.

20.

Klatt

, Brenchley

. Th17 cell dynamics in HIV infection. Curr Opin HIV AIDS, 2010; 5,2:135–140.

21.

Mehandru

, Poles

, Tenner-Racz

et al. Lack of mucosal immune reconstitution during prolonged treatment of acute and early HIV-1 infection. PLoS Med, 2006; 3,12:e484.

22.

Macal

, Sankaran

, Chun

et al. Effective CD4+ T-cell restoration in gut-associated lymphoid tissue of HIV-infected patients is associated with enhanced Th17 cells and polyfunctional HIV-specific T-cell responses. Mucosal Immunol, 2008; 1,6:475–488.

23.

Favre

, Lederer

, Kanwar

et al. Critical loss of the balance between Th17 and T regulatory cell populations in pathogenic SIV infection. PLoS Pathog, 2009; 5,2:e1000295.

24.

Cecchinato

, Trindade

, Laurence

et al. Altered balance between Th17 and Th1 cells at mucosal sites predicts AIDS progression in simian immunodeficiency virus-infected macaques. Mucosal Immunol, 2008; 1,4:279–288.

25.

Arthos

, Cicala

, Martinelli

et al. HIV-1 envelope protein binds to and signals through integrin alpha4beta7, the gut mucosal homing receptor for peripheral T cells. Nat Immunol, 2008; 9,3:301–309.

26.

Kader

, Wang

, Piatak

et al. Alpha4(+)beta7(hi)CD4(+) memory T cells harbor most Th-17 cells and are preferentially infected during acute SIV infection. Mucosal Immunol, 2009; 2,5:439–449.

27.

Kader

, Bixler

, Piatak

, Lifson

, Mattapallil

. Anti-retroviral therapy fails to restore the severe Th-17: Tc-17 imbalance observed in peripheral blood during simian immunodeficiency virus infection. J Med Primatol, 2009; 38,Suppl 1:32–38.

28.

Favre

, Mold

, Hunt

et al. Tryptophan catabolism by indoleamine 2,3-dioxygenase 1 alters the balance of TH17 to regulatory T cells in HIV disease. Sci Transl Med, 2010; 2,32:32ra36.

29.

Swingler

, Mann

, Jacque

et al. HIV-1 Nef mediates lymphocyte chemotaxis and activation by infected macrophages. Nat Med, 1999; 5,9:997–103.

30.

Jacotot

, Ravagnan

, Loeffler

et al. The HIV-1 viral protein R induces apoptosis via a direct effect on the mitochondrial permeability transition pore. J Exp Med, 2000; 191,1:33–46.

31.

, Friedman

, Wang

, Metelev

, Pardee

. Induction of apoptosis in uninfected lymphocytes by HIV-1 Tat protein. Science, 1995; 268,5209:429–431.

32.

Banda

, Bernier

, Kurahara

et al. Crosslinking CD4 by human immunodeficiency virus gp120 primes T cells for activation-induced apoptosis. J Exp Med, 1992; 176,4:1099–1106.

33.

Raffatellu

, Santos

, Verhoeven

et al. Simian immunodeficiency virus-induced mucosal interleukin-17 deficiency promotes Salmonella dissemination from the gut. Nat Med, 2008; 14,4:421–428.

34.

Gorny

, Williams

, Volsky

et al. Human monoclonal antibodies specific for conformation-sensitive epitopes of V3 neutralize human immunodeficiency virus type 1 primary isolates from various clades. J Virol, 2002; 76,18:9035–9045.

35.

Kaur

, Tuen

, Virland

et al. Antigen stimulation induces HIV envelope gp120-specific CD4(+) T cells to secrete CCR5 ligands and suppress HIV infection. Virology, 2007; 369,1:214–225.

36.

Cocchi

, DeVico

, Garzino-Demo

, Arya

, Gallo

, Lusso

. Identification of RANTES, MIP-1 alpha, and MIP-1 beta as the major HIV-suppressive factors produced by CD8+ T cells. Science, 1995; 270,5243:1811–1815.

37.

Garzino-Demo

, Moss

, Margolick

et al. Spontaneous and antigen-induced production of HIV-inhibitory beta-chemokines are associated with AIDS-free status. Proc Natl Acad Sci USA, 1999; 96,21:11986–11991.

38.

El Hed

, Khaitan

, Kozhaya

et al. Susceptibility of human Th17 cells to human immunodeficiency virus and their perturbation during infection. J Infect Dis, 2010; 201,6:843–854.

39.

Baba

, Nishimura

, Kanzaki

et al. A small-molecule, nonpeptide CCR5 antagonist with highly potent and selective anti-HIV-1 activity. Proc Natl Acad Sci USA, 1999; 96,10:5698–5703.

40.

Robbins

, Roderiquez

, Peden

, Norcross

. Human immunodeficiency virus type 1 infection of antigen-specific CD4 cytotoxic T lymphocytes. AIDS Res Hum Retroviruses, 1998; 14,16:1397–1406.

41.

Abdelwahab

, Cocchi

, Bagley

et al. HIV-1-suppressive factors are secreted by CD4+ T cells during primary immune responses. Proc Natl Acad Sci USA, 2003; 100,25:15006–15010.

42.

Grant

, Lama

, Anderson

et al. Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. N Engl J Med, 2010; 363,27:2587–2599.

43.

Alfano

, Crotti

, Vicenzi

, Poli

. New players in cytokine control of HIV infection. Curr HIV/AIDS Rep, 2008; 5,1:27–32.

44.

Jacobson

, Pilaro

, Smith

. Rational interleukin 2 therapy for HIV positive individuals: Daily low doses enhance immune function without toxicity. Proc Natl Acad Sci USA, 1996; 93,19:10405–10410.

45.

Bolesta

, Kowalczyk

, Wierzbicki

et al. Increased level and longevity of protective immune responses induced by DNA vaccine expressing the HIV-1 Env glycoprotein when combined with IL-21 and IL-15 gene delivery. J Immunol, 2006; 177,1:177–191.

46.

Sereti

, Dunham

, Spritzler

et al. IL-7 administration drives T cell-cycle entry and expansion in HIV-1 infection. Blood, 2009; 113,25:6304–6314.

47.

Mastroianni

, d'Ettorre

, Forcina

, Vullo

. Teaching tired T cells to fight HIV: Time to test IL-15 for immunotherapy? Trends Immunol, 2004; 25,3:121–125.

48.

Levy

, Durier

, Krzysiek

et al. Effects of interleukin-2 therapy combined with highly active antiretroviral therapy on immune restoration in HIV-1 infection: A randomized controlled trial. AIDS, 2003; 17,3:343–351.

49.

Abrams

, Levy

, Losso

et al. Interleukin-2 therapy in patients with HIV infection. N Engl J Med, 2009; 361,16:1548–1559.

50.

Ndhlovu

, Sinclair

, Epling

et al. IL-2 immunotherapy to recently HIV-1 infected adults maintains the numbers of IL-17 expressing CD4+ T (T(H)17) cells in the periphery. J Clin Immunol, 2010; 30,5:681–692.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB