Reactions and Receptivity to Framing HIV Prevention Message Concepts About Pre-Exposure Prophylaxis for Black and Latino Men Who Have Sex with Men in Three Urban US Cities

Abstract

Men who have sex with men (MSM) of color are disproportionately affected by HIV in the United States. Pre-exposure prophylaxis (PrEP) using antiretroviral medications is a newer biomedical prevention modality with established efficacy for reducing the risk of acquiring HIV. We conducted formative qualitative research to explore audience reactions and receptivity to message concepts on PrEP as part of the development of prevention messages to promote PrEP awareness among black and Latino MSM in the United States. In 2013, 48 black and 42 Latino (total study sample = 90) mixed HIV serostatus MSM from Chicago, Ft. Lauderdale, and Kansas City participated in either an individual interview or focus group discussion. Men were recruited online and at community-based organizations in each city. We elicited feedback on the comprehensibility, credibility, and relevance of two draft messages on PrEP. The messages included efficacy estimates from iPrEx, a phase III clinical trial to ascertain whether the antiretroviral medication tenofovir/emtricitabine disoproxil fumarate (commercially known as Truvada^®) could safely and effectively prevent HIV acquisition through sex among MSM and transgender women. With participants' consent, the interviews and focus groups were recorded and transcribed. The data were then summarized and analyzed using a qualitative descriptive approach. The majority of men were unfamiliar with PrEP. It was suggested that additional information about the medication and clinical trials establishing efficacy was needed to enhance the legitimacy and relevancy of the messages. Participants sought to form an opinion of PrEP that was grounded in their own interpretation of the efficacy data. However, confusion about nonadherence among clinical trial subjects and individual versus average risk limited comprehension of these messages. Thematic overlaps suggest that message believability was connected to participants' ability to derive meaning from the PrEP efficacy data. Despite being concerned that other MSM would interpret the messages to mean that condom use was unnecessary while taking PrEP, participants themselves primarily understood PrEP as a supplement rather than a replacement for condoms. Based on their experience with taking antiretroviral medication, HIV-positive men considered condom use a more feasible form of HIV prevention than PrEP. Participants' responses suggest that more information about PrEP and the clinical trial would support the legitimacy of PrEP and the messages as a whole. These details may enhance believability in the concept of PrEP and reinforce confidence in the validity of the efficacy result.

Introduction

Since 2000, men who have sex with men (MSM) have been the only risk group in the United States in which HIV incidence is increasing,¹ accounting for 65% of estimated new HIV infections in 2011.² In 2014, the largest estimated number and percentage of new diagnoses of HIV infection among MSM in the United States occurred among black MSM (11,201; 38%), followed by whites (9008; 31%), and Hispanics/Latinos (7552; 26%).³ Young black MSM ages 13–29 have the greatest number of estimated incident infections (9108) than any other subgroup by race/ethnicity, age, and sex.³ Existing approaches to HIV prevention—behavioral risk reduction, promotion of condom use, and counseling and testing—have been insufficient for reducing HIV incidence among MSM. Results from recent large-scale randomized controlled trials of antiretroviral (ARV) medications for prevention [i.e., pre-exposure prophylaxis (PrEP)] have emerged as an effective new biomedical prevention modality.⁴ In 2012, Ackers et al.⁵ showed an overall HIV incidence of 2.7 per 100 person years among MSM (N = 4684) participating in an HIV vaccine study in 47 US cities, and highest incidence among young MSM and those reporting unprotected sex, recreational drug use, and a history of a sexually transmitted infection. Ackers et al.⁵ called for a renewed focus on effective interventions for ARV uptake and adherence for prevention of HIV.

The safety and efficacy of PrEP in HIV seronegative MSM were established in the multinational “iPrEx” study: a double-blind, randomized controlled trial comparing a once-daily oral tablet containing emtricitabine and tenofovir disoproxil fumarate (FTC-TDF) to placebo in 2499 MSM, where FTC-TDF resulted in an overall 44% reduction in the incidence of HIV infection after a median of 1.2 years.⁶ Notably, when taken consistently, PrEP has been shown to reduce the risk of HIV infection in people who are at high risk by 95%. However, the efficacy of ARVs for prevention is closely tied to uptake and adherence. Evidence suggests that PrEP efficacy decreases exponentially with fewer doses of PrEP per week, with maximum efficacy achieved when six to seven doses are taken per week, but still high protection with at least four doses per week.⁷ Continuing to develop and test innovative behavioral interventions that increase knowledge, motivations, and skills to access a variety of behavioral and biomedical HIV prevention options is necessary to decrease the population burden of disease among MSM.^8

–13

Formative research is essential to the development of effective health messages.¹⁴ Message concepts—partially formulated message ideas that convey the main elements to be represented in the finished product—are pretested with members of the target audience before the content is finalized. Pretesting provides direction for the generation of appropriate terminology, illuminates audience understanding of claims and evidence, and identifies potential sources of confusion.¹⁵ Refining the message concepts based on the pretest findings ensures that the final version of the message reflects the audience needs and positioning. Informed by this approach, we conducted qualitative interviews and focus groups to explore audience reactions and receptivity to message concepts on PrEP with black and Latino MSM in three US cities.

Methods

Participants and procedures

The Messages4Men study was carried out in 2013 as part of a multi-phase research initiative to develop an HIV prevention messaging campaign for black and Latino MSM in the United States. It took place in three cities: Chicago, Illinois, Fort Lauderdale, Florida, and Kansas City, Missouri.

Individuals were recruited from community-based organizations, street-based outreach, and through social networking websites and local editions of partner seeking websites in each city. A member of the research team screened men for the following criteria: (1) 18 years of age or older; (2) identified as black and/or Latino; (3) engaged in sex with another man in the past 12 months; and (4) lived and/or worked in one of the three cities. To capture the perspectives of MSM with different sexual risk profiles, equal numbers of HIV-positive, HIV-negative “higher-risk,” and HIV-negative “lower risk” MSM were enrolled; “higher risk” was defined as condomless anal sex in the past 3 months with a male partner. Once determined eligible, participants were asked about their HIV status and recent sexual practices. As recruitment proceeded, individuals were selected based on these characteristics. Participants were scheduled for either an individual interview or a focus group depending on their availability.

Study visits were conducted in English or Spanish by trained research staff in a private space. After rescreening for eligibility into the study, participants underwent a detailed verbal informed consent process before data collection commenced. Interviews and focus groups lasted 60–90 min and were audio-recorded with participant consent. Separate focus groups were held for HIV-infected, HIV-uninfected “higher-risk,” and HIV-uninfected “lower-risk” MSM. Each group contained six to nine men. At the end of the interview or focus group, participants had the option to receive a free rapid HIV test (OraQuick ADVANCE^® Rapid HIV-1/2 Antibody Test) and were given a $50 gift card for participating. To preserve confidentiality, data were de-identified, stored in a secure location, and securely transferred electronically to investigators involved in the analysis. The study was approved by the Institutional Review Board at John Snow, Inc., Boston, MA.

Interviews and focus group discussions

Participants were presented with nine brief (i.e., two-three sentences) messages on HIV prevention-related topics, including PrEP, HIV testing, condom efficacy, HIV serosorting, and antiretroviral medication. The present analysis focuses on responses from two preliminary messages about PrEP (Table 1), including PrEP informational messages to HIV-positive men in reference to their HIV-negative partners' use.

Table 1.

Messages Presented to Black and Latino MSM Participants (N = 90)

Message 1	You can reduce your risk of getting HIV if you take an anti-HIV pill every day. A recent study was done of gay and bisexual men who did not have HIV and were asked to take an anti-HIV pill every day. Even though some of the men did not take the pill every day, the group as a whole reduced their chance of getting HIV by 44%.
Message 2	You can reduce your risk of getting HIV if you take an anti-HIV pill every day. A recent study was done of gay and bisexual men who did not have HIV and were asked to take an anti-HIV pill every day. For the men who took the pill every day, the group as a whole reduced their chance of getting HIV by 95%.

MSM, men who have sex with men.

Efficacy estimates were based on the iPrEx clinical trial of tenofovir with emtricitabine for HIV prevention in MSM.⁶ Message 1 cited the overall efficacy finding of 44%; message 2 cited the 95% efficacy finding for the subgroup of MSM with detectable drug levels. Participants were presented with the messages and then asked the following questions: (1) What does this information mean to you? (2) How believable is this information to you? and (3) How might this information change what you usually do to prevent HIV infection? Response clarification was directed through the use of probes, which were tailored to the individual or group dynamics as per the natural flow of conversation.¹⁶ Participants received additional information about PrEP and the iPrEx trial results at the end of the study visit.

Data analysis

The interviews and focus group discussions were analyzed using a qualitative descriptive approach.¹⁷ Two research staff members per site listened to the audio-recordings and generated transcriptions and detailed written summaries using illustrative quotes to elucidate key concepts. The summaries were triangulated by a separate member of the research team who compared them with their own notes of the audio-recordings. The finalized summaries were compiled and analyzed. The audio-recordings were reviewed as needed to clarify meaning or provide additional insight. A process of analytical memo writing and continuous within-case and across-case comparisons helped to conceptualize and refine the emergent themes.¹⁷

Results

Across each site, participants ranged from 19 to 61 years old. In terms of educational attainment, about 1/3 of the sample had completed a college degree or higher (32%), about 1/3 had completed some college (36%), and about 1/3 had earned a high school diploma or lower (32%). The sample included 33 HIV-infected and 57 HIV-uninfected MSM. Following a framework for message concept pretesting,¹⁷ the qualitative data are presented according to four assessment categories: (1) Preexisting Awareness of PrEP; (2) Comprehension; (3) Believability; and (4) Perceived Impact on Behavior. Illustrative quotes are provided for individual themes.

Preexisting awareness of PrEP

Most men in the study had no prior knowledge of PrEP. Initial responses to the messages focused on PrEP as a new, unfamiliar concept. “You can take a pill? There's a pill (smirks)? I did not know this. This is new information.” “I didn't know there was a pill to prevent HIV. I'm shocked and surprised. I'm learning something new.” As their first point of exposure to PrEP, participants considered the messages too general and worried that the meaning could be misconstrued.

I feel like it's a very vague statement. That kinda makes me feel like, ‘Oh great there's a cure out there but I need further information’. I think other individuals may not take it that way, and be like, ‘There's a cure!’ Great, I'm gonna be barebacking everything, which is not good.

Many requested additional information on the clinical trial (e.g., number of people enrolled; location of study sites) and the medication (e.g., the name; side effects; how it worked in the body). “But how? What is the mechanism? This is a message that is not giving me enough information.” Participants suggested that providing more information would enhance the legitimacy and relevancy of the messages for MSM learning about PrEP for the first time. “The name of the study needs to be included to make it more believable.” “I need more info. I need the name of the pill to make it real.”

Comprehension

The messages begin with a claim about PrEP: You can reduce your risk of getting HIV if you take an anti-HIV pill every day, followed by iPrex efficacy data—which is intended to substantiate the claim. Rather than take the claim at face value, participants sought to form an opinion of PrEP that was grounded in their own interpretation of the clinical trial results. For instance, despite the fact that the failure rate of PrEP was not made explicit in either message, it featured prominently in several participants' understandings of the intervention. The “5%” failure level in Message 2 was considered a reason for using condoms in conjunction with PrEP. “I would still use protection with random partners because I do not want to be in the 5%.” Participants acknowledged that PrEP did not completely protect them from HIV, but the failure rate was small enough to support its potential utility. In Message 1, the failure rate had a very different implication. Participants generally interpreted “56%” as a high level of risk. Based on this evaluation, most concluded that PrEP was an inadequate strategy for preventing HIV. “Based on the information presented in this message, this (PrEP) is not a good option because there is still a 56% chance of getting HIV.” Although the claims about PrEP were the same, different meanings for the efficacy data in each message yielded two distinct understandings of PrEP.

While the iPrEx efficacy data were an important indicator of meaning in the messages, in some instances they caused confusion. This was particularly the case for Message 1. Large numbers of participants were perplexed that not all MSM in the clinical trial took PrEP on a regular basis. “What kind of study is that where some men took it and others didn't?” Without understanding the reason for this, many struggled to derive meaning from the efficacy estimate. “The meaning of the percentage is not clear. What does the 44% mean for an individual person?” As the response implies, a contributing factor to the confusion was that participants did not distinguish a difference between the concepts of average and individual risk.

The term “anti-HIV pill” also caused confusion. Some thought “anti-HIV pill” referred to medication that eliminated the risk of HIV transmission. These participants struggled to reconcile their understanding of the term with the efficacy estimate—which showed that PrEP was only partially effective.

This is very…(sighs) confusing to me, I see it as… the anti-HIV pill…means that I can go have sex with an HIV-positive person? So generally, it (HIV risk) just reduces by 44%, meaning someone in the group must have gotten it. Regardless if they took the pill every day or not.

Believability

In many ways, the believability of the messages hinged upon participants' ability to make sense of the iPrEx efficacy data. For example, in Message 2 most participants interpreted “95%” as a significant reduction in HIV transmission risk. Not only did the positive evaluation of the figure provide evidence of the intervention's efficacy, it also connoted credibility to the message as a whole. “Totally believable, the data is great!” “I believe it because of the 95% success rate.” “Out of 100—that is a pretty significant reduction, it sounds like it's right.” The definitive tones of these statements convey a sense of certainty—not only in the veracity of the meanings derived from the data but also in the ability of the estimates to represent fact. Message 1 was treated with far more skepticism. The perceived incoherence of the data delegitimized both the clinical trial and the message.

This is the most absurd message. It isn't worth sharing with anyone. It doesn't have any scientific validity specifically the statement ‘even though some of the men did not take the pill every day.’

For both messages, a group of participants seemed to be looking for a “truth” about PrEP beyond the quantitative data presented to them. Suspicions were raised as to whether other factors caused the reduction of HIV transmission risk among clinical trial subjects. “I need to know how risk was reduced. Condom use? Monogamy?” “I want to believe that the 95% is reduced by the pill alone but I'm on the fence.” Similarly, participants in Ft. Lauderdale emphasized a distinction between the “real world” and the clinical trial environment—suggesting that the favorable conditions of the research setting contributed to the success of PrEP. “Where the study took place probably helped with getting a 95% reduction [in risk], but in the real world this might not be the case.”

There were a few men who did not believe in the concept of PrEP and denied the accuracy of the messages altogether. Having an anti-HIV pill isn't real, it's not a true statement.” “Not really believable. I know what HIV is, there can't be just a pill you take to protect yourself from HIV.” Some of these participants said they were mistrustful of the messages because they did not contain enough information. “It's too good to be true. It sounds like ‘snake oil’. There is more to this. I wouldn't believe it due to the fact that there isn't enough info.”

Perceived impact on behavior

A salient theme throughout the interviews and focus groups was a concern that MSM would engage in high-risk sexual risk behaviors if they were taking PrEP. Participants talked about the “false sense of security” that PrEP could provide if MSM believed they no longer were at risk for HIV and/or STDs. As the messages did not state whether MSM in the clinical trial used condoms in conjunction with taking PrEP, concerns were raised about whether the message would be interpreted to mean that PrEP supplanted the need to use condoms.

This message would influence people in a negative way, it promotes unprotected sex.

What's this? It is license not to use condoms. You can reduce it by 95%. There is still 5%, there is still a risk. It makes you think it's ok to have unprotected sex because you're ok with just taking the pill.

Risk compensation was also raised as a potential issue for MSM who were taking PrEP. “You can have as much sex as you want and won't get HIV because of the pill. You can branch out and do more, have unprotected sex, a lot more promiscuity.” In light of these concerns, many men wanted the messages to underscore the importance of engaging in other types of risk reduction strategies in concurrence with PrEP. “It needs to be clearer and specify that the condom should always still be used.”

Despite thinking that other MSM would interpret the messages to mean that condom use was unnecessary while taking PrEP, this was not the way the majority of participants understood them. With few exceptions, PrEP was described as a supplement, rather than a replacement for existing risk reduction strategies (e.g., condoms). “It would help. Everything that can assist in not getting HIV…I would still use a condom. I would be able to add the anti-HIV pill to it (condom).” “Your best bet, condom and a pill.” The use of PrEP without condoms was also described in terms of harm reduction. “Some people don't like to use a condom. If they know they can take a pill every day it would help. Something is better than nothing.”

The experience of taking antiretroviral medication gave HIV-positive MSM a unique perspective on PrEP uptake and utilization. Overall, there was a sense that condom use was an easier and simpler HIV prevention strategy than taking a daily dose of PrEP. “This doesn't make any sense, why would you take a pill when you can just use a condom?” Medication adherence and cost were raised as potential concerns.

Pills are very expensive and the regimen itself is very difficult to adhere to. From the point of view of the negative person, it would not be fair to have that person stuck to the regime forever.

Further, participants suggested that HIV-negative MSM would find PrEP less appealing if they were aware of the challenges associated with taking a medication daily.

An uninformed HIV-negative person may be very excited, but for us living with HIV we know the reality of having an incurable disease. We know that taking a pill every day is not easy.

Discussion

Pretesting message concepts for comprehension, believability, and relevance with members of the target audience are an important aspect of the health message design process.¹⁵ Following this approach, we conducted qualitative interviews and focus group discussions to pretest two preliminary messages (that could appear through both larger social marketing efforts and at the individual level, via paper or technology-delivered educational materials) as part of the development of a health messaging intervention on PrEP for black and Latino MSM in three US cities.

Our findings provide insight into how participants derived meaning from the messages. Overall, there was a reluctance to accept the messages' claim: You can reduce your risk of getting HIV if you take an anti-HIV pill every day. Rather than take the claim at face value, it was important for participants to form an opinion of PrEP based on their own interpretation of the efficacy data. PrEP was considered far more acceptable in Message 2 than Message 1. This is in keeping with a 2014 study among MSM, which found that an efficacy estimate for the subgroup of iPrEx trial subjects with detectable drug levels was looked upon more favorably than that of the sample as a whole.¹⁸ Not surprisingly, 44% versus 95% is a (relatively) easy comparison for most people. Of note, in the qualitative guide used for this study, there were also messages about condoms being 70% effective.¹⁹ There are two important observations with respect to this message as follows. (1) Seventy percent was perceived as “pretty good effectiveness” by some and “much lower than I thought” by others and (2) although we asked MSM to “forget” previous messages when presented with the next, some people saw the 95% for PrEP in contrast to the 70% for condoms. Interestingly, it did not seem to shake the view that condom usage (and concurrent messaging about it) with use of PrEP was essential. This highlighted the sense of condoms as HIV prevention “religion” that would stick around for a long time, regardless of the proclaimed effectiveness of a biomedical prevention method. There is a general lack of research (for which we are aware) on MSM at risk for HIV or HIV-infected MSM with a strong belief in condom usage. For HIV-infected MSM in particular, this may stem from an internal sense of self-responsibility for protection of their sexual partners, or perhaps for some, an awareness of the risk of potential legal implications for not using condoms with their sexual partners; neither was explicitly mentioned by any of the participants. Further research to disentangle this phenomenon in the era of PrEP is warranted.

Further, two factors contributed to participants' understanding of PrEP as an effective HIV prevention strategy: a positive evaluation of the efficacy estimate and a sense of confidence in their ability to derive meaning from it. Conversely, skepticism toward PrEP emanated from confusion around the efficacy data and a perception that the messages did not contain enough information. Reluctance to accept message claims and the need to draw one's own conclusion about PrEP may be an expression of medical mistrust.

Stemming from a history of discrimination in healthcare and research, HIV-specific medical mistrust is prevalent among racial and ethnic minorities in the United States and has been shown to impact HIV prevention and treatment behaviors.^20
–22 As a biomedical intervention whose efficacy is evidenced by clinical research, PrEP may be regarded with particular suspicion by black and Latino MSM. To address mistrust, messages about PrEP must be perceived as straightforward, comprehensible, and legitimate.

When interpreting these results, the following limitations should be kept in mind. As discussed in the methods, these data were collected as part of a broader study of message concepts on both behavioral and biomedical aspects of HIV prevention, and as part of the development of a messaging intervention to promote HIV prevention awareness among black and Latino MSM in the United States. It is important to mention that data were examined in aggregate, and even though no across site, serostatus-specific, or educational attainment differences emerged in reported findings, the sample size may have been too small to conclusively determine this. In addition, social desirability may have led to a mischaracterization or overexaggeration of views regarding messages. However, this was likely minimized within this sample because each agency we partnered with is perceived as relatively safe space by MSM.

Participants' responses suggest that more information about PrEP and the clinical trial would support the legitimacy of PrEP and the messages as a whole. These details may enhance believability in the concept of PrEP and reinforce confidence in the validity of efficacy results. Additional information about concurrent condom use may help reduce the potential of the message being misconstrued by MSM in the community. Notably though, participants themselves viewed PrEP as a supplement to condom use. Further, including efficacy results in messages assumes that the audience has skills to make sense of clinical research findings; this assumption needs to be fully explored in the context of biomedical prevention messaging. Finally, participants' blurring of individual versus average risk and confusion around subject nonadherence indicate that future messaging should be mindful when using epidemiological/clinical terms and more focused on individual-level risk. Of great importance to PrEP messaging among MSM: It is difficult to provide additional information to enhance legitimacy and believability while ensuring content is simple and easy to understand. If comprehension is compromised, so too are legitimacy and believability. Conducting formative research, as done here, is the first step in ensuring a balance.

Funding

This research was supported by the Centers for Disease Control and Prevention grant number U01PS0053307-01 (PI: M.M).

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Johnson

, Hall

, Hu

, et al. Trends in diagnoses of HIV infection in the United States, 2002–2011. JAMA, 2014; 312:432–434.

Herbst

, Raiford

, Carry

, et al. Adaptation and national dissemination of a brief, evidence-based, HIV prevention intervention for high-risk men who have sex with men. Morb Mortal Wkly Rep (MMWR), 2016; 65:42–50.

Center for Disease Control and Prevention (CDC). HIV Surveillance Report; (2015): Vol. 26. Available at: www.cdc.gov/hiv/library/reports/surveillance/ (Last accessed June 10, 2016 ).

Jiang

, Yang

, Ye

, et al. Pre-exposure prophylaxis for the prevention of HIV infection in high risk populations: A meta-analysis of randomized controlled trials. PLoS One, 2014; 9:e87674.

Ackers

M-L

, Greenberg

, Lin

, et al. High and persistent HIV seroincidence in men who have sex with men across 47 U.S. cities. PLoS One, 2012; 7:e34972.

Grant

, Lama

, Anderson

, et al. Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. N Engl J Med, 2010; 363:2587–2599.

Anderson

, Glidden

, Liu

, et al. Emtricitabine-tenofovir exposure and pre-exposure prophylaxis efficacy in men who have sex with men. Sci Transl Med, 2012; 4:151ra125.

Mimiaga

, Case

, Johnson

, et al. Pre-exposure antiretroviral prophylaxis (PrEP) attitudes in high risk Boston area MSM: Limited knowledge and experience, but potential for increased utilization after education. J Acquir Immune Defic Syndr, 2009; 50:77–83.

Chan

, Rose

, Maher

, et al. AIDS Patient Care STDs, 2015; 29:597–605.

10.

Mutchler

, McDavitt

, Ghani

, et al. Getting PrEPared for HIV prevention navigation: Young black gay men talk about HIV prevention in the biomedical era. AIDS Patient Care STDS, 2015; 29:490–502.

11.

Eaton

, Driffin

, Bauermeister

, et al. Minimal awareness and stalled uptake of pre-exposure prophylaxis (PrEP) among at risk, HIV-negative, black men who have sex with men. AIDS Patient Care STDS, 2015; 29:423–429.

12.

Mimiaga

, Closson

, Kothary

, et al. Sexual partnerships and considerations for HIV antiretroviral pre-exposure prophylaxis utilization among high-risk substance using men who have sex with men. Arch Sex Behav, 2014; 43:99–106.

13.

Mayer

, Oldenburg

, Novak

, et al. Early adopters: Correlates of HIV chemoprophylaxis use in recent online samples of US men who have sex with men. AIDS Behav, 2015; 20:1489.

14.

Thompson

, Parrott

, Nussbaum

(Eds.) The Routledge Handbook of Health Communication, (2nd ed.); New York, NY: Routledge, 2011.

15.

Rice

, Atkin

. Public Communication Campaigns, (4th ed.). Thousand Oaks, CA: Sage Publications, 2013.

16.

Lindlof

, Taylor

. Qualitative Communication Research Methods, (3rd ed.); Thousand Oaks, CA; Sage Publications, 2011.

17.

Sandelowski

. What's in a name? Qualitative description revisited. Nurs Health, 2010; 33:77–84.

18.

Underhill

, Morrow

, Colleran

, et al. Explaining the efficacy of pre-exposure prophylaxis (PrEP) for HIV prevention: A qualitative study of message framing and messaging preferences among US men who have sex with men. AIDS Behav, 2016; 20:1514–1526.

19.

Smith

, Herbst

, Zhang

, Rose

. Condom effectiveness by consistency of use among men who have sex with men (MSM) in the United States. J Acquir Immune Defic Syndr, 2015; 68:337–344.

20.

Ross

, Essien

, Torres

. Conspiracy beliefs about the origin of HIV/AIDS in four racial/ethnic groups. J Acquir Immune Defic Syndr, 2006; 41:342–344.

21.

Bogart

, Wagner

, Galvan

, Banks

. Conspiracy beliefs about HIV are related to antiretroviral treatment nonadherence among African American men with HIV. J Acquir Immune Defic Syndr, 2010; 53:648–655.

22.

Bogart

, Galvan

, Wagner

, Klein

. Longitudinal association of HIV conspiracy beliefs with sexual risk among black males living with HIV. AIDS Behav, 2011; 15:1180–1186.