Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use

Abstract

Background

People with HIV are more likely than the general population to have a substance use disorder (SUD), which can impact the HIV care continuum. HIV service organizations (HSOs) can implement SUD interventions but may need assistance from support systems like the AIDS Education and Training Center (AETC) network. We assess the fit of strategies AETCs may use to help HSOs integrate SUD interventions.

Method

We invited 74 of 91 AETCs (81.3%) to participate. Using a real-time Delphi approach, 64 AETCs (86.5% of those invited) rated the (a) importance of, (b) feasibility of, (c) readiness to offer, (d) scalability of, (e) pressure to offer, and (f) current need for 10 strategies their AETC could use to help HSOs integrate SUD interventions. Items were examined via confirmatory factor analyses. Responses were summed to create the Setting-Strategy Fit index score. We conducted pairwise t-tests to examine differences in scores between strategies, plotted the mean importance ratings for each strategy against the mean ratings for other criteria to review the strategies’ relative viability, and conducted bivariate and multiple regression analyses to examine correlates of the scores.

Results

The items of the Setting-Strategy Fit index showed good internal consistency and model fit. Generally, strategies were considered somewhat important but AETCs felt very little pressure to offer them. Two strategies (disseminating information, providing access to asynchronous training) exceeded the “important” threshold. One strategy (disseminating information) was considered viable for also having high feasibility. Overall, AETCs were only somewhat ready to provide the strategies, which were perceived as only somewhat feasible or currently needed.

Conclusions

Although AETCs recognized the importance of several strategies for helping HSOs integrate SUD interventions, their responses resulted in only one having good fit. These findings can guide efforts to further prepare AETCs to support HSOs and to end the HIV epidemic.

Plain Language Summary

Strategies AIDS Education and Training Centers Could Use to Help HIV Service Organizations Implement Substance Use Interventions.

Why was the study done?

Having a substance use disorder (SUD) can complicate care for people with HIV by reducing their engagement in services. HIV service organizations (HSOs) serve people with HIV but not all of them offer services to address SUD. We assessed the fit of different strategies that AIDS Education and Training Centers (AETCs), which provide technical assistance to HSOs, could use to help HSOs implement SUD interventions.

What did the researchers do?

We engaged 74 AETC representatives nationally to rate 10 strategies. They were asked about (a) the importance of the strategies, (b) the feasibility of offering them, (c) their readiness to offer them, (d) the scalability of the strategies, (e) pressure they receive to offer them, and (f) the current need for them. Responses were summed into an index score reflecting the strategies’ fit for AETCs to help HSOs integrate SUD interventions. We looked for differences in scores between strategies and plotted the scores for each strategy to identify which are most promising.

What did the researchers find?

Two of the strategies (sharing information about the SUD interventions and providing access to asynchronous trainings) exceeded the threshold for being “important.” One of those (sharing information) was also considered “feasible” for AETCs to offer, making it the only potentially viable strategy. Overall, AETCs were only somewhat ready to provide the strategies, which they felt very little pressure to offer.

What do the findings mean?

Although AETCs recognized the importance of different strategies for helping HSOs implement SUD interventions, they only identified one strategy as having good fit. These findings can guide future efforts to further prepare AETCs to support HSOs and to end the HIV epidemic.

Keywords

substance use disorder HIV Delphi implementation strategies intermediary organizations

Background

Substance use disorders (SUDs) are prevalent among people with HIV (PWH). Nearly half of PWH in the United States have an SUD (Hartzler et al., 2017; Liu et al., 2024), which is almost three times that of the general population (SAMHSA, 2023). The most common SUDs among PWH are for marijuana (31%), alcohol (19%), methamphetamines (13%), cocaine (11%), and opioids (4%). In a study exploring SUD prevalence among PWH, organizations serving PWH perceived the prevalence to be even higher at 42.3%, 41.9%, 32.2%, 28.1%, and 34.6% for each SUD, respectively (Garner et al., 2021). SUDs impact every stage of the HIV care continuum, including increased risk of new infections, delayed testing and diagnosis, and reduced adherence to medication and viral suppression (Ahmed et al., 2024; Arnsten et al., 2002; Azar et al., 2010; Friedman et al., 2009; Hall et al., 2013; Hendershot et al., 2009; King et al., 2009; Lesko et al., 2023; Lucas et al., 2001; Ma et al., 2024; Malta et al., 2008; Mimiaga et al., 2020; Paschen-Wolff et al., 2020; Patrick et al., 2012; Przybyla et al., 2022; Reback et al., 2021; Taylor et al., 2023). Considering the impact that different SUDs have on the care continuum, as well as their prevalence among PWH, a real-time Delphi study that engaged PWH, staff at HIV service organizations (HSOs), and HIV/AIDS Planning Council members identified alcohol use disorder, methamphetamine use disorder, and opioid use disorder as the three SUDs with the highest population-level negative impacts (Garner et al., 2021).

Addressing SUDs among PWH is critical to “Ending the HIV Epidemic,” according to the comprehensive initiative that seeks to reduce the number of new HIV infections in the United States by 90% by 2030 (HHS, 2021). This initiative emphasizes integrating substance use and HIV care by increasing HIV testing in substance use prevention and treatment programs, increasing access to syringe services and preexposure prophylaxis, and supporting those who have not yet reached viral suppression (HHS, 2021). Integrating SUD interventions into HSOs is key to supporting PWH and reducing the number of new HIV infections (Haldane et al., 2017). Although many HSOs in North America report offering substance use-related education, screening, and referral (71%; Parcesepe et al., 2020), few PWH receive substance use treatment (Burnam et al., 2001; Durvasula & Miller, 2014; Goldstein et al., 2005). HSOs can help address unmet service needs by integrating SUD interventions into settings where PWH already access services.

Fortunately, several effective SUD interventions are promising for integration into HSOs. A recent real-time Delphi study of HSOs assessed the Setting-Intervention Fit of nine evidence-based interventions (i.e., their perceived fit for integration within HSOs) on six criteria: whether they were fundable, implementable, retainable, sustainable, scalable, and timely (Garner et al., 2022). Overall, three psychosocial interventions (motivational interviewing, cognitive behavioral therapy, and contingency management) had higher Setting-Intervention Fit scores compared to six pharmacological interventions (acamprosate, injectable buprenorphine, oral buprenorphine, disulfiram, injectable naltrexone, and oral naltrexone). Helping HSOs implement psychosocial interventions to treat SUD will likely help them better support PWH and improve outcomes along the HIV care continuum.

National efforts to support adoption and scale-up of effective SUD interventions in HSOs, however, have been limited. The Health Resources and Services Administration's AIDS Education and Training Center (AETC) Program is a national network of HIV experts who provide training and technical assistance to healthcare providers serving PWH. The network comprises eight regional offices and their local partners that administer training programs. AETCs could serve as intermediaries (i.e., organizations that build the capacity within a system to implement and sustain best practice programs or services; Proctor et al., 2019) to help support integration of effective SUD interventions into HSOs, but historically this has not been a major focus.

To date, most AETC efforts related to SUD have been limited to disseminating information. According to the AETC National Coordinating Resource Center's Resource Library, 69 of hundreds of active resources (e.g., reports, webinar recordings, short courses) are categorized under “SUDs,” with four specific to implementing SUD interventions (AETC Resources). AETCs may better support integration of SUD interventions in HSOs by enhancing partnerships with the Substance Abuse and Mental Health Services Administration-funded regional Addiction Technology Transfer Centers (ATTCs) to offer a broader array of strategies to bridge the gap between knowledge and practice. However, the extent to which strategies (i.e., methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice; Proctor et al., 2013) beyond disseminating information would be feasible for AETCs to offer is unknown.

This article aims to understand the Setting-Strategy Fit for 10 strategies that an AETC may use to help HSOs integrate effective SUD interventions. Previous research by Waltz and colleagues on experts’ ratings of the importance and feasibility of discrete strategies provided insight into the general applicability of the Expert Recommendations for Implementing Change (ERIC) strategies (Waltz et al., 2015). That study found a strong relationship between importance and feasibility ratings by plotting strategies in four quadrants. Strategies tended to fall in the high importance and high feasibility or low importance and low feasibility quadrants. In the current study, we sought to further understand the applicability of a subset of strategies by assessing them from the perspective of intermediary AETC representatives. Furthermore, we build on previous findings by assessing the subset of strategies across four additional dimensions (readiness, scalability, tension for change, and timeliness) to provide a more comprehensive understanding of strategy fit. Our goal was to identify the most promising strategies that AETCs can use to support HSOs to integrate effective psychosocial SUD interventions.

Method

Setting and Participants

In April 2021, we emailed initial invitations to complete a screener survey to 91 AETC representatives from across the United States (Figure 1). We identified these representatives from a public directory that consisted of individuals in leadership positions, such as Program or Clinical Directors, Program Managers, andTraining Coordinators ( AETC Directory ). Participants were required to be at least 18 years of age as well as employed by and able to represent their regional AETC office or local partner. We invited eligible participants to complete the study. In cases where there was more than one eligible representative from a particular regional office or local partner, we randomly selected one respondent to participate on behalf of their organization. We ultimately invited representatives of all eight regional offices and 66 of the local partners to participate (81.3% of AETCs). We sent a $110 gift card to participants after they completed study activities.

Figure 1.

Participant Flow Diagram.

Study Design

This study builds on our previous work to (a) further develop the real-time Delphi method and (b) to identify promising strategies for supporting integration of effective SUD interventions into HSOs (Garner et al., 2021 , 2022; Gordon & Pease, 2006). Delphi methods leverage a ground-up approach to generate consensus on a complex topic among a group of experts by asking them to complete rounds of questionnaires, after each of which participants are provided a summary of how others responded. Consensus is developed over the course of the rounds of questionnaires and feedback (Linstone & Turoff, 1975). In its original form, the Delphi process can be time-consuming and costly (multiple rounds of sending out questionnaires, summarizing responses, and sharing results back). A real-time Delphi, however, enables the process to occur instantaneously so that participants can review and respond to the most recent information entered by other participants. For the current study, we developed an innovative web-based real-time Delphi platform. Participants were asked to review information (see the strategies section), respond to questions (see the variables section), provide reasoning for their response selections, review the anonymous responses and reasoning provided by other participants, and subsequently adjust their own responses if so inclined. We asked participants to engage with the platform at least twice over a 2-week period. RTI International's Institutional Review Board approved all study procedures.

Strategies

Many strategies to facilitate implementation of interventions have been identified (EPOC, 2015; Powell et al., 2015). We narrowed our focus by engaging HIV, SUD, and implementation practitioners and researchers from the project's Guiding Coalition, which included representatives of AETCs and ATTCs (Kotter, 2012). Five coalition members (two who represent training and technical assistance centers, four who identify as SUD researchers, and four who identify as implementation researchers) formed a strategies subgroup and led the effort to identify the most relevant strategies to ask AETC representatives to consider. Starting with the 73 strategies listed in the ERIC taxonomy (Powell et al., 2015), the subgroup identified 25 strategies that (at least some) local AETC partners were already using or could use to support organizations in implementing various interventions. Through discussions with the broader coalition, the list was narrowed to 10 strategies across the phases of (E) exploration, (P) preparation, and (I) implementation (Aarons et al., 2011) that vary in the resources they require. The final set of strategies included three exploration phase strategies: (E1) disseminating information about the evidence-based intervention, (E2) conducting a formal assessment, and (E3) obtaining a formal commitment; four preparation phase strategies: (P1) developing an implementation plan, (P2) providing access to asynchronous training, (P3) conducting synchronous training, and (P4) assessing provider proficiency; and three implementation phase strategies: (I1) providing clinical consultation, (I2) providing implementation support, and (I3) facilitating an ongoing learning collaborative.

We developed infographics for each of the 10 strategies so that participants would have the same basic understanding to inform their responses. The infographics specified the goal of the strategy, the actions an AETC would need to take to carry out the strategy (including who they would need to involve at HSOs receiving support), what skills AETC staff would need to deliver the strategy, and the level of financial resources and time an AETC would need to devote to delivering the strategy (Supplemental File 1, Supplemental Figures 1–3). The Guiding Coalition's strategies subgroup developed the content for each infographic and then revised it based on feedback from the broader coalition. Additionally, we created animated versions of the infographics with voiceover to help aid understanding of the presented information.

Variables

We collected participant characteristics (e.g., age, sex, race/ethnicity, knowledge about their AETC) and AETC characteristics (e.g., AETC region and type, staffing, number of HSOs and PWH in jurisdiction) online via the screening process. This included whether supporting integration of SUD interventions in HSOs was part of their AETC's current work plan, the extent to which their AETC currently shared responsibility with its regional ATTC for supporting integration of SUD interventions in HSOs, and their AETC's current relationship with their regional ATTC. These measures and response options are presented in Table 1.

Table 1.

AETC Characteristics

	Frequency or mean	Percentage or SD
AETC region (n = 64)
Pacific	11	17.2
Northeast/Caribbean	10	15.6
South Central	10	15.6
Mountain West	8	12.5
New England	7	10.9
Southeast	7	10.9
Mid-Atlantic	6	9.4
Midwest	5	7.8
Respondent level of knowledge regarding their AETC (n = 64)
Not knowledgeable at all	0	0.0
Knowledgeable to a small extent	0	0.0
Knowledgeable to a moderate extent	11	17.2
Knowledgeable to a great extent	53	82.8
AETC staffing
Number of full-time staff (n = 64)	9.1	49.8
Number of part-time staff (n = 63)	5.0	12.9
AETC jurisdiction
Number of HSOs in jurisdiction (n = 59)	120.3	303.3
Number of people with HIV in jurisdiction (n = 57)	28,629.7	44,669.6
Extent to which respondent believes it is important for SUD-related services to be integrated into HSOs (n = 61)
Not at all	0	0.0
To a minor extent	1	1.6
To a moderate extent	10	16.4
To a great extent	50	82.0
Extent to which SUD-related services are currently integrated into HSOs in respondent's AETC jurisdiction (n = 61)
Not at all	0	0.0
To a minor extent	24	39.3
To a moderate extent	32	52.5
To a great extent	5	8.2
Extent to which helping HSOs integrate SUD-related services is expected by HRSA and/or respondent's regional AETC office (n = 60)
Not at all	4	6.7
To a minor extent	14	23.3
To a moderate extent	29	48.3
To a great extent	13	21.7
Extent to which helping HSOs integrate SUD-related services is supported by HRSA and/or respondent's regional AETC office (n = 59)
Not at all	0	0.0
To a minor extent	14	23.7
To a moderate extent	21	35.6
To a great extent	24	40.7
Extent to which helping HSOs integrate SUD-related services is rewarded by HRSA and/or respondent's regional AETC office (n = 60)
Not at all	10	16.7
To a minor extent	27	45.0
To a moderate extent	18	30.0
To a great extent	5	8.3
Helping HSOs integrate SUD-related services is part of respondent's local AETC's current work plan (n = 61)
No	29	47.5
Yes	25	41.0
I don’t know	7	11.5
AETC phase of implementation for helping HSOs improve their integration of SUD-related services (n = 60)
(Exploring) Perceive a need to help HSOs integrate SUD-related services and have identified possible solutions	29	48.3
(Preparing) Have decided to help HSOs integrate SUD-related services	6	10.0
(Implementing) Have begun to help HSOs integrate SUD-related services	13	21.7
(Sustaining) Routinely help HSOs to integrate SUD-related services	10	16.7
Do not perceive a need to help HSOs integrate SUD-related services	2	3.3
Extent to which respondent believes the responsibility for helping HSOs integrate SUD-related services should be shared between your local AETC and the ATTC (n = 61)
% ATTC responsibility	60.3	18.7
% AETC responsibility	39.7	18.7
Extent to which respondent believes the responsibility for helping HSOs integrate SUD-related services is currently shared between your local AETC and the ATTC (n = 60)
% ATTC responsibility	53.6	32.9
% AETC responsibility	46.4	32.9
AETC relationship with ATTC (n = 61)
No relationship	20	32.8
Networking relationship	19	31.2
Cooperating relationship	15	24.6
Coordinating relationship	0	0.0
Collaborating relationship	7	11.5

Note. AETC = AIDS Education and Training Center; ATTC = Addiction Technology Transfer Centers; HRSA = Health Resources and Services Administration; HSO = HIV service organizations; SUD = substance use disorder.

We then deployed the real-time Delphi to identify the most promising strategies for AETCs to support integration of psychosocial SUD interventions within HSOs. After they reviewed the information describing each strategy, we asked participants to think about their AETC and the HSOs they serve and to rate the extent to which (a) the strategy was important, (b) the strategy was feasible, (c) the AETC was currently ready to offer the strategy, (d) the strategy was scalable, (e) the AETC felt pressure to offer the strategy, and (f) how many of the HSO's served by the AETC currently needed the strategy (Supplemental File 2). A 4-point scale was used for each of the criteria, representing importance, feasibility, readiness, scalability, tension for change, and timeliness. These criteria were identified by the Guiding Coalition's strategies subgroup. For each criterion, items were developed through a process of reviewing similar or related measures, identifying or further developing appropriate questions and response options, and refining wording to reflect language used by AETC staff. Input from the broader coalition to further refine the items promoted content and face validity of the items. Summed together, these six criteria form the Setting-Strategy Fit Index (possible range: 0 to 18, with higher scores representing greater fit).

Statistical analysis

For each of the 10 strategies, we used the six items to estimate confirmatory factor analysis (CFA) models assuming ordinal data fit using Mplus 8 (Muthén & Muthén, 1998–2017) and the Mplus Automation R package (Hallquist & Wiley, 2018). Psychometric results included the fit of the CFA models, CFA factor loadings, and Cronbach's alpha estimated using the psych R package (Revelle, 2022) using polychoric correlation matrices estimated using the misty R package (Yanagida, 2023). Fit was determined from the following criteria: Root Mean Square Error of Approximation (RMSEA) less than 0.05 (less than 0.08 still acceptable), Comparative Fit Index and Tucker–Lewis Index (TLI) greater than 0.95 (greater than 0.90 still acceptable), and significant chi-squared tests (p-values less than .05).

After reviewing the psychometric results, we calculated mean scores for each criterion item and the Setting-Strategy Fit Index score. We conducted pairwise t-tests to test for significant (p < .05) differences in the overall mean scores between strategies. To identify differences in fit for each strategy between the regional and local levels (which have varying scopes and resources), we calculated the difference in each regional AETC office's index score and the mean index score among their corresponding local partners.

To further review the fit of each strategy for AETCs, we plotted the mean importance ratings for each strategy against the mean ratings for each of the other five items. We prioritized the importance criterion in this way because if a strategy is not perceived as important by AETCs, they would be unlikely to offer it and the other criteria would be moot. Each plot was divided into four sections to visualize which strategies might be most viable across dimensions. We colored the section of the plots containing strategies that scored above the threshold for importance (2 or higher) and for the second criterion green (“go” zone), indicating consensus on their high relevance and suitability. Conversely, we left the section with strategies that scored below the threshold for both criteria white, suggesting lower relevance and suitability (“no-go” zone). We colored the section with strategies scoring above the threshold for importance but below the threshold for the other criterion blue, and the section with strategies scoring below the threshold for importance but above the threshold for the other criterion yellow.

Finally, we conducted multiple regression analyses to examine the extent to which characteristics of the AETC were significant predictors of each respective Setting-Strategy Fit Index score. In the regression models, an average of 5% of individuals had missing data on one or more model variables and were excluded from the analysis.

Results

Representatives from all eight regional offices and 56 of the local partners completed the study (Figure 1), resulting in responses from 70.3% of AETCs overall and 86.5% of those invited. Participants had an average age of 50.4 years (11.4 SD) and most identified as female (65.6%), White (76.6%), and not Hispanic (85.7%; data not shown). All participants reported a moderate (17.2%) or great (82.8%) extent of knowledge regarding their AETC.

Nearly all participants believed to a moderate or great extent (98.4%) that it is important for SUD-related services to be integrated into HSOs (Table 1). However, most reported that SUD-related services were currently integrated only to a moderate (52.5%) or minor extent (39.3%) within HSOs in their area. Furthermore, only 21.7%, 40.7%, and 8.3% of the participants reported that their AETC is expected, supported, and rewarded to a great extent to help HSOs integrate SUD-related services, respectively.

All but two participants perceived a need to help HSOs integrate SUD-related services; 48.3% reported being in the exploration phase, 10.0% in the preparation phase, 21.7% in the implementation phase, and 16.7% in the sustainment phase. On average, participants felt that ATTCs should hold more responsibility (60.3% out of 100% total responsibility) than AETCs (39.7% out of 100%) for helping HSOs integrate SUD-related services; the ATTCs’ current share of responsibility was believed to be on average 53.6%. However, 32.8% of participants reported that their AETC has no relationship with their regional ATTC and only 36.1% reported that their AETC has a cooperating or more extensive relationship.

Setting-Strategy Fit

Measure Properties

Psychometric properties of the Setting-Strategy Fit Index by strategy are presented in Supplemental Table 1 (Supplemental File 3). The items developed for this study showed good internal consistency across the 10 strategies, as evidenced by Cronbach's alpha coefficients greater than .79. Although point estimates for RMSEA were above 0.05 for most strategies, acceptable model fit was indicated for half the strategies by confidence intervals that included values below 0.05. CFI values were greater than 0.90 for all strategies, and TLI values were greater than 0.95 for seven of the 10 strategies. Additionally, chi-square tests produced p-values less than .05 for half the strategies. Response norms for each of the six criteria by strategy are available in Supplemental Table 2 (Supplemental File 3). Within each strategy, importance received the highest ratings followed by feasibility. Readiness, scalability, and timeliness had relatively similar ratings within each strategy. Finally, tension for change had the lowest ratings. Skewness values typically range from −1 to 1, indicating only a moderate degree of skewness.

Measure Findings

Figure 2 presents the unadjusted Setting-Strategy Fit Index scores for each strategy to help integrate psychosocial SUD interventions into HSOs, including the average contributions of the six items that comprise the index. Disseminating information about the evidence-based intervention had the highest ratings for each dimension resulting in the highest overall score (9.24; although it just surpassed the midpoint of the potential range of scores) and was the only strategy that had an unadjusted score that was significantly different from all other strategies. Providing access to asynchronous training modules had the second highest unadjusted score (6.97) and was significantly different from all other strategies except conducting a formal assessment, which had the third-highest unadjusted score (6.70).

Figure 2.

Total Setting-Strategy Fit Index Scores and Dimension Contributions

The highest ratings across the 10 strategies were for importance (with 1.70 of 3 being the lowest importance rating for obtaining a formal commitment) and lowest for tension for change (with 0.55 of 3 being the highesttension rating for disseminating information about the evidence-based intervention). In general, the strategies were considered somewhat feasible (ratings ranging from 0.93 to 2.15), but few respondents rated their AETCs as ready to offer them (only two strategies achieved a rating >1), considered the strategies timely (only four strategies achieved a rating >1), or indicated the strategies were scalable (only three strategies achieved a rating >1).

We plotted the importance ratings for each strategy against the ratings for each of the other criteria in Figure 3. Only two strategies (disseminating information about the evidence-based intervention and providing access to asynchronous training) surpassed the important threshold (>2 of 3), although the other strategies were clustered just below the threshold. Only one strategy, disseminating information about the evidence-based intervention, which requires lower effort, basic skills, and few resources (Supplemental File 1, Supplemental Figures 1–3), surpassed the threshold for any of the other criteria (i.e., the strategy was perceived as feasible, that AETCs were ready to offer and could offer it at scale, that AETCs felt pressure to provide it, and that many HSOs needed it)—placing it in the “go” zone only for having high importance and feasibility.

Figure 3.

“Go”/”No-Go” Quadrants

Setting-Strategy Fit Index Correlates

Table 2 displays the difference in Setting-Strategy Fit Index scores between the average score among local partners within a region and the regional AETC office by strategy. Positive values indicate that the average score among local partners was higher than the regional office's score, while negative values indicate that the regional office's score was higher. In general, regional AETC offices had higher scores for all three exploration phase strategies and three of the four preparation phase strategies. For the implementation phase, the local partners tended to close the gap and had less of a difference in scores.

Table 2.

Differences in Setting-Strategy Fit Index Scores Between Regional AETC Offices and Their Corresponding Local Partners^a

	Exploration phase strategies			Preparation phase strategies				Implementation phase
Region	E1	E2	E3	P1	P2	P3	P4	I1	I2	I3
Mid-Atlantic	−4.00	−6.00	−3.00	−4.20	−6.75	−3.20	−0.40	−4.40	−4.20	−3.40
Midwest	−6.00	−5.00	−4.33	−6.33	−5.33	−2.33	−6.67	−1.00	−2.00	2.00
Mountain West	2.00	0.86	5.14	1.50	−2.00	2.33	3.33	4.33	3.67	4.50
New England	1.00	−3.00	−3.17	−1.50	2.83	3.00	1.40	4.60	4.80	4.00
Northeast/Caribbean^b	−5.44	(Missing)	(Missing)	(Missing)	(Missing)	(Missing)	(Missing)	(Missing)	(Missing)	(Missing)
Pacific	−3.80	0.00	0.10	−0.70	−0.10	−1.00	−1.00	−0.30	−2.00	−1.50
South central	−3.00	1.78	−1.00	−0.78	−2.22	−2.13	1.38	−0.63	1.43	−1.43
Southeast	−2.67	−5.83	−4.33	−2.20	−0.60	−1.20	0.60	−0.20	−0.40	−1.60

Note. AETC = AIDS Education and Training Center; E1 = disseminate information about the evidence-based intervention; E2 = conduct a formal assessment; E3 = obtain a formal commitment; P1 = develop an implementation plan; P2 = provide access to asynchronous training; P3 = conduct synchronous training; P4 = assess provider proficiency; I1 = provide clinical consultation; I2 = provide implementation support; I3 = facilitate an ongoing learning collaborative.

Differences are calculated by subtracting the regional office's score from the average score among local partners in that region. Positive values indicate that the average score among local partners is higher than the regional office's score, while negative values indicate that the regional office's score is higher.

Missing data within this region prevents differences from being calculated for nine of the 10 strategies.

Table 3 summarizes the results of the multivariate regression analysis examining organizational characteristics as predictors of the Setting-Strategy Fit Index score for each strategy. Across regions, the Midwest AETCs were significantly negatively associated with Setting-Strategy Fit Index scores for one strategy in each phase (obtaining a formal commitment during exploration, assessing provider proficiency during preparation, and providing implementation support during implementation). Compared to the local partners, regional AETC offices were positively associated with index scores for all three exploration phase strategies (disseminating information about the evidence-based intervention, conducting a formal assessment, obtaining a formal commitment) and one of the preparation phase strategies (developing an implementation plan); however, this was only statistically significant for one of the strategies (conducting a formal assessment). AETCs with a work plan that included supporting integration of SUD interventions at HSOs had a positive effect across all strategies and was significantly associated with predicting index scores for two exploration phase (disseminating information about the evidence-based intervention, conducting a formal assessment) and one preparation phase strategy (developing an implementation plan).

Table 3.

Multivariate Regression Results of AETC Characteristics on Setting-Strategy Fit Index Scores for 10 Strategies

	E1	E2	E3	P1	P2	P3	P4	I1	I2	I3
	Coef (se)	Coef (se)	Coef (se)	Coef (se)	Coef (se)	Coef (se)	Coef (se)	Coef (se)	Coef (se)	Coef (se)
Intercept	10.368 (0.628)	7.749 (0.694)	5.859 (0.755)	6.903 (0.702)	7.355 (0.8)	6.54 (0.816)	5.086 (0.7)	6.357 (0.91)	5.651 (0.823)	4.62 (0.75)
Census region
Northeast	Ref.
South	−0.438 (0.918)	−0.857 (0.984)	−0.754 (1.088)	−0.445 (1.005)	−0.811 (1.138)	−0.406 (1.134)	−0.383 (0.973)	−0.077 (1.323)	−1.069 (1.195)	0.284 (1.089)
Midwest	−0.125 (1.413)	−0.34 (1.594)	−3.584 (1.732)**	0.538 (1.613)	−0.502 (1.836)	−0.909 (1.868)	−3.034 (1.603)*	−0.514 (2.097)	−0.971 (1.888)	−3.288 (1.72)*
West	−1.371 (0.956)	−0.648 (1.042)	−0.684 (1.131)	−0.048 (1.058)	−1.417 (1.172)	−1.428 (1.194)	−1.027 (1.024)	−0.75 (1.38)	−1.18 (1.241)	−0.839 (1.13)
AETC type
Local	Ref.
Regional	1.768 (1.214)	2.383 (1.29)*	1.514 (1.43)	1.607 (1.303)	0.729 (1.503)	0.363 (1.527)	−0.113 (1.31)	−0.214 (1.676)	−0.375 (1.505)	−0.382 (1.372)
AETC size
Small (<4 staff)	Ref.
Medium (4–5 staff)	−0.824 (0.959)	1.526 (0.977)	−0.498 (1.088)	−0.043 (0.991)	−1.136 (1.156)	−0.537 (1.162)	−1.047 (0.997)	0.632 (1.279)	0.187 (1.149)	−0.399 (1.047)
Large (≥6 staff)	−0.336 (1.05)	0.845 (1.071)	−0.593 (1.187)	−0.247 (1.099)	−0.126 (1.253)	−0.489 (1.277)	−0.737 (1.096)	0.075 (1.416)	−0.36 (1.278)	−0.467 (1.164)
Work plan includes SUD
Not included	Ref.
Included	2.155 (0.746)***	2.104 (0.762)***	1.287 (0.833)	1.933 (0.786)**	1.11 (0.909)	0.973 (0.919)	0.826 (0.789)	1.263 (1.016)	1.221 (0.925)	0.871 (0.843)
Number of HSOs served
≤20 HSOs	Ref.
>21 HSOs	0.3 (0.726)	0.927 (0.745)	0.275 (0.816)	0.194 (0.768)	−0.159 (0.88)	−1.164 (0.889)	−0.241 (0.763)	−1.266 (0.996)	−0.589 (0.905)	−0.725 (0.825)
Relationship with ATTC
No relationship	Ref.
Networking or higher relationship	0.929 (0.818)	1.146 (0.853)	0.78 (0.927)	0.647 (0.888)	1.237 (1.02)	0.586 (1.032)	0.363 (0.886)	1.01 (1.173)	0.935 (1.068)	1.109 (0.973)

Note. AETC = AIDS Education and Training Center; ATTC = Addiction Technology Transfer Centers; HSO = HIV service organizations; SUD = substance use disorder; E1 = disseminate information about the evidence-based intervention; E2 = conduct a formal assessment; E3 = obtain a formal commitment; P1 = develop an implementation plan; P2 = provide access to asynchronous training; P3 = conduct synchronous training; P4 = assess provider proficiency; I1 = provide clinical consultation; I2 = provide implementation support; I3 = facilitate an ongoing learning collaborative.

* p ≤ .10, ** p ≤ .05, *** p ≤ .01.

Discussion

Prior efforts have sought to integrate SUD-related services and HIV care in a variety of settings (Falade-Nwulia et al., 2023; Haldane et al., 2017; Weiss et al., 2011). AETC representatives from across the United States reported that SUD-related services should be integrated into HSOs specifically and recognized the importance of several strategies to support such integration. While the strategies included in this study were considered somewhat feasible, AETCs reported that they were generally not ready to offer them, did not consider them timely or scalable, and did not feel pressure to offer them. Notably, the strategies with the highest overall scores (disseminating information about the evidence-based intervention and providing access to asynchronous training) are asynchronous and one-directional. Strategies like these are often planned and conducted once, then archived for passive engagement. In this way, these types of strategies can be easily sustained and require fewer resources but are limited in their ability to provide active and tailored support to HSOs. The next set of strategies with overall scores in the middle—conducting formal assessments, conducting synchronous trainings, and providing ongoing clinical consultation—often require more interactive planning and delivery and slightly more resources. Four of the five lowest-scoring strategies—obtaining a formal commitment, developing an implementation plan, providing ongoing implementation support, and facilitating a learning collaborative—require AETC staff to have strong facilitation skills.

Inclusion of SUD in AETCs’ work plans was correlated with greater fit for a subset of strategies, aligning with research regarding intention to change as a facilitator (Webb & Sheeran, 2006). Internal and external leadership frequently impact work plan priorities, for example through expecting, supporting, and rewarding staff for helping HSOs integrate effective SUD interventions. AETC representatives in this study perceived the Health Resources and Services Administration and/or regional AETC office expectations, support, and rewards for this to be relatively low at the present time. In addition, they reported that most of the responsibility for helping HSOs integrate SUD-related services lies with ATTCs. Importantly, limited resources to support integration of SUD interventions into HSOs may explain their exclusion from work plans, as well as the low readiness and scalability of the strategies. Further elucidation of what leads AETCs to collaborate with ATTCs and to address SUD in their work plans could facilitate improved fit.

Disseminating information about evidence-based interventions was the only strategy to reach a “go” zone for having high importance and feasibility. Our findings on AETCs’ perceived importance and feasibility matched the expert ratings reported by Waltz et al., (2015) for this strategy only. Waltz et al., included several training strategies that were largely rated as both important and feasible and in the “go” zone. We included two training strategies—providing access to asynchronous training and conducting synchronous training—that were both close to the important threshold but considered only somewhat feasible. Likewise, Waltz et al., mapped the ERIC equivalents of conducting a formal assessment (assess for readiness and conduct local needs assessment), developing an implementation plan (develop a formal implementation blueprint), and providing ongoing implementation support (use an implementation advisor) to the “go” zone based on expert ratings, while AETC perceptions led us to map them to the “no-go” zone. Experts engaged by Waltz et al., rated obtaining formal commitments, providing clinical supervision, and creating a learning collaborative as either important or feasible, whereas ratings from AETC representatives in our study kept these strategies in the “no-go” zone for not meeting the thresholds for important or feasible. Waltz et al., did not rate assessing provider proficiency. These differences suggest that the context in which AETCs operate is less conducive for enacting strategies than the likely broader viewpoint of the implementation science and clinical experts who participated in the Waltz et al. study.

Many strategies can be used to facilitate implementation of interventions (EPOC, 2015; Powell et al., 2015). As the body of evidence on the effectiveness of different strategies grows, there has been limited exploration of intermediaries’ ability to offer them. We extended Waltz et al.,’s “go/no-go” method to illustrate which strategies are most viable for AETCs across dimensions, beyond importance and feasibility. The Setting-Strategy Fit Index can help intermediaries more comprehensively consider the fit of different strategies for their context. However, the perceived fit of strategies should be balanced with their relative effectiveness in bringing about actual change in practice when prioritizing which strategies to offer. Information regarding poor fit of effective strategies may be used by agencies (e.g., the Health Resources and Services Administration) that fund and oversee intermediaries (e.g., AETCs) to make strategic investments and prioritize resources that motivate uptake of effective strategies to better support implementing organizations (e.g., HSOs).

Limitations

To reduce burden on participants, our study focused on only 10 strategies that could reasonably fall within the scope of AETCs to support HSOs integrate evidence-based interventions. However, AETCs could offer other strategies that were not assessed for fit in this study. We also asked about strategies to support implementation of psychosocial interventions for SUD broadly, rather than specific interventions for specific substances. Future research may assess whether there are differences in which strategies are best suited for supporting integration of pharmacological interventions and which are best suited for integration of interventions by substance. Additionally, this study focuses on the perspectives of AETCs in the United States, and the findings may not be directly applicable to other countries with different systems for delivering care to PWH or that have different infrastructure and resources to support implementation of evidence-based interventions. Furthermore, we did not engage HSO representatives in the study; therefore, we do not know how well-suited these strategies are from their perspective.

At least three additional limitations related to the analyses for this study should be considered. First, although we extensively explored the psychometric properties of the items developed to assess the Setting-Strategy Fit of various strategies, the items have not been formally validated to assess their robustness in varied contexts. Second, the use of the term “pressure” to assess tension for change may have been too strong or confusing for participants, resulting in especially low scores for that criterion. The wording may have skewed the responses and prevented us from adequately capturing whether AETCs experienced tension for change related to the different strategies. Last, while the analysis examined the correlates of Setting-Strategy Fit, it did not fully account for potential confounding variables (e.g., resource availability, organizational culture, previous experiences with similar strategies) that could influence the results.

Conclusions

Improving the integration of SUD interventions within HSOs is among the targets included in the National HIV/AIDS Strategy (House, 2021). Building on our team's prior research using the real-time Delphi method to assess the fit of nine evidence-based interventions for SUD within HSOs across the United States (Garner et al., 2022), we used the real-time Delphi method to assess the fit of 10 strategies that AETCs may use to support integration of psychosocial SUD interventions within HSOs. The exploration phase strategy of disseminating information about evidence-based interventions had the best fit for AETCs. Although simply disseminating information is feasible and can increase awareness about an intervention, it may not be the most effective strategy for promoting adoption and successful implementation within an HSO. This strategy will serve as the control strategy for a national dissemination experiment conducted by our team to test the incremental effectiveness of exploration facilitation—a single session strategy guided by the four guiding principles of the effective and cost-effective Implementation and Sustainment Facilitation strategy (i.e., engage, focus, evoke, and plan; Garner et al., 2020; Hinde et al., 2023)—for supporting adoption and implementation of brief motivational interviewing for substance use among HSOs.

Supplemental Material

sj-docx-1-irp-10.1177_26334895251343647 - Supplemental material for Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use

Supplemental material, sj-docx-1-irp-10.1177_26334895251343647 for Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use by Sheila V. Patel, Sarah Philbrick, Michael Bradshaw, Heather J. Gotham, Hannah K. Knudsen, Tom Donohoe, Stephen Tueller and Bryan R. Garner in Implementation Research and Practice

Supplemental Material

sj-docx-2-irp-10.1177_26334895251343647 - Supplemental material for Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use

Supplemental material, sj-docx-2-irp-10.1177_26334895251343647 for Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use by Sheila V. Patel, Sarah Philbrick, Michael Bradshaw, Heather J. Gotham, Hannah K. Knudsen, Tom Donohoe, Stephen Tueller and Bryan R. Garner in Implementation Research and Practice

Supplemental Material

sj-docx-3-irp-10.1177_26334895251343647 - Supplemental material for Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use

Supplemental material, sj-docx-3-irp-10.1177_26334895251343647 for Supporting integration of substance use interventions in HIV service organizations: Assessing the fit of 10 strategies for AIDS Education and Training Centers to use by Sheila V. Patel, Sarah Philbrick, Michael Bradshaw, Heather J. Gotham, Hannah K. Knudsen, Tom Donohoe, Stephen Tueller and Bryan R. Garner in Implementation Research and Practice

Footnotes

Acknowledgments

This work was possible because of the participation of AETC representatives. We thank the following members of our Guiding Coalition who shared their expertise to inform this study: Beth Rutkowski, Cindy Bolden Calhoun, Melissa Grove, Nicolé Mandell, Tom Freese, and Ted Gordon. We are grateful for Katie Loyd who programed the real-time Delphiand for Dilsey Davis, Rebecca Hipp, Shari Lambert, and Robert Walton, and Carter Reedy who created infographics and animated videos to communicate key information about the included strategies to participants. We further extend our thanks to Christina Rodriguez, Loretta Bohn, Teyonna Downing, and Nunzio Landi for their editorial, formatting, and graphic support for this manuscript.

Authors’ Contributions

BRG conceived the study; SVP led development of the study with support from HJG, HKK, and TD with oversight from BRG; ST and MB developed the analytic plan with input from BRG and SVP; MB conducted the analyses with oversight from ST; SVP, SP, and MB drafted the manuscript; all coauthors reviewed the draft manuscript and provided substantive feedback; and SVP finalized the manuscript based on coauthor input.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the National Institute on Drug Abuse via R01-DA044051 (PI Garner). The funding body had no role in the design of the study; collection, analysis, and interpretation of data; or writing of the manuscript.

Ethics Approval and Consent to Participate

All study procedures were approved by RTI International's institutional review board.

Availability of Data and Materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

ORCID iDs

Sheila V. Patel

Sarah Philbrick

Michael Bradshaw

Heather J. Gotham

Hannah K. Knudsen

Bryan R. Garner

Supplemental Material

Supplemental material for this article is available online.

References

Aarons

G. A.

Hurlburt

Horwitz

S. M.

(2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7

AETC Directory . Rutgers school of nursing. Retrieved January 31 from https://aidsetc.org/directory

AETC Resources . (2023). Rutgers school of nursing. Retrieved December 1 from https://aidsetc.org/resources

Ahmed

Nijhawan

A. E.

Gao

Ahn

Chow

J. Y.

(2024). Association between substance abuse and mental illness symptoms screener (SAMISS) scores and HIV care continuum outcomes in people newly diagnosed with HIV in the US south. AIDS Behav, 28(5), 1731–1740. https://doi.org/10.1007/s10461-024-04287-1

Arnsten

J. H.

Demas

P. A.

Grant

R. W.

Gourevitch

M. N.

Farzadegan

Howard

A. A.

Schoenbaum

E. E.

(2002). Impact of active drug use on antiretroviral therapy adherence and viral suppression in HIV-infected drug users. Journal of General Internal Medicine, 17(5), 377–381. https://doi.org/10.1046/j.1525-1497.2002.10644.x

Azar

M. M.

Springer

S. A.

Meyer

J. P.

Altice

F. L.

(2010). A systematic review of the impact of alcohol use disorders on HIV treatment outcomes, adherence to antiretroviral therapy and health care utilization. Drug and Alcohol Dependence, 112(3), 178–193. https://doi.org/10.1016/j.drugalcdep.2010.06.014

Burnam

M. A.

Bing

E. G.

Morton

S. C.

Sherbourne

Fleishman

J. A.

London

A. S.

Vitiello

Stein

Bozzette

S. A.

Shapiro

M. F.

(2001). Use of mental health and substance abuse treatment services among adults with HIV in the United States. Archives of General Psychiatry, 58(8), 729–736. https://doi.org/10.1001/archpsyc.58.8.729

Durvasula

Miller

T. R.

(2014). Substance abuse treatment in persons with HIV/AIDS: Challenges in managing triple diagnosis. Behavioral Medicine, 40(2), 43–52. https://doi.org/10.1080/08964289.2013.866540

EPOC. (2015). Effective practice and organisation of care taxonomy. Retrieved 1 January 2021 from https://epoc.cochrane.org/epoc-taxonomy

10.

Falade-Nwulia

Agee

Kelly

S. M.

Park

J. N.

Schwartz

Hsu

Schweizer

Jones

Keruly

Shah

Lesko

C. R.

Lucas

G. M.

Sulkowski

(2023). Implementing a peer-supported, integrated strategy for substance use disorder care in an outpatient infectious disease clinic is associated with improved patient outcomes. International Journal of Drug Policy, 121, Article 104191. https://doi.org/10.1016/j.drugpo.2023.104191

11.

Friedman

M. S.

Marshal

M. P.

Stall

Kidder

D. P.

Henny

K. D.

Courtenay-Quirk

Wolitski

R. J.

Aidala

Royal

Holtgrave

D. R.

(2009). Associations between substance use, sexual risk taking and HIV treatment adherence among homeless people living with HIV. AIDS Care, 21(6), 692–700. https://doi.org/10.1080/09540120802513709

12.

Garner

B. R.

Gotham

H. J.

Chaple

Martino

Ford Ii

J. H.

Roosa

M. R.

Speck

K. J.

Vandersloot

Bradshaw

Ball

E. L.

Toro

A. K.

Griggs

Tueller

S. J.

(2020). The implementation and sustainment facilitation strategy improved implementation effectiveness and intervention effectiveness: Results from a cluster-randomized, type 2 hybrid trial. Implement Res Pract, 1, Article 2633489520948073. https://doi.org/10.1177/2633489520948073

13.

Garner

B. R.

Gotham

H. J.

Knudsen

H. K.

Zulkiewicz

B. A.

Tueller

S. J.

Berzofsky

Donohoe

Martin

E. G.

Brown

L. L.

Gordon

(2021). The prevalence and negative impacts of substance use disorders among people with HIV in the United States: A real-time Delphi survey of key stakeholders. AIDS and Behavior, 26(4), 1183–1196. https://doi.org/10.1007/s10461-021-03473-9

14.

Garner

B. R.

Knudsen

H. K.

Zulkiewicz

B. A.

Tueller

S. J.

Gotham

H. J.

Martin

E. G.

Donohoe

Toro

A. K.

Loyd

Gordon

(2022). The setting-intervention fit of nine evidence-based interventions for substance use disorders within HIV service organizations across the United States: Results of a national stakeholder-engaged real-time Delphi study. Journal of Acquired Immune Deficiency Syndromes, 90(S1), S206–s214. https://doi.org/10.1097/qai.0000000000002981

15.

Goldstein

R. B.

Rotheram-Borus

M. J.

Johnson

M. O.

Weinhardt

L. S.

Remien

R. H.

Lightfoot

Catz

S. L.

Gore-Felton

Kirshenbaum

Morin

S. F.

Group

N. H. L. T.

(2005). Insurance coverage, usual source of care, and receipt of clinically indicated care for comorbid conditions among adults living with human immunodeficiency virus. Medical Care, 43(4), 401–410. https://doi.org/10.1097/01.mlr.0000156850.86917.f8

16.

Gordon

Pease

(2006). RT Delphi: An efficient, “round-less” almost real time Delphi method. Technological Forecasting and Social Change, 73(4), 321–333. https://doi.org/10.1016/j.techfore.2005.09.005

17.

Haldane

Cervero-Liceras

Chuah

F. L. H.

Ong

S. E.

Murphy

Sigfrid

Watt

Balabanova

Hogarth

Maimaris

Buse

Piot

McKee

Perel

Legido-Quigley

(2017). Integrating HIV and substance use services: A systematic review. Journal of The International Aids Society, 20(1), Article 21585. https://doi.org/10.7448/IAS.20.1.21585

18.

Hall

H. I.

Holtgrave

D. R.

Tang

Rhodes

(2013). HIV transmission in the United States: Considerations of viral load, risk behavior, and health disparities. AIDS and Behavior, 17(5), 1632–1636. https://doi.org/10.1007/s10461-013-0426-z

19.

Hallquist

M. N.

Wiley

J. F.

(2018). Mplusautomation: An R package for facilitating large-scale latent variable analyses in Mplus. Structural Equation Modeling, 25(4), 621–638. https://doi.org/10.1080/10705511.2017.1402334

20.

Hartzler

Dombrowski

J. C.

Crane

H. M.

Eron

J. J.

Geng

E. H.

Christopher Mathews

Mayer

K. H.

Moore

R. D.

Mugavero

M. J.

Napravnik

Rodriguez

Donovan

D. M.

(2017). Prevalence and predictors of substance use disorders among HIV care enrollees in the United States. AIDS and Behavior, 21(4), 1138–1148. https://doi.org/10.1007/s10461-016-1584-6

21.

Hendershot

C. S.

Stoner

S. A.

Pantalone

D. W.

Simoni

J. M.

(2009). Alcohol use and antiretroviral adherence: Review and meta-analysis. Journal of Acquired Immune Deficiency Syndromes, 52(2), 180–202. https://doi.org/10.1097/QAI.0b013e3181b18b6e

22.

HHS. (2021). What is ending the HIV epidemic in the U.S.? Office of Infectious Disease and HIV/AIDS Policy, United States Department of Health and Human Services. Retrieved January 14 from https://www.hiv.gov/federal-response/ending-the-hiv-epidemic/overview

23.

Hinde

J. M.

Garner

B. R.

Watson

C. J.

Ramanan

Ball

E. L.

Tueller

S. J.

(2022). The implementation & sustainment facilitation (ISF) strategy: Cost and cost-effectiveness results from a 39-site cluster randomized trial integrating substance use services in community-based HIV service organizations. Implement Res Pract, 3, Article 26334895221089266. https://doi.org/10.1177/26334895221089266

24.

House

T. W.

(2021). National HIV/AIDS strategy for the United States 2022-2025. https://www.whitehouse.gov/wp-content/uploads/2021/11/National-HIV-AIDS-Strategy.pdf

25.

King

W. D.

Larkins

Hucks-Ortiz

Wang

P. C.

Gorbach

P. M.

Veniegas

Shoptaw

(2009). Factors associated with HIV viral load in a respondent driven sample in Los Angeles. AIDS and Behavior, 13(1), 145–153. https://doi.org/10.1007/s10461-007-9337-1

26.

Kotter

J. P.

(2012). Leading change. Harvard Business Review Press. https://books.google.com/books?id=xpGX1EWL_EMC

27.

Lesko

C. R.

Falade-Nwulia

O. O.

Pytell

J. D.

Hutton

H. E.

Fojo

A. T.

Keruly

J. C.

Moore

R. D.

Chander

(2023). Joint effects of substance use disorders and recent substance use on HIV viral non-suppression among people engaged in HIV care in an urban clinic, 2014–2019. Addiction, 118(11), 2193–2202. https://doi.org/10.1111/add.16301

28.

Linstone

H. A.

Turoff

(1975). The Delphi method: Techniques and applications. Addison-Wesley Publishing Company, Advanced Book Program. https://books.google.com/books?id=52xHAAAAMAAJ

29.

Liu

Manavalan

Siddiqi

Cook

R. L.

Prosperi

(2024). Comorbidity burden and health care utilization by substance use disorder patterns among people with HIV in Florida. AIDS and Behavior, 28(7), 2286–2295. https://doi.org/10.1007/s10461-024-04325-y

30.

Lucas

G. M.

Cheever

L. W.

Chaisson

R. E.

Moore

R. D.

(2001). Detrimental effects of continued illicit drug use on the treatment of HIV-1 infection. Journal of Acquired Immune Deficiency Syndromes, 27(3), 251–259. https://doi.org/10.1097/00126334-200107010-00006

31.

Ma, J., Luu, B., Ruderman, S. A., Whitney, B. M., Merrill, J. O., Mixson, L. S., Nance, R. M., Drumright, L. N., Hahn, A. W., Fredericksen, R. J., Chander, G., Lau, B., McCaul, M. E., Safren, S., O'Cleirigh, C., Cropsey, K., Mayer, K. H., Mathews, W. C., Moore, R. D., … Delaney, J. A. C. (2024). Alcohol and drug use severity are independently associated with antiretroviral adherence in the current treatment era. AIDS Care, 36(5), 618–630. https://doi.org/10.1080/09540121.2023.2223899

32.

Malta

Strathdee

S. A.

Magnanini

M. M.

Bastos

F. I.

(2008). Adherence to antiretroviral therapy for human immunodeficiency virus/acquired immune deficiency syndrome among drug users: A systematic review. Addiction, 103(8), 1242–1257. https://doi.org/10.1111/j.1360-0443.2008.02269.x

33.

Mimiaga

M. J.

August Oddleifson

Meersman

S. C.

Silvia

Hughto

J. M. W.

Landers

Brown

Loberti

(2020). Multilevel barriers to engagement in the HIV care continuum among residents of the state of Rhode Island living with HIV. AIDS and Behavior, 24(4), 1133–1150. https://doi.org/10.1007/s10461-019-02677-4

34.

Muthén

L. K.

Muthén

B. O.

(1998–2017). Mplus user’s guide. Muthén & Muthén.

35.

Parcesepe

A. M.

Lancaster

Edelman

E. J.

DeBoni

Ross

Atwoli

Tlali

Althoff

Tine

Duda

S. N.

Wester

C. W.

Nash

(2020). Substance use service availability in HIV treatment programs: Data from the global IeDEA consortium, 2014-2015 and 2017. PLoS One, 15(8), Article e0237772. https://doi.org/10.1371/journal.pone.0237772

36.

Paschen-Wolff

M. M.

Campbell

A. N. C.

Tross

Choo

T.-H.

Pavlicova

Jarlais

D. D.

(2020). DSM-5 substance use disorder symptom clusters and HIV antiretroviral therapy (ART) adherence. AIDS Care, 32(5), 645–650. https://doi.org/10.1080/09540121.2019.1686600

37.

Patrick

M. E.

O'Malley

P. M.

Johnston

L. D.

Terry-McElrath

Y. M.

Schulenberg

J. E.

(2012). HIV/AIDS risk behaviors and substance use by young adults in the United States. Prevention Science, 13(5), 532–538. https://doi.org/10.1007/s11121-012-0279-0

38.

Powell

B. J.

Waltz

T. J.

Chinman

M. J.

Damschroder

L. J.

Smith

J. L.

Matthieu

M. M.

Proctor

E. K.

Kirchner

J. E.

(2015). A refined compilation of implementation strategies: Results from the expert recommendations for implementing change (ERIC) project. Implementation Science, 10(1), Article 21. https://doi.org/10.1186/s13012-015-0209-1

39.

Proctor

Hooley

Morse

McCrary

Kim

Kohl

P. L.

(2019). Intermediary/purveyor organizations for evidence-based interventions in the US child mental health: Characteristics and implementation strategies. Implementation Science, 14(1), Article 3. https://doi.org/10.1186/s13012-018-0845-3

40.

Proctor

E. K.

Powell

B. J.

McMillen

J. C.

(2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8, Article 139. https://doi.org/10.1186/1748-5908-8-139

41.

Przybyla

Ashare

R. L.

Cioffi

Plotnik

Shuter

Seng

E. K.

Weinberger

A. H.

(2022). Substance use and adherence to antiretroviral therapy among people living with HIV in the United States. Tropical Medicine and Infectious Disease, 7(11), Article 349. https://doi.org/https://www.mdpi.com/2414-6366/7/11/349

42.

Reback

C. J.

Rünger

Fletcher

J. B.

(2021). Drug use is associated with delayed advancement along the HIV care continuum among transgender women of color. AIDS and Behavior, 25(1), 107–115. https://doi.org/10.1007/s10461-019-02555-z

43.

Revelle

(2022). psych: Procedures for personality and psychological research. In (Version 2.2.9). Northwestern University. https://CRAN.R-project.org/package=psych

44.

SAMHSA. (2023). Key substance use and mental health indicators in the United States: Results from the 2022 National Survey on Drug Use and Health (HHS Publication No. PEP23-07-01-006, NSDUH Series H-58). https://www.samhsa.gov/data/

45.

Taylor

S. W.

McKetchnie

S. M.

Batchelder

A. W.

Justice

Safren

S. A.

O’Cleirigh

(2023). Chronic pain and substance use disorders among older sexual minority men living with HIV: Implications for HIV disease management across the HIV care continuum. AIDS Care, 35(4), 614–623. https://doi.org/10.1080/09540121.2022.2076801

46.

Waltz

T. J.

Powell

B. J.

Matthieu

M. M.

Damschroder

L. J.

Chinman

M. J.

Smith

J. L.

Proctor

E. K.

Kirchner

J. E.

(2015). Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: Results from the expert recommendations for implementing change (ERIC) study. Implementation Science, 10, Article 109. https://doi.org/10.1186/s13012-015-0295-0

47.

Webb

T. L.

Sheeran

(2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132(2), 249–268. https://doi.org/10.1037/0033-2909.132.2.249

48.

Weiss

Netherland

Egan

J. E.

Flanigan

T. P.

Fiellin

D. A.

Finkelstein

Altice

F. L.

Collaborative

f. t. B.

(2011). Integration of buprenorphine/naloxone treatment into HIV clinical care: Lessons from the BHIVES collaborative. JAIDS Journal of Acquired Immune Deficiency Syndromes, 56, S68–S75. https://doi.org/10.1097/QAI.0b013e31820a8226

49.

Yanagida

(2023). misty: Miscellaneous functions. In (Version 0.4.7). https://CRAN.R-project.org/package=misty

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

5.91 MB

0.03 MB

0.04 MB