Utilizing the Practical Robust,Implementation and Sustainability Model to Explore the Primary Care Setting Contextual Determinants of Implementing Patient-Reported Outcomes Measures to Support Opioid Use Disorder Care

Study Design

Our qualitative contextual inquiry design sought to identify and describe the relevant information about the primary care context where the intervention package (PROMs) will be delivered. This information will be used to guide future planning, implementation, and sustainment phases. An intervention package is all components of the intervention including the theoretic foundation, features of the intervention design, intent of the intervention, and implementation strategy components such as training and protocols (Kilbourne et al., 2007). We grounded our approach in Mielke, Leppla et al.'s (2022) description of the six components of a contextual inquiry: (1) choosing a theory, model or framework, (2) identifying relevant contextual factors, (3) involving stakeholders/partners, (4) developing a study design, (5) determining how the context influences the intervention package co-design and implementation strategies, and (6) reporting the contextual analysis.

In our case, dissemination and implementation science theories, models, and frameworks played a key role in the research process by providing a “systematic structure for the development, management, and evaluation of [dissemination and implementation] efforts, linking study aims, design measures, and analytic approaches” (Tabak et al., 2023, p. 492). A determinant framework to specify the contextual factors influencing implementation was most appropriate to support the planning and design of an intervention package (Nilsen, 2015). Thus, to address Mielke, Leppla, et al.'s first component, the research team chose PRISM to guide the research since it includes multi-level contextual domains for both clinical care teams (i.e., leaders, clinicians, staff) and patients in alignment with the study aim. In terms of the second component of Mielke, Leppla, et al.'s contextual analysis, we selected the perspectives and characteristics of clinic teams (across multiple roles) and the implementation and sustainability infrastructure as the key PRISM contextual elements to assess as our interviews and focus groups were time-constrained and there was insufficient time to address all PRISM domains (Glasgow et al., 2024). For Mielke, Leppla, et al.'s third and fourth components, we collaborated with our clinic/community partners to select the following qualitative methods to gather rich information about the context: individual interviews with implementation partners and focus groups with community members. Finally, components five and six are reflected in the presentation of findings relevant to the design of the implementation package and the overall reporting of the contextual analysis, following the Consolidated Criteria for Reporting Qualitative Research (Tong et al., 2007).

Setting

The Denver Health and Hospital Authority is an integrated safety-net health care system comprised of a hospital and 11 community health centers serving nearly 25% of the local population in Denver, CO, USA. The system has developed a comprehensive approach to addressing substance use in the target population, including having substance use treatment counselors (herein referred to as “counselors”) embedded within the primary care clinics to support prescribing clinicians and deliver behavioral health interventions for OUD (Bacon et al., 2025). The system is a recognized learning health system (Agency for Healthcare Research and Quality, 2019) with robust infrastructure to track population-level outcomes (Bacon et al., 2025; Prieto et al., 2019).

Study Population

Our target study population included those who would be directly involved with implementation of PROMs: (1) administrators who manage clinical programs, (2) clinicians, including physicians, advanced practice practitioners, and counselors, who provide primary-care based OUD care and therefore understand clinical feasibility, and (3) patients who receive primary care-based OUD care. We also aimed to include input from community members of people with lived experience of substance use (but not necessarily currently receiving care) or who have an interest in the care of people who use substances (Gilgoff et al., 2026). Community members with these characteristics are engaged in a standing community advisory meeting (CAM) hosted by the health system to gather feedback and comments about ongoing quality improvement projects and grant proposals (Bacon et al., 2025).

Data Collection

Our approach was to first conduct individual interviews among the three implementation partners: (1) administrators, (2) clinicians, and (3) patients. We then conducted focus groups with community members who participate in the CAM.

Individual Partner Interviews—Recruitment and Conduct

Individuals from our three implementation partner groups (administrators, clinicians, and patients) were recruited from email outreach and flyers to participate in 60-min semi-structured individual interviews. Participation required English-language proficiency and there were no additional inclusion or exclusion criteria. Administrators and clinicians were recruited via email outreach. Patients were informed about the study by their clinician and either provided permission for the study team to contact them or were provided with the study team's contact information. One investigator (JDP), who provides integrated OUD care in primary care, recruited patients from his patient panel. In addition, eight other clinicians who also provide integrated OUD care were asked to support recruitment by informing potentially eligible patients. Patients who expressed interest were contacted by phone. Each potential participant was contacted up to three times to schedule interviews, or if the scheduled interview was missed, to attempt to reschedule.

At the start of each interview potential participants were screened by the study's authors to verify eligibility, and the study and procedures were described in detail. No participants opted out of participating at this stage and all participants provided verbal consent and interviews were audio recorded. All interviews were conducted via phone or virtually (Zoom^®) by a single author (JDP), who has qualitative research methods training. Zoom^® AI transcriptions were referenced as needed if the recordings were unclear. Participants received a $30 gift card for participating.

CAM Member Focus Groups—Recruitment and Conduct

The CAM is an existing group that meets monthly, and the topic of the meeting is sent to an email list of community members who requested inclusion in the CAM 1 week ahead of time. Consistent with CAM protocols, we invited the first 25 community member participants who expressed interest to attend the focus group, which was held at the same time as the normal CAM meeting; those who attended received a $35 remuneration and a complimentary lunch. For the focus groups, to reach optimally sized groups for dialogue, community member participants were randomly separated into two equal-sized groups, where the study protocol was explained, and verbal consent was obtained. The groups were not audio recorded as it was outside CAM norms. Therefore, each focus group had a leader (JDP and SB [see acknowledgments]) and two coinvestigators (AG, LK, BF, and KD) who took contemporaneous notes which were used for analysis.

Interview Guide Development

We developed a semi-structured interview and focus group guide based on selected contextual domains from PRISM: (1) the multi-level perspectives of the intervention, (2) multi-level partner characteristics, and (3) implementation and sustainability infrastructure (Table 1). We did not include the PRISM domain of the external environment because it was out of scope of what members of the partner groups could meaningfully address and prioritized domains that would directly inform the initial design of the intervention package.

OPEN IN VIEWER

Table 1.

Working Definitions of PRISM Domains.

PRISM Domains	Working Definition
PRISM multi-level perspectives
Value proposition	Partner perspectives on the potential value of implementing PROMs to the organization and greater clinical community.
Form features	Describe the form features, which are the modifiable aspects of the collection, input, and use of PROMs in the context of measurement-based care.
Impact on care	Since measurement-based care using PROMs is new in the context of OUD treatment, we sought perspectives from the partners on what impacts, both positive and negative, implementing PROMs would have on individual patient care
Recovery domains	Partner perspectives on what domains of recovery should be assessed through the PROMs, including how or if the domains should change over time.
PRISM multi-level partner characteristics
Current practices	The current practices of how OUD treatment outcomes are assessed.
PRISM implementation & sustainability infrastructure
Existing infrastructure	Administrator and clinician input on how to fit the implementation of PROMs into the existing infrastructure and how to increase institutional support.

Note. PROM = patient-reported outcome measure; OUD = opioid use disorder; PRISM = Practical, Robust Implementation and Sustainability Model.

We explored PRISM Perspectives of the intervention in four distinct ways. First, we explored the value proposition of the perceived goals (“core functions”) for PROMs for OUD. Second, according to the Form-Function guidance for complex intervention, we also explored the form features since the characteristics of an evidence-based intervention that can be adapted while maintaining fidelity to the core functions (Perez Jolles et al., 2019). Third, we sought to describe potential unanticipated outcomes of PROMs, both positive and negative. Finally, as there are many existing PROMs for OUD care which are diverse in what aspects of recovery are assessed, we sought perspectives from participants on what they felt were the most important recovery domains to assess.

Related to the PRISM domain of “Characteristics” of our three partner groups, the interview guide was developed to elicit how outcomes are currently monitored to understand how PROMs may be integrated with existing outcome monitoring. Questions about the PRISM domain of “Implementation and Sustainability Infrastructure” sought to uncover how to fit PROMs into the existing primary-care infrastructure and how to increase institutional support. The Implementation and Sustainability Infrastructure domains were only assessed among administrators and clinicians, as they relate to the health system infrastructure which is relatively unknown to patients/community members.

Analysis

We analyzed the interview data using rapid thematic analysis and iteratively made refinements to the interview guide (Gale et al., 2019; Hamilton & Finley, 2019). This approach was optimal because our goal was to describe PRISM contextual factors which suited deductive analytic approach. We also needed to iteratively analyze interviews and focus groups quickly to accommodate both the competing demands on research staff and to meet the health system's timeline for integrating a PROM into the health record. Finally, funding constraints required a pragmatic method that allowed for efficient and economical use of team resources (Gale et al., 2019). Rapid thematic analysis took place in three key steps utilizing a matrix template (Schexnayder et al., 2023). First, a researcher (JDP, SL, or AG) conducted initial listening and created a summary document or note-taking of each interview. All interviews and focus groups were independently summarized by a single researcher. Next, a second researcher (JDP, SL, or AG) listened to the interview and reviewed and independently refined, edited, and expanded the summary of the first researcher. Next, these two researchers met to review the summaries and reconcile any differences, identify preliminary themes within each of the contextual domains, and create definitions. Each partner group (administrators, clinicians, and patients) were analyzed separately to identify preliminary themes within each group.

Preliminary themes which were similar between groups were then harmonized by the analytic team (JDP, AG, KA, SL). We held group sessions with the team to read the preliminary theme titles, definitions, and representative quotes. The team held group meetings to review the preliminary themes, combining those which were felt to be similar, and renamed the themes and adjusted the definitions.

The analytic team (JDP, AM, AG, KA) reviewed the preliminary themes and selected those which we felt were most important to triangulate with community members to strengthen the credibility and improve the representativeness of perspectives (Yin, 2011, p. 81). The study team reviewed the now harmonized themes and, through group discussion, identified those themes that were either (A) unexpected or (B) highly relevant to informing the intervention package and (C) related to topics that the community members could provide meaningful insight. Using the same rapid thematic analysis method, we identified emergent themes from the community member focus groups to clarify the themes from individual interviews.

The study team (JDP, AM, AG, BF, KA) identified key themes relevant to the planning and design of a PROM implementation package and applied an additional level of organization by categorizing the themes in two ways. First, the team grouped themes into de novo categories. One investigator (JDP) developed an initial categorization, which was then iteratively reviewed by team members (AM, AG, KA). Once the themes’ categories were finalized, the team collaboratively named and described the categories. Second, themes were organized according to the PRISM contextual domains that guided the development of the interview questions. From October 2023 through July 2024, interviews and focus groups were conducted alongside an iterative rapid analysis. A local institutional review board determined this research exempt.

Category 1: Maintaining Patient Trust as PROMs Integrate into Workflow

Themes in this category addressed the practical considerations of implementing PROMs, including building trust, integrating PROMs into care, and refining workflows to ensure that measurement supports clinical decision-making without compromising the therapeutic alliance between patients and their clinicians. All participant groups acknowledged that implementation of PROMs would impact the dynamic between patients and clinicians during care. While all groups felt the impact could be quite positive during the actual clinic visit, patients and community members had concerns about possible unintended consequences. Administrators conceived of PROMs as a less judgmental way to obtain information [Standardization Minimizes Judgement] and community members agreed. As one community member said, “questionnaires will be good, as sometimes those going in for the first time are not as comfortable sharing with doctor in that first visit.” Clinicians similarly focused on how PROMs would impact patient visits, indicating their belief that the results would improve the timeliness and efficiency of visits by showing a “red flag” when patients need additional support and help reduce missed opportunities for intervention [Improve Timeliness and Efficiency of Care]. Community members expanded the definition of this theme by imagining PROMs as bridges—tools that could connect patients to new clinicians and increase the efficiency of care.

While administrators and clinicians generally viewed PROMs positively, patients identified potential concerns. Patients generally felt comfortable sharing sensitive information with their clinician, since that is routine in clinical care [Trust and Transparency of Results]. As one patient noted, they already “trust them [clinicians] with my life” (Patient 31). But patients also emphasized the importance of transparency in how results are used to inform current and future care. One patient explained that patients might not give accurate responses if honest responses (e.g., returning to drug use) meant losing access to medications or being given a lower dose. Community members similarly expressed concern that results could negatively influence how future clinicians treat them, for example, not being prescribed pain medications even if they were clinically appropriate.

Measurement-based care involves sharing the results with patients and using the results to guide clinical decision-making. Clinicians emphasized the need for implementation strategies of education, training, and support on how to interpret and discuss PROM results with patients [Improve Knowledge]. PROMs would be a new clinical outcome in this context, and thus support would be needed to ensure that clinicians know how to access results, interpret scores, and how to use the results to make care decisions. But patients put considerably less stock into the scoring of their recovery status and indicated that they did not need to see their scores [Result Review Undesired]. As one patient stated, “As long as I’m not doing drugs, I don’t care about scores” (Patient 8), suggesting that while reviewing PROM results may be a helpful tool for clinicians, they may be of less inherent value to patients. This theme is also related to Category 3 themes describing the implementation considerations (see below).

Category 2: Perspectives on PROM Content and Use

Themes in this category focus on the measurement of outcomes for OUD treatment and how PROMs could help assess the unique experiences, preferences, and goals of patients. These themes emphasize a shift to normalize and routinize outcome monitoring in OUD care. Administrators viewed results as a tool for evaluating system-wide interventions and improving population-level outcomes [Making Routine Outcome Measurement Routine]. “You only change what you measure” (Administrator 73), one administrator remarked, underscoring the idea that system-level improvements hinge on quantifiable data. Community members echoed this theme, linking measurement not just to evaluation, but to service delivery.

Clinicians felt that the collection of PROMs should focus on health-related outcomes [Health-Related Outcomes]. Administrators and clinicians emphasized that PROMs could help add context to the process outcomes, which are routinely collected in the electronic health record [Contextualize Process Outcomes]. The process outcome measures included receipt of medication for OUD (e.g., days of dosing), being retained in care for a specific period (e.g., 90-days or 6-months), and naloxone co-prescribing. However, an administrator noted that these are “not rich data sources of recovery” (Administrator 10) and simply represent the currently available data.

A shared perspective between patients, clinicians, and reinforced by the community members was that PROMs provide meaningful outcomes for patients [Meaningful Outcomes for Patients]. Clinicians and patients alike described how tracking outcomes important to patients helped identify progress that might otherwise go unnoticed. One community member captured this sentiment saying, “It's really important to revel or celebrate what a person has done to achieve improvements or changes to make their life better or more sustainable.” For patients whose recovery journeys often unfold slowly, PROMs could offer tangible reassurance of progress. An administrator stated, “Recovery is not a straight line – it's a circle” (Administrator 7) and PROMs can be impactful at any time in care.

Reflecting the diversity of patient goals, a common theme among all participant groups was that PROMs need to assess the many distinct domains of recovery [Multidimensional Assessment]. Community members reflected that PROMs could assess aspects of a patient's life that go beyond simply asking about the presence or absence of substance use. This ideal of collecting outcome data that is multidimensional and individualized to the patient is juxtaposed to the current practice where patients and clinicians also noted that urine drug toxicology results are the outcome routinely collected in routine OUD care [Urine Drug Test]. Urine drug testing is a guideline-recommended practice and may be deemphasized if PROMs are collected (Substance Abuse and Mental Health Services Administration, 2020). PROMs may also provide important contextual data to aid in the interpretation of toxicology results and would be more applicable for patients whose treatment goals do not include abstinence.

Category 3: Implementation Considerations for Clinics and Teams

These themes represent the pragmatic factors that would influence the implementation of PROMs.

All participant groups agreed that integrating PROM collection into the electronic health record would be ideal [Integrated into the Electronic Health Record]. Collecting PROMs through patient portals or clinical workflows would ensure results of PROMs could be viewed in multiple places in the electronic health record. Additionally, integrating PROMs into the electronic health record could include longitudinal visualization and allow for interpretation in the context of other results, such as toxicology results or medication dosing history. As noted above for the theme “Result Review Undesired,” (in Category 1), it may be that initial efforts for integration in the electronic health record focus on how to make PROMs usable for clinicians.

When considering who should be responsible for collection of PROMs, administrators and clinicians (which include counselors) reported counselors are best suited, rather than the prescribing clinician who is often responding to multiple patient medical needs [Team/Counselor Responsibility]. One administrator put it plainly: “We cannot pile more on primary care” (Administrator 78), with a counselor adding that having counselors collect PROMs might even improve the honesty of responses by reducing perceived power differentials. Indeed, prescribing clinicians (e.g., physicians and advance practice practitioners) reported that they already strongly rely on counselors’ assessments when caring for patients.

All participant groups reflected on the current practices and noted that there was no consistent method for collecting data on patient recovery [No Consistent Collection Presently]. A clinician reflected on their practice, stating, “there is random free-text I put in my clinic note” (Clinician 19) and administrators noted that no system-level efforts have been attempted to collect data about patient recovery. This theme along with the “Improve Knowledge” theme (in Category 1) highlights how capturing PROMs in this context is a new endeavor that would require clinician and patient support to improve implementation. One way to improve implementation highlighted by clinicians and administrators was the importance of integrating the collection of PROMs into existing systems to reduce burden of collection [Leverage Existing Systems]. A clinician emphasized that in a busy primary care clinic, it was important not to slow down clinic flow. Administrators echoed this concern and suggested that integrating PROMs into existing workflows could enhance implementation.