Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision

Abstract

Background

Implementation strategies are theorized to work well when carefully matched to implementation determinants and when factors—preconditions, moderators, etc.—that influence strategy effectiveness are prospectively identified and addressed. Existing methods for strategy selection are either imprecise or require significant technical expertise and resources, undermining their utility. This article outlines refinements to causal pathway diagrams (CPDs), a method for articulating the causal process through which implementation strategies work and offers illustrations of their use.

Method

CPDs are a visualization tool to represent an implementation strategy, its mechanism(s) (i.e., the processes through which a strategy is thought to operate), determinants it is intended to address, factors that may impede or facilitate its effectiveness, and the series of outcomes that should be expected if the strategy is operating as intended. We offer principles for constructing CPDs and describe their key functions.

Results

Applications of the CPD method by study teams from two National Institute of Health-funded Implementation Science Centers and a research grant are presented. These include the use of CPDs to (a) match implementation strategies to determinants, (b) understand the conditions under which an implementation strategy works, and (c) develop causal theories of implementation strategies.

Conclusions

CPDs offer a novel method for implementers to select, understand, and improve the effectiveness of implementation strategies. They make explicit theoretical assumptions about strategy operation while supporting practical planning. Early applications have led to method refinements and guidance for the field.

Plain Language Summary Title

Advances to the Causal Pathway Diagramming Method to Enhance Implementation Precision

Plain Language Summary

Implementation strategies often fail to produce meaningful improvements in the outcomes we hope to impact. Better tools for choosing, designing, and evaluating implementation strategies may improve their performance. We developed a tool, causal pathway diagrams (CPD), to visualize and describe how implementation strategies are expected to work. In this article, we describe refinements to the CPD tool and accompanying approach. We use real illustrations to show how CPDs can be used to improve how to match strategies to barriers, understand the conditions in which those strategies work best, and develop generalizable theories describing how implementation strategies work. CPDs can serve as both a practical and scientific tool to improve the planning, deployment, and evaluation of implementation strategies. We demonstrate the range of ways that CPDs are being used, from a highly practical tool to improve implementation practice to a scientific approach to advance testing and theorizing about implementation strategies.

Keywords

implementation strategy mechanisms of action theory implementation determinants implementation outcomes causal pathway

Introduction

Implementation strategies are theorized to work well when carefully matched to targeted determinants, including barriers and facilitators, and when factors that influence strategy effectiveness—such as preconditions and moderators—are prospectively identified and addressed. Yet, selecting a strategy appropriate for a specific determinant within a given setting remains a vexing problem in implementation practice and research (Balis & Houghtaling, 2023). The field is lacking in theory and empirical evidence that informs which implementation strategies are capable of addressing specific barriers and facilitators (Wang et al., 2023). Historically, systematic assessment of strategy-determinant alignment has rarely occurred (Flottorp et al., 2013; Wolk et al., 2015), and evidence for implementation strategy effectiveness reveals that outcomes are often modest at best (Flodgren et al., 2011; Forsetlund et al., 2009; Giguere et al., 2020; Glidewell et al., 2022; Ivers et al., 2012). In response, there have been growing efforts to develop approaches to match strategies to determinants and to identify the mechanisms that explain how implementation strategies work (Carey et al., 2019; Fernandez et al., 2019; Geng et al., 2022; Smith et al., 2020).

In 2018, our team developed causal pathway diagrams (CPDs), a method for visually representing the causal operation of implementation strategies (Lewis et al., 2018). CPDs are rooted in the use of directed acyclic graphs (DAGs) and path diagrams in statistics (Garson, 2013; Stage et al., 2004), structural equation modeling (Bowen & Guo, 2011; Hoyle, 1995), and causal inference (Pearl, 1998, 2009a, 2009b). CPDs adapt these visualizations to the needs of implementation science by making explicit the factors—and their roles—implicated in an implementation strategy's operation. By representing the chain of events through which strategies work, CPDs can support selection of strategies that are conceptually aligned with determinants and can fill gaps in our understanding of strategy functioning. To achieve these goals, CPDs draw on existing theory and empirical evidence to identify entities involved in a strategy's operation and the roles they play. When relevant theory and evidence are not available, however, CPDs can also provide a structure to stimulate novel theorizing to guide empirical testing of how implementation strategies work.

Our team has applied CPDs within three National Institute of Health (NIH) grants (P50CA244432, P50MH126219, and R01CA262325) and has supported over one hundred researchers and practitioners in using CPDs in their own work through workshops and consultation. This has allowed us to apply CPDs to a broad set of implementation problems, with diverse interventions, strategies, and settings. Through these applications, our team has made refinements to the structure of CPDs and process for developing them and has clearly outlined the functions which they are most equipped to address. This article offers a refined method that centers determinants in the causal chain, details how evidence, theory, and practice-based experience inform the CPD elements, illustrates how to generate CPDs for multifaceted strategies, and details a comprehensive set of functions that CPDs serve. We describe the method through three applications that illustrate diverse uses of CPDs.

Method

Method Overview

CPDs are a tool for planning implementation initiatives and research, that explicitly represent researchers’ or practitioners’ (hereafter referred to as “implementers”) hypotheses about how implementation strategies operate. They are box-and-arrow graphical representations that indicate the role each element in the diagram (e.g., determinant, proximal outcome) plays in the hypothesized causal process. They capture the implementers’ current understanding—based on theory and experience—of the process through which an implementation strategy is thought to influence the intended implementation outcomes. CPDs enable implementers to check the face validity of their assumptions (i.e., “can I convince myself this strategy would effectively bring about change?”), evaluate the theoretical and empirical support for their assumptions (“is there evidence or theory to suggest this strategy can address this barrier?”), decide what should be measured to understand if the strategy is working, and identify additional strategies or supporting activities needed to ensure the preconditions for the strategy's effectiveness will be met. CPDs are intended to be thoroughly practical.

Elements of CPDs

A basic CPD consists of two parts: (a) the “stem” (central part of Figure 1), describing the main causal process through which a strategy operates and (b) the “leaves” (top/bottom boxes of Figure 1), which represent “effect modifiers”—factors that influence whether, and to what extent, the process represented by the stem unfolds successfully. Specification of elements in a CPD should draw on all available information, including theory, prior empirical evidence, and implementers’ clinical and site-specific experience. Figure 2 illustrates how a CPD can be used to articulate how decision support prompts work.

Figure 1

CPD Template

Figure 2

A CPD for Decision Support Prompts to Improve Depression Screening

CPD Stem

The stem visually represents the intended sequence of impacts of an implementation strategy. It begins, on the left, with the strategy, and ends, on the right, with the distal implementation outcome the strategy is intended to achieve. The stem will typically consist of the following elements (Table 1).

Table 1

Terms and Definitions

Term	Definition
Implementation strategy	Methods used to improve the adoption, fidelity, or sustained use of an evidence-based treatment, practice, or service.
Mechanism	The process through which an implementation strategy operates to affect an implementation outcome.
Determinant	Commonly referred to as barriers or facilitators, a factor that has an enabling or hindering influence on an implementation outcome. Determinants are often the targets that implementation strategies are intended to affect.
Proximal outcome	The most immediate, observable outcome of an implementation strategy.
Distal implementation outcome	The downstream implementation outcome that the implementation strategy is ultimately intended to achieve.
Precondition	A factor that is necessary for an implementation strategy to exert its influence on an implementation outcome.
Moderator	A factor that can strengthen or weaken the influence of an implementation strategy.

Implementation strategy: Simplest CPDs describe the functioning of a single/discrete implementation strategy, but CPDs can also describe the operation of a multifaceted implementation strategy. The strategy is represented at the left end of the diagram, and the arrows pointing from the strategy to the right represent the causal influence of that strategy. CPDs can vary in their degree of abstraction and generalizability. Most commonly, CPDs describe the functioning of a concrete operationalization of a strategy—how the strategy will be deployed in a particular setting. We recommend using strategy specification guidelines (Presseau et al., 2019; Proctor et al., 2013) because the way a strategy is operationalized often influences how it works. In addition, more abstract CPDs can be built to understand the causal functioning of an implementation strategy based on the components that commonly comprise that strategy. In this way, CPDs can be used to develop generalizable theories of strategy operation.

Mechanism: This is the process through which a strategy is thought to impact the targeted determinant. It describes how and why a strategy works. In the example in Figure 2, the decision support prompts are thought to address clinicians’ forgetfulness due to divided attention by increasing the cognitive salience of the need to screen. Implementation strategies may work through a single or multiple mechanisms.

For some strategies, extensive theoretical literature exists that describes how the strategy is hypothesized to work (e.g., opinion leadership; Rogers, 1995). In such cases, the mechanism specified in the CPD should be grounded in existing theory. Other times, however, little existing theory can guide mechanism specification. In these cases, implementers will need to synthesize empirical literature and their own experience to articulate a mechanism through which they think the strategy operates. This articulation constitutes, de facto, new (albeit lightweight) theorizing, serving as a first step in furthering the field's understanding of how hereto untheorized strategies work.

To avoid conflation between the activities that constitute an implementation strategy and a mechanism, we recommend that mechanisms are conceptualized as the changes that the employment of a strategy produces in its recipients, which, in turn, mediate the downstream implementation outcomes. Recipients are not limited to individuals, they can include entities such as teams, organizations, or systems depending on the level at which the strategy operates. This conceptualization of mechanisms is similar to the conceptualization used in realist evaluation (Lacouture et al., 2015).

Determinant: Determinants are factors that influence implementation outcomes, but the way in which they exert their influence differs. We distinguish three ways in which determinants exert their influence: determinants can (a) act as barriers or facilitators, directly influencing the implementation outcomes; (b) function as moderators of how well an implementation strategy works to influence implementation outcomes; and (c) function as preconditions that are necessary for a strategy to work. Note that these distinctions reflect differences in the causal role that determinants may play in different initiatives and settings, rather than inherent differences in the constructs themselves. That is, a determinant, such as leadership support, can operate as a barrier or facilitator, moderator, or a precondition, depending on the context and implementation strategy of interest. A key strength of CPDs is that they help make explicit the roles that different determinants play in relation to a specific strategy in a particular setting. CPDs center the determinants that act as barriers or facilitators in the Determinant box in the diagram stem; determinants that play the role of moderators or preconditions are represented in the CPD leaves.

While most settings have several determinants of implementation, each basic CPD should center the barrier (or facilitator) the strategy being analyzed is intended to address. In Figure 2, the barrier being addressed is clinicians’ forgetting to screen for depression due to competing attentional demands during a clinic visit. A strategy might be poised to resolve multiple barriers. In this case, we recommend creating CPDs that represent the causal paths through which each of those barriers might be addressed by that strategy, as each process may involve distinct preconditions or moderators that will need to be accounted for.

Proximal outcomes: These are a sequence of observable effects that, if the strategy is operating as intended, will occur en route to the distal implementation outcome (Brenner et al., 1995). They enable early assessment of whether the strategy is working and allow implementers to identify if and where the impact of an implementation strategy is stalling. Identifying a robust set of proximal outcomes supports the development of a measurement battery to assess the process and success of an implementation initiative. Implementers should choose proximal outcomes that help assess aspects of a strategy's operation about which they have greatest uncertainty or that provide early signs of whether the strategy is working, to allow for implementation adjustment.

Distal implementation outcome: This refers to the global intended effect of an implementation initiative, such as the rate of depression screening in an entire clinic. A strategy's impact on a distal outcome is mediated by its proximal outcomes. While implementation strategies are intended to have downstream effects on health outcomes, they are not sufficient, in and of themselves, to bring about change in health outcomes and, therefore, these outcomes are not centered in the causal pathway. Instead, CPDs typically end with implementation outcomes (e.g., fidelity or adoption) that can more directly be achieved by the implementation strategy being theorized. Similarly, CPDs typically only include implementation outcomes that the strategy can plausibly bring about without requiring additional implementation strategies. For instance, while training can exert influence on adoption, it is unlikely to ensure the sustained use of a practice without additional implementation strategies.

CPD Leaves

The CPD leaves represent factors (i.e., determinants) that facilitate or impede a strategy's operation. We describe two types of such effect modifiers: preconditions and moderators.

Preconditions: These are factors that must be present, but might not be, for a part of the strategy's causal process to take place (Mackie, 1965). If not present, preconditions block the strategy from having any effect. Preconditions can operate at different points along the causal chain. In Figure 2, the first precondition is that the prompt is noticeable (e.g., a foreground window that the clinician can see), which is necessary for the strategy to activate its mechanism (reminding the clinician of the need to screen). The second precondition, availability of the depression measure, operates further down the causal path when a clinician tries to administer a screener. If forms are unavailable in the exam rooms, the clinician cannot screen even if the reminder worked well.

Although any given implementation strategy may have many preconditions, only those that plausibly might not be present in the clinical setting should be included. Identifying preconditions makes it possible to devise activities (e.g., a restocking process) or additional implementation strategies that ensure the preconditions are met.

Moderators: Moderators refer to factors that can either facilitate or impede the functioning of an implementation strategy. Like preconditions, moderators can operate at different points along the causal pathway. In Figure 2, the first moderator, habituation to electronic health record (EHR) prompts, describes a factor that could reduce the ability of prompts to increase screening salience due to decreased attention clinicians pay to EHR alerts when they get too many. The second, clinic visit duration, indicates a factor that may influence whether a screener is administered even if clinicians remember to screen.

Identifying moderators is important for two reasons. First, it enables implementers to incorporate additional strategies that would either potentiate (for facilitating moderators) or weaken (for impeding moderators) the influence of those moderators. For instance, if the perceived value of depression screening is identified as a moderator of prompts, a patient testimonial (additional strategy) could be incorporated to increase clinicians’ perceptions of screening importance to potentiate prompt effectiveness. Moreover, identifying moderators can inform how strategies are sequenced, with the strategy focused on increasing perceptions of screening importance coming before the EHR reminders are set to be activated. Second, identification of potential moderators may point to the need to measure them, so that variations in a strategy's effectiveness can be understood.

CPDs Integrate and Support the Development of Theory

CPDs should use existing theory to inform causal accounts of how strategies work. Most commonly, CPDs draw on determinant frameworks to identify barriers and facilitators that directly influence implementation success and determinants that influence whether or how well an implementation strategy works (i.e., moderators and preconditions). Theorizing about the mechanisms and proximal outcomes (i.e., mediators) of implementation strategies may require looking outside of the implementation science theory. Often this includes drawing on classic social, cognitive, or organizational theories, that originate in other fields, but have relevance to implementation science (Nilsen, 2015).

Sometimes, however, developing a causal explanation for how strategies work requires novel theorizing, either because no theoretical accounts of a strategy exist or to fill gaps in existing theory. Such novel theorizing may draw on theories more distant to implementation science—for instance, dual-process models (Houlihan, 2018; Kahneman, 2011) which can explain how an implementation strategy can activate fast, heuristic cognitive processes to guide behavior—or it may involve synthesizing empirical findings about a strategy to derive a mechanism that may be responsible for observed phenomena. In either case, using CPDs to structure such novel causal accounts enables the formulation of clear hypotheses about a strategy's functioning that can be assessed and refined through subsequent empirical and theoretical research.

Principles of Constructing CPDs

Center CPDs on specific questions: CPDs should help answer targeted questions, such as: Which strategy is a better match for barrier X? What can I measure to assess the earliest signs that strategy X is working? The complexity and focus of a CPD should be a function of the question it is trying to answer. Starting with clear questions helps determine the needed level of complexity and what elements to include. Developing a CPD may also surface new questions to center in additional CPDs. This is often the case when a precondition or moderator is surfaced in a CPD. The next question becomes, what additional strategy might ensure this precondition is met? In this way, a precondition in one CPD can become a targeted determinant in another.

Start with whatever information you currently have or understand: CPDs do not have to be developed linearly. One may begin with a known barrier to identify strategies to remove that barrier or with the distal implementation outcome of interest. When available, theory should be central to the CPD development process. Existing theories can help implementers understand how an implementation strategy will function, such as its mechanisms or moderators, but their guidance may be limited, since much existing theory focuses on determinants that impede implementation rather than on how strategies address determinants. Thus, it is often necessary to also draw on empirical evidence, collect new evidence, and draw on stakeholder experiences to articulate how a strategy may work. In Additional File 1, we present a series of steps and questions that are commonly used to develop a CPD.

Keep it simple; add complexity only when it's helpful: Implementation initiatives are anything but simple. However, much of the value of CPDs for supporting decision-making comes from parsimonious diagrams like the one in Figure 2. Start with diagrams that show the basic functioning of each strategy being considered, or a set of CPDs that provide good coverage of prioritized barriers. Add complexity only when more complex representations answer important questions that simpler representations leave unanswered.

Key Functions

Table 2 describes the key functions of CPDs for facilitating implementation theory, research, and practice. These functions outline what implementation science problems CPDs seek to address. In the Results section, we present three case studies that demonstrate how CPDs can be used to fulfill three of these key functions.

Table 2

Key Functions of CPDs

Function	Description
1. Match implementation strategies to determinants	Articulate how an existing or novel implementation strategy is expected to work to assess the plausibility of whether it can address determinants necessary to bring about the intended outcomes.
2. Understand the conditions under which an implementation strategy will work	Identify preconditions that must be present for an implementation strategy to work and moderators that may strengthen or weaken how well a strategy works to understand if it will work in a particular setting and how to maximize its effectiveness.
3. Design and optimize multifaceted implementation strategies	Articulate how each discrete strategy that comprises a multifaceted strategy is expected to work to assess barrier coverage, redundancy, strategy interdependence, and sequencing.
4. Develop causal theories of how implementation strategies work	Develop generalizable accounts, using existing theory, empirical evidence, and novel theorizing of how the common components of an implementation strategy are expected to work.
5. Measure implementation strategy functioning	Articulate how an implementation strategy is expected to work to inform what and when to measure early and distal indicators of strategy effectiveness.
6. Diagnose why an implementation strategy did not work as intended	Retrospectively articulate how an implementation strategy, including moderators and preconditions that may have affected how well it worked.

Note. CPD = causal pathway diagram.

Results

Function 1: Matching Implementation Strategies to Determinants

Function goal: CPDs can function as a practical tool for supporting selection or design of implementation strategies to address locally relevant determinants. Implementation strategies are often mismatched with critical determinants (Waltz et al., 2019), resulting in strategies that have little or no impact on implementation outcomes. The CPD stem centers on the question, can this implementation strategy plausibly resolve a targeted barrier to bring about change in the implementation outcome? Once the main causal process for the implementation strategy is described, the CPD leaves can inform strategy selection by identifying whether conditions exist that could prevent the strategy from working as expected or could strengthen or weaken how well it works.

Background: Appa Health is a digital mental health (DMH) tool for adolescents. The treatment model consists of (a) DMH tools based on evidence-based interventions (EBIs), such as cognitive behavioral therapy (CBT), delivered in short-form videos from “influencers” with >1 million TikTok followers and (b) near-peer lay-mentors with relatable lived experience providing supportive accountability, via text and video chat, for learning and using the EBI. Appa found good youth engagement with mentoring, but suboptimal use of CBT-informed DMH tools and videos. While 80% of teens communicated with mentors weekly, only 50% viewed videos. Eight determinants of adolescent engagement with video-based EBP content surfaced using a rapid evidence review and focus group with users. Appa Health aimed to prioritize key determinants and select strategies to address them.

Causal pathway diagramming: Representatives from Appa Health and researchers participated in four 90-min sessions over 4 weeks to develop CPDs to evaluate alternative strategies to address key determinants. Before beginning CPD development, they used the Miro Board “Lean Coffee” template (Miro, 2022), combined with the Nominal Group Technique (Harvey & Holmes, 2012), to prioritize barriers to target. CPDs were then used to vet alternative strategies for targeting barriers.

CPD stem: What strategies can plausibly address the determinants of adolescent engagement with the EBP content? CPD development began by focusing on the main causal process through which candidate strategies would work. The team articulated the distal outcome that motivated the study, adolescent engagement with DMH content, and the proximal outcome that would precede adolescent engagement. For each CPD stem, they included a single barrier, beginning with “numerous steps to access DMH content.” After brainstorming viable strategies, they voted on a subset to compare using diagramming. Each strategy was clearly operationalized to provide enough detail for the team to brainstorm the mechanism, describing how the strategy would work. The team could not produce a compelling mechanism for several strategies, and these were eliminated due to poor plausibility that they could address the determinant and intended outcomes.

CPD leaves: Under what conditions would those strategies work best? Next, preconditions were added to each CPD stem to compare the conditions necessary for each strategy to work (e.g., teens must have access to a phone). If preconditions could not be met, strategies were eliminated. They then added moderators to assess if there were factors that could enhance (e.g., mentors providing accountability support) or interfere with how well the strategy would work.

Repeating the process: This process was repeated, centering each prioritized barrier. Through this process, the team identified two strategies that could plausibly address five unique barriers (Figure 3), enabling the team to develop a strategy package that was both parsimonious and likely high-impact. Appa Health used these CPDs to redesign how they release and remind users of the DMH content to improve their engagement.

Figure 3

CPD Depicting Strategies for Addressing Barriers to Youth Digital Mental Health Engagement

Function 2: Understand the Conditions Under Which an Implementation Strategy Works

Function goal: There is considerable variation in whether an implementation strategy brings about improvements in outcomes, and this variation is often attributed to differences in the context in which implementation occurs (May et al., 2016; Squires et al., 2015). Clarifying these conditions can inform which settings a strategy is best suited to, if additional strategies or supports are needed to ensure a strategy works as intended, and how a strategy should be carried out to fit the conditions in a setting. This illustration demonstrates how CPDs can be used to inform how an implementation strategy is operationalized to maximize effectiveness. Existing frameworks describe what characteristics of strategies should be operationalized (Presseau et al., 2019; Proctor et al., 2013). CPDs complement these by identifying effect modifiers that should guide strategy operationalization decisions.

Background: Human papillomavirus (HPV) testing alone is a newly recommended strategy for cervical cancer screening for women aged 30–65 years (Curry et al., 2018), and can be completed at home using self-collected samples. While mailing HPV self-sampling kits to overdue women increases screening uptake by 50% versus usual care (Tiro et al., 2019; Winer et al., 2018), overall screening rates remain low. Screening barriers include patient understanding of HPV testing and perceived accuracy of self-sampling (Tiro et al., 2019). The Patient-Centered Approach to Tailoring HPV Self-Sampling for Cervical Cancer Screening (PATH) study aimed to address those barriers using patient educational brochures and determine whether the brochure content required tailoring.

Causal pathway diagramming: PATH used CPDs to ensure implementation strategies contained in the brochure (e.g., educational materials, testimonials) were addressing targeted barriers and to determine how those strategies should be operationalized to address moderators (e.g., age, vaccination status) discovered during the formative focus groups. CPDs were developed for each barrier surfaced through prior work: knowledge, perceived risk, and perception of test accuracy for home testing (Figure 4).

Figure 4

Sample CPDs for the PATH Study

CPD Stem: What Content Within Educational Brochures Is Likely to Address the Barriers?

For each of the three prioritized barriers, the PATH team hypothesized about the mechanisms through which brochure content could resolve the barrier. For instance, for perceived risk, the team postulated the mechanism “increasing personal relevance of risk information.” They used the CPD stem to compare various messaging that could activate this mechanism, to choose messaging that would be most likely to address targeted barriers. This allowed the PATH team to operationalize the content of the educational brochures to maximize effectiveness.

CPD Leaves: Will the Effectiveness of Different Operationalizations Depend on Characteristics of the Recipients?

Subsequent CPDs informed tailoring of brochure content to different groups of recipients. To investigate moderators that may influence brochure effectiveness, the PATH team held four focus groups with Kaiser Permanente Washington (KPWA) health plan members (see Additional File 2 for prototype examples). Rapid analysis (Alison Hamilton, 2013; Hamilton & Finley, 2019) of focus groups indicated the need to tailor materials by age (below and above 45 years) and HPV vaccination status. Based on these findings, the team created tailored versions of the brochure that included different content (e.g., images accompanying testimonials) or additional elements (e.g., information about the need to screen even if one is vaccinated) that addressed the identified moderators.

This process allowed PATH to operationalize brochure content in a manner that was both aligned with hypothesized mechanisms and that addressed key recipient characteristics (age and vaccine status) found to influence how the brochure was perceived by recipients. The resulting brochures are currently undergoing optimization through a factorial experiment.

Function 3: Developing Causal Theories of Implementation Strategies

Function goal: There is a significant body of theoretical literature describing what influences implementation (Nilsen, 2015), but few theoretical accounts describe how and under what circumstances implementation strategies work. Such generalizable causal theories of implementation strategies can aid the field in testing and clarifying what implementation strategies work, for which outcomes, and under what circumstances.

Background: Our team's NIH grant [grant name masked] is developing theories of implementation strategy functioning based on existing evidence and implicit or explicit theories held by researchers (MASKED). This approach utilizes CPDs to generate micro-theories of how implementation strategies function, agnostic to context, for 30 commonly used strategies. The micro-theories can help practice stakeholders plan and evaluate implementation efforts and support researchers in conducting mechanisms-focused research. We illustrate this approach for audit and feedback (A&F), which is among the most widely used and tested strategies, with over 140 clinical trials (Ivers et al., 2012). Despite its widespread use and establishment of meta-laboratories to optimize A&F (Grimshaw et al., 2019), no empirical studies or conceptual articles have established the mechanisms of how A&F works.

Causal pathway diagramming: Our team used CPDs to structure the development of a micro-theory of A&F operation, informed by a synthesis of the extant literature and conversations with subject matter experts (Figure 5).

Figure 5

CPD for Audit and Feedback

CPD Stem: What Implementation Outcome Does A&F Impact and How Does It Achieve That Outcome?

We began by reviewing the literature and speaking to subject matter experts to operationalize A&F. To enhance the generalizability of the micro-theory, this operationalization captured core activities of A&F without operationalizing details of its deployment (e.g., timing, frequency), the context in which it would be deployed, or the performance behavior it would be targeting. The remainder of the CPD stem was informed by a review of conceptual articles, systematic reviews, and practical articles for statements about (1) the purpose or goal of A&F—useful for identifying the determinant (i.e., barrier) that it addresses or the distal outcome it aims to attain; (2) the rationale for the strategy—useful for identifying the barrier the strategy could be deployed to overcome; and (3) any clues about the strategy's plausible mechanism(s) and proximal outcomes. We propose that A&F addresses the barrier of “unknown or inaccurate perception of performance” by activating the mechanism “highlighting performance discrepancy.” That is, we hypothesize that A&F works by making salient for clinicians the discrepancy between their actual clinical performance and desired or expected clinical performance. This ultimately has downstream impact on clinicians’ guideline adherence.

CPD Leaves: Under What Conditions Does A&F Work Best?

Next, we reviewed the literature for statements about determinants that influence whether or not, or how well, A&F works to improve guideline adherence. The literature suggested several plausible moderators related to the strategy features (e.g., clarity of feedback message), the strategy recipient (e.g., behavioral control), and the organizational context (e.g., constructive continuous quality improvement approach).

This micro-theory, and others that are being developed, will inform instrument development and validation efforts to measure plausible mechanisms of common implementation strategies.

Discussion

CPDs offer an approach to advancing the precision in the field's selection, understanding, optimization, and testing of implementation strategies. This article provides principles to inform the development of CPDs and summarizes their key functions to aid users in adopting the method. Through showcasing three CPD use cases, we demonstrate the utility of this approach for grappling with challenging questions in implementation science, including, “how do implementation strategies work,” “what determinants and outcomes are they effective in addressing,” and “under what conditions do they work?”

Dissemination of the early version of the method sparked questions about the intended users and the degree of complexity CPDs can incorporate. The illustrations presented begin to address these issues. They demonstrate how CPDs can be leveraged by diverse stakeholders for varying purposes, ranging from industry partners striving to improve engagement with their DMH app to implementation scientists developing micro-theories of how implementation strategies work. We illustrate how CPDs can incorporate complexity, including layering strategies whose effectiveness is interdependent (e.g., functions 1 and 2), depicting a strategy's ability to influence multiple barriers (e.g., function 1), and how multiple implementation strategies can collectively contribute to improving a single implementation outcome (e.g., functions 1 and 2). Our team has developed a toolkit with additional illustrations and resources to support users in developing CPDs that can we accessed at https://www.impscimethods.org/.

We also commonly encounter the question of how CPDs fit with similar methods that attempt to match strategies to determinants or articulate strategies’ mechanisms of action. CPDs can often be used in combination with existing methods and, in some instances, may be warranted instead of similar approaches. For instance, existing tools to match strategies to determinants, such as the ERIC–CFIR matching tool (Waltz et al., 2019), can identify several possible strategy-determinant pairings that can be vetted using CPDs. However, the CFIR matching tool offers little consensus among experts regarding which strategies are most appropriate for which barriers, reflecting the limited evidence on this topic (Waltz et al., 2019). By using CPDs to articulate the mechanisms through which a strategy impacts a determinant, they can provide greater conceptual certainty about the plausibility of strategy-determinant pairings.

Similarly, CPDs can be used in combination with existing tools that link strategies to mechanisms. For instance, the “Theory and Techniques Tool” summarizes the evidence for the links between behavior change techniques (BCTs) and mechanisms (Connell et al., 2019; The Theory and Techniques Tool, n.d.). Implementers can draw on this evidence to inform strategy-mechanism linkages in their CPDs, especially for strategies that leverage BCTs, such as reminders. CPDs can also extend this tool by providing a structure to articulate the full causal process through which BCTs influence implementation outcomes.

Logic models, such as the implementation research logic model (IRLM) (Smith et al., 2020), describe hypothesized causal processes through which program activities impact outcomes. Logic models provide a helpful roadmap for planning and executing projects, but often lack specificity about the causal chain to evaluate the unique contribution and plausible effectiveness of each strategy. For instance, Sales et al. (2021) note that the IRLM does not capture the mechanism by which determinants are addressed, requiring a “logical leap” between identifying a determinant and selecting an implementation strategy to address it. CPDs can be used to carefully vet individual strategies for selection, while the IRLM can be used for summarizing, planning, and executing the resulting strategy package.

Like CPDs, other approaches seek to elucidate the mechanisms of implementation strategies as well as their downstream effects on outcomes. “Mechanism mapping” uses DAGs to articulate the processes through which a strategy works to understand why a strategy did or did not work in an implementation initiative (Geng et al., 2022). A key strength of mechanism mapping is that it can provide a comprehensive account of the relationships between an implementation strategy and the clinical setting in which that strategy is deployed, including the feedback loops that may be present. However, this comprehensiveness can also lead to a high level of complexity, limiting the method's practical application, especially among implementation practitioners. CPDs can be used to simplify this complexity, by articulating the role of each node identified through mechanism mapping, and zoom in on individual strategies to ensure their appropriateness and the availability of the conditions needed for their effective functioning.

Similar to mechanism mapping, realist evaluations aim to provide comprehensive accounts of how implementation strategies operate. They do so by identifying context–mechanism–outcome (CMO) configurations that link strategy activities, characteristics of the setting, and the achieved outcomes (Sarkies et al., 2023). However, surveys of the realist literature have consistently found that many CMO configurations conflate mechanisms, program elements, and context (Lacouture et al., 2015; Lemire et al., 2020), limiting their ability to generate theoretical insights about strategy mechanisms. Given the similarities in how realist evaluation and CPDs conceptualize mechanisms, CPDs have the potential to increase clarity and precision of realist analyses by providing a way to explicitly describe the role each concept plays in CMO configurations.

Limitations

Healthcare systems are inherently complex. Even rigorously developed CPDs will inevitably miss important information, including elements that are difficult to operationalize, or complex and chaotic processes. The more narrow or atomistic our models become, the less we may be able to see the functioning of the whole system (Popper, 1966)—we lose the forest for the trees. CPDs assume linear causality (or linear loops) that produce specified phenomena, but systems can be non-linear and phenomena emergent, with novel or self-sustaining properties. Although the adage, “all models are wrong, but some are useful,” is perhaps overused, CPDs accept these limitations and seek to turn them on their head. Given the inherent complexity of implementation, CPDs aim to help implementors tame that complexity by making sense of the causal mechanisms underlying implementation efforts and manipulating those mechanisms to maximize effectiveness.

Conclusion

Understanding how, when, and under what circumstances implementation strategies work is critical to the advancement of practical, generalizable knowledge about how to improve health care. The CPD method offers a useful structure and set of principles for advancing this line of work. While no method will fully capture the complexity inherent in implementation processes, the approach we present in this article can facilitate precision in our exploration and understanding of how, when, and under what conditions implementation works.

Supplemental Material

sj-pdf-1-irp-10.1177_26334895241248851 - Supplemental material for Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision

Supplemental material, sj-pdf-1-irp-10.1177_26334895241248851 for Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision by Predrag Klasnja, Rosemary D. Meza, Michael D. Pullmann, Kayne D. Mettert, Rene Hawkes, Lorella Palazzo, Bryan J. Weiner and Cara C. Lewis in Implementation Research and Practice

Supplemental Material

sj-pdf-2-irp-10.1177_26334895241248851 - Supplemental material for Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision

Supplemental material, sj-pdf-2-irp-10.1177_26334895241248851 for Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision by Predrag Klasnja, Rosemary D. Meza, Michael D. Pullmann, Kayne D. Mettert, Rene Hawkes, Lorella Palazzo, Bryan J. Weiner and Cara C. Lewis in Implementation Research and Practice

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by funding from the National Cancer Institute (grant numbers R01CA262325 and P50CA244432) and the National Institute of Mental Health (grant number P50MH126219).

ORCID iDs

Michael D. Pullmann

Kayne D. Mettert

Cara C. Lewis

Supplemental Material

Supplemental material for this article is available online.

References

Alison Hamilton

(2013). Qualitative methods in rapid turn-around health services research. U.S. Department of Veterans Affairs. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780

Balis

L. E.

Houghtaling

(2023). Matching barriers and facilitators to implementation strategies: Recommendations for community settings. Implementation Science Communications, 4(1), Article 144. https://doi.org/10.1186/s43058-023-00532-1

Bowen

N. K.

Guo

(2011). Structural equation modeling. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195367621.001.0001

Brenner

M. H.

Curbow

Legro

M. W.

(1995). The proximal-distal continuum of multiple health outcome measures: The case of cataract surgery. Medical Care, 33(4), AS236–AS244. http://www.jstor.org/stable/3766632

Carey

R. N.

Connell

L. E.

Johnston

Rothman

A. J.

De Bruin

Kelly

M. P.

Michie

(2019). Behavior change techniques and their mechanisms of action: A synthesis of links described in published intervention literature. Annals of Behavioral Medicine, 53(8), 693–707. https://doi.org/10.1093/abm/kay078

Connell

L. E.

Carey

R. N.

De Bruin

Rothman

A. J.

Johnston

Kelly

M. P.

Michie

(2019). Links between behavior change techniques and mechanisms of action: An expert consensus study. Annals of Behavioral Medicine, 53(8), 708–720. https://doi.org/10.1093/abm/kay082

Curry

S. J.

Krist

A. H.

Owens

D. K.

Barry

M. J.

Caughey

A. B.

Davidson

K. W.

Doubeni

C. A.

Epling

J. W.

Kemper

A. R.

Kubik

(2018). Screening for cervical cancer: US Preventive Services Task Force recommendation statement. JAMA, 320(7), 674–686. https://doi.org/10.1001/jama.2018.10897

Fernandez

M. E.

Ten Hoor

G. A.

Van Lieshout

Rodriguez

S. A.

Beidas

R. S.

Parcel

Ruiter

R. A.

Markham

C. M.

Kok

(2019). Implementation mapping: Using intervention mapping to develop implementation strategies. Frontiers in Public Health, 7, Article 158. https://doi.org/10.3389%2Ffpubh.2019.00158

Flodgren

Parmelli

Doumit

Gattellari

O’Brien

M. A.

Grimshaw

Eccles

M. P.

(2011). Local opinion leaders: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 2019(8), Article CD000125. https://doi.org/10.1002/14651858.CD000125.pub4

10.

Flottorp

S. A.

Oxman

A. D.

Krause

Musila

N. R.

Wensing

Godycki-Cwirko

Baker

Eccles

M. P.

(2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science, 8(1), 1–11. https://doi.org/10.1186/1748-5908-8-35

11.

Forsetlund

Bjorndal

Rashidian

Jamtvedt

O’Brien

M. A.

Wolf

Davis

Odgaard-Jensen

Oxman

A. D.

(2009). Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 2009(2), Article CD003030. https://doi.org/10.1002/14651858.CD003030.pub2

12.

Garson

G. D.

(2013). Path analysis. Statistical Associates Publishing.

13.

Geng

E. H.

Baumann

A. A.

Powell

B. J.

(2022). Mechanism mapping to advance research on implementation strategies. PLoS Medicine, 19(2), Article e1003918. https://doi.org/10.1371/journal.pmed.1003918

14.

Giguere

Zomahoun

H. T. V.

Carmichael

P. H.

Uwizeye

C. B.

Legare

Grimshaw

J. M.

Gagnon

M. P.

Auguste

D. U.

Massougbodji

(2020). Printed educational materials: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 8(8), Article CD004398. https://doi.org/10.1002/14651858.CD004398.pub4

15.

Glidewell

Hunter

Ward

McEachan

R. R.

Lawton

Willis

T. A.

Hartley

Collinson

Holland

Farrin

A. J.

(2022). Explaining variable effects of an adaptable implementation package to promote evidence-based practice in primary care: A longitudinal process evaluation. Implementation Science, 17(1), 1–24. https://doi.org/10.1186/s13012-021-01166-4

16.

Grimshaw

Ivers

Linklater

Foy

Francis

J. J.

Gude

W. T.

Hysong

S. J.

(2019). Reinvigorating stagnant science: Implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Quality & Safety, 28(5), 416–423. https://doi.org/10.1136/bmjqs-2018-008355

17.

Hamilton

A. B.

Finley

E. P.

(2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, Article 112516. https://doi.org/10.1016/j.psychres.2019.112516

18.

Harvey

Holmes

C. A.

(2012). Nominal group technique: An effective method for obtaining group consensus. International Journal of Nursing Practice, 18(2), 188–194. https://doi.org/10.1111/j.1440-172x.2012.02017.x

19.

Houlihan

(2018). Dual-process models of health-related behaviour and cognition: A review of theory. Public Health, 156, 52–59. https://doi.org/10.1016/j.puhe.2017.11.002

20.

Hoyle

R. H.

(1995). Structural equation modeling: Concepts, issues, and applications. Sage.

21.

Ivers

Jamtvedt

Flottorp

Young

J. M.

Odgaard-Jensen

French

S. D.

O’Brien

M. A.

Johansen

Grimshaw

Oxman

A. D.

(2012). Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (6), Article CD000259. https://doi.org/10.1002/14651858.CD000259.pub3

22.

Kahneman

(2011). Thinking, fast and slow (1st ed.). Farrar, Straus and Giroux.

23.

Lacouture

Breton

Guichard

Ridde

(2015). The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation. Implementation Science, 10, Article 153. https://doi.org/10.1186/s13012-015-0345-7

24.

Lemire

Kwako

Bohni-Nielsen

Christina

Donaldson

Leeuw

(2020). What is this thing called a mechanism? Findings from a review of realist evaluations. New Directions for Evaluation, 2020(167), 73–86. https://doi.org/10.1002/ev.20428

25.

Lewis

C. C.

Klasnja

Powell

B. J.

Lyon

A. R.

Tuzzio

Jones

Walsh-Bailey

Weiner

(2018). From Classification to Causality: Advancing Understanding of Mechanisms of Change in Implementation Science. Front Public Health, 6, 136. https://doi.org/10.3389/fpubh.2018.00136

26.

Mackie

J. L.

(1965). Causes and conditions. American Philosophical Quarterly, 2(4), 245–264. https://www.jstor.org/stable/20009173

27.

May

C. R.

Johnson

Finch

(2016). Implementation, context and complexity. Implementation Science, 11(1), Article 141. https://doi.org/10.1186/s13012-016-0506-3

28.

Miro. (2022). Miro Board. https://miro.com/

29.

Nilsen

(2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10, Article 53. https://doi.org/10.1186/s13012-015-0242-0

30.

Pearl

(1998). Graphs, causality, and structural equation models. Sociological Methods & Research, 27(2), 226–284. https://doi.org/10.1177/0049124198027002004

31.

Pearl

(2009a). Causal inference in statistics: An overview. Statistics Surveys, 3, 96–146. https://doi.org/10.1214/09-SS057

32.

Pearl

(2009b). Causality. Cambridge University Press. https://doi.org/10.1017/CBO9780511803161

33.

Popper

K. R.

(1966). Of clouds and clocks an approach to the problem of rationality and the freedom of man. Washington University.

34.

Presseau

McCleary

Lorencatto

Patey

A. M.

Grimshaw

J. M.

Francis

J. J.

(2019). Action, actor, context, target, time (AACTT): A framework for specifying behaviour. Implementation Science, 14(1), 1–13. https://doi.org/10.1186/s13012-019-0951-x

35.

Proctor

E. K.

Powell

B. J.

McMillen

J. C.

(2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8(1), 1–11. https://doi.org/10.1186/1748-5908-8-139

36.

Rogers

E. M.

(1995). Diffusion of innovations (Vol. 4). Free Press.

37.

Sales

A. E.

Barnaby

D. P.

Rentes

V. C.

(2021). Letter to the editor on “the implementation research logic model: A method for planning, executing, reporting, and synthesizing implementation projects” (Smith JD, LI DH, Rafferty MR. The implementation research logic model: A method for planning, executing, reporting, and synthesizing implementation projects. Implement sci. 2020;15 (1):84. https://doi.org/10.1186/s13012-020-01041-8). Implementation Science, 16(1), Article 97. https://doi.org/10.1186/s13012-021-01169-1

38.

Sarkies

Francis-Auton

Long

Roberts

Westbrook

Levesque

J. F.

Watson

D. E.

Hardwick

Sutherland

Disher

Hibbert

Braithwaite

(2023). Audit and feedback to reduce unwarranted clinical variation at scale: A realist study of implementation strategy mechanisms. Implementation Science, 18(1), Article 71. https://doi.org/10.1186/s13012-023-01324-w

39.

Smith

J. D.

D. H.

Rafferty

M. R.

(2020). The implementation research logic model: A method for planning, executing, reporting, and synthesizing implementation projects. Implementation Science, 15(1), 1–12. https://doi.org/10.1186/s13012-020-01041-8

40.

Squires

J. E.

Graham

I. D.

Hutchinson

A. M.

Michie

Francis

J. J.

Sales

Brehaut

Curran

Ivers

Lavis

Linklater

Fenton

Noseworthy

Vine

Grimshaw

J. M.

(2015). Identifying the domains of context important to implementation science: a study protocol. Implementation Science, 10(1), 1–9. https://doi.org/10.1186/s13012-015-0325-y

41.

Stage

F. K.

Carter

H. C.

Nora

(2004). Path analysis: An introduction and analysis of a decade of research. The Journal of Educational Research, 98(1), 5–13. https://doi.org/10.3200/JOER.98.1.5-13

42.

The Theory and Techniques Tool . (n.d.). Retrieved January 24, 2024, from https://theoryandtechniquetool.humanbehaviourchange.org/about

43.

Tiro

J. A.

Betts

A. C.

Kimbel

Buist

D. S.

Mao

Gao

Shulman

Malone

Beatty

Lin

(2019). Understanding Patients’ perspectives and information needs following a positive home human papillomavirus self-sampling kit result. Journal of Women’s Health, 28(3), 384–392. https://doi.org/10.1089/jwh.2018.7070

44.

Waltz

T. J.

Powell

B. J.

Fernández

M. E.

Abadie

Damschroder

L. J.

(2019). Choosing implementation strategies to address contextual barriers: Diversity in recommendations and future directions. Implementation Science, 14(1), 1–15. https://doi.org/10.1186/s13012-019-0892-4

45.

Wang

Wong

E. L.

Nilsen

Chung

V. C.

Tian

Yeoh

E. K.

(2023). A scoping review of implementation science theories, models, and frameworks - an appraisal of purpose, characteristics, usability, applicability, and testability. Implementation Science, 18(1), Article 43. https://doi.org/10.1186/s13012-023-01296-x

46.

Winer

R. L.

Tiro

J. A.

Miglioretti

D. L.

Thayer

Beatty

Lin

Gao

Kimbel

Buist

D. S.

(2018). Rationale and design of the HOME trial: A pragmatic randomized controlled trial of home-based human papillomavirus (HPV) self-sampling for increasing cervical cancer screening uptake and effectiveness in a US healthcare system. Contemporary Clinical Trials, 64, 77–87. https://doi.org/10.1016/j.cct.2017.11.004

47.

Wolk

C. L.

Powell

B. J.

Beidas

R. S.

(2015). Contextual influences and strategies for dissemination and implementation in mental health. In P. Nathan (Ed.), Oxford Handbooks Online (pp. 1–20). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199935291.013.12

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.64 MB