Abstract
The Model for Understanding Success in Quality (MUSIQ) is a framework designed to understand the contextual factors that can influence healthcare quality improvement (QI) initiative implementation. The MUSIQ calculator was originally developed to help clinics identify contextual factors that may affect QI success. This retrospective study adapted the MUSIQ calculator to serve as an evaluative tool for practice facilitators engaged in a colorectal cancer screening initiative. Eight primary care clinics were scored in 6 contextual factors categories using the adapted MUSIQ calculator. Average MUSIQ scores were used to identify high and low contextual factors clinics, and their colorectal cancer screening rates were assessed across a 4-year period of active engagement with the colorectal cancer screening initiative. There were statistically significant, strong, correlations between overall contextual factors scores and colorectal cancer screening rates across all 4 years. There was a statistically significant difference between screening rate changes during the period of active engagement and high or low contextual factors scores (P = .047). There were statistically significant correlations between 3 contextual factors and colorectal cancer screening rate changes with “Microsystem” and “QI Support” having the strongest associations (P < .001). Low or high contextual factors classification statistically significantly predicted colorectal cancer screening rate changes across the observed timeframe (P = .047). By adapting existing tools with a strong track record of success, like MUSIQ, practice facilitators can identify potential challenges earlier in the QI process providing an important opportunity to intervene to prompt greater success.
Introduction
Practice facilitation is a strategy used to assist healthcare organizations increase their capacity to implement and sustain quality improvement (QI) initiatives based on current evidence.1 -4 It is frequently used in the primary care setting 3 and has been found to be effective in enhancing care delivery, particularly with chronic disease management and preventive services, like cancer screening. 4 Practice facilitation has been shown to increase colorectal, breast, and cervical cancer screening rates within primary care,1,2,4 highlighting its importance to the field of cancer prevention and control. It is an established strategy to support cancer screening QI initiatives in primary care2,4,5 with organizations like the Agency for Healthcare Research and Quality (AHRQ) offering a formal training curriculum to support this work. 6
Practice facilitators work with primary care clinics to adopt evidence-based guidelines which can be used to improve process and outcome measures. 3 Facilitators use interpersonal communication and health information technology skills in this work. 3 Their tasks can include providing technical assistance through facilitation of Plan-Do-Study-Act (PDSA) cycles, data collection support, project management, training, and consulting services. 1 Practice facilitators serve a unique role in primary care. They are external to the clinics they work with but frequently build long-term, trusting relationships with staff and providers, providing them with a crucial opportunity to make a lasting impact in healthcare. 7 Despite the importance of practice facilitation to QI and the varied roles of facilitators, research into the field is limited 3 and additional tools and structures to support this work are needed. 8
Study Framework
The Model for Understanding Success in Quality (MUSIQ) is a framework designed to understand the contextual factors that can influence QI success in healthcare. 9 MUSIQ identifies 3 domains for clinical practices to evaluate: (1) setting, (2) environment, and (3) leadership. 10 These domains were derived from a systematic review of evidence and the framework has been shown to be a useful tool to understand QI success. 11 MUSIQ was initially proposed as a tool for practice facilitators to use during PDSA cycles with clinics. 9 To support this use, the Cincinnati Children’s Hospital Medical Center research team developed a calculator for clinics to complete as a way to self-reflect on contextual factors that may inhibit implementation of evidence-based QI strategies. 9 The calculator was built in Excel and computes scores for each contextual factor, such as external motivators, QI team norms, and specific microsystems. Clinic staff respond to prompts in each contextual factors subscale, assigning a value of 1 (lowest score) to 7 (highest score) based on the level of agreement, with total scores for each factor provided in the summary tab of the tool. The premise is that low scores will highlight areas of potential challenge for the clinic during QI efforts.
Study Overview and Aims
The West Virginia Program to Increase Colorectal Cancer Screening (WV PICCS) is designed to increase colorectal cancer screening rates in rural primary care using a practice facilitation approach as part of the Centers for Disease Control and Prevention (CDC) Colorectal Cancer Control Program. Since 2015, WV PICCS has partnered with 50 primary care clinics throughout West Virginia, located in the rural Appalachian region of the United States, to implement evidence-based interventions to increase colorectal cancer screening rates. WV PICCS uses skilled practice facilitators to engage in QI work with partner clinics. Facilitator activities include regular communication with clinic staff, providing targeted training, working through clinic workflows and processes using PDSA cycles, and supporting the use of health information technology.
WV PICCS engaged an evaluator to monitor clinic progress using a mixed methods approach grounded in the CDC program evaluation framework. 12 Through this evaluation process, particularly bi-annual interviews with practice facilitators on implementation progress, the WV PICCS team noticed similar characteristics in the partner clinics that were able to significantly improve their colorectal cancer screening rates during their participation. An evaluative study was proposed to determine if there was a systematic way for WV PICCS practice facilitators to identify partner clinic characteristics most associated with improved performance. The premise of the study was to identify key contextual factors associated with better performance retroactively so potential challenges with future partner clinics could be recognized earlier in the practice facilitation process. The MUSIQ calculator was proposed as a tool to better understand the contextual factors surrounding each partner clinic. The team determined that the MUSIQ calculator would need to be adapted to fit programmatic needs, including being tailored to be completed by WV PICCS practice facilitators as opposed to clinic staff to provide a greater level of external objectivity. The study hypothesis was that partner clinics with higher overall contextual factors would have greater colorectal cancer screening rate increases over the duration of their participation with WV PICCS.
The aims of this study were to:
Adapt the MUSIQ calculator to be used by practice facilitators evaluating primary care clinic performance in a colorectal cancer screening QI program.
Compare primary care clinic contextual factors scores with annual colorectal cancer screening performance.
Assess whether the adapted MUSIQ calculator could be used as a tool to screen, predict, and evaluate clinic performance in a colorectal cancer screening QI program.
Methods
Ethics
Evaluation of WV PICCS using aggregate clinic-level data is covered by the West Virginia University Institutional Review Board protocol #2010148957.
Participants
Nine West Virginia primary care clinics provided aggregate colorectal cancer screening rates at baseline prior to engaging with WV PICCS (2020), during 2 years of active implementation activities (2021–2022), and after 2 years of maintenance support (2023–2024) and served as the participant pool for this retrospective, evaluative study. All but one clinic participating in WV PICCS during this timeframe was included in analysis (Table 1). The excluded clinic was unable to provide a baseline colorectal cancer screening rate therefore their data could not be similarly compared with the other clinics for this analysis. Of the clinics included in this study, 7 clinics were Federally Qualified Health Centers and 1 was a safety-net free clinic (Clinic D).
Participant Clinic Characteristics.
Colorectal cancer screening rate data is for 50 to 75-year-old patients as screening guidelines to lower the starting age to 45 years did not occur until 2021.
Low contextual factors clinics (below cohort average overall MUSIQ score).
Instrumentation
The WV PICCS team actively involved in the adaptation of the MUSIQ calculator included 2 practice facilitators along with the program’s evaluator, director, and principal investigator. They received access to the MUSIQ calculator from Dr. Heather Kaplan, the researcher who led the team that designed the framework, in February 2023. The evaluator reviewed the MUSIQ calculator as designed by Dr. Kaplan and her team and made initial edits to the wording to shift from clinic self-reflection to evaluation by external practice facilitators. The evaluator also added average scores for each subscale, differing from the original version. For instance, where the original tool broke down “QI Team” items into 9 different elements that were scored, the adaptation used a comprehensive score from 1 to 7 to rate “QI Team” performance. Focusing on a single score for 6 different subscales (“QI Team,” “Microsystem,” “QI Support,” “Organization,” “Environment,” and “Other”) made the tool more practical for the WV PICCS team to identify specific areas of clinic strength and weakness. The “QI Team” section had 16 items focused on the clinic team working on WV PICCS. For instance, “[m]ost members of the QI team have worked on improvement projects before.” The “Microsystem” section had 15 items focused on 3 different teams within the primary care setting (clinical, clerical, referral). For instance, “[t]his microsystem values teamwork, communication, and a commitment to quality improvement.” The “QI Support” section had 2 items focused on data reporting, like “[e]xisting information systems allows the clinic to easily pull data specifically needed for this QI project.” The “Organization” section had 8 items focused on clinic administration and leadership involvement. For instance, “[t]he senior executives in this clinic are directly involved in quality improvement activities.” The “Environment” section had 3 items focused on external pressures, such as, “[p]ressures or incentives from outside the organization motivated them to undertake this specific QI project.” Finally, the “Other” section had 1 item, “[a] specific event prompted the launch of this QI project.” Each subscale score was further averaged to arrive at a clinic’s overall contextual factors score, differing from the original version that did not assign an overall score. After these adaptations were made by the evaluator, the tool was sent to all members of the WV PICCS team for review. Edits were made to item language to reflect the use of the calculator specifically for WV PICCS. For instance, clinic participation in WV PICCS was identified as the triggering QI event, so each clinic received a score of 7 for the “Other” contextual factors subscale.
Scoring Procedures and Analysis
After the adapted MUSIQ calculator was finalized, each clinic was scored. Internal reliability of this scoring process was accomplished with a primary coding team consisting of 3 individuals who worked with the clinics (2 practice facilitators and program director) meeting to come to consensus about overall clinic characteristics and scoring approach.
Annual colorectal cancer screening rates for the 8 clinics were compared to MUSIQ subscale and overall contextual factors scores. An average contextual factors score was calculated by averaging all clinics’ contextual factors scores. Clinics with overall contextual factors scores above the clinic average were identified as having “high contextual factors” and those below the clinic average were identified as having “low contextual factors.” Low and high contextual factors clinics were grouped for analysis. All statistical analyses were completed using SPSS v28.
Results
Overall Contextual Factor Scores
Four clinics were identified as having “high contextual factors” and 4 clinics were identified as having “low contextual factors” based on their overall scores being above or below the clinic average (Table 1). Scores ranged from 6.17 (out of a possible 7) to 3.77. Average colorectal cancer screening rate changes were calculated for clinics within their classification groups of high or low contextual factors for between-group comparison (Table 2). High contextual factors clinics saw a 16.7% increase in colorectal cancer screening rates during the same period that low contextual factor clinics saw a 35.9% decrease. High contextual factors clinics had a more positive colorectal cancer screening linear trendline compared to clinics with lower contextual factors scores (Figure 1). A 1-way ANOVA was conducted to determine if colorectal cancer screening rate changes over the 2020 to 2024 period were different for clinics determined to have high (n = 4) or low (n = 4) contextual factors. Rate changes increased for high contextual factor clinics (13.61 ± 5.26) compared to decreasing for low contextual factors clinics (−32.00 ± 36.32). This result was statistically significant, F (1,6) = 6.180, P = .047. Point-biserial correlations found statistically significant, strong, associations between high and low contextual factors classification and colorectal cancer screening rates all 4 years of clinic involvement with WV PICCS—2021(rpb (6) = 0.835, P = .010); 2022 (rpb (6) = 0.856, P = .007); 2023 (rpb (6) = .813, P = .014); 2024 (rpb (6) = .904, P = .002).
Clinic Colorectal Cancer Screening Rate Changes (2020-2024) by Overall Contextual Factors Grouping Score.
Colorectal cancer screening rate changes by contextual factors score grouping with trendlines.
Subscale Contextual Factors Scores
Point-biserial correlations were used to assess associations between subscale category classification for low contextual factors or high contextual factors and each subscale (“QI Team,” “Microsystem,” “QI Support,” “Organization,” “Environment”). “Other” was excluded from this analysis as all clinics received the same score based on it being determined that participation in WV PICCS served as an equal triggering event. There was a statistically significant correlation between 3 of the 5 individual contextual factors—“Microsystem” (rpb (6) = .923, P < .001), “QI Support” (rpb (6) = .926, P < .001), and “Environment” (rpb (6) = .752, P = .031). “Organization” (rpb (6) = .600, P = .116) and “QI Team” (rpb (6) = .695, P = .056) did not have a statistically significant association with low or high contextual factors.
Contextual Factors Predicting Colorectal Cancer Screening Rate Changes
A simple linear regression was run to assess the linear relationship between contextual factors scores in the categories of “QI Team,” “Microsystem,” “QI Support,” “Organization,” and “Environment” and colorectal cancer screening rates after 4 years of engagement with WV PICCS. Contextual factors scores accounted for 98.1% of variation in the 2024 colorectal cancer screening rates with adjusted R2 = 93.3%, a large size effect. The overall contextual factors score statistically significantly predicted colorectal cancer screening rates after 4 years of WV PICCS participation, F (5, 2) = 20.583, P = .047.
Discussion
Findings suggest that clinic contextual factors play a significant role in colorectal cancer screening rate changes when engaged in a practice facilitation-based QI initiative. Clinics with high contextual factors scores were associated with improved colorectal cancer screening rate changes over the observed period. In contrast, clinics with low contextual factors scores were associated with decreased colorectal cancer screening rates over the same timeframe. This suggests that given similar experiences working with WV PICCS, different outcomes can be anticipated related to the unique contextual factors of each clinic. This aligns with literature suggesting the critical role that healthcare organization contextual factors play in QI initiative success. 13
While 3 contextual factors subscales were statistically significantly correlated with changes in colorectal cancer screening rates over the observed period, “Microsystem” and “QI Support” had the strongest and largest associations. “Microsystem” items assess how different teams (eg, clerical, clinical) engage with the QI initiative. This matches existing literature that emphasizes the importance of a team-based approach for QI initiative success. 14 As a practical screening tool, practice facilitators that identify areas of clinic weakness can intervene earlier, perhaps improving overall QI initiative success. For instance, if a clinic has been found to have challenges in the “Microsystem” subscale, the practice facilitator could engage the clinic in an early training program such as AHRQ’s TeamSTEPPS. 15 “QI Support” which highlights the data reporting aspect of QI initiatives aligns with previous research on the importance of electronic health records support for successful colorectal cancer screening initiatives. 16
Results suggest that using the adapted MUSIQ calculator may be an important screening, evaluative, and predictive tool for individuals engaged in QI practice facilitation within primary care. The adapted MUSIQ calculator may be a useful tool to help practice facilitators: (1) identify clinics ready to participate in QI programs, (2) predict overall success while participating in QI programs, (3) identify potential challenges early in the practice facilitation process, and (4) provide more precise evaluation of QI program performance.
Limitations and Future Directions
While these findings suggest the adapted MUSIQ calculator could be an effective planning and evaluation tool for QI practice facilitation in primary care, findings are limited as they are focused on a small, state-specific sample of clinics engaged in 1 QI program. Furthermore, the potential for bias in the evaluations of clinics must be considered as a limitation. While there were 3 individuals who worked collectively to arrive at scoring consensus, they had been exposed to several years of clinic-level colorectal cancer screening data which may have unintentionally influenced their evaluations. Future studies should test the effectiveness of using the MUSIQ calculator earlier in the evaluation process to limit potential scoring bias along with tailoring the adapted version of the tool to different QI programs and populations to assess for scalability and effectiveness.
Conclusion
As the use of practice facilitators in primary care QI initiatives continues to grow, practical tools to support their work and evaluate outcomes are needed. By adapting existing tools with a strong track record of success, like MUSIQ, QI programs can foster innovative ways to identify potential challenges earlier in the practice facilitation process providing an important opportunity to intervene early and prompt greater QI success.
Footnotes
Acknowledgements
The MUSIQ Excel Calculator by Cincinnati Children’s Hospital Medical Center is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Author Contributions
Dannell Boatman: conceptualization; methodology; analysis; project administration; writing—original draft; Susan Eason: investigation; writing—review & editing; Kelcie Sturgeon-Danley: investigation; writing—review & editing; Catherine Whitworth: writing—review & editing; Stephenie Kennedy-Rea: conceptualization; writing—review & editing.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We acknowledge the Centers for Disease Control and Prevention, for its support of the West Virginia University staff, and the printing and distribution of the monograph under cooperative agreement NU58DP006768 awarded to West Virginia University. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CDC.
Ethical Approval
This study is covered by West Virginia University Institutional Review Board protocol #2010148957.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.