Outcomes of an Equity,Quality Improvement,and Patient Safety (EQuIPS) Curriculum in Community Hospital-Based Family Medicine Residency Program

Introduction

The Accreditation Council for Graduate Medical Education (ACGME) requires training in Equity, Quality Improvement, and Patient Safety (EQuIPS) in residency and fellowship programs. The actual requirements are interspersed in all the milestones of all six ACGME core competencies. Equity, safe and quality health care underpin the Quadruple aim, yet medical errors account for 250 000 to 400 000 U.S. deaths annually, making them third leading cause of death.^1,2 Quality of care and equity are twins; one is not achieved without the other. Health equity has long been a central focus of public policy and public health, for example, Healthy People 2020 initiative. However, a study examining trends from 1993 to 2017 found minimal progress in advancing health equity in the United States over the past 25 years.^3,4 Studies suggest that U.S. patients receive approximately only about half of the recommended care, despite the healthcare system being the most expensive globally. ⁵ The 1999 report To Err is Human ⁶ brought attention to medical errors prevalence and their impact on healthcare outcomes, which shifted the focus from individual blame to systemic issues. The COVID-19 pandemic further emphasized the importance of ethical decision-making and communication, yet formal training in these areas remains limited. ⁷ Despite recognizing leadership as a key competency, studies show that many faculty and students feel it's not adequately taught to future physicians. ⁸

Studies by Stevens et al ⁹ and Warm et al ¹⁰ demonstrated that residents exposed to QI education during their training delivered higher-quality care in their future practice. While studies like this highlight the value of integrating QI education into residency programs, there remains a notable gap in the literature that reports measurable outcomes related to equity, QI, and patient safety within graduate medical education (GME) curricula, especially using the Kirkpatrick model of evaluation.

Quality improvement in healthcare is driven by systematic, data-based strategies supported by organizations like ACGME, which mandates that residency programs teach QI through its Practice-Based Learning and Improvement (PBLI) competency. Additional support comes from groups like Agency for Healthcare Research & Quality (AHRQ) and Institute for Healthcare Improvement (IHI), which provide research, tools, and training to enhance patient safety and care quality. Despite these resources, there are many challenges in developing an effective training program that address the complexities of patient safety and quality improvement. The healthcare system is always evolving and its quality is affected by many different factors including social determinants of health (SDOH), care coordination and the application of QI tools across diverse settings. Residency programs, especially community programs with limited time and resources, may struggle to provide comprehensive QI education while balancing other aspects of residency training.

Additionally, fear of retribution, including legal consequences or criticism from colleagues, may discourage residents from reporting mistakes, ultimately limiting the opportunity for learning and improvement. ¹¹

To overcome these barriers, medical educators must design improved curricula that empower residents with the skills to improve patient safety. Shorter, more focused curricula, with straightforward methods for assessing residents’ progress, can be an effective way to teach practical QI skills. ¹²

We introduced and implemented an innovative, longitudinal and integrated curriculum on EQuIPS. It is focused on teaching residents the core concepts and skills necessary to address healthcare disparities, enhance quality of care, and participate in patient safety work. By equipping residents with knowledge and tools to navigate the complexities of healthcare systems, programs like EQuIPS are aimed at improving both patient outcomes and the overall healthcare experience. This study reports quantitative analysis of the outcomes of EQuIPS using the Kirkpatrick model in a GME program.

Methods

The Curriculum as Intervention

Grounded in ACGME milestones, the curriculum requires mandatory Quality Improvement reading modules, a Personal Wellness Quality Improvement (PWQI) project and IHI Patient certificate to teach residents essential skills for delivering equitable care, lead quality improvement initiatives, and improve patient safety. The changes in scholarly involvement after EQuIPS rollout indicated the impact of the curriculum on resident behavior and leadership. Table 1 shows the content area with learning objectives for the EQuIPS curriculum.

OPEN IN VIEWER

Table 1.

EQuIPS (Equity, Quality Improvement/Processes, and Patient Safety) Longitudinal Curriculum With Their Learning Objectives.

Content Area	Learning Objectives
Introduction to Equity, SDOH and Health Care Improvement	List the Institute of Medicine's (IOM) six aims for improvement Describe common national and global healthcare disparities Define health equity, distinguish between health equity and equality IV. List Social Determinants of Health (SDOH) and barriers to healthcare V. Analyze the SDOH to barriers in your practice critically List Deming's System of Profound Knowledge
Model for Improvement	List the Principles for setting SMART aim Distinguish between vision, goal, AIM, and SMART aim Define measures, and distinguish between outcome, process, and balancing measure, apply measures in a mock case Identify change concepts for intervention selection Practice testing changes with an iterative plan, do study, and act (PDSA) cycles
Quality tools (total of 2 sessions)	List the seven quality tools and describe their uses Create an Ishikawa diagram (cause and effect diagram) (mock case) Create a Driver diagram (mock case) Create a Process map (mock case) pre intervention and post intervention
Utilizing Data for Improvement	Describe the probability and non-probability sampling methods for data collection Practice entering baseline data in statistical software Demonstrate high impact Intervention using an impact effort matrix
Statistical Process Control (SPC) = 2 sessions)	List vital aspects of the run chart List crucial elements of control charts (I-MR) _ Distinguish between common and special cause variation Introduction to Statistical Process Control Analyze and apply common and special cause variations to sample data
Intervention Iterative (PDSA)	Define Implementation (Pilot) List the components of PDSA Distinguish between Scale and Spread
Communicate and Share	Demonstrate communicating results of SPC Integrate visual graphics for presentations
Culture of Safety and Just culture	Organizational Culture of safety Differentiate latent failures and active failures in Swiss cheese Model Psychological safety Reporting adverse events List the principles of Just culture
Morbidity &Mortality	Learn the principles of conducting Morbidity and Mortality Learn the cause and effect diagram (5 Why's)
Root Cause Analysis	X. Describe the salient features of RCA2

The quality improvement project training was packaged into a 13-step structured roadmap for scholarly activity with previously proven successful implementation,^13–16 A summary of the alignment of the 13-step structured roadmap with Quality Improvement projects is shown in Figure 1. Kern's model for curriculum development was used to align new and existing elements of this curriculum in overarching training of family medicine. It is shown in Figure 2 as a schematic overview of the curriculum development process with the six steps of the Kern's cycle.

OPEN IN VIEWER

Figure 1.

The Alignment of Quality Improvement and Patient Safety Projects With the 13-Step Structured Roadmap for Scholarly Activity.

OPEN IN VIEWER

Figure 2.

Schematic overview of the curriculum development process with the six steps of Kern's Cycle.

This curriculum was rolled out as multifaceted bundled intervention. The implementation of this bundle was also part of the personnel requirements reported previously by Waheed et al. ¹³

Data Collection

This study evaluated three resident cohorts in chronological order based on rollout and implementation of the curriculum. Those who graduated: (a) before EQuIPS implementation (2016-2018 = “pre-EQuIPS”), (b) during its rollout (2019-2020 = “Rollout Phase”), and (c) after full EQuIPS implementation (2021-2024 = “post-EQuIPS”). Baseline demographics, including race (Asian, White, African American, Latino, Other), gender (male, female), and medical school background (LCME accredited, osteopathic, international), were compared across phases using the Chi-squared test. Meaningful resident engagement and accomplishment was assessed by evaluating the extent of scholarly output via Kirkpatrick levels. The Kirkpatrick model is a recognized framework for evaluating curriculum outcomes (11). Engagement and accomplishment was assessed using the Kirkpatrick evaluation model, focusing on Level 2 (demonstration of theoretical knowledge through completion of Institute for Healthcare (IHI) Patient Safety certificates, required QI modules, and personal wellness QI projects), Level 3 (behavioral change through participation in clinical/system-based QI projects, RCA Square activities, and M&M presentations), and Level 4 (project leadership and dissemination through regional/national presentations or peer-reviewed publications). Detailed definitions for each Kirkpatrick level are provided in Table 2. Kirkpatrick Level 1 was not included in our data collection given that it measures only reaction to the curriculum. The evaluation of the curriculum by the residents is optional for accreditation. By the design there would be no data for pre-EQuIPS and minimal for Rollout Phase for Kirkpatrick Level 1, there would be no meaningful comparison between the groups. It would not have added any more value to this study. Kirkpatrick Level 2 evaluates knowledge component. For this study, granular data on individuals was not collected. Rather the outcome was measured as dichotomous variable of completion or no-completion of IHI certificate, required QI modules, and personal wellness QI project. Our analysis focused on Kirkpatrick Levels 2, 3, and 4, while Level 1 was excluded since it reports reaction to the curriculum and done by all residents for all curricula. The data used for this study was already collected in the residency management software New Innovations. It was collected anonymously without any identifying information as count data. Although the data was presented to the Clinical Competency Committee for every resident per standard procedures, achievement of different Kirkpatrick levels did not need any additional judgement of an evaluator or score by an evaluator. Residents’ accomplishments were validated in the residency management suite, New Innovations, by the supervising attending or the program coordinator.

OPEN IN VIEWER

Table 2.

Outcomes of the Curriculum Evaluation on Kirkpatrick Model.

Kirkpatrick Level	Outcomes of the Curriculum Evaluation	P-Value
Level 2: Demonstrate theoretical knowledge of concepts of EQuIPS	Completion of IHI certificate^ for Patient Safety, Completion of Quality Improvement required reading modules, Completion of Personal Wellness Quality Improvement (PWQI) project	<.001*
Level 3: Demonstrate Behavioral Change related to QI & PS Projects in the program	Documented involvement in Traditional QI projects in the clinic, hospital, or system or documented involvement of 3-2-1 team project, Perform RCA Square and present in Morbidity & Mortality Meeting	<.001*
Level 4: Demonstration of leadership or key involvement in a completed project	Local, Regional, National presentation or peer-reviewed publication of QI project from the clinic (3-2-1 team structure)	<.001*

1

P-value calculated using Poisson regression analysis.

^ IHI certification requires post-test for the modules indicating the development of theoretical knowledge of the concepts. There was no certification before the implementation of the EQuIPS, only partial during the implementation, and 100% post-implementation.

* Statistical significance (P < .05).

Results

Descriptive demographic analysis showed 14 of total residents in the pre-implementation, 12 residents in the Rollout phase, and 25 residents in the post-implementation phase. The Demographic analysis between the three groups was done by using the Chi-Squared test. There were no significant differences between the three groups with regards to Gender and Race among the three groups as shown in Table 3. There were significantly more graduates from International Medical Schools in the post-implementation group as compared to the other two while more Osteopathic graduates were observed in the pre-implementation group (P < .001) shown in Table 3.

OPEN IN VIEWER

Table 3.

Comparison of the Three Groups by Race, Gender, and Medical School Attended.

Variable	Category	Pre-EQuIPS Implementation, n (%)	During EQuIPS Rollout, n (%)	Post-EQuIPS Implementation, n (%)	P-Value
Race	Asian	3 (5.9%)	6 (11.8%)	14 (27.5%)	.072
	White	9 (17.6%)	4 (7.8%)	4 (7.8%)
	African American	1 (2.0%)	2 (3.9%)	5 (9.8%)
	Latino	0 (0.0%)	0 (0.0%)	2 (3.9%)
	Other	1 (2.0%)	0 (0.0%)	0 (0.0%)
Medical School	LCME Accredited	0 (0.0%)	0 (0.0%)	3 (5.9%)	<.001*
	Osteopathic Accredited	14 (27.5%)	12 (23.5%)	9 (17.6%)
	International Graduate	0 (0.0%)	0 (0.0%)	13 (25.5%)
Gender	Male	6 (11.8%)	6 (11.8%)	12 (23.5%)	.928
	Female	8 (15.7%)	6 (11.8%)	13 (25.5%)

1

P-value calculated using Chi-squared test or Fisher's exact test where assumptions of Chi-squared test were not met because of the observed counts.

*

Statistical significance (P < .05).

Percentages are calculated based on the total sample size (N = 51).

The SPC 3-phase analysis, as shown in Figure 3, revealed a significant shift in the process indicating resident engagement and involvement in interdisciplinary EQuIPS projects (Kirkpatrick Level 3) and the completion of projects presented at regional/national conferences or published in peer-reviewed journals (Kirkpatrick Level 4) following the implementation of the EQuIPS curriculum. The number of EQuIPS project presentations at regional/national levels and peer-reviewed publications were 0.36 per resident per graduation year at the baseline. This increased to 1.3 per resident per graduation year in the roll-out period and to further increase at 4.9 per resident per graduation year in the post-implementation phase.

OPEN IN VIEWER

Figure 3.

U-Control Chart of the Rate QI Projects per Resident per Graduation Year. UCL, Upper Control Limit; LCL, Lower Control Limit. The Blue Lines Indicate the Phases along the Time Scale on X-axis. The Average Number of QI Projects in the Three Phases Steadily Increased, but U Control Charts Adjusted for the Denominator as shown by the Variable UCL and LCL (Zigzag Top and Bottom Red Lines).

The Kruskal–Wallis test revealed a significant difference in scholarly research involvement (Levels 2 and 3 outcomes) among the three rollout groups for the EQuIPS curriculum (Pre-EQuIPS Implementation, During EQuIPS Rollout and Post-EQuIPS Implementation), with a P-value of <.001. Pairwise comparisons using the Wilcoxon rank sum test with Bonferroni correction further confirmed significant differences between all group pairs: Pre-EQuIPS Implementation versus During EQuIPS Rollout (P = .002), Pre-EQuIPS Implementation versus Post-EQuIPS Implementation (P < .001), and During EQuIPS Rollout versus Post-EQuIPS Implementation (P < .001). This can also be visualized in the 3-phase analysis using the U-control chart shown in Figure 3. It indicates a steady increase in the number of QI and PS projects per resident per graduation year. Poisson regression also mirrored these results indicating that the implementation of the EQuIPS program had a significant impact on increasing scholarly activity among residents. The Poisson regression plot is shown in Figure 4. It presents a Poisson regression model examining the number of scholarly activities involving EQuIPS projects over time. Poisson regression is appropriate for this analysis because the outcome variable represents count data, and the model estimates the expected rate of scholarly activities as a function of the predictor(s), assuming a log-linear relationship. The regression plot demonstrates a positive association between time (or implementation phase) and the number of scholarly activities, with the fitted regression line indicating an increasing expected count of EQuIPS-related scholarly outputs. This suggests that as the EQuIPS initiative matured, participation in scholarly activity increased in a sustained and systematic manner rather than occurring sporadically. The observed data points cluster reasonably close to the fitted line, indicating that the Poisson model provides an adequate fit to the observed counts. The absence of extreme deviations suggests no substantial overdispersion or model instability evident from the visual inspection alone, supporting the use of a Poisson framework for inference. From an interpretive standpoint, the upward trajectory of the regression line implies that the implementation of EQuIPS was associated with a progressive increase in scholarly productivity, consistent with a growing culture of scholarship and structured support for resident and faculty engagement. The log-scale nature of Poisson regression means that changes are multiplicative rather than additive, indicating that each incremental increase in the predictor corresponds to a proportional increase in expected scholarly output. Overall, this regression plot supports the conclusion that EQuIPS implementation was associated with a significant and sustained increase in scholarly activities, reinforcing the effectiveness of the program in promoting scholarship within the training environment (Figure 4).

OPEN IN VIEWER

Figure 4.

Regression Plot of Number of Scholarly Activities Involving EQuIPS Projects on Poisson Distribution.

Table 4 summarizes the results of Poisson regression analysis which showed that as compared to the baseline there was 215% increase in the number of EQuIPS-related scholarly activities in the post-implementation phase.

OPEN IN VIEWER

Table 4.

Summary of the Results of Poisson Regression Analysis.

Comparison Level	Incidence Rate Ratio	95% Confidence Interval
Phase 1/0	5.46	1.86-16.02
Phase 2/1	3.94	2.49-6.23
Phase 2/0	21.50	7.77-59.4

Phase 0 represents the pre-EQuIPS, phase 1 represents the rollout phase, and phase 2 represents post-EQuIPS.

Discussion

Research has consistently shown that early training in quality improvement (QI) methodologies improves patient outcomes and positively influences resident performance. Many studies, including the work by Warm et al, ¹⁰ have shown that residents exposed to QI education during their residency training deliver higher-quality care in clinical practice. However, most of these studies have been conducted in large academic medical centers.

Given the lack of studies evaluating QI curricula in community-based residency programs, our study is among the first to utilize measurable outcomes through the Kirkpatrick model to assess the impact of a structured QI curriculum on equity, patient safety, and overall quality improvement. The application of the Kirkpatrick model provided a structured and comprehensive framework for evaluating the effectiveness of our curriculum within the family medicine residency program. By focusing on Levels 2 (Learning), 3 (Behavior), and 4 (Results), we were able to move beyond surface-level satisfaction metrics and assess meaningful educational and clinical outcomes. The statistically significant findings across these levels suggest that the curriculum not only increased residents’ knowledge and skills (Level 2) but also translated into observable changes in clinical behavior (Level 3) and measurable improvements in patient care outcomes or system-level processes (Level 4). Excluding Level 1 (Reaction) was intentional, as all residents consistently met this threshold, and its inclusion would not have contributed value to our analysis. Using the Kirkpatrick model made our evaluation stronger by tying the results to clear, recognized levels of educational impact, which helped make our findings more understandable and meaningful. Unlike traditional QI curricula that focus primarily on process improvement projects or improvements of certain KPIs, the EQuIPS uniquely integrates principles of health equity, systems-based practice, and patient safety within a single longitudinal framework. This integration differentiates EQuIPS by embedding QI learning directly into residents’ clinical workflows while emphasizing equity-driven outcomes. This framework may be a useful tool for other residency programs seeking to systematically evaluate the effectiveness of multifaceted curricular interventions.

Our findings reveal statistically significant outcomes. It has transformed our residency program's training by creating a culture of continuous growth and improvement. Inspired by its success, this curriculum is now being adapted across other residency and fellowship programs across two sponsoring institutions. The results highlight the potential of targeted QI education to improve both resident competency and patient-centered outcomes. It can be adapted to other programs to achieve similar results.

Over the past two decades, quality improvement (QI) has become a core competency in physician training. A collective case study of four residency programs at one institution examined how QI is taught to residents. ¹⁷ It revealed that programs with hybrid QI curricula, combining both didactic and experiential learning, lead to higher resident engagement, skills, and attitudes toward QI. In contrast, programs with didactic-only curricula face challenges, including the perception that research-focused scholarly projects hinder QI involvement. Common barriers to QI involvement included limited time, mentorship, and pressure to prioritize research over QI. The study also found that learning climate and local safety culture significantly influences residents’ QI learning. Therefore, based on the previous findings, our study utilized both didactic and experiential learning in QI training, creating an environment conducive to QI activity.

Despite the U.S. healthcare system's substantial expenditure, there remains a need to ensure that investments lead to measurable improvements in healthcare delivery. However, the complexity of the healthcare environment demands a strong focus on system-based practice. This can only be done by using QI tools to respond to SDOH and improve care coordination within the system. By training our residents with QI skills, they will be able to recognize and work towards solving issues within the healthcare system.

Another aspect to note is that regardless of field of specialization, physicians fundamentally serve in leadership roles within healthcare teams and are held accountable for patient care outcomes. Yet, despite the essential role of leadership in clinical practice, there is still no universally accepted definition of “clinical leadership.” Research highlights this gap in clarity and training. One study found that while 79% of survey respondents view leadership as a core competency for physicians, 55% feel that leadership skills are not adequately taught during medical education. Additionally, 82% believe that stronger leadership abilities would enhance their effectiveness as physicians, and 61% do not agree that leadership training is consistently integrated across the medical curriculum. ⁸ EQuIPS addresses this gap by embedding leadership skill development directly within QI and patient safety initiatives, allowing residents to gain leadership experience through managing real improvement projects and team-based interventions. This integrated approach strengthens both leadership capacity and QI competency. It allows demonstration of leadership growth through hands-on participation in system-level improvement and dissemination within the system and outside world as presentations at local, regional, and national presentations as well as peer-reviewed publications.

This study has several limitations that should be acknowledged. First, quasi-experimental design without randomization introduces potential selection bias and limits causal inference. While demographic variables such as race and gender were comparable across cohorts, a significant difference in medical school background (eg, higher proportion of international medical graduates in the post-implementation group) may have influenced outcomes, particularly scholarly engagement. Second, the study was conducted at a single community-based residency program, which may limit generalizability to larger academic centers or other specialties. However, the Poisson regression model adjusted to control the confounders. Third, while the use of the Kirkpatrick model provided a structured framework for evaluation, Level 4 outcomes focused on scholarly output rather than direct clinical or patient safety outcomes. Additionally, outcome measures relied on data from residency management software rather than independent validation, which may be susceptible to reporting or confirmation bias. While the EQuIPS curriculum was rolled out and implemented, some other interventions related to scholarly activity were also being implemented. Those interventions may have acted as confounding and enhanced the effect and association of Level 4 outcomes. Lastly, the curriculum's long-term impact on clinical practice and patient outcomes after residency remains unknown and warrants future longitudinal follow-up studies.

Overall, our study highlights the value of implementing a structured, outcomes-based QI curriculum in a community hospital setting. A successful QI curriculum in such settings must be concise, focused, and practical. It should include tools for assessing resident attainment of learning objectives, offer hands-on opportunities for residents to lead their own QI initiatives, and provide simple yet effective evaluation methods to gauge the curriculum's impact. Creating a curriculum aimed at introducing QI concepts to residents early on in their training and using objective scales to assess the outcomes can better prepare future physicians to lead improvement efforts. Furthermore, by addressing barriers and tailoring the curriculum to meet local needs, it will benefit the entire healthcare system. Future work should examine the long-term impact of QI training on clinical practice, patient safety, and healthcare outcomes, helping to ensure that residency programs adequately prepare residents to lead efforts in improving care quality.

Conclusion

The implementation of EQuIPS curriculum in our community-based family medicine residency program was associated with a statistically significant and educationally meaningful shift in how residents’ engagement in scholarly activities related to equity, quality improvement and patient safety as assessed by using the Kirkpatrick model. Other GME programs could potentially use this approach when considering scholarly activity and curricula related to EQuIPS-related topics.

Supplemental Material