How national healthcare change initiatives balance emergent and deliberate change: A principles-focused evaluation

Abstract

Principles-focused evaluation reflects on the change process itself through examination of its underlying principles. The Centre for Advancing Collaborative Healthcare & Education (CACHE) worked to build interprofessional education programs and tools that attended to the Team Primary Care (TPC) principles. Our internally directed principles-focused evaluation, presented here, asks how CACHE adhered to these principles in the programs and tools it delivered to the TPC project. The article’s main contribution is the creation of a new concept, organizational critically reflective practice, which describes an approach health leaders can use to mitigate the limitations of short-term initiatives while pursuing transformational change. We propose specific tools and steps that will help health leaders attempting to enact organizational critically reflective practice.

Introduction

Leaders across the Canadian healthcare system have identified primary care as both a system in crisis^1-4 and the backbone of equitable, universal, single-payer healthcare.^5-7 Inequity in primary care permeates the Canadian system⁸ and correlates with poorer health for persons from equity-deserving backgrounds including Indigenous people, recent immigrants, and low-income households.^8-10 The Team Primary Care (TPC) initiative was a pan-Canadian effort proposing the most likely solution to the crisis comes from interprofessional teamwork based on clear principles.¹¹ While the TPC principles evolved over the course of the initiative, ultimately, the final principles grounding the work were social accountability; truth and reconciliation; psychological health and safety; equity, diversity, inclusion, and accessibility; and interprofessionalism (see Figure 1).¹²

Figure 1.

Team Primary Care principles as of March, 2024.

Our Centre, the Centre for Advancing Collaborative Healthcare and Education (CACHE), a leading organization in the advancement of interprofessional education, practice, research, systems, and policy, was a core partner in TPC. The Centre has long embodied interprofessionalism and we continue to learn, practice, and integrate the other TPC principles. To model critical yet generative reflection, this article describes our organization’s own attempt to more fully realize all five TPC principles in our work.

Self-critique is a sensitive and challenging scholarly modality. Evaluating transformative initiatives in complex systems such as primary care is doubly challenging.^13,14 Our principles-focused evaluation represents an attempt to model reflexivity, to challenge ourselves, and, through this process, offer insights to health leaders collaborating in other transformational change projects. Principles-Focused Evaluation (PFE) is a useful approach to such work because PFE explores the developmental process of complex change initiatives.¹⁵ It is first and foremost a learning opportunity; one that elevates the importance of principles to the ongoing design and development of education. Rather than asking traditional outcomes-oriented evaluation questions, such as how an academic group like CACHE impacted primary care across an entire country over the course of single year, our principles-focused evaluation asked: to what extent did CACHE as an organization adhere to the full scope of the TPC principles in the programs and tools it delivered during the TPC project? Most importantly, what lessons could be gleaned from the CACHE experience so leaders and collaborating organizations in the next phases of the work can best attend to these principles in the future?

Methods

Design and recruitment

We followed an emergent design combining the precepts of constructivist grounded theory^16,17 and principles-focused evaluation.¹⁸ Using a constant comparative approach, we iteratively collected and analyzed data based on the following question: to what extent have the TPC principles been adhered to the programs and tools CACHE delivered for TPC?

Reflexivity

Principles-focused evaluation uses ideas from participant observation and developmental evaluation where the researcher both participates in and documents the ongoing work.^19,20 In this way, the analysis was shaped both by the perspectives of the CACHE team and the experiences of the evaluators involved in developing educational resources for interprofessional education in primary care.

Data collection and analysis

The data collected for this study included: 15 semi-structured interviews with leaders and educators from across CACHE and TPC; 3 focus groups with CACHE researchers and educators; 3 field observations from CACHE design meetings and implementation sessions; and 18 reflective memos.

Analysis of the data proceeded iteratively. We targeted sampling of key participants timed to milestones in the development and implementation process of CACHE’s educational intervention. Sampling of educators happened in two phases over the course of the project: once before development and implementation of new tools and teaching sessions over the course of the year and again after the implementation process. The coding and analysis of data followed constructivist grounded theory processes from the earliest interviews, and we refined the emergent categories throughout the analysis process until we agreed by consensus that sufficiency had been reached.^21,22

Results

The CACHE team worked to find a balance between achieving clear deliverables and pursuing transformational change in accordance with the TPC principles. Ultimately, supporting principles of interprofessionalism and psychological health and safety was so intrinsic to the organization that little adjustment was needed. But attending to principles of social accountability; truth and reconciliation; and equity, diversity, inclusion, and accessibility required developmental effort and attention.

Description of the core category, balancing deliberate and emergent change, is accompanied below by its subcategories (Table 1). Data from focus groups are depicted with the subscript_(FG#) and data from interviews with the subscript_(I#).

Balancing deliberate and emergent change

Our participants described juggling attempts to develop new tools and programs that adhered to the principles while also attempting to adapt existing programs to meet deadlines. Some participants spoke explicitly about this tension, calling it a “kind of dance between emergence vs having the corporate structure tell you”_(I16) what kind of work to do. In the case of emergence, participants focused on the complex nature of the planned change: “in enacting [the] principles, we don’t want to be tokenistic. We don’t want to appropriate. We want to be meaningful, we want to be thoughtful.”_(FG1) Rather than immediate implementation of existing educational interventions, participants acknowledged “there’s so much learning that needs to happen in order to do the work.”_(I1) Working from this approach, some expressed a need for both time to reflect and a network of resources: “to do that work justice, to not be not accidentally inappropriate or tick box [or] tokenistic, we need some help.”_(FG1) In contrast, clearly focused systemic efforts towards a unified goal, or for partners to simply be “informed that ‘those are the principles’,”_(FG1) may have “helped all teams be in better alignment from the start.”_(I5) Our participants attempted to build tools that and programs that reflected the principles despite early ambiguity about both the principles themselves and the resources available to support the work.

Recognizing time pressure

Tight timelines pressed the balance of the work towards directed implementing tools (such as needs assessment streaming) and programs (such as professional development sessions) before optimal space for deep reflection about the principles could be afforded. When our participants were considering whether to implement existing tools vs attempting a principles-focused needs assessment, consultation process, or fundamental re-thinking “there were a lot of things that we couldn’t do, [that] we had to scratch, because timeline-wise we couldn’t do it.”_(I2) Working on a large project with multiple partners led to compromises on implementation because “we would get a lot of requests and the deadline would be, like, tomorrow.”_(I1) With the recognition that “enabling emergence is… very, very hard when the deliverables are so tight,”_(I14) our participants leaned towards modifying existing tools rather than reimagining the tools to attend specifically to the TPC principles “because we [were] feeling that pressure of delivering and meeting all the milestones and the reporting deadlines.”_(I7)

Searching for clarity

When pressed for time, our participants searched for ways to incorporate principles-oriented material into their educational tool development in a modified way. A first priority became clarifying which principles to work towards, to “make sure that we were living the principles and that it wasn’t just implicit in our heads”_(I9) while acknowledging “our uncertainty as a team as to what were themes, what were principles, what was the difference? Who’s cross-cutting? Who isn’t? Even that was a bit messy.”_(FG1) Identifying, for example, what measures to take to include equity or reconciliation into educational tools, which groups held responsibility for such changes, and whether the guiding principles may change lent significant complexity both “because we don’t know what we don’t know”_(FG2) about how to best work towards the principles themselves and because “there was no concrete… documentation in terms of ‘these are the principles, this is what we want to focus on’.”_(I3) Our participants described pushing forward building new educational tools under conditions of uncertainty was “a bit chaotic,”_(I2) “felt like we were running on the spot,”_(FG1) and like “treading water”_(I7) while also acknowledging their trust in emergence: “we kept saying, as a team, that clarity will come. We’ve got to trust the process.”_(FG1)

Revising expectations

As the project progressed, our participants began to reimagine the meaning and legacy of the work. The complexity of collaborating under conditions of uncertainty leads to recognition that “we were going to have to pivot.”_(I2) Initial hopes for multi-partner education interventions shifted towards hopes for future collaboration: “people who were on board and got it, and whose strengths align with the strengths of this team, and the ethos of this project, they’ll continue the work.”_(I12) Here, “emergence was helpful because… what we thought was going to be needed wasn’t actually really needed.”_(I16) The initial framing of the project, as an act of principles-focused implementation, turned towards building a culture to support future change.

Deferring to future change

Recognizing the complexity of system change accompanied a shift in our participants away from immediate implementation and towards imagining a new future for interprofessional education in primary care: “We had a way of seeing and a way of doing and it was challenged. In the end, I think it moved. I actually think we moved. I don’t think we moved completely as far as ‘everything is said and done’, but I’m all about balance and context.”_(I8) Through involvement in TPC, “I think the three [ideas] that we’re paying more and more attention to as a Centre would be truth and reconciliation, equity and diversity, and social accountability.”_(I9) Participants spoke about “what TPC has brought us. It’s brought a lot of different organizations together”_(I11) around shared principles creating “a coalition of the willing,”_(I16) to reimagine interprofessional education in Canadian primary care in the future. As the formal project came to a close, participants spoke about new dedication to addressing complex primary care problems that simply could not be resolved in the initially planned timeframe.

Table 1.

Representative quotes depicting core category and subcategories.

Balancing deliberate and emergent change	I05: “I think it’s important to put on record that there’s friction, but the friction and conflict doesn’t necessarily have to be bad… In the end, I think it helps us all improve, and I think one of the things we need to focus on moving forward is finding the right balance about doing things right [versus] just doing something”
	I08: “That’s probably another [thing] to consider because there are financial implications to spending a lot of time planning something and having to totally redo something vs using existing things”
Recognizing time pressure	I12: “If people are feeling pressured or pressed to get all these things done, then the things that feel hard might get pushed aside, not necessarily because it’s not important but because it feels harder to get involved with or explore because you have all these other things that you need to get done”
	FG1: “I think one of our challenges is—as we are trying to kind of go forward and really meet these intense timelines—there’s a lot of this desire, from TPC, for all of these cross-cutting themes to really be multi-directional and to see stuff about EDIA or psychological health and wellness in our… Interprofessionalism [initiatives]”
Searching for clarity	FG1: “I was asked to do something quickly, but no one knew what I was being asked to do. So it was like, ‘do it fast, but no one knows what we’re doing’. It’s really hard to do the unknown. And so I think at the starting point everyone was trying to just figure it out… Maybe the lack of clarity of the expectations coming from TPC at the top from the start made that trickle-down effect to us really challenging…”
	I07: “How do you define principle? What does that mean in a 65-partner grant? So, I think there are some differences there. I think the different professions use those terms in slightly different ways. Researchers use them a little bit different than clinicians, different than educators. So, maybe going through those kinds of bumps with different players was also making it more complicated”
Revising expectations	I02: “We were going to have to pivot just because of the timeline…. I think we had our plan, and that had to change because the greater [project] team did not have a plan in place…And everything that we were talking about then was very different than what our original grant… there has been a lot of adapting”
	I07: “Way back in Jan we had hoped that we could do a session…and embed part of [our collaborators] modules with our modules. That was kind of our pie in the sky idea at the time. We also wanted to be very cautious and acknowledge that we don’t have anyone concretely on our team that has an indigenous background. So, not knowing what we don’t know is a risk… We had also wondered, what are ways that we can truly partner with our indigenous colleagues? What does that mean?...Those were questions that came up certainly early on, and have changed as the project has evolved and we’ve understood more what is possible and what isn’t”
Deferring to future change	I05: “I think my biggest concern is sustainability… Our team was like, okay, let’s offer some of our professional development. But once again, if that only happens over the year, the impact becomes very small again. We need to change our healthcare system for more than just a year… And how do those people working on the reserves who are caring for a million and a half patients who don’t have an interprofessional team, who are doing it all on their own, whose clients are coming in so far to get them, how do we make sure that those people get the support they need moving forward in a sustainable way, and we give people across this country access to healthcare? That’s what keeps me up”
	I16: “Recognizing that we weren’t going to change the world in 17 months. From the beginning it was: we will bring some people together, we’ll have more people understanding about team-based care and primary care… Are we going to change team-based care [in 17 months]? Absolutely not, but we’ll have increased… the coalition of the willing”

Discussion

We fused a grounded theory approach with principles-focused evaluation to explore how CACHE adhered to all five principles of TPC. Our findings suggest that, in national-scale healthcare change initiatives, achieving time-dependent deliverables and allowing for the time-independent emergence necessary for transformative change may inherently be in tension with one another. We propose that health leaders may learn from the CACHE experience and mitigate the impact of this tension through the practice of what we have called organizational critically reflective practice.

Principles under tension

Efforts at reforming modern healthcare practices suffer from a failure to consider the healthcare system as a complex adaptive system.^13,23 Healthcare systems can be inherently unpredictable, embedded within other systems, and behave based on internal but ever-evolving rules.²⁴ Despite this complexity, health leaders are still required to attempt to address inefficiencies, inadequacies, and inequities. In this vein, Team Primary Care was funded based on the premise that the Canadian primary care system is in crisis^1,6 and the solution is most likely to come from teamworking based on principles of social accountability; truth and reconciliation; psychological health and safety; equity, diversity, inclusion, and accessibility; and interprofessionalism.^5,7,11

A crisis, especially one where health inequities are deepened by the healthcare system itself,⁸ obliges reform with evidence of effectiveness. Demonstrating clear evidence of effectiveness in complex adaptive systems remains challenging,²⁵ but the findings of this study may account for some of these evidentiary challenges. The participants in this study were well-funded experts and national leaders in interprofessional education in healthcare. Nevertheless, even with this expertise, the expectation to demonstrate deliberate action towards a deliverable outcome in the short-term was in direct conflict with their recognition of the time-independent nature of transformational change. As suggested by the action scales model,²⁶ if evidence of effectiveness is required over the short-term then the findings of this study suggest it may be less likely that said change can be transformative. This inherent tension likely exists because assessing for deliberately sought short-term change does not adequately allow for emergence and poses intractable problems for the logic models and selection timing that characterize attempts to evaluate programs within complex adaptive systems.²⁵ Health leaders pursuing transformative change may be well served by seeking out resources that nudge collaborating organizations to co-create principles mitigating the tension between emergent and planned work earlier in the process.

Organizational critically reflective practice

Mitigating the impact of this inherent tension may be possible through the practice of what we will call organizational critically reflective practice. Reflexivity is the work of recognizing the limitations of our knowledge and critical reflection is the practice of challenging harmful processes and power dynamics.²⁷ Our findings support previous management scholarship suggesting that an organization undertaking well-intentioned reform without reflexivity runs the risk of reinforcing the very patterns of thought it seeks to transform.^28,29 Organizational critically reflective practice, therefore, describes a process that both precedes and accompanies decisions about principles and outcomes in reform initiatives grounded in social accountability. Before, during, and after reform initiatives we recommend collaborating organizations attempt to tilt the balance of their focus towards emergence by considering the following practices:

i. Take deliberate steps to understand how the organization interprets the prospective principles of the change project at the grassroots level early in the process.

ii. Reflexively question the organization’s current practices regarding the principles while deliberately seeking out voices within the organization that may be missing.

iii. Develop consensus around qualitative outcomes that reflect the principles and continuously reconsider potential harms of the reform during the process.

These suggestions are drawn from the literature on reflexivity and critically reflective practice in interprofessional education and research³⁰ and reimagine organizational fitness for change as an educational process.^14,31 The concept of organizational critically reflective practice relies on valuing trust, respect, humility, and accountability to encourage an inward curiosity about well-intentioned work in areas as sensitive and complex as social accountability; truth and reconciliation; and equity, diversity, inclusion, and accessibility. These practices need to occur at all levels of an organization from values and visions through to operational processes. Organizations new to critically reflective practice may benefit from adapting existing resources in addition to taking the steps described above. The 3Rs (representation; roles/relationships; and responsibilities)³² is one such resource drawn from literature on critically reflective practice²⁷ and adapted over the course of the TPC initiative³³ to help organizations continuously recalibrate their efforts. Given the inherent challenges of navigating complex power relationships in socially accountable reform projects, in the absence of critically reflective work, health leaders that rely primarily on short-term deliverables risk implementing reforms that recapitulate the very structures they hope to dismantle.

Conclusions

Health leaders designing reform initiatives should consider the inherent tension between deliverables and transformative change. The findings of this study suggest that deliberately sought short-term change efforts may catalyze future transformation but cannot be expected to deliver transformative change without allowing for emergence. Early attention to emergence of principles through organizational critically reflective practice may mitigate some of the impact of this inherent tension. We hope that organizations collaborating on large healthcare reform initiatives use the insights gleaned from this internal evaluation to focus earlier, more thoroughly, and more iteratively on the individual and organizational learning required to fully realize the principles in future initiatives.

Footnotes

Acknowledgements

The authors wish to thank the TPC Secretariat for their vision; the TPC evaluation team for their guidance; Dr. Michael Patton for his valuable insights; and the CACHE team for their bravery, brilliance, and hard work. The lead author is supported by an investigator award through the Department of Family & Community Medicine (University of Toronto) and a fellowship through AMS Healthcare.

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Team Primary Care (TPC) was supported by Employment and Social Development Canada through a grant to the Foundation for Advancing Family Medicine, co-led by the College of Family Physicians of Canada and the Canadian Health Workforce Network.

Ethical approval

The study was approved by the University of Alberta Research Ethics Board (Pro00128443).

ORCID iDs

Tavis Apramian

Kathryn Parker

Zeenat Ladak

References

Tara

Michael

Mylaine

, et al. Results from the OurCare survey: public perspectives on primary care in Canada. Ann Fam Med. 2023;21(Supplement 3):5232. doi:10.1370/afm.22.s1.5232

Canadian Medical Association . Improving Accountability in Health Care for Canadians. Ottawa, ON: Canadian Medical Association; 2023.

Ross

Green

. We Need Health System Solutions Now. Ottawa, ON: Canadian Medical Association and College of Family Physicians of Canada; 2024.

Martin

Pinto

. Fixing the Primary Care Crisis is Underway. Now We Need Action. Toronto, ON: Toronto Star; 2023.

College of Family Physicians of Canada . A Prescription for Primary Care: A Collaborative Approach to a Common Challenge. Mississauga, Ontario. 2023. https://www.cfpc.ca/CFPC/media/Resources/Health-Policy/2023-Rx-for-Primary-Care.pdf

Flood

Thomas

McGibbon

. Canada’s Primary Care Crisis: Federal Government Response. Los Angeles, CA: Sage Publications; 2023:327-332.

Green

. Priorities and opportunities for 2024. Can Fam Physician. 2024;70(1):72. doi:10.46747/cfp.700172

Eissa

Rowe

Pinto

, et al. Implementing high-quality primary care through a health equity lens. Ann Fam Med. 2022;20(2):164-169.

Bushnik

Tjepkema

Martel

. Socioeconomic disparities in life and health expectancy among the household population in Canada. Health Rep. 2020;31(1):3-14.

10.

Shah

Slater

Frymire

, et al. Use of the health care system by Ontario First Nations people with diabetes: a population-based study. Canadian Medical Association Open Access Journal. 2020;8(2):E313-E318.

11.

Bourgeault

Oandasan

McMurchy

Sunnar

. Team primary care: training for transformation in Canada. 23rd International Conference on Integrated Care Antwerp, Flanders, 22-24 May 2023. Int J Integrated Care.

12.

Bourgeault

Oandasan

Sunner

McMurchy

. Special Issue of Healthcare Management Forum - Editorial. Healthcare Management Forum. In press. 2024.

13.

Ratnapalan

Lang

. Health care organizations as complex adaptive systems. Health Care Manag. 2020;39(1):18-23.

14.

Dickson

. Health reform in Canada: enabling perspectives for health leadership. Healthc Manag Forum. 2016;29(2):53-58. doi:10.1177/0840470416628635

15.

Wolfe

Long

Brown

. Using a principles-focused evaluation approach to evaluate coalitions and collaboratives working toward equity and social justice. N Dir Eval. 2020;2020(165):45-65.

16.

Apramian

Cristancho

Watling

Lingard

. (Re)Grounding grounded theory: a close reading of theory in four schools. Qual Res. 2017;17(4):359-376.

17.

Charmaz

. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis. 2nd ed. Los Angeles, CA: Sage Publications; 2014.

18.

Patton

. Principles-focused Evaluation: The Guide. New York, NY: Guilford Publications; 2017.

19.

Parker

. A better hammer in a better toolbox: considerations for the future of programme evaluation. Med Educ. 2013;47(5):440-442.

20.

Patton

. State of the art and practice of developmental evaluation. Developmental Evaluation Exemplars. 2016:1-24.

21.

LaDonna

Artino

Jr Balmer

. Beyond the guise of saturation: Rigor and qualitative interview data. J Grad Med Educ. 2021;13(5):607-611. doi:10.4300/JGME-D-21-00752.1

22.

Nelson

. Using conceptual depth criteria: addressing the challenge of reaching saturation in qualitative research. Qual Res. 2017;17(5):554-570.

23.

Kuziemsky

. Decision-making in healthcare as a complex adaptive system. Healthc Manag Forum. 2016;29(1):4-7. doi:10.1177/0840470415614842

24.

Plsek

Greenhalgh

. Complexity science: the challenge of complexity in health care. BMJ. 2001;323(7313):625-628.

25.

Luna Pinzon

Stronks

Dijkstra

, et al. The ENCOMPASS framework: a practical guide for the evaluation of public health programmes in complex adaptive systems. Int J Behav Nutr Phys Activ. 2022;19(1):33. doi:10.1186/s12966-022-01267-3

26.

Nobles

Radley

Mytton

Whole Systems Obesity programme team . The action scales model: a conceptual tool to identify key points for action within complex adaptive systems. Perspect Public Health. 2022;142(6):328-337.

27.

Wright

Kuper

. The divergence and convergence of critical reflection and critical reflexivity: implications for health professions education. Acad Med. 2019;94(8):1122-1128.

28.

Cunliffe

. “On becoming a critically reflexive practitioner” redux: what does it mean to be reflexive? J Manag Educ. 2016;40(6):740-746.

29.

Raelin

. Toward an epistemology of practice. Acad Manag Learn Educ. 2007;6(4):495-519.

30.

Baker

Phelan

Snelgrove

Varpio

Maggi

. Recognizing and responding to ethically important moments in qualitative research. J Grad Med Educ. 2016;8(4):607-608.

31.

Steinke

Dastmalchian

Blyton

Hasselback

. Organizational change strategies within healthcare. Healthc Manag Forum. 2013;26(3):127-135. doi:10.1016/j.hcmf.2013.05.002

32.

Centre for Advancing Collaborative Healthcare & Education (CACHE) . Reflexive Questions for Educators & Facilitators. Toronto, ON: University of Toronto at University Health Network; 2024.

33.

Patton

State of the Art and Practice of Developmental Evaluation: Answers to Common and Recurring Questions. Developmental Evaluation Exemplars: Principles in Practice. In: Patton

McKegg

Wehipeihana

. Guilford Press, 2015.