Saving 20,000 Days and Beyond using breakthrough improvements: lessons from an adaption of the Breakthrough Series Collaborative

Abstract

Objectives

To report on selected findings from an evaluation of two consecutive quality improvement campaigns that adapted the Breakthrough Series Collaborative model to the broad topic of reducing demand for hospital care and reflect on lessons learned from their adaption of the model for subsequent collaborative improvement efforts.

Methods

We conducted a series of semi-structured interviews with Campaign sponsors and Collaborative team leaders as part of the broader realist evaluation of the two Campaigns. In addition, follow-up semi-structured interviews with Campaign sponsors and implementers were undertaken three years after the evaluation concluded (ex post evaluation interviews) to understand which adaptions to the Breakthrough Series Collaborative model had been the most influential.

Results

The interviews explored two features that differentiated the Campaigns from other Breakthrough Series Collaboratives. Firstly, the Campaigns enabled a diverse range of improvement solutions to be tested which had implications for the collaborative nature of the learning collaboratives. Secondly, two sequential Campaigns were implemented that incorporated common elements and provided the opportunity for the transfer of knowledge from one Campaign to the next.

Conclusions

Given widespread pressure to transform health care into a learning system, this paper provides a practical example of using cumulative insights to encourage the sustainability of collaborative improvement efforts. These insights centre on the gains from spreading improvement methodology throughout the organization and learning how to select and support successful collaborative teams.

Keywords

breakthrough series learning collaboratives learning system quality improvement

Introduction

The Breakthrough Series (BTS) framework is a change model that builds on collaborative learning to achieve quality improvement in health care. Developed by the Institute for Healthcare Improvement (IHI) in the USA,^1–3 its central tenet is the idea that small-scale changes have the potential to accumulate into large effects. A key element of the framework is the Model for Improvement,⁴ which incorporates four critical components of successful process improvement: aims that are specific and quantifiable, appropriate measures that can be tracked over time, key changes that will bring about the sought after improvement, and a series of testing cycles (known as Plan Do Study Act or PDSA cycles) during which the key changes are refined and applied by teams to their organizations.^1,4

The BTS framework has been used widely to support and accelerate health care improvement. In this paper, we present the experience of Counties Manukau District Health Board (CMDHB), which is one of 20 District Health Boards in New Zealand that are responsible for planning and funding health services for their populations. In 2011, CMDHB launched the first of two consecutive quality improvement campaigns primarily to address a projected increase in demand for hospital bed capacity⁵ while also striving to ‘give back’ healthy and well days to the local community. The first 20,000 Days Campaign (October 2011–July 2013) sought to reduce hospital demand, equating to 20,000 bed days by returning 20,000 well and healthy days to the community,⁶ followed by the subsequent Beyond 20,000 Days Campaign (July 2013–June 2014), which aimed to continue returning healthy and well days to the population of Counties Manukau.⁷

The Campaigns were subject to an independent evaluation, which focused, largely, on how the Campaigns achieved improvements and to provide formative lessons for future campaigns.^8,9 In particular, evaluative attention focused on the social processes and mechanisms that enabled (or hindered) the work of a wide diversity of collaborative teams, and it concluded that each Campaign was likely to be moving the system towards better management of demand. However, the scale at which this was happening was likely to be continuously shifting as the work of the most successful collaborative teams spread to more sites.

In this paper, we build on the findings of the independent evaluation of the Campaigns, reflecting on their dynamic and sequential nature, along with the distinctive application of the BTS model, to elaborate on the lessons learned from their adaption of the model for subsequent collaborative improvement efforts.

Methods

Study context

The 20,000 Days and the Beyond 20,000 Days Campaigns have been described in detail elsewhere.^6,7 In brief, collaboratives that adopt the BTS framework typically have a single clinical focus in an area where evidence exists regarding best practice but is not widely implemented.¹⁰ In contrast, teams in the CMDHB Campaigns were tasked with generating improvement ideas that linked to the overarching goal of reducing hospital demand. The two Campaigns involved up to 29 Collaboratives comprised of 8–10 members, each testing a diversity of quality improvement solutions that aimed to: (i) reduce hospital admissions and length of stay; (ii) increase patient access to community support; and (iii) reduce harm to patients and subsequent readmissions. Each Collaborative selected a specific measurable aim to be achieved during the Campaign they were part of. In total, 24 teams maintained momentum until the end of the Campaigns (10 teams in the 20,000 Days Campaign and 14 teams in the Beyond 20,000 Days Campaign).

Table 1.

Lessons learned from the 20,000 Days Campaign that informed the implementation of the Beyond 20,000 Days Campaign.

Approach

A realist evaluative approach was adopted to understand more about what worked in the Campaigns and in what contexts.¹¹ It used a mixed-methods design with three phases of data collection undertaken between March 2013 and November 2014: (1) a review of Campaign planning records from the first Campaign and semi-structured interviews with key informants to develop an initial logic model of the rationale and expectations underpinning the initiative; (2) semi-structured interviews with Campaign sponsors and team leaders, a questionnaire emailed to all participants, and secondary analysis of teams’ dashboards of quantitative outcomes and measures to assess the first Campaign; and (3) semi-structured interviews with team leaders, in-depth case studies of four teams, and a secondary analysis of the overall system-level dashboard to evaluate the second Campaign. Analyses involved an iterative process of testing against the initial logic model to progressively develop a more refined theory about how the Campaigns achieved their results; the details of the evaluation and key findings are reported elsewhere.^8,9,12

Following the realist emphasis on understanding how and why people respond to interventions,¹¹ we were further interested in how the ‘practical wisdom’ derived from operating a diversity of collaborative teams was applied in subsequent Campaigns. The term practical wisdom refers to the dynamic adaptation shown by those involved in the design and implementation of the Campaigns.¹³ To build this understanding, we drew on, first, semi-structured interviews with four Campaign sponsors in 2013 and 2014 explaining how they adapted their processes between the Campaigns. These interviews were initially used to develop a logic model of how the Campaigns were expected to achieve their effects for the main evaluation. We revisited these and explored commentary on how processes were adapted over time. Second, we used semi-structured interviews with five Collaborative team leaders in 2013 and 2014 who participated in both Campaigns. Again, these interviews were used as case study material to inform the main evaluation. We revisited these and investigated reflections of what was different between the Campaigns. Third, we carried out follow-up semi-structured interviews with five Campaign sponsors and implementers in 2017. These ex-post evaluation interviews occurred three years after the main evaluative reports. The questions in these interviews were shaped by our in-depth exploration of the adaptions that had occurred between the Campaigns, drawn from the above-mentioned interviews from 2013 and 2014. Specifically, they explored the significant features of the 20,000 Days and Beyond 20,000 Days Campaigns that distinguished them from other BTS Collaboratives, and how subsequent CMDHB initiatives^14,15 were influenced by what was learnt from these first two Campaigns.

All interviews lasted between 30 and 45 minutes and were held over the phone or in person. All interview participants provided informed consent either verbally or in writing.

Results

Figure 1 describes the key findings from the main evaluation. The findings confirm that the Campaigns benefited from a culture responsive to change, visible support from senior management and team leaders who motivated, rather than just managed their collaborative teams. Constraints centred on a lack of consensus about what changes would be most likely to reduce hospital demand, and confidence in the measures used by teams to track change.¹²

Figure 1.

Summary of how the Collaborative teams worked to achieve the Campaigns' outcomes.

As those implementing the Campaigns gained an understanding of what characterized successful and unsuccessful teams, future Campaign activity was adapted in light of these findings. In what follows, we report on two features that dominated the adaptive thinking of those implementing the Campaigns. These features were initially flagged in the main evaluation and selected for in-depth follow-up in the ex post interviews: (1) The implications of testing a diverse range of improvement solutions rather than a single clinical best practice or bundle of best practices; and (2) the ways in which implementation insights transferred from one Campaign to the next.

A diversity of improvement solutions

The CMDHB Campaigns encouraged participants to come up with their own project ideas for redesigning care processes that would reduce hospital demand and keep people well in the community. This resulted in a plethora of different ‘solutions’ that varied not only in the topic area chosen for the intervention but also in the development of process and outcome indicators required to measure progress:

… these were very different from standard collaboratives, which … have a single proven methodology and you’re spreading it. These were, in a way, a different beast. They were an innovation collaborative. We were testing many flowers and identifying what worked. (Campaign sponsor, 2017 ex post interview)

One consequence of teams pursuing such different improvement efforts was the lack of opportunities for benchmarking among teams. While teams were all trying to manage demand, the intermediate measures developed to track progress of, for example, early stroke discharge, were highly distinctive to this area and could not be compared to measures developed by other teams, such as those looking to reconcile medicines management. Instead, interview participants reflected that the collaborative group effort became focussed on the shared experience of learning and applying improvement methods, rather than the exchange of ideas and data on how to implement the same best practice. Not only did this alter the nature of the collaborative activity experienced during these learning collaboratives, with teams focussed primarily on the methods of achieving sustainable improvements in practice. It also prompted changes to team members’ daily work. The nature of this change is encapsulated well in the following quote:

The tools we learned, and it is hard to reflect on now as they have started to be part of the culture, this is how we test using the PDSA cycles; this is how we measure, these are now part of the air that we breathe. (20,000 Days Collaborative team leader)

Testing multiple quality improvement ideas rather than a small number of predefined best practices in the CMDHB Campaigns had advantages and disadvantages. In particular, while the problem of acute demand for hospital care and potential areas for innovation and improvement were canvassed at the outset, responsibility for developing specific aims, changes and measures of expected outcomes lay with the collaborative teams themselves:

The challenge which was so different … is this idea that we didn’t have any solutions like the traditional IHI [Breakthrough Series Collaboratives]

… We were trying to come up with the solutions as part of the work … (Campaign sponsor, 2017 ex post interview)

One implication of individual teams being relied upon to undertake much of this design work was that certain conditions needed to be met to support the application of the Model for Improvement. These included the need to release a high number of team members from each team to attend learning sessions to learn about PDSA testing cycles, as well as ensuring adequate infrastructure to allow teams to begin regular cycles of PDSAs early in their change programmes. In the first Campaign, a small number of collaborative teams that had already been in existence for some time were rolled into the work of the Campaign. However, these teams had formed a degree of belief that their idea for change was ready to spread, so rather than spend time testing the change idea, they wanted to move quickly to implementing change. By the second Campaign, Campaign sponsors had become more adept at recognizing the different subcultures that were more conducive to testing change.

Two sequential Campaigns: Learning what works

A second distinctive feature of the CMDHB Campaigns was having two sequential Campaigns with elements that flowed from one to another. This included a shared commitment to managing hospital demand, and a reliance on the same organizational support and branding. The openness to ensuring key lessons were transferred from one Campaign to another is captured by the following reflection:

… what we were doing at a higher level in the Campaign was in effect doing a PDSA cycle ourselves. We weren’t sure how this was all going to pan out. We definitely, in the true spirit of improvement, reflected a lot on what had worked and what hadn’t worked in the first Campaign to help inform what we then did in the follow up one. (Campaign sponsor, 2017 ex post interview)

Lessons included learning from teams that failed in the first Campaign to enable more comprehensive selection processes to be implemented in the second Campaign. Given the Campaigns were operating in a constrained financial environment where there were limited funds to do anything differently, there was the potential to gather a variable range of improvement suggestions under the broad topic of reducing hospital demand. The ‘self-selection’ of both teams and interventions in the first Campaign included projects that were already underway. More active selection was implemented in the second Campaign as teams presented their change ideas that were then judged on the basis of their ‘fit’ with the Campaign’s objective. This selection process aimed to balance the energy of those proposing improvement ideas with a robust mechanism for assessing which proposals were most likely to result in improved design.

Lessons from teams that succeeded in the first Campaign included the need to support even more robust measurement practices in the second Campaign to ensure the sustainability of improvement efforts in terms of having an intervention picked up and funded as mainstream business. Successful collaborative teams were able to access time-limited funding set up for their improvement ideas. To secure permanent funding, business case requirements in the second Campaign increased efforts by teams to quantitatively link their improvement to the larger Campaign goal of giving back healthy and well days to the local community and thereby demonstrate potential costs and savings. Table 1 provides detail on lessons learned according to different aspects of the Campaigns.

Ex post interviews highlighted several additional emphases for subsequent CMDHB improvement efforts that built on the earlier lessons learned. These included incorporating co-design with patients as a fundamental component of any initiative that seeks to engage effectively with patients and their families, understand what is important to them and design improved services that meet their needs:

The routine use of co-design with patients and their families, we’ve learnt that that should just be a standard part of the work now. (Campaign implementer, 2017 ex post interview)

The need to give more attention to robust collaborative team measurement was also a recurring theme, with participants highlighting the need for better problem definition and a more disciplined use of the methodology in order to cogently link data and aims:

What we learnt was that we tried to get more teams to have a more specific understanding of what their problem was and clearer baseline data. (Campaign implementer, 2017 ex post interview).

Finally, and more generally, CMDHB’s approach of adapting and learning what works across successive waves of Campaign activity was identified as having built a confidence among local health professionals in tailoring and applying the tools of improvement science to multiple contexts, thereby promoting the sustainability of this way of working across the sector. As one participant commented:

We absolutely, over the whole series of collaboratives, were aiming to build a network, build a movement, build a capacity and capability … (Campaign sponsor, 2017 ex post interview).

Discussion

It is not uncommon to find that successful health care quality improvement initiatives fail to transfer to different settings due to local contextual factors.¹³ Moreover, many methodologies do not facilitate the exploration of the contexts underlying how and why quality improvements fail or succeed in different settings. The Breakthrough Series Collaborative has been widely used as a vehicle for improving the quality of health care, using collaborative learning to achieve breakthrough improvements in the quality and safety of health services. Our results highlight what has worked and not worked when the BTS model was applied to the broad topic of managing hospital demand within a New Zealand District Health Board.

Our realist evaluative findings confirmed much that was shown in other research regarding the importance of having the right organizational context for improvement, voluntary participation and the ability to work as a team.^16–19 However, what proved of more interest and prompted a further investigation was an examination of the ways in which the evaluation findings influenced subsequent quality improvement efforts at CMDHB. These insights focus on the practical wisdom that was deployed in the design and running of subsequent Campaigns.¹³ This practical wisdom involved the dynamic adaption of the BTS model in two key aspects.

First, while the BTS model has been adopted on a large scale by various organizations,^20,21 it is relatively unusual to incorporate a diversity of improvement solutions within the scope of a single initiative. In single topic campaigns, information sharing among teams across institutions and within the same institution is expected to drive improvement through a cooperative group effort, or in some circumstances, competition may be the main motivator for change.²² However, given the diversity of improvement solutions being tested in the CMDHB Campaigns, there was little opportunity for benchmarking between teams. As Øvretveit et al.²³ explain, when a broad topic is selected for a quality improvement collaborative, each team is likely to focus on different change ideas and there may not be ‘clear or directly comparable examples of best practice, change concepts, or good research evidence’ (p. 347). What did serve to unify the teams in the CMDHB Campaigns, however, and provide essential peer stimulus was the shared experience of applying the BTS methodology to the broad topic of reducing demand for hospital care. This had benefits for spreading knowledge of quality improvement methodology throughout the organization, creating a network of health professionals with the methodological expertise to bring about sustainable improvements in their practice.

Commentators on quality collaboratives have pointed out that a common set of measures is useful for monitoring and comparing improvement.²³ In traditional Breakthrough Series Collaboratives, the specific content for the collaborative is developed by expert faculty and presented to participants at the start of the initiative as part of a predefined change package that includes collective wisdom around measurement.¹ However, given there was more uncertainty in the CMDHB Campaigns regarding what changes would have the most impact on managing hospital demand than typically experienced in pre-defined single topic campaigns, so the complexity of measurement tasks undertaken by collaborative teams was increased. Adequate time and support are therefore needed to develop appropriate measures to assess the effectiveness of quality improvement collaboratives in such circumstances.

Second, in transitioning between the two Campaigns, there was a concerted attempt to carry forward what had been learned from the first Campaign. This involved building on successes in the 20,000 Days Campaign, as well as learning from failures to ensure that the Beyond 20,000 Days Campaign had the ability to select and support successful collaborative teams in their work to produce sustainable quality improvements in practice. Further, examination of how key learnings continue to influence and shape the design of more recent Campaigns, reinforced the cumulative role of CMDHB’s initiatives for growing new models of care and spreading knowledge of improvement methodology, thereby encouraging the sustainability of collaborative improvement efforts across the sector.

Recognizing that health care systems are complex adaptive systems that alter according to their own localized rhythm, the Campaigns avoided mandating solutions from the top. In this aspect, they match advice to nudge health care systems towards change.²⁴ Those leading the Campaigns harnessed the creativity of those with an idea for improvement, but also needed to marshal the resulting diversity of ideas into a collective effort that would have a recognizable impact on the Campaigns’ goals. Caution has been raised that allowing a thousand flowers of quality interventions to bloom is not always efficient.²⁵ However, in this situation any inefficiency from teams that failed was converted into learning for future campaigns.

Footnotes

Acknowledgements

We wish to offer a very appreciative thank you to those involved in both Campaigns who shared their insights and experiences. This paper draws from the findings of two evaluation reports and follow-up that involved a wider team comprising David Mason, Janet McDonald, David Grayson and David Codyre as well as research support provided by Madeline Fuller. We thank them for their insightful contributions.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: DD, JG and LV were employed by the organisation delivering the Campaigns. DD and JG had direct involvement in delivering the Campaigns.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Counties Manukau District Health Board, which commissioned an independent evaluation of the Campaigns. Approval was sought for the publication of these results as part of the contract for services.

ORCID iD

Lesley Middleton

References

Institute for Healthcare Improvement. The Breakthrough Series: IHI’s collaborative model for achieving breakthrough improvement. Cambridge, MA: Institute for Healthcare Improvement, 2003.

Kilo

CM.

A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement’s Breakthrough Series. Qual Manag Health Care 1998; 6: 1–14.

Kilo

CM.

Improving care through collaboration. Pediatrics 1999; 103: 384–393.

Langley

Moen

Nolan

, et al. The improvement guide: a practical approach to enhancing organizational performance. 2nd ed. San Francisco, CA: Jossey-Bass Publishers, 2009.

Counties Manukau District Health Board. CMDHB bed model – 2009/10 base year (01/09/2011). Auckland: CMDHB, 2011.

Awatea

. 20,000 Days Campaign: from small revolutions to big change. Auckland: Counties Manukau Health, 2013.

Awatea

. Beyond 20,000 Days campaign. Auckland: Counties Manukau Health, 2014.

Middleton

Mason

Villa

, et al. Evaluation of the 20,000 Days Campaign. Wellington: Health Services Research Centre, Victoria University of Wellington, 2014.

Middleton

Villa

Mason

, et al. 20,000 Days and Beyond: Evaluation of CMDHB’s quality improvement campaigns. Wellington: Health Services Research Centre, Victoria University of Wellington, 2015.

10.

de Silva

Improvement collaboratives in health care: evidence scan no. 21. London: The Health Foundation, 2014.

11.

Pawson

Tilley

Realistic evaluation. London: Sage, 1997.

12.

Middleton

Dowdle

Villa

, et al. Saving 20 000 Days and Beyond: a realist evaluation of two quality improvement campaigns to manage hospital demand in a New Zealand District Health Board. BMJ Open Qual 2019; 8.

13.

Dixon-Woods

The problem of context in quality improvement. London: The Health Foundation, 2014.

14.

Awatea

. Manaaki Hauora – supporting wellness campaign. Auckland: Counties Manukau Health, 2016.

15.

Awatea

. Safety in Practice, http://koawatea.co.nz/project/ppc-quality-improvement-resources/ (2015, accessed 23 April 2019).

16.

Kaplan

Brady

Dritz

, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q 2010; 88: 500–559.

17.

Wilson

Berwick

Cleary

PD.

What do collaborative improvement projects do? Experience from seven countries. Jt Comm J Qual Saf 2003; 29: 85–93.

18.

Dixon-Woods

McNicol

Martin

Ten challenges in improving quality in healthcare: lessons from the Health Foundation’s programme evaluations and relevant literature. BMJ Qual Saf 2012; 21: 876–884.

19.

Schouten

Grol

Hulscher

Factors influencing success in quality-improvement collaboratives: development and psychometric testing of an instrument. Implement Sci 2010; 5: 84.

20.

Berwick

Calkins

McCannon

, et al. The 100,000 lives campaign: setting a goal and a deadline for improving health care quality. JAMA 2006; 295: 324–327.

21.

The Health Foundation. Learning report: Safer Patients Initiative. Lessons from the first major improvement programme addressing patient safety in the UK. London: The Health Foundation, 2011.

22.

Dainty

Scales

Sinuff

, et al. Competition in collaborative clothing: a qualitative case study of influences on collaborative quality improvement in the ICU. BMJ Qual Saf 2013; 22: 317–323

23.

Øvretveit

Bate

Cleary

, et al. Quality collaboratives: lessons from research. Qual Saf Health Care 2002; 11: 345–351.

24.

Braithwaite

Changing how we think about healthcare improvement. BMJ 2018; 361: k2014.

25.

Dixon-Woods

Martin

GP.

Does quality improvement improve quality?

Future Hosp J 2016; 3: 191–194.