Abstract
This article describes a rapid implementation research project with the Stockholm health care system to assist the system to respond to the COVID-19 pandemic. It uses this example to illustrate some ways in which implementation research and knowledge can contribute to improving service responses to the pandemic and its consequences as these evolve over the coming months. A sub-specialty of rapid implementation science is proposed to provide practical assistance and as one way to develop implementation research.
Plain language abstract
This article describes a rapid implementation research project with the Stockholm health care system to assist the system to respond to the COVID-19 pandemic. It uses this example to illustrate some ways in which implementation research and knowledge can contribute to improving service responses to the pandemic and its consequences as these evolve over the coming months. A sub-specialty of rapid implementation science is proposed to provide practical assistance and as one way to develop implementation research.
When a pandemic arrives, service delivery systems make many changes. Some are evidence based, many are not. Managers and front-line practitioners implement and modify the changes. Can implementation scientists help them make the changes more quickly and effectively? What can researchers offer in the next months? For many services, changes in service delivery are directed from above and the details of implementation are left to the service. Their aim is rapid implementation using fast and effective ways to make the changes. Research into implementation is low priority for managers and front-line practitioners. They do not usually seek help from researchers, for various reasons. In the case presented here, we, the researchers, had a long running partnership with the service delivery system. We were assertive in “selling” implementation science to management. Our experience was that we could use implementation methods and research to help our health system’s response to the pandemic. But also, that we found limitations to the science that we and others can now address.
The purpose of this article is to share our experience and encourage colleagues to contribute to improving the responses of our services as the pandemic and its consequences evolve over the coming months. A sub-specialty of rapid implementation science is proposed for providing practical assistance and developing implementation research. The article describes the emergency response implementation project that we undertook. We concentrate on the challenges we faced as implementation researchers and our resolutions. Findings from the first 5 months of the project are summarized, and the interim report will be available in September 2020 (Øvretveit et al., 2020).
Background
Stockholm healthcare
About 2.3 million people live in Stockholm. Most hospital and many community health services are publicly provided and owned by the Stockholm regional government. About 60% of primary care services are private and there are a few private hospitals. Many health-related services are provided or purchased by the 26 municipalities in the region, including many services for older people, such as private home care visiting services and residential care homes.
Stockholm is unique in that all the public non-acute health care services are owned by the region and managed within the “Stockholm healthcare” organization (Stockholms läns sjukvårdsområde [SLSO], 2020). This primary and community health care organization includes a diverse range of service delivery units and medical home care services, including 70 primary care centers, services for psychiatry, rehabilitation, addiction, and specialist “hospital at home” and palliative care. The public acute care hospitals are managed by the regional government, separately to the SLSO.
The implementation research project
Region Stockholm government activated a full emergency management system for the first time on February 29, 2020, in response to the rising number of our residents infected with SARS-CoV-2. The first COVID-19 death was reported on March 11, the same day that the World Health Organization (WHO) declared a global pandemic. Although we had little information about the best prevention and treatment of the disease, our contacts with Italian colleagues showed that we could be overwhelmed, not least because the Swedish government did not institute a “lock-down” as did other European countries (Our World in Data, 2020b). Yet COVID-19 mortality per capita each day between March and July was almost the same as the United Kingdom, which did have a strict “lock-down.” What explains this? How did our health services respond? What did we learn about implementation?
The author works 50% time as a research and development officer for the SLSO primary and community health care organization, and 50% time as a professor of implementation, improvement, and evaluation at the nearby Karolinska Institutet, a medical university; there are 500 m between the SLSO and the medical university offices. With a background in these fields, I saw a role for applied research and to help implement the changes being made. Although retrospective implementation studies are needed, I was not willing to stand on the side-lines as my clinical colleagues prepared for the expected increase in infections and illness. My plan for rapid implementation research was welcomed by the managing director of the SLSO primary and community service delivery organization, and he and I submitted an ethics application that was fast tracked and approved in 10 days. The research started in the last weeks of March and will continue until next year. This partnership and action research approach gave us access to information and interviews that would not have otherwise been available as this historic series of events unfolded.
Implementation research challenges and resolutions
Challenge 1: defining which research is useful, but feasible, given the resources
The first step was to define more precisely the scope and objectives following the “start with the end in mind” principle. The Karolinska medical management center approach to applied research views practical and scientific contribution as mutually reinforcing and of equal importance. Our methods are designed to deliver both, although there are sometimes challenges in balancing the different time perspectives. One objective of the COVID-19 rapid implementation response project is to contribute to implementation science. The other is to provide timely research-informed assistance and reports to the emergency and operational management teams of SLSO.
Our most valuable resources were our health system digital data bases and the national data bases, as well as our established 3-year working relationships with health system staff. Our other resources for the period described here from March to July 2020 were seven “volunteer” researchers. All had other full-time work and added this work and paused other tasks where possible. This team was a joint academic/health system team, so several of us, including myself, were employees of the health system. As principal investigator, one of my leadership challenges was to maintain the enthusiasm for the project, honor the high ambitions of the team, and recognize all the different possibilities for the study, but limit the data and analysis to what was feasible and useful. I did this by summarizing the options raised in the team discussions and then itemizing the hours and data we had available and the target dates for reports to management. How did we focus the research and decide which data to collect, given these resources, time frames, and assets?
Methods and concepts
Given the aims of providing implementation research-informed assistance to the management team and contributing to the science, we found two sets of ideas useful for focusing the research. The first was a version of the learning or improvement science testing iteration cycle.
Rapid learning experimental cycle
We found a generic rapid learning experimental cycle (RLEC) useful for our study, and our service providers found it useful to track and improve their changes. This generic method has five steps: (1) Define problem; (2) Decide data indicating problem solved; (3) Design and implement solution; (4) Review data and revise solution; and (5) Repeat as situation changes. We used this “3D-2R” model to guide our documentation and assessment of the adaptations made to the intervention and to the implementation actions of the different changes that staff made to their practices, service organization and facilities, and to support systems. The difference between this model and the more well-known Plan-Do-Study-Act (PDSA; see Leis & Shojania, 2017, for an updated narrative review) quality improvement cycle is the emphasis on the first two steps of defining the problem and deciding the data that would indicate it as solved.
Implementation science concepts
We found the simple distinction between intervention and implementation useful both to define the project and to continually remind ourselves where we could most add value with our knowledge about implementation. Our practitioner colleagues assumed “intervention” meant only a new treatment or diagnostic method. We found the term, “the new better way (perhaps),” useful for identifying and communicating the many types of changes that were being made, and quickly. These changes included new ways to practice telemedicine, infection control practices, physical changes to buildings and people flows, as well as changes to the content of IT systems. The word “perhaps” provided a reminder that some changes were not supported by strong evidence or were adapted to the service and population from proven versions tested elsewhere.
The terminology made it easier to separate actions or methods used to establish the “new better way” into everyday operations—the implementation actions. Methods used to help people take up the new better way in everyday work typically included training, practicing the training (e.g., for the many steps to don and doff protective personal equipment [PPE]), feedback on performance, and highlighting incentives to perform the new better way.
These concepts also highlighted that many different changes were being made at the same time. Should we choose a sample of primary care centers and document and evaluate how they implemented changes? Or should we document and assess how all the service within the SLSO organization implemented different changes? Which of these or other changes should we concentrate on, given our limited resources and the objectives of the research?
Challenge 2: describing and reporting implementation
We chose to focus on the implementation of the emergency management system (the intervention): what was done to implement the new emergency management groups and establish their working with the routine operations management system (implementation actions). The idea was that, if emergency management needed to be re-established later, we would have documented how it was implemented for the first time, and what worked well and less well. This could help in August to prepare for the winter. In addition, there is limited research into implementing new management structures and systems, and even less knowledge about implementing emergency management systems.
Implementation science has found that contextual influences help and hinder implementation of the intervention and its performance (McCormack et al., 2002; Øvretveit et al., 2011). Also, research suggests that some context influences are specific to certain interventions: for example, for a computer application intervention, the type of hardware and computer system and privacy rules are important contexts helping or hindering the implementation and operation of the computer application intervention. Other contextual influences are important for implementing practices to reduce the number of falls experienced by older people in a care home.
The focus on the emergency management system as the intervention helped us to theorize which contextual influences would help and hinder implementing and operating the system. The emergency response highlighted that the context mostly influences the implementation actions. Then, over the longer term, context has a greater effect on sustainment or continual operation of the intervention. The Consolidated Framework for Implementation Research website (https://cfirguide.org/) provided ideas and guidance that help to specify the inner and outer contextual influences that we needed to consider.
What then would be the outcomes of a successful or unsuccessful implementation of an emergency management system? We found an implementation outcomes model useful to differentiate different outcomes of the emergency management system for the primary and community health services (Proctor et al., 2011). The immediate outcomes of implementation are whether and how quickly changes to work practices, organization, and support systems are accomplished and sustained. We found useful reminders of the many different implementation actions that could be used from the compilation of 73 implementation strategies (Powell et al., 2015) and the list of 93 behavior changes (Michie et al., 2013).
These implementation science concepts helped us to outline a logic model of context, implementation, and outcomes that helped us to decide which data to collect about the emergency management system for the SLSO primary and community health services. We then developed a data collection strategy that started on March 27, 2020, with our first weekly survey of all our service delivery managers (110 units).
Illustrations of how the implementation science concepts helped us structure questions are as follows:
Context, which includes the regional government requirements of the SLSO organization to establish emergency management and to coordinate all other community services, and the hindering context of inadequate and timely data about service utilization and prognosis of likely demand on services, as well as changes to patient portals to enable easier use of smart phones for telemedicine;
Implementation actions to establish emergency management, which included formal instructions to service delivery unit managers to make certain changes, formulating guidance to service delivery unit managers and clinical staff (e.g., when to refer patients to acute hospital), establishing a special purchasing and supply center with volunteers to obtain and distribute PPE;
Implementation outcomes of the implementation of emergency management included whether managers reported receiving clear instructions and guidance as well as sufficient PPE, as evidenced in the weekly surveys to the managers.
Challenge 3: data for management decisions
Which data to collect
Our resources, the project focus, and the concepts described above helped us to concentrate on using already collected data and to ensure low burden data collection from the weekly survey and interviews by avoiding collecting data not essential for achieving the project objectives. One set of data was the weekly survey reports from heads of service delivery units. This included what helped and hindered implementing the changes they needed to make, as well as questions about implementing actions to maintain staff health. We used the internal secure health system intranet to ask questions about context and outcomes of the changes, using nine multiple-choice questions with free text at the end of each and taking 15 min to complete. Response rates varied from 49% to 29% for 14 weeks analyzed so far. Interviews were carried out with all members of the emergency management team and included questions about preparations for the winter and how to establish emergency management and systems more quickly and effectively in the future. Other data sources were documents recording the twice daily emergency management team meetings and the one daily operations management team meeting, as well as emergency plans and other documents.
Many implementation science projects collect data to quantify implementation outcomes. In this rapid implementation study, we needed to collect data for this purpose, but also to provide reports to the management team to enable them to track the care provided and predict demand on the units. We found the national data about these subjects was not sufficiently up to date and was inconsistent with our local data. We established a system for combining various information sources to provide frequent reports. These data included hospital admissions sourced from hospital data systems, intensive care unit occupancy, and infection rates in older peoples’ homes. To estimate predicted demand, we worked with one local hospital center that was using the Penn University open-source modeling system and were able to compare the projections from this model over time with actual data from all the hospitals that we worked with and adjust the model (Weissman et al., 2020).
We found limitations to implementation science in methods available researchers to use in partnership with health care services for rapid implementation research of the type we sought to carry out. Drawing on quality improvement project work, we found our version of iterative testing using the RLEC was useful. We also discovered the value of a project team that combined university researchers and health care system staff with a principal investigator employed 50% time with both organizations. One area to develop implementation science is through reporting and assessing different rapid implementation research methods and structures: others may have responded like we did and developed other methods and structures and we would hope this and other implementation science journals could report these.
Implementation outcomes: infection and mortality?
The focus of this article was on implementation of emergency management and our research continues. One question we are investigating is the possible impact of effective emergency management on COVID-19 mortality rates. As with most implementation research, there are attribution challenges in estimating how much implementation actions influenced implementation outcomes and then later health outcomes.
The article noted earlier that Sweden was unusual in not instituting a “lock down,” as shown in the COVID-19 Government response Stringency Index (Our World in Data, 2020b). Did Sweden’s “life as normal, with precautions” result in high infection and excess mortality compared to other European countries? Evidence shows COVID-19 deaths per million as significantly higher than other Nordic countries, but consistently equal to the United Kingdom from March to July 2020 (Our World in Data, 2020a). At present, the explanations of why COVID-19 deaths were not considerably higher in Sweden are speculative. It is possible infection rates were comparable, but an effective primary and community health care response with e-health and other support for self-care at home with monitoring and fast transfer to hospital when necessary may have reduced further infection and mortality.
One hypothesis we are investigating is whether what appears to be an effective implementation of emergency management in primary and community care may have prevented higher mortality. There is evidence that it did prevent our hospitals from being overwhelmed. One finding from our study was to highlight again the importance of a systems understanding: what happens in hospitals depends on what happens in primary and community health services. This was starkly demonstrated by the high infection and death rates in long-term care homes. There are simple evidence-based methods to prevent this (CDC, 2020). The failure is in implementation. The ethical and economic imperatives are for a “zero infection in long-term care homes” program.
Conclusion
Implementation research traditionally studied how empirically tested changes to practice, organization, and policy have been taken up in different health and welfare services. More recently, this research field has studied how changes proven elsewhere have been adapted for different situations. The COVID-19 pandemic raised the question of whether knowledge within implementation science could help with implementing, planning, or evaluating the many changes made during 2020 in response to the pandemic. Our rapid implementation research with region Stockholm health care system revealed useful concepts and methods from implementation science for designing a study and assisting the response. It also showed limitations in the science for understanding and assisting an emergency response and in the methods within implementation science for rapid research into proven and unproven changes in a rapidly changing situation. One of the aims of this article is to invite discussion in different forums about how implementation science can contribute new and useful knowledge and practice for emergency responses, and especially for the evolving crises that our services are experiencing in the next months. One view is that we are best equipped to carry out systematic research, using existing methods and concepts which may mean retrospective research. Another view is that rapid implementation research is needed with an implement-ology of emergency response and a sub-specialty of rapid implementation research. Both may be required.
Footnotes
Author contributions
The author, John Øvretveit, is the named principal investigator in the ethics application and acknowledges the contributions of the following colleagues to the project: Mats Brommels, associate Karolinska Institute, MMC, Vibeke Sparring, senior administrator, research and development, SLSO, Clara Hellner, professor Karolinska Institute and director of research and development, SLSO, Mikael Ohrling, director SLSO, Karin Solberg Carlsson, Karolinska Institute, MMC, Hakan Uvhagen, development officer SLSO, and editors Cara Lewis and Sonja Schoenwald.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The author and some research team members were employed by the Stockholm Region Government and by the Karolinska Institutet medical university and conducted the research in addition to their other funded projects and work assignments. One research member was retired and not employed by either organization. Because the author and some of the researchers were dually employed by the university and the Stockholm Region Government, there is a potential conflict of interest and potential for bias in their investigation of how their employers implemented emergency management. This commentary offers the author’s perspectives on the project and process but does not present findings, and procedures to manage potential bias will be described in a forthcoming research report.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.