How public hospitals respond to and use a national serious and sentinel events report: A qualitative study in New Zealand

Abstract

How and if public hospital leaders in a national health system use an annual Serious and Sentinel Events (SSE) report, an aim of which is to stimulate improvements in health care quality and patient safety, is an important question that is under-researched. This exploratory qualitative study in New Zealand using semi-structured interviews was undertaken in response. Interviewees included 29 representatives in patient safety leadership roles from 20 hospital districts, each of whom were recommended by their Chief Executives. Four themes describing factors contributing to the use of the SSE report were identified: response to the report itself; perceived use of and value of the report as a quality improvement tool; collaboration amongst hospitals around the findings; and, the priority given to improving quality within respondents' healthcare organisations. This article provides examples of these themes and how they relate to the use of the SSE report as a quality improvement tool. The article concludes that an annual SSE report has the potential to be a very useful tool for health care leaders in addressing SSEs. However, it also suggests that the report is underutilised and consequently some of this potential is lost. This may be explained by hospital capacity to absorb information from, and respond to, the SSE report.

Keywords

adverse events New Zealand public hospitals patient safety quality serious and sentinel events

Introduction

Annually, over 200,000 people enter New Zealand (NZ) public hospitals in the hope they will receive life-improving treatments.¹ For the most part, timely and appropriate care is given, but around 12% of patients can expect to be harmed in the process of care sometimes resulting in serious disability or even death.² New Zealand is not alone in this. The 1995 Quality of Australia Health Care Study found that 16.6% of admissions resulted in preventable adverse events.³ Adverse events in British hospitals have shown that 10.8% of patients experience an adverse event when receiving medical treatment.⁴

Since 2008, NZ has produced an annual Serious and Sentinel Event (SSE) report. While the collating and dissemination by a government-funded agency of SSEs in a national report is unusual internationally, such reporting to a central agency is routine and via a range of mechanisms in other high-income countries.⁵ For example, in Britain, serious events are reported to the National Reporting and Learning System (NRLS) an aim of which is to facilitate analysis of risk, to drive learning and improve patient safety. Since 2010, it has been mandatory to report all serious patient safety incidents which come via the NRLS to the Care Quality Commission (CQC) as part of the CQC registration process.⁶ Australia has its Australian Incident Management System, while in the USA the Joint Commission on Accreditation of Healthcare Organisations is the key agency involved in incident reporting.^7,8

Responsibility for New Zealand's SSE report presently resides with the Health Quality and Safety Commission (HQSC) which has developed policy requiring each of NZ's 20 District Health Boards,^a which govern and operate public hospitals, to report all SSEs.⁹ However, historically this has been difficult to monitor because adverse event reporting was voluntary in the NZ health system. The aim of the SSE report is to shine light on serious errors in the effort to improve healthcare quality and safety and stimulate learning across the NZ health sector. The SSE report provides information on falls, clinical management and medication errors. These incidents are represented as a percentage of all patients. Until 2010, the report included out-patient suicides, but the HQSC now considers these events to be different from other SSEs reported, because the report is focused on incidents that have an identifiable cause and are for the most part preventable.⁹ Once an adverse event has been identified and reported within the hospital, the severity assessment code (SAC) is then used to rate the severity of the event on a scale of 1–4.⁹ It is mandatory for all SAC rated 1 or 2 events to be reported to the HQSC within 14 working days.⁹ The hospital is then required to undertake a root cause analysis to identify the events that led to the event happening.^10,11 The completed report must be reported to the HQSC with recommendations to prevent such incidents from reoccurring.¹¹ The report previously provided an additional file, which contained as detailed as possible information on each identified SAC1 and 2 event.⁹ However, since the release of the 2011–2012 report, each hospital district now provides this information themselves on their website.

Adverse events do not inevitably signal poor quality of care, nor does their absence necessarily indicate good quality of care.¹² Often the occurrence of adverse events boils down to faults in management systems within the organisation and more often than not similar incidents are occurring at other hospital organisations. It might be assumed that the SSE report serves as a means to improve hospital systems and that having comparative data would help hospital districts to learn from each other. Yet, little is known about whether an SSE report actually stimulates change and is a useful exercise, both within NZ and in the international context. This article, therefore, reports on a study designed to assess this question.

Methods

Study design

An exploratory qualitative study was undertaken following the release of the 2010–2011 report by the HQSC. The objectives of the study explored through interviews were:

‐How hospital districts responded to the SSE report?

‐Whether the SSE report was used as a quality improvement tool and what was its perceived value?

‐Whether there was convergence or variation in responses to adverse events across hospital districts?

‐Whether improvement of quality and patient safety was a collaborative activity across hospital districts?

Study population

The study was conducted across the 20 hospital districts. Individuals of interest were those employed by each district with a leadership role in quality and patient safety. The CEO of each district was contacted and asked to recommend an appropriate person within their hospitals to be interviewed. These staff ideally had responsibilities for ensuring quality and patient safety within the organisation, including the reporting of SSEs and implementing changes to organisational processes. Each recommended individual was contacted by telephone.

Data collection

Interviews were semi-structured to allow discussion to proceed based on participant responses. Interviews were tape recorded and then transcribed verbatim. Ethical approval was provided by the University of Otago.

Analysis

Interview transcripts were imported into the qualitative data analysis programme, NVivo 10¹³ which allowed for iterative thematic coding of interview data using a general inductive approach.^14,15 Initial codes followed topics led by the interview questions. Themes were then assembled by collating codes into preliminary themes. This organised the data by making connections between a major category and its subcategory enabling the recognition of themes within the data.¹⁶ From here, all data related to the provisional themes were gathered together. In the next phase of analysis, all interviews were re-analysed to ensure that examples of each theme had not only been accounted for and compared but also that the themes were relevant to the codes derived from the initial data set.^14,16 It was apparent during coding that key information provided described the context in which SSEs took place. This information allowed for further insight on the use of the SSE report, including how it is perceived and used as an improvement tool. The four key themes that emerged are described in the results.

Results

Twenty-six individuals from the 20 different hospital organisations were interviewed. Interviews ranged from 30 to 90 minutes with the average interview being 47 minutes. There were 18 different job titles amongst interviewees. Twelve reported straight to the Chief Executive, one was the Chief Executive and the remaining 13 reported to superiors, one or two levels below the Chief Executive. Interviewees had been in their current positions from 3 months up to 11 years. Interviewees generally had a nursing, medical or allied health background but two had come from the quality and patient safety industry.

Response to the SSE report

Participants generally suggested that their response to the annual release of the SSE report was limited and amounted to the comparing incidents from one organisation to another and the distribution of the report around their hospital district as illustrated in the following quote:

Well we already know our own information, because we have already provided it and I guess it is quite typical to be comparing how we are doing with other [hospitals]. Having said that you don't know whether you have got less or more events because you have actually had less or more or whether it is to do with the reporting and capturing of adverse events. – Hospital P

However, some found the report useful for locating where advice from other hospital districts might be sought:

It gets distributed really widely, we talk about where there might be differences and whether it is worth making contact somewhere if there is something we are struggling with. – Hospital Q

A number of participants also discussed how the release of the report provides an opportunity for reflection on prior improvement processes and is a reminder to go back and review how these changes are progressing within the organisation:

From our point of view once we respond to on going issues, like we go back and try to make sure that the recommendations were carried out are still going. – Hospital T

Use as a quality improvement tool

Many interviewees noted that the SSE report was used as a quality improvement tool, but its use varied significantly:

It certainly informs our decision making at the clinical board as to what we should be concentrating on, so yes it does. – Hospital H

Aspects of it I would say we analyse and say yes that's a trigger for improvement opportunity. – Hospital C

However, three DHB representatives commented that the report was not used to improve quality, because not enough information was given in it:

Probably not yet. I don't think we are sharing enough depth in them because people, we all, we all have similar events and not many are that different actually. What you don't see in the report in the format that it is in now is say for example if I had an incident where a certain drug is mistaken for another drug and it's given and the patient dies, and I learn another [hospital district] had a similar event, I want to be able to go somewhere where I can see not only the story but actually the recommendations in place. I don't want to reinvent the wheel, so there needs to be enough information in the national report. – Hospital J

Three hospital districts also discussed how elements of the report were used as improvement opportunities but that the report was not a core component or a driver of their improvement efforts:

Look, it informs one component of what we do but it's not a major one. We wouldn't use the national reporting system to bring about change. For one thing it only comes out once a year, so why would you wait for an annual report. We would actually just get on with existing networks to just get on and do that. – Hospital D

We are with or without it. We are doing it as business as usual anyways but specifically in relation to the report it's distributed through out the organisation, directorates and clinical governance meetings table it and discuss it and I also look at it as an analysis oversight point of view and report on that. – Hospital O

Discussions surrounding the use of the SSE report as a quality improvement tool also brought up questions about the value of the report. There were mixed responses about how valuable the report was, but the majority said they found it useful. The case study section was considered to be the most valuable part of the report:

I find the case study stuff the most valuable and see where we stand nationally, but that tends to raise more questions than answers. – Hospital Q

Participants reflected that they felt that the report was important to the public because it allowed open transparency to see what was occurring locally and nationally in hospitals

I think that it is humanising hospitals. I think from a public perspective it is putting it out there saying actually we do get things wrong from time to time but we are trying to reduce the times that those sorts of events are occurring and I think that it raises the profiles for staff to actually say you know your best has to be your best every single day of the week… when patient care is at the forefront you have to make sure that you are on your game all the time and the minimum is not good enough. – Hospital L

Problems with reporting and acting on SSEs

Interviewees raised various issues with the SSE reporting process, but suggested that the physical system (electronic or paper based) did not affect the actual reporting of events

Many years ago we said if it's so hard to fill out the paper work, why don't you just lift the phone and leave a message. And people didn't do that either, so there will always be some people that report and some people that don't. – Hospital B

Common influences related to the number of adverse events reported included organisational culture, lack of feedback, recognition of an incident itself, fear of repercussions, job title, individual personalities, loyalties and time constraints:

As you are presumably aware the vast majority of incidents don't get reported, reporting is a very poor way of actually trying to establish any idea of the rate of frequency of incidents. You need to do audits to actually establish that. Umm so the kinds of this that actually trigger voluntary reporting tend to be driven very much by the consequences of the incident. So an incident that has more serious consequences is more likely to be reported, but that might not be the most important incident to report because sometimes everyday incidents that have minor impact are still things because they are common are actually really important to do something about and also certain kinds of people are in the habit of reporting. – Hospital D

Comments were made regarding the lack of feedback about what happens once an event has been reported

When I first got here people would think that nothing would happen so what's the use of writing a report. Now we have got a process where they get all the reports back monthly and they actually see things that are happening. So they are starting to think that it is a useful process. – Hospital G

I think it is important that people actually get feed back and that the fact that they are reporting stuff and it's going into a black hole and nothing happens or gets done can discourage people from continuing to report. – Hospital P

Is quality improvement a priority?

Interviewees suggested that the culture of the organisation itself was a major influence on the priority given to improving quality and patient safety and perceptions of the SSE report. Terms used to describe culture included improving, fiscally aware, good, very focused, very strong, just culture, changing and caring, and growing positivity. One interviewee suggested of the culture at their hospital district:

Umm not at all good. There are a few problems I think. 1) I don't think the clinicians have been very well engaged with the management team, I don't think that the clinicians see that every body is working to the same aim or should be so people have not worked together as effectively as they should have done and evidently that has a negative impact. 2) In terms of seeing quality and safety as the absolute front runner key thing that we should all be focused on, I think it would be fair to say that neither clinicians nor managers feel that way and in fact you know when I come to give my quality report to the board when I first got here I got a five minute slot at the end of a two hour meeting that was mostly about performance, that horrified me. When I was at my old job overseas I was the first item on the agenda and talked for over an hour. – Hospital A

Areas relating to the role of the board in the prioritising of SSEs and financial limitations were brought up by a number of interviewees. Financial drivers taking precedence over quality improvement efforts were raised as well as concerns around the backgrounds of board members and their understanding of the serious need to build a culture of quality and safety and put this at the top of their agendas:

The main focus at board meetings is finances. They definitely override patient safety and quality. – Hospital H

However, not all participants agreed with this statement:

The main focus at board meetings would be money but that does not override patient safety and quality improvement. It just means that somebody has to champion these portfolios or needs for the organisation. – Hospital F

The need for quality improvement to be a priority means that there needs to be adequate opportunities for professional development. Multiple hospitals felt that this was an area that needed improvement. Many organisations had orientation days for all new staff and held training seminars throughout the year but these were not compulsory.

There are seminars run a few times a year. We don't force anyone to participate but we advertise them. – Hospital F

Discussion

This study found that, rather than having a transformational impact on NZ's healthcare system and providers, the response to an annual SSE report was limited bringing into question its use as a quality improvement tool. This counters assertions made in the literature around the need for wider dissemination of information on adverse events.^17–19 Our findings also indicated that more could be done by NZ's hospital districts in using information from the SSE report which would be to their own and their patients' benefit.

As described in this article, the use of the SSE report elicited mixed views from our research participants. While many saw a limited impact, there were some positives including that it had raised the profile of adverse events in showing these were important and not taboo with significant implications for the individual patients harmed in the process of care. Respondents had indicated that the report helped boost transparency including opening up discussions at national meetings and other forums and facilitating a degree of cross-sectoral learning and reflection on incidents. This said, a surprising finding was that that a majority of hospital districts said they were unable to use information in the SSE report from their counterparts as it lacked the depth required for sufficient lesson-drawing and response generation. A number also said that it was an annual report and that quality improvement is on going. Again, this is not ideal and counters one of the objectives of the report, which is to allow for shared and continuous learning.⁹

What would be required for the SSE report to have more impact and to produce the transformational change intended of it? First, the place of quality improvement staff in the organisation and structure of the hospital or any health organisation is vital.²⁰ The participants in this study all felt that the executive level should have a major focus on quality and patient safety. In many hospital districts, this was the case, but in some the position of quality and safety in the organisational hierarchy was still lower than evidence suggested it should be.^20–22 This meant reduced capacity for information transfer from the hospital frontlines to the executive level and vice versa, and reduced potential for SSE report impact and response.

Second, culture is a major driver of quality improvement and improved patient safety.^23–26 The literature affirms that to become safer, hospitals need to build cultures of quality and safety that are committed to respect, clear communication, full disclosure, apology, support, resolution and learning for staff, patients and families when harm has occurred.^23–27 In 2000, Leape and Berwick (2000) identified that building a culture of safety that involves learning, trust, curiosity, systems thinking and executive responsibility would be difficult. Results from our study confirm that multiple hospital districts were struggling to develop an ‘ideal culture’, but a small number were more developed. Reasons for this could be due to the size of the organisation, the allocations of resources within and, very importantly, leadership. According to Leape and Berwick, changing even a few practices and policies necessitates that all personnel share one vision and take personal responsibility for safety, which requires vision and leadership.²⁷

Third, feeding into culture is the provision of education and training opportunities for continuous professional development.^28,29 In the 2010–2011 SSE report, the HQSC commented on how hospital districts provided education and training to their employees.⁹ From the participants of this study, it was found that some organisations have more advanced opportunities for professional development than others. Basic orientation training into the organisation for new employees was the only formal training provided by some hospitals. Others provided periodic opportunities for staff to voluntarily attend seminars. Resource issues again influenced a hospital's ability to provide professional development to its staff, suggesting that it would be beneficial for a national approach to training in quality and patient safety.

Of course, a further explanation and intertwined with the aforementioned three points, is provided by the voluminous literature around organisational learning and capacity to absorb knowledge from external sources.^30–33 Studies have variously found that organisational response to external quality improvement initiatives or data can be characterised by conflict and tension and is often haphazard and inadequate.³⁴ The general explanations for this include the basic commitment within the organisation to resourcing an appropriate response, leadership around this, and presence of organisational objectives which conflict with the demands for response to externally generated information. When multiple organisations across a health system are involved, the absorption and response challenges are exacerbated.³⁵ Thus, any preconception that information from a national report, as discussed in this article, will result in system-wide change may be naïve. Theories of implementation in complex adaptive systems, which portray change as circuitous and contextual,^36,37 may assist and help with the practical process of garnering an appropriate response in cases such as New Zealand's SSE report.

Methodological issues and limitations

This exploratory qualitative study provided a useful overview of responses to the annual SSE report in each of NZ's 20 hospital districts. Study participants were limited to those at the highest levels of each organisation meaning that there was limited capacity to investigate how those at the front-lines of care delivery responded to the SSE report. However, our belief is that the respondents were a data rich group with a good understanding of each of their organisations' approaches to quality and safety and its SSE report response. In this regard, we believe that they represented the broad views of their respective organisations, including general attitudes towards adverse events, quality and safety. Different lengths of tenure can impact on level of organisation specific knowledge on quality improvement and patient safety. However, only three interviewees were recent appointees and had moved from similar roles in prior jobs. It is therefore unlikely to have had an impact on the results of this study. Obviously, this study has provided an initial insight into SSE report responses. Further research involving a wider number of interviewees at different levels of their hospitals could be useful, along with quantitative approaches such as a cross-sectional survey of the healthcare professional workforce.

Conclusion

The main finding of this study is that an annual SSE report has a limited role in improving quality and patient safety in the NZ context. The report has been of value to the public by increasing community awareness of what is occurring in our public hospitals. It has also provided a bridge for national groups to talk about and discuss systems improvement in varying contexts. Very importantly, this study provides a contribution to the literature in which there is limited understanding of how healthcare organisations use national information about SSEs and about the possible impact of such reporting on healthcare improvement.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Note

References

Ministry of Health. Services delivered: Patient discharge and case-weight information, Wellington: Ministry of Health, 2013.

Davis

Lay-Yee

Briant

. Adverse events in New Zealand public hospitals I: occurrence and impact. New Zeal Med J 2002; 115(1167): U271.

Wilson

Runciman

Gibberd

. The quality in Australian Health Care Study. Med J Australia 1995; 163(9): 458–471.

Vincent

Neale

Woloshynowych

. Adverse events in British hospitals: Preliminary retrospective record review. Br Med J 2001; 322(7285): 517–519.

Gauld

. The new health policy, Maidenhead: Open University Press, 2009.

Keenan

. Never events, London, UK: Care Quality Commission, 2013. .

Duckett

Willcox

. The Australian health care system, Melbourne, Australia: Oxford University Press, 2011. .

Cheng

Yuan

Sun

. International comparative analyses of incidents reporting systems for healthcare risk management. J Evidence-Base Med 2011; 4(1): 32–47.

Health Quality and Safety Commission. Making our hospitals safer – serious and sentinel events 2010/2011, Wellington: Health Quality and Safety Commission, 2012.

10.

Health Quality and Safety Commission. Root cause analysis for clinical incidents: A practical guide, Wellington: Health Quality and Safety Commission, 2012.

11.

Health Quality and Safety Commission. New Zealand health and disability services – National reportable events policy 2012, Wellington: Health Quality and Safety Comission, 2012.

12.

Brennan

Leape

Laird

. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. New Engl J Med 1991; 324: 370–376.

13.

QSR International. Nivio10, QSR International Pty Ltd, 2014. .

14.

Pope

Mays

. Qualitative research in health care, Oxford, UK: John Wiley & Sons, 2008. .

15.

Thomas

. A general inductive approach for analyzing qualitative evaluation data. Am J Eval 2006; 27: 237–246.

16.

Liamputtong

. Research methods in health: foundations for evidence-based practice, Melbourne, Australia: Oxford University Press, 2013. .

17.

Leape

Lawthers

Brennan

. Preventing medical injury. QRB Qual Rev Bull 1993; 19: 144–149.

18.

McKinsey and Company. Management in healthcare: Why good practice really matters, London, UK: Centre for Economic Performance, The London School of Economics and Political Science, 2010. .

19.

Woodward

. Will the future continue to repeat the past? Qual Safe Health Care 2005; 14: 74.

20.

Dorgan

Layton

Bloom

. Management in healthcare: Why good practice really matters, London: McKinsey and Company/London School of Economics, 2010.

21.

Bigda-Peyton T. Making change happen in large organisations: Tope down or Bottom Up?. Available at: http://c.ymcdn.com/sites/www.npsf.org/resource/resmgr/LLI/LLI_PPT_031913_web.pdf (2010, accessed 21 August 2015).

22.

Conway

. Getting boards on board: engaging governing boards in quality and safety. Joint Commission J Qual Patient Safe 2008; 34: 214–220.

23.

Gauld

. Continuity amid chaos: Health care management and delivery in New Zealand, Dunedin, New Zealand: University of Otago Press, 2003. .

24.

Lucian Leape Institute. Through the eyes of the workforce: Creating joy, meaning, and safer health care. Available at: https://aneconomyofmeaning.wordpress.com/2010/07/02/making-change-happen-in-large-organizations-top-down-or-bottom-up/ (2013, accessed 21 August 2015).

25.

Seddon

Merry

. How safe are our hospitals? J New Zeal Med Assoc 2002; 115: 1–4.

26.

Leape

Berwick

. Safe health care: are we up to it? Br Med J 2000; 320: 725–726.

27.

Leape

Berwick

. Five years after To Err Is Human: What have we learned? J Am Med Assoc 2005; 293: 2384–2390.

28.

Paterson-Brown

. Improving patient safety through education. Br Med J 2011; 342: d 214. .

29.

Cronenwett

Sherwood

Barnsteiner

. Quality and safety education for nurses. Nurs Outlook 2007; 55: 122–131.

30.

Scanlon

Darby

Rolph

. Use of performance information for quality improvement. Health Services Res 2001; 36: 619–641.

31.

Johnsen

. What does 25 years of experience tell us about the state of performance measurement in public policy and management? Public Money Manage 2005; 25: 9–17.

32.

Ferlie

Crilly

Jashapura

. Knowledge mobilisation in healthcare: a critical review of health sector and generic management literature. Social Sci Med 2012; 74: 1297–1304.

33.

Freeman

. Using performance indicators to improve health care quality in the public sector: A review of the literature. Health Service Manage Res 2002; 15: 126–137.

34.

Gardner

Dowden

Togni

. Understanding the uptake of continuous quality improvement in Indigenous primary health care: Lessons from a multi-site case study of the Audit and Best Practice for Chronic Disease project. Implement Sci 2010; 5: 1–14.

35.

Walsh

. Understanding what works – and why – in quality improvement: the need for theory-driven evaluation. Int J Qual Health Care 2007; 19: 57–59.

36.

Room

. Complexity, institutions and public policy: Agile decision-making in a turbulent world, Cheltenham: Edward Elgar, 2011.

37.

Van de Ven

Polley

Garud

. The innovation journey, Oxford: Oxford University Press, 1999.