Abstract
Public hospitals are required to have quality systems in place to meet accreditation standards, achieve government performance expectations and continually improve care. However, previous study suggests that there has been limited success in the implementation of effective quality systems. Using document review, self-evaluation and qualitative data from interviews and focus groups of 270 board members, managers and staff we explored the implementation of quality systems in eight Australian public hospitals. Using normalisation process theory, we found that the hospitals took a technical, top-down approach to quality system implementation and did not provide staff with opportunities for socialization of the technology that enabled them to normalise the quality work. ‘Quality’ was consistently described as an ‘extra’ set of tasks to do, rather than a means to creating sustained, safe, quality care. Despite enormous goodwill and positive intent, a lack of understanding of how to effect change in the complexity of hospitals has led the boards and senior managers in our sample to execute a technical, top-down approach based on compliance and reactive risk.
Background
Health services aim to ensure consistently safe, high quality care, but this remains a challenge in public hospitals in Australia and around the world. Health care is a complex high-risk industry, with many steps and people involved in the simplest care episode. 1 Problems with safety of care in key areas such as infections and medications persist, and evidence-based guidelines are inconsistently applied.2,3 Quality programs have evolved slowly, limited by a lack of consistent focus, data and resources, and minimal attention paid to this aspect of the professional role in clinical education. 4 While there is a strong focus on quality systems in government policy 5 there has been little research on implementation. The purpose of this study is to document the state of quality system implementation in eight Australian hospitals from the perspective of board members, managers and front line staff. Informed by normalisation process theory (NPT), the study triangulated data from eight hospitals, drawing on document, interview and focus group data.
Implementation of hospital quality systems
Efforts to improve hospital care quality have been influenced by widely publicised failures of care delivery,6–10 long-standing models of external accreditation,5,11 and an increasing focus on improving organisational performance in the health sector. 12 These forces promote the need for robust quality systems. Although not conclusive, there is evidence of a positive relationship between quality systems and organisational performance. 13 Hospitals must demonstrate effective quality systems to satisfy these imperatives.
Early definitions of a quality system largely focused on internal operations. For example, Feigenbaum 14 defined it as the operating work structure, documented in effective procedures for the workforce, the machines and the information to assure both customer quality satisfaction and economical operations. More recent definitions stress the need for quality systems to encompass strategy and organisational culture, in addition to internal operating processes, structures and measurement (p. 8). 15 This aligns with the definition of a hospital quality system used in this study: ‘A systematic, coordinated, organisation-wide program of planning, governance, mind-set, behaviours, tools, change, measurement, evaluation and action to achieve and maintain the organisation’s vision of a great experience for each consumer’ (p. 8). 16
A large literature outlines quality system components,17–19 antecedents12,20 and barriers. 21 There is less evidence on implementation, and despite the need for effective involvement of hospital human resources in quality system implementation,21,22 there are few robust studies on quality system implementation from the perspective of managers and staff.
Normalisation process theory (NPT) has been successfully applied to complex intervention studies, with a particular focus in the health sector.
23
Four process constructs underlie NPT, providing a useful framework for analysis of the implementation of a new technology such as a hospital quality system:
Processes of coherence, which focus on individual and communal sense-making, Processes of cognitive participation, directed to engagement and building commitment, Processes of collective action used to enact the new practice, and Processes of individual and communal reflexive monitoring to assess the effects of the new practice.23–25
NPT fits well with system analysis within hospitals, as they are complex organisations. 1 Health professionals act relatively independently, working to their own rules. 26 As a result, complex systems run on relationships and unwritten rules and staff strongly resist the imposition of policies and changes that work against these relationships. Complex systems are adaptive, evolving due to the variety of forces on them. 27 This reinforces the importance of the social processes that influence the implementation of organisation-wide systems in hospitals. NPT provides an ideal framework enabling in-depth analysis of the various processes that hospital workers employ to adopt, ignore or even thwart the implementation of the quality system within their organisation.
Method
Eight hospitals in one state in Australia volunteered to participate in a qualitative study, tracking the implementation of their quality systems. The sample included one metropolitan specialist health service, two large multi-campus metropolitan health services, two regional public health services, one regional private hospital and two rurally-based district health services. The sample was restricted to one state, as, while there are overarching national health policy directions in Australia, the states have relative independence in the management of the hospitals to achieve the national goals.
This paper reports on the findings from the first round of data collection during February, March and April 2015. Over 270 health service managers, staff and board members across the eight hospitals participated in interviews or focus groups (Figure 1). The participants were distributed among the participating hospitals as expected by the size of the hospitals; the smallest rural hospital had 13 participants and the largest health service had 61 participants.
Study participants in interviews or focus groups by employment category.
Three interviews and 52 focus groups were completed by the two researchers. The interviews were conducted to accommodate key participants who were unable to attend a focus group. The structured questions included:
Describe the components of your organisation’s quality system. What drives quality of care improvements in your organisation? What difference does the organisational quality system make to the quality of patient care? What is your role in the provision of high quality care? How do you know this? How does the quality system assist you in the provision of high quality care? What is your organisation’s definition of high quality care? What level of quality care do patients receive in your health service today? How do you know this? Has the quality of care improved over the past 6 months? How do you know? What would be helpful to you in further improving the care your patients receive?
As outlined by Denzin and Lincoln 28 both researchers took extensive notes during the interviews and focus groups and independently coded the data in relation to emerging themes. Together the researchers completed data reduction. 29 The data were then displayed in spreadsheets to simplify analysis and verification. Descriptive categories were developed for each emerging theme, and data fragments were assigned codes and a site identifier to enable individual site analysis as well as cross-site comparison. Only themes agreed by both researchers were included in the final qualitative database. At this stage, analysis was further developed through the normalisation process theoretical concepts, including making sense of ambivalent and contradictory statements and practices. The final themes were reported to the research sites for verification.
The researchers jointly reviewed board quality committee minutes and internal and external quality reports, enabling triangulation with the qualitative data. The hospital committee responsible for the governance of the quality and safety program completed a self-assessment of their improvement capability using the IHI Improvement Capability Self-Assessment Tool 30 and the Quality System Maturity Scale. 16 The IHI Tool rates six quality system components of leadership for improvement, results, resources, workforce and human resources, data infrastructure and management, and improvement knowledge and competence on the 5-item scale: 1. Just beginning; 2. Developing; 3. Making progress; 4. Significant impact; and 5. Exemplary. The Quality System Maturity Scale provides a self-assessment of the implementation of the entire quality system in relation to the following five stages: 1. Informal improvement; 2. Compliance; 3. Reactive risk; 4. Proactive improvement; and 5. Strategic creation.
The study received approval from the Human Research Ethics Committee of La Trobe University and participating health services in January to April 2015.
Results and discussion
The results suggested that none of the sites had a fully implemented quality system in place that both managers and staff would rate as successful in supporting them to provide consistently safe, high quality care. Using the Quality System Maturity Scale three sites self-rated at or above the 4. Proactive improvement level; the other five indicated they were still moving from 2. Compliance and 3. Reactive risk to 4. Proactive improvement. None of the participating hospitals self-rated stage 5. Strategic creation. In the IHI Tool self-assessment, the participating quality committees identified the greatest areas of weakness in Workforce and Human Resources (mean score 3.1), Improvement Knowledge and Competence (mean score 3.0) and Resources (mean score 3.0), which were assessed as ‘making progress,’ and Data Infrastructure and Management, which received the lowest scores as ‘developing’ (mean score 2.7). Leadership for Improvement (mean score of 3.4) and Results (mean score 3.3) were seen as slightly more developed. The qualitative data from the interviews and focus groups suggested that hospital staff identified similar weaknesses, but staff also perceived the Leadership and Results were not as well developed as the quality committees self-assessment suggested. Despite the acknowledgement of the weaknesses of resources in the self-evaluation, lack of resources did not emerge as a qualitative theme.
In all hospitals, the board and senior management reported aspirational visions for high quality care, variously defined. Senior managers, board and quality committee members provided strategic and operational documents that they thought were sufficient to demonstrate the leadership and communicate the quality agenda to front line staff. Six of the participating hospitals self-rated their Leadership for Improvement on the IHI Tool 30 as having ‘significant impact.’ However, in all cases, the vision was not well understood by local managers and front line staff and was not embedded at point of care. Most of the board members and senior managers were not aware that their vision and messages for high quality care had not penetrated throughout the organisation.
The health professionals and their local managers reported lack of communication about quality and quality improvement expectations and implementation. In the absence of this guidance, staff identified their organisation’s quality system as the tasks associated with compliance, predominantly incident reporting, reactive risk management, meetings and mandatory training. Without a clear purpose for these activities, and translation to point of care, carrying out these tasks was seen by staff as an end in itself, labelled as ‘doing quality,’ a finding in all of the participating hospitals.
Staff described quality improvement at point of care as rigid and mechanistic, largely comprising policy and procedure, standards, audits and incident reports. They felt that the focus was not on continuous improvement, but focused on monitoring the status quo, with occasional incremental changes, which may or may not be sustained. All hospitals had some quality system elements in place which drove high quality care in some services, but none demonstrated a comprehensive quality system that drove consistently high quality care across all services, for all consumers.
The NPT theoretical framework suggests that organisational interventions are embedded through four social processes:
Processes of coherence, with individual and communal sense-making Processes of cognitive participation, fostering engagement and commitment Processes of collective action Processes of individual and communal reflexive monitoring.23–25 The analysis in relation to each of the four social processes is outlined below.
Processes of coherence, with individual and communal sense-making
The board members and senior managers believed that they provided quality and safety leadership for staff, directing us to plans, frameworks and extensive policies and procedures aimed at quality care. However, few of the middle managers and almost none of the front line staff were able to describe their hospital’s approach to quality improvement. Illustrative quotes are provided. We think that there is something written somewhere [about the vision for quality of care]…but we have demonstrated [in this focus group] that we do not know the expectations regarding the delivery of high quality care. (Health professionals MHS8 FG#1) Perhaps no one knows how it [the components of the quality system] all fits together. (Health professionals MHS6 FG#2) Everyone has their own individual definition of safe, quality care. (Health professionals REHS2 FG#4) We came to this focus group because we would like to get our ‘head around the process of quality’ in this organisation and we haven’t understood it to date. (Health professionals REHS3 FG#1) What we are auditing is not what we want to know. (Health professionals RUH5 FG#2)
Participants complained of the burden of quality and safety paperwork, and did not appear energised by improvement work, despite wanting to and believing that they provided high quality care. The quality systems were described as focusing more on process than people, and were not constructed to use the knowledge, judgement and experience of health care professionals to enact a stated vision at point of care. The quality system makes busy people busier. (Managers MHS7 FG#3 and Managers MHS8 FG#3) Standards, evidence and incidents makes busy people, busier. (Health professionals RUHS5 FG#2) Everyone is doing their quality part, plus their job. (Health professionals REHS2 FG#2) Doctors with good quality brains do not like the bureaucracy of quality. (Health professionals MHS8 FG#2)
Processes of cognitive participation fostering engagement and commitment
Four socio-technical principles suggest that workers need to be involved in change processes,
32
be clear about the overall objective,
33
have access to information about their operations and noted variations, and be given the opportunity to address operating issues in their immediate workplace.
32
While we expected these principles to drive implementation of the quality systems, in contrast we heard about top-down constructs, with expectations for staff dictated from the executive through the line managers, or via the quality director/manager to staff. In fact, in many of the hospitals, the health professionals perceived the top-down ‘quality directives’ changed so frequently that it had become easier just to ignore them all. The quality team does not have much to do with the quality we provide. (Health professionals MHS7 FG#3) Whatever the flavour of the month is for quality gets dropped on us. (Health professionals MHS7 FG#5) The quality system is the schedule of audit reports we are required to provide. (Health professionals RUH3 FG#2) We do not hear a lot from the Board – we feed things up, but they do not feed back. (Health professionals MHS8 FG#2) It is difficult to go against what the quality team have laid out for us. (Health professionals MHS7 FG#4) It is difficult to make changes. The most common quality outcome is – we will re-audit in 6 months. (Health professionals REHS3 FG#4)
This finding is concerning, as complex adaptive systems do not perform well with highly structured systems, 34 with evidence that simple rules and loose coupling are better suited for successful performance in organisations with unpredictable environments. 35 In our study, hospital staff was not provided with opportunities for cognitive participation, but were ‘ruled’ by quality system activities driven by compliance requirements. This created a rule-based environment with no engagement by the staff actually providing the care.
While rules have their place in health care, highly trained professionals create work-arounds that better suit the situation. 36 In complex adaptive organisations, the unwritten rules shared by staff are generally more powerful in driving behaviour than explicit rules imposed through policies and procedures. 37 ‘Despite the fact that many guidelines and policies exist, these are often viewed as recommendations rather than strictly enforced rules’ (p. 26). 38 There is increasing evidence that the imposition of rules to improve safety may be at odds with staff perceptions of what they require to provide high quality care. 39 The top-down compliance-based models observed in this study did not enhance manager and staff engagement with their quality systems. In fact, they reported that ‘quality’ often interfered with their ‘real work.’ There was little evidence of cognitive participation or commitment to the hospital’s quality work by staff.
It is not that staff did not understand or appreciate the need for policy and standards. In fact, many commented that the National Safety and Quality Health Service Standards had contributed to real improvements in key clinical risk areas. Rather, it was the over-reliance on the standards as the major component of the quality system, and the way in which they were implemented, that appeared to create staff pushback with the quality system. Our findings suggest that the accreditation process, with the strong focus on compliance, may make it difficult to garner collective action on quality, due to the natural resistance of health care staff to over-regulation, and therefore engagement with a quality system largely focused on meeting accreditation standards. This finding provides support for Grepperud’s 40 contention that accreditation may be socially inefficient for public hospitals.
Processes of collective action
Despite the evidence that achieving and maintaining high quality care requires collective staff participation and commitment,22,41 the organisations in our sample did not routinely explore how quality and safety had penetrated throughout the hospital. Complex systems are complex because they require individuals to work in interdependent and interconnecting processes, and improving these processes requires an interdisciplinary approach. However, interaction in health services, particularly in response to change, tends to occur mainly among members of the same professional groups. That is, nurses interacting with nurses and doctors interacting with doctors. These ‘tribes’ give the people within them an important sense of belonging, but it is hard to foster collective action. 42 In our study the various hospital groups were allowed to hold their own perspectives and approaches to the provision of high quality care, with no mechanisms to facilitate collective understanding and implementation.
As outlined by Glouberman and Mintzberg, 43 each of the four major hospital stakeholder groups (i.e. governors, managers, medical professionals and other health professionals) interacts within their own worlds, holding their own values and using their own jargon. The hospital groups, both within a profession and within a hierarchical level, talk mainly to each other, reinforcing their own values, perceptions and unwritten rules. This lack of shared values, tools and even work makes it more difficult to develop the shared context 44 and behavioural norms 45 for implementation. Further, a recent systematic review identified the need for ‘an infrastructure for staff-manager interactions on quality strategy’ (p. 13). 46 Our data suggest that these hospitals have not invested in this necessary collective action to embed high quality care throughout the organisation.
Processes of individual and communal reflexive monitoring
In every focus group we heard that the participants had a ‘sense’ that quality was improving, but except for selected areas of focus, such as infection rates and falls, there were limited data available to know if quality was improving. We don’t know where we would go to get the data [about quality improvement] – we have no idea. (Health professionals MHS8 FG#3) Don’t know that any of us has the evidence that the quality of care has improved. (Health professionals ReHS2 FG#4 and Managers RUHS4 FG#2) We have a sense of well-being - improvement is hard to measure anyway. (Managers RUHS5 FG#1) There is not enough time for us to gather the data to be able to know if the quality is improving. (Health professionals RGHS2 FG#4) We have had some wins, but we are not sure why. (Managers REHS1 FG#1) We can all say we are doing things for quality, but we are not sure that it is making a difference. (Health professionals REHS1 FG#3)
All of the participating hospitals had externally mandated performance indicators, often presented in complicated quality of care dashboards. Yet our findings suggest that these were rarely useful in helping managers and staff reflect on the quality of care they provided at point of care. Pfleuger suggests that this is because current measurements which tend to be based on accounting principles ‘generate less rather than more information about quality, provide representations of quality which are oriented away from the reality of practice on the front line, and create an illusion of control while producing areas of unknowability’ (p. 186). 47 Others stress the need to augment the current indicators with softer intelligence. 48 Our findings support the assertion that the existing methods of accounting for quality do not provide a true understanding of the level of quality of care. The use of imposed quality performance indicators may in fact limit the ability of hospital staff to engage in reflexive performance monitoring in relation to their own practice or to that of their work group as a whole.
Communal reflective processes have been suggested to help staff deal with the tensions experienced in complex systems. 49 Hospital staff ‘value reflective processes where they have space to discuss quality in practice’ (p. 130). 49 Our data suggest that there were limited opportunities for reflexive monitoring. Board members and senior managers did not tell us that they had limited data to measure quality and monitor quality improvement and it was difficult to ascertain if they fully appreciated the implications. As a result middle managers and their staff did not have the information and feedback they required to fully engage with the quality system to influence their behaviours. We suggest that the lack of an organisation-wide definition of quality made it difficult for the hospitals to be clear about the indicators they needed to track.
Limitations
While we collected data from many individuals across the participant hospitals, these data represent a snapshot in time. We were not able to collect observational ethnographic data of interactions among the hospital staff and longitudinal data that might assist in exploring the implementation processes in greater depth. Further study is required.
Conclusions
This is the first study to examine health service quality system implementation in Australian public hospitals in depth from the perspective of board members, senior and middle managers and front line care staff. From our review of quality and safety documents, self-evaluations and discussions with 270 board members and staff through interviews and focus groups, we have reviewed the status of quality system implementation in our sample of eight Australian hospitals. Quality systems are documented in plans, policies and procedures, and there is much activity, but hospital staff generally see quality improvement as tasks outside (and not connected with) their care delivery roles. Quality systems are primarily focused on compliance and finding errors, and do not contribute to consistent and sustained high quality care across the dimensions of quality.
There are pockets of strategic, integrated and proactive pursuit of excellence at point of care, but even in these hospitals, staff understanding weakened the further they were down the hierarchy. Our study confirmed that limited opportunities for cross-discipline social interaction around quality did not enable staff to normalise quality into their day-to-day activities. Quality was seen as an ‘extra’ set of tasks to do, rather than a means to creating sustained, safe, quality care.
It was concerning that, in the absence of useful data, board members and senior managers chose to be optimistic about the care provided, believing that ‘all our staff come to work to provide good care.’ This optimism arose from comfort seeking behaviours that focused on a few indicators such as positive accreditation results, small numbers of serious adverse events and positive consumer satisfaction feedback. But these indicators did not measure improvement, and none of our participants could confirm whether the quality of care provided by their hospital had improved in the past year. None could clearly describe the quality of care their consumers were likely to experience during a routine admission, but were positive about what they ‘hoped’ it would be.
Despite enormous goodwill and positive intent, it appears that a lack of understanding of how to effect change and improvement in the complexity of health care has led the boards and senior managers in our sample to execute a technical, top-down approach based on compliance and reactive risk. We argue the lack of progress in high quality care delivery suggests that this approach is making incremental rather than transformational gains to quality and safety and the technical aspects of implementation must be augmented with greater understanding and attention to the social aspects of implementation.
Motivating all staff in a complex health care organisation to achieve a common goal for excellence at point of care is challenging. Despite the consistent lack of implementation we found board members and senior managers overestimated their ability to influence local staff action. This was further complicated by an apparent lack of normalisation processes to embed the quality system technology, and an overemphasis on policies, procedures and rules. Those with knowledge of complex systems call for approaches that vest quality of care initiatives in those providing the care – ‘not blindly following rules’ (p. 420), 39 yet our data suggests these hospitals had a much greater focus on the technical aspects of implementation than essential social aspects. Unless we can enhance the understanding and actions of decision makers in hospitals about what supports and encourages meaningful staff engagement with creating safe, quality care, this is unlikely to change. Without attention to social processes of implementation, even the most elegantly documented technical quality systems are unlikely to effect the changes at point of care that health care consumers want – and need.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
