Quality improvement strategies and tools: A comparative analysis between Italy and the United States

Abstract

Italian and American hospitals, in two different periods, have been urged by external circumstances to extensively redesign their quality improvement strategies. This paper, through the use of a survey administered to chief quality officers in both countries, aims to identify commonalities and differences between the two systems and to understand which approaches are effective in improving quality of care. In both countries chief quality officers report quality improvement has become a strategic priority, clinical governance approaches, and tools—such as disease-specific quality improvement projects and clinical pathways—are commonly used, and there is widespread awareness that clinical decision making must be supported by protocols and guidelines. Furthermore, the study clearly outlines the critical importance of adopting a system-wide approach to quality improvement. To this extent Italy seems lagging behind compared to US in fact: (i) responsibilities for different dimensions of quality are spread across different organizational units; (ii) quality improvement strategies do not typically involve administrative staff; and (iii) quality performance measures are not disseminated widely within the organization but are reported primarily to top management. On the other hand, in Italy chief quality officers perceive that the typical hospital organizational structure, which is based on clinical directories, allows better coordination between clinical specialties than in the United States. In both countries, the results of the study show that it is not the single methodology/model that makes the difference but how the different quality improvement strategies and tools interact to each other and how they are coherently embedded with the overall organizational strategy.

Keywords

Italy quality United States of America

Introduction

Quality improvement (QI) has become an increasingly important issue over the past few decades, with many countries establishing the provision of high quality, safe health care as a national priority. One of the World Health Organization’s priorities is establishment of effective strategies for ensuring quality of patient care within each healthcare system.¹ Consequently, policymakers of industrialized countries share an interest in implementing quality management practices to deliver appropriate care to patients, achieve positive clinical outcomes, avoid unnecessary clinical complications, and ensure efficient use of resources (e.g. Blaise and Kegels,² Kunkel and Westerling,³ Macinati,⁴ Makai et al.,⁵ Pejović et al.,⁶ Simou and Koutsogeorgou,⁷ Cohen et al.,⁸ McGlynn et al.,⁹ Wachter,¹⁰ Leape and Berwick,¹¹ and Ross et al.¹²).

This paper aims to analyze the strategies employed by Italian and American hospitals to improve quality of care. We administered in Italy a survey instrument used by Cohen et al.⁸ to analyze the QI strategies employed by American hospitals in reaction to the publication of the US Institute of Medicine (IOM) report¹³ that documented numerous deficiencies in patient care quality and safety.^9–12 In a subsequent report the following year the IOM¹³ called for a redesigned US healthcare system to resolve patient safety errors. Shortly after the publication of the IOM reports, US hospitals began to design different types of QI and patient safety initiatives.^14–17

The US healthcare system is large and complex, with approximately 4900 community hospitals and a mix of private and public insurance programs. Since 2010, the healthcare system has undergone major reforms in new service delivery models (e.g. accountable care organizations containing networks of hospitals and physicians) and new payment models that reward providers for their performance in delivering more effective and less costly care.

In contrast, the Italian healthcare system is a publicly-funded system that guarantees universal coverage. It has three tiers: the central government at the top level; 21 regional governments in the middle; and approximately 200 local health authorities (LHA) with their own hospitals and other healthcare facilities (as well as 100 independent hospitals not under the jurisdiction of a LHA^a). Since 2001, the system has undergone massive decentralization, with regions now having jurisdiction on all healthcare issues including promoting policies to improve quality.⁴ Italian hospitals, like those in the US although almost a decade later, implemented new QI initiatives after a series of national and regional programs stressing the critical importance of improving quality.¹⁸

Despite the growing literature in QI systems, it is unclear which strategies and tools have been actually used by hospitals and with which results. In fact most of the scientific studies have focused their attention on the evaluation of the application and effectiveness of single specific QI tools and methodologies. However, as shown by Walshe¹⁹ in his bibliometric analysis, over the last two decades, we have seen the successive rise and fall of a number of concepts, ideas or methods in healthcare QI (QI); tools, methodologies or even terms often rise in popularity, are used for three or four years and then fall out of use or fashion again.¹⁹ It can be seen, for example, that “medical audit” and “clinical audit” did so in the early and mid-1990s and that a similar burst of interest in total quality management (TQM) was followed by a somewhat more sustained interest in continuous QI.¹⁹ In recent years the buzzwords are represented by terms such as lean thinking and six sigma. These results show that it is not the single methodology that makes the difference but how the different QI strategies and tools interact with each other and how they are coherently embedded with the overall organizational strategy.

The present paper, unlike most previous studies in the field, aims to carry out a more comprehensive assessment of the QI systems looking at the interdependence between the different tools and components with the aim to understand which features work and which do not. We try to accomplish this goal by comparing two different systems that, even though in different periods in time, because of external pressures, have gone thorough massive reforms and innovation in their QI systems.

Other studies^20–27 have shown in the past that comparing practices, innovative organizational solutions or QI strategies, adopted in several countries, can actually provide useful insights to both managers and policy makers. Hussey et al.,²⁸ for example, have compared international data on the quality of medical care in Australia, Canada, New Zealand, England, and the United States. They found that each country had at least one area of care where it could learn from international experiences and one area where its experiences could teach others. Ferlie and Shortell²¹ investigated deficiencies in care in the United Kingdom and the United States and proposed four levels of interventions to improve the quality.

Sluijs et al.²² analyzed the relationship of government quality policy and quality management in health care by comparing two countries with different quality policies (Netherlands and Finland). The objective of Wagner et al.’s²⁶ research was to investigate agreement and disparities in the implementation of quality management activities in the Netherlands, Hungary, and Finland. The results indicated that government legislation or financial reimbursement can stimulate the implementation of quality management activities more than voluntary and bottom-up approaches.

Having said that—in order to partially fill the gap found in the scientific literature—the paper aims to address the following two research questions:

What QI strategies have Italian and American hospitals implemented in response to external pressures to improve quality?

What characteristics of the QI systems implemented in the two countries actually facilitate improvement of patient care quality?

In the next sections, we present the methodology, findings, discussion, and policy and managerial implications.

Research methodology

Sampling and survey administration

In the US survey, the sampling included all short-term, nonfederal, general service hospitals that had at least 25 beds in 2004. The survey was sent to the universe of 4237 hospitals. The final sample contained 470 hospitals, a response rate of 11%. The sample is similar to that of the population except for overrepresentation of large hospitals and teaching hospitals and underrepresentation of small hospitals (including nonmetropolitan hospitals and non-profit hospitals). Although this is a less than desirable response rate, as discussed in more detail in the original paper reporting on this survey⁸ and in a recent paper addressing the issue of potential bias from non-response,²⁹ valid results can nevertheless be obtained from surveys with low response rates.

In the Italian survey, the questionnaire was mailed to the quality manager or general manager of all 229 public healthcare providers (teaching hospitals (AOU), independent trust (AO), research hospitals (IRCCS) and local health authorities (LHA)) with a cover letter explaining the purpose of the study, followed a few months later by a telephone call. The survey was conducted between September 2011 and September 2012 with a response rate of 27% (Table 1). Before analyzing the data non-response bias tests were carried out in accordance with Oppenheim.³⁰ A t test was conducted to examine whether there were systematic differences between early and late responses. No statistically significant differences in the means scores were found between early and late responses for any of the variables included in the study, thus revealing that non-response bias was not a problem in this study. All the questionnaires were fully completed and, thus, none of them was excluded.

Table 1.

Respondents’ characteristics in Italy and the US.

	Italy(n = 62) n (%)	USA (n = 470) n (%)
Size (beds)
25–99	–	145 (30.9)
100–399	8 (13)	237 (50.4)
Over 400	54 (87)	88 (18.7)
Hospital type
Independent trust	36 (58)	403 (85.7)
Research hospital	–	–
Teaching hospital	2 (3)	67 (14.3)
Local health authority	24 (39)	–
Location (US)
Central (Midwest)	12 (19)	136 (28.9)
North (Northeast)	42 (68)	66 (14.1)
South (South)	8 (13)	182 (38.8)
- (West)	–	86 (18.2)

Sample hospitals were similar to the population of Italian hospitals in terms of institutional arrangement (independent trust versus LHA versus teaching hospital) except for underrepresentation of hospitals from the south.

Questionnaire

This study employed a survey method to collect data; in particularly, we adapted to the Italian context a questionnaire that was first developed and tested by Cohen et al.⁸ in a previous study carried out in US hospitals.³¹ This American survey was based in part on previously established surveys^17,32 and endorsed by the Health Research and Educational Trust of the American Hospital Association and the US Department of Veterans Affairs Health System.^8,31

As explained in the introduction the two surveys were administered in two different time frames; this circumstance does not undermine the robustness of the study since the objective of the paper is to evaluate the different reactions of the two systems in response to external pressures, that occurred in the years immediately preceding each survey, to improve quality and safety of care provided to patients.

The survey followed the detailed procedures outlined by Dillman.³³ First, the questionnaire was tested in five different pilot case studies and slightly changed to consider the specificities of Italian context regarding, for example, the different hospital organizational structure or the different national and regional institutions in charge of QI policies. The Italian version of the questionnaire was a translation of the original validated American research instrument. Specifically, the questions were compiled in English, and then translated into Italian and back translated in English by two independent translators to ensure that the content of the questions remained unchanged.

The distribution procedures involved (a) sending a pre-notice mail announcing the survey; (b) sending another mail containing the questionnaire³⁴; and (c) three follow-ups by telephone calls. The questionnaire was sent by e-mail to all chief quality officers (CQOs) of Italian public Independent Trusts and LHA. We ensured to respondents that the completed questionnaire was directly received by the researcher and, in doing so, we protected respondent anonymity and reduced evaluation apprehension by assuring respondents that there were no right or wrong answers.³⁵

The questionnaire included the following five sections with a total number of 51 questions. Specifically, the survey was structured as follow:

hospital quality managers and their role (3 questions);

hospital approach to QI (11 questions);

quality tools and methodologies (17 questions);

quality measures and information system (14 questions);

assessment of patient care quality (6 questions).

For most questions, a 5-point scale was employed, which asked respondents to indicate their agreement with statements about the hospital and/or its practices (response categories ranged from strongly disagree to strongly agree). For other questions there was a list of options where the respondents were asked to select the one/s used in their organization. Respondents had also the chance to select the option “I don't know” if he or she were not able to identify an opinion by not answering to a question.

As mentioned in the introduction, we have designed this survey not only to depict the state of QI initiatives in the two countries but also to understand which models and approaches work better in improving quality of care. In order to accomplish this second goal the survey contains two different types of questions:

a first type of question assesses, in objectives terms, if the characteristics of the two systems are consistent with the standards and best practices found in the scientific literature;

a second set of questions assess, on the other hand, the perceptions and evaluations of CQOs about the level of quality of care offered in their organizations and the actual impact of models and tools implemented at their hospital.

The comparison of the actual characteristics of the QI systems with the guidelines commonly found in the quality literature and with the perceptions of CQOs allowed us to draw some conclusions regarding the impact of different QI models and tools reported in the discussion.

Statistical analysis

The analyses performed followed the approach of Cohen et al.⁸ and were conducted in two steps: (i) descriptive and (ii) bivariate analyses.

First, for all the questions of the survey we performed descriptive statistics by calculating percentages and means to explore the trends and type of QI strategies (in terms of tools, methodologies and approaches).

Second, bivariate analyses were carried out in order to examine the association between the nurse-to-bed ratio and the “Quality managers’ assessments of hospital quality” questions. For each of the five questions—“patient care today versus what it should be,” “patient care today versus 3 years ago,” “comfort in having a family member treated without being present to monitor the care,” “the hospital’s QI focus has resulted in major performance gains,” and “patient satisfaction is where it should be”—we combined the top two categories and the bottom three categories.

Statistical significance was interpreted using p values of less than .05. Data was analyzed using SPSS.

Findings

Hospital quality managers and their roles

The first section of the survey investigated the quality managers’ responsibilities and their roles in the organization. First, quality managers were asked to declare whether quality was a priority in their hospital strategic and/or business plans. In both countries CQOs reported QI to be considered an important organizational goal. Nearly all hospitals (98% IT; 93% US) included QI as a priority in their strategic and/or business plans and included quality related goals in their formal budgeting process. In addition, 96% of respondents in the US and 70% in Italy reported having a quality council or other high-level management committee whose primary purpose was to oversee and coordinate QI activities.

Table 2 shows the responsibilities of Italian and US hospital quality managers. In both countries, the roles and responsibilities of these managers extend beyond the traditional focus on quality assurance (as opposed to QI), which consisted primarily of medical audit and other retrospective reviews, to more proactive areas such as institutional accreditation and control of ISO 9000 standards for performance measurement (93% US; 55% IT), patient safety (83% US; 45% IT), patient satisfaction measurement (50% US; 44% IT), and clinical risk management (56%: US; 44% IT).

Table 2.

Quality managers and their roles.

	USA	Italy
Quality manager responsibilities
Quality improvement	100%	94%
Accreditation	93%	63%
Patient safety	83%	45%
Infection control	55%	13%
Patient satisfaction	50%	44%
Clinical risk management	56%	44%
Operations	NA	3%
Clinical pathways	NA	49%
Time of affiliation with the hospital
More than 10 years	55%	21%
Between 3 and 10 years	25%	38%
Less than three years	20%	41%

NA: not available.

However, in Italy, the role of the CQO is more limited than in the US. Hot issues such as patient safety and clinical risk management were among CQO responsibilities in less than half of hospitals, infection control in only 13%, and operations management in 3%.

Table 2 shows the time of affiliation of quality managers. Most US CQOs (80%) reported long affiliations with their hospitals, with 55% having worked there more than 10 years compared to only 21% of Italian respondents.

Hospital approach to QI

In this section the survey asked about the hospitals’ approach to QI (Table 3) and the presence and scope of training programs specifically dedicated to quality (Figure 1).

Table 3.

Hospital approach to quality improvement.

		Total	Agree or strongly agree		Neither agree nor disagree
Question	Country	n	n	%	n	%
Progress toward achieving hospital-wide quality goals is tracked and communicated to clinical staff.	USA	467	445	95	6	1
	Italy	62	60	97	2	3
People and processes are in place to identify, analyze, and act upon all adverse events to prevent future occurrences.	USA	467	440	94	4	1
	Italy	62	58	94	4	6
Corrective action is taken if progress toward achieving hospital-wide quality goals is not adequate.	USA	469	407	87	26	6
	Italy	62	57	92	2	3
People and processes are in place to identify, analyze, and act upon near misses to prevent future occurrences.	USA	468	401	86	29	6
	Italy	62	53	85	4	6
Patient care processes are standardized, where and when appropriate	USA	469	375	80	38	8
	Italy	61	49	80	7	11
There is little coordination of QI efforts across departments and workgroups	USA	466	369	79	50	11
	Italy	62	24	39	38	61
The hospital’s structure and work processes impede coordination across departments and workgroups	USA	468	303	65	74	16
	Italy	60	7	12	52	87
Clinicians involve patients and families in efforts to improve patient care	USA	468	303	65	46	10
	Italy	59	31	53	14	24
The hospital regularly communicates achievement of hospital-wide quality goals to the general public	USA	468	159	34	183	39
	Italy	61	32	52	25	41

Figure 1.

Staff involvement, training, and use of quality improvement (QI) methods.

Managers were asked to indicate their agreement regarding nine statements about the hospital quality practices and processes using a 5-point scale. To describe the phenomena we combined the top two response categories (strongly agree and agree) and the bottom three categories (neither agree nor disagree, disagree, and strongly disagree). Italian and US quality managers agreed or strongly agreed on many aspects of QI related to patient safety: “progress toward achieving hospital-wide QI goals is tracked and communicated to clinical staff” (95% US; 97% IT); “people and processes are in place to identify, analyze, and act upon all adverse events to prevent future occurrences” (94% US; 94% IT); “corrective action is taken if progress toward achieving hospital-wide QI goals is not adequate” (87% US; 92% IT); “people and processes are in place to identify, analyze, and act upon near misses to prevent future occurrences” (86% US; 85% IT); and “patient care processes are standardized, where and when appropriate” (80% US; 80% IT).

Differences in responses to the above questions between the two countries were small. In contrast, 79% of US CQOs perceived “little coordination of QI efforts across departments and workgroups” and 65% thought the hospital’s structure impeded such coordination compared to only 39% and 12% of Italian managers, respectively.

Both in the US and in Italy, relatively less attention has been paid to data dissemination to the public (34% US; 52% IT) and to involvement of patients and families by clinicians in QI efforts (65% US; 53% IT). This circumstance is at odds with what has been reported in the scientific literature.^8,17,19,21 where it has been claimed that empowerment of patients would enhance patient satisfaction and compliance. In addition, the recent US Affordable Care Act mandates that hospitals set up patient advisory groups.

In this section the survey has also examined: (i) the scope of training programs specifically dedicated to quality; and (ii) the use QI methods and tools in the daily work. The data (Figure 1) show significant differences between Italy and the US regarding the typical target of these training programs: senior manager (77% US; 25% IT), administrative staff (70% US; 14% IT), physicians (32% US; 25% IT), and nurses (45% US; 32% IT).

Quality tools and methodologies

Table 4 shows the tools and methods used by Italian and American hospitals in QI projects. We asked quality managers to rate whether a specific quality tool/method was: “used widely or hospital-wide”, “used moderately” or “used minimally”. The most commonly used clinical QI strategies in Italy and the US were related to actions to prevent adverse drug events (70% US; 47% IT), clinical pathways (47% in both cases) and disease- or condition-specific QI projects (50% US; 39% IT). In general, the similarities between the two systems were:

Table 4.

Quality tools and methods.

		Used widely or hospital-wide		Used moderately		Used minimally or not at all
Question	Country	n	%	n	%	n	%
Actions to prevent adverse drug events	USA	322	70	106	23	34	7
Actions to prevent adverse drug events	Italy	29	47	24	39	6	10
Disease- or condition-specific QI projects	USA	231	50	177	38	55	12
Disease- or condition-specific QI projects	Italy	24	39	26	42	11	18
Clinical pathways	USA	219	47	170	37	74	16
Clinical pathways	Italy	29	47	22	35	9	15
Multidisciplinary rounds	USA	159	34	121	26	183	40
Multidisciplinary rounds	Italy	15	24	20	32	21	34
Specific strategies to reduce the number of patients assigned to each nurse	USA	142	31	153	34	162	35
	Italy	13	21	32	52	11	18
Pharmacists placed in patient care units	USA	141	31	84	18	236	51
Pharmacists placed in patient care units	Italy	6	10	5	8	46	74
Chronic disease registries	USA	43	9	112	25	299	66
Chronic disease registries	Italy	18	29	25	40	14	23
Benchmarking with other hospitals	USA	326	71	105	23	31	7
Benchmarking with other hospitals	Italy	6	11	29	51	22	39
Benchmarking within the hospital	USA	297	64	119	26	45	10
Benchmarking within the hospital	Italy	14	25	23	40	20	35
Management walks around to identify quality problems or issues	USA	184	40	144	31	135	29
	Italy	44	75	13	22	2	3
Patient flow improvement strategies	USA	181	39	187	40	95	21
Patient flow improvement strategies	Italy	10	18	18	32	29	51
Work process redesign and reengineering	USA	154	33	195	42	111	24
Work process redesign and reengineering	Italy	23	40	23	40	12	21
Learning best practices of other industries	USA	154	33	162	35	146	32
Learning best practices of other industries	Italy	10	15	28	42	29	43
Profiling of individual provider performance	USA	137	30	140	31	180	39
Profiling of individual provider performance	Italy	5	9	4	7	45	83
Patient advisory groups	USA	23	5	86	19	349	76
Patient advisory groups	Italy	9	16	23	40	25	44

Clinical governance approaches and tools, such as disease-specific QI projects and clinical pathways. There is widespread awareness that clinical decision making must be supported by protocols and guidelines.

Drug management as a source of medical errors. In both countries, however, the role of pharmacists in QI is relatively small.

Adoption rates of QI techniques widely used in the manufacturing industry are very low: in Italy, only 6% of hospitals used statistical process control (SPC), 5% used six sigma and 10% used lean thinking techniques. Similar results were found in American hospitals: 19% SPC, 6% six sigma, and 6% Lean.

QI initiatives aimed to reduce nurses’ workloads were consistent in both countries. The shortage of nurses in both systems is of great concern^36–38 particularly as the linkage between nurses’ shortage and quality of care has been demonstrated by several studies.^27,39–41 Particularly the work of Aiken et al.²⁷ has clearly shown that the association between nursing and the quality and safety of hospital care is remarkably similar across Europe and the US.

In US hospitals, the nurse-to-patient ratios were significantly associated with two of the quality manager assessment questions. For the “patient care today compared with what it should be” question, the mean ratio for hospitals with high ratings (above or well above expectations) was 1 nurse to 5.0 patients while the mean ratio for hospitals with lower ratings was 1 nurse to 5.4 patients (p < .01). For the “patient satisfaction” question, the mean ratios were similar (1 nurse to 5.0 patients in hospitals with high ratings versus 1 nurse to 5.4 patients in those with lower ratings, p < .01). Nurse vacancy rates showed statistically significant relationships with three of five assessment questions. Mean vacancy rates for hospitals with high versus low ratings on the “patient care today versus what it should be” question were 5.7% and 8.3%, respectively (p < .001); mean rates for hospitals with high versus low ratings on the “family member” question were 6.2% and 9.6% (p < .001); and rates on the “patient satisfaction” question were 5.9% and 7.8% (p < .01).

In Italian hospitals, the nurse-to-bed ratios were significantly associated with two of the quality manager assessment questions. For the “patient care today compared with what it should be” question, the mean ratio for hospitals with high ratings (above or well above expectations) was 1.6, while the mean ratio for hospitals with lower ratings was 1.5 (p < .01). For the “Hospital focus on improving patient care has resulted in major performance gains” question, the mean nurses-to-beds ratios were 1.7 in hospitals with high ratings vs. 1.5 in those with lower ratings, p < .01).

A major difference between US and Italian hospitals was the use of data for benchmarking purposes. Italian hospitals typically were more reluctant to do benchmarking analysis both within the hospital and with other organizations. In contrast, for the US, benchmarking with other hospitals (70%) and benchmarking within the hospital (64%) were the top two QI activities widely used.

The survey also explored two other aspects of QI activities:

the influence of external institutions on hospital QI strategies; and

the adoption of health information technology.

Quality managers were asked to characterize on a 5-point scale from very negative to very positive the influence of different nationally prominent QI initiatives and organizations on their hospitals’ QI efforts. In the US, the CMS’s QI Organizations and the IHI’s 100,000 Lives Campaign were, respectively, the organizations and initiatives most often cited (89%, 88%) by respondents as having had very positive or somewhat positive influence on their hospitals. In Italy, the Regions and the Ministry of Health were the organizations most often cited (respectively, 76%, 74%) as having had very positive or somewhat positive influence on their hospitals.

Figure 2 summarizes the results regarding the adoption of health information technology (HIT) tools in the US and Italy. In the US, quality managers reported that medication management systems (e.g. Pyxis) and picture archiving and communications systems (PACS) were the most commonly adopted HITs, having been implemented hospital-wide in 69% and 51% of hospitals, respectively. In Italy, the most commonly implemented information technologies were PACS (68%) and inpatient electronic medical record systems (EMRs) (37%). The data show that, despite intense debate about the possible impact of information technology on healthcare delivery, many information technologies had not spread widely in either country at the time of the surveys. It should be noted, however, that at least in the US the adoption of HITs, especially EMRs, has increased substantially since this time.⁴²

Figure 2.

Health information technology.

Quality measures and information systems

Table 5 reports the quality measures most commonly reported to hospital boards of trustees, senior managers, department managers, clinical staff, and others employees in the US and Italy.^b In Italy, the measures most commonly reported to boards of trustees included: patient satisfaction results (85% of hospitals); QI project results (79%); overall hospital risk-adjusted mortality rates; hospital-acquired infection rates (66%); and waiting times for outpatient clinic appointments (92%). In US hospitals, the most commonly reported measures were: patient satisfaction results (95%); QI project results (89%); hospital-acquired infection rates (84%); adverse events (82%); and medication error rates (81%).

Table 5.

Quality of care measures and reporting.

		% of hospitals in which measures are reported to …					% of hospitals in which the measure is not monitored
		Board of trustees	Senior managers	Department managers	Clinical staff	Other employees
Staff	Country	%	%	%	%	%
Patient satisfaction results	USA	95	97	97	90	66	1
Patient satisfaction results	Italy	85	32	74	61	58	10
QI project results	USA	89	97	95	83	50	1
QI project results	Italy	79	40	68	48	47	16
Hospital-acquired infection rates	USA	84	92	89	78	33	2
Hospital-acquired infection rates	Italy	66	11	53	39	31	27
Adverse events	USA	82	93	83	63	22	0,7
Adverse events	Italy	60	18	37	35	27	31
Medication error rates	USA	81	91	89	74	23	2
Medication error rates	Italy	44	10	39	34	32	37
Overall hospital risk-adjusted mortality rates	USA	70	77	50	39	15	13
Overall hospital risk-adjusted mortality rates	Italy	66	19	39	23	18	26
Near-miss events	USA	56	84	75	52	22	6
Near-miss events	Italy	42	13	35	37	37	50
Unplanned hospital readmission rates	USA	46	71	57	40	16	16
Unplanned hospital readmission rates	Italy	39	8	39	16	8	53
Disease-specific mortality rates	USA	41	59	46	38	12	30
Disease-specific mortality rates	Italy	40	6	40	21	13	47
Emergency department diversion rates	USA	40	71	59	36	12	23
Emergency department diversion rates	Italy	45	10	35	18	15	42
Individual physician profiles of quality performance	USA	37	50	11	28	14	17
Individual physician profiles of quality performance	Italy	16	8	16	10	8	76
Waiting times for procedures, tests, and test results	USA	35	68	77	53	19	23
Waiting times for procedures, tests, and test results	Italy	92	37	74	40	40	6
Procedure-specific mortality rates	USA	32	51	38	31	11	35
Procedure-specific mortality rates	Italy	18	3	19	11	6	69
Unplanned ICU readmission rates	USA	19	38	46	30	5	43
Unplanned ICU readmission rates	Italy	15	5	19	11	6	71

Quality measures that went unmonitored by more than 20% of US hospitals included: waiting times for outpatient clinic appointments (45%); unplanned ICU readmission rates (43%); procedure-specific mortality rates (35%); disease-specific mortality rates (30%); and emergency department diversion rates (23%). In Italy, more than 20% of hospitals did not measure: overall hospital risk-adjusted mortality rates (26%); adverse events (31%); medication error rates (37%); emergency department diversion rates (42%); disease-specific mortality rates (47%); near miss events (50%); unplanned hospital readmission rates (53%); procedure-specific mortality rates (69%); unplanned ICU readmission rates (71%); and individual physician profiles of quality performance (76%).

Italian hospitals exhibit a lower rate of reporting quality measures, many of which are reported only to the top managers. This restricted information flow is at odds with QI principles. Also, some indicators are not collected, such as: individual physician performance; errors in drug prescription; and patient flow indicators (e.g. “unplanned hospital readmission rates” or “ICU bumps”).

Italian hospitals pay greater attention than their American counterparts to waiting times. This may reflect, in part, greater concern about equity issues in Italy than in the US Furthermore, in Italy, waiting time is a critical issue since it represents a way of “rationing” the demand for care, which is free at the point of consumption.

Assessment of patient care quality

The last section of the questionnaire aimed to evaluate the performance of QI systems in hospitals (Table 6). The CQOs were asked to evaluate:

Table 6.

Quality managers' assessments of hospital quality.

		Well below expectations		Below expectations		Meets expectations		Above expectations		Well above expectations
Question 1	Country	n	%	n	%	n	%	n	%	n	%
Patient care today compared with what it should be	USA	–	–	85	18	207	45	149	32	19	4
Patient care today compared with what it should be	Italy	–	–	10	16	49	80	3	–	–	–
		Much worse		Worse		About the same		Better		Much better
Question 2	Country	n	%	n	%	n	%	n	%	n	%
Patient care today compared with 3 years ago	USA	1	–	13	3	43	10	206	48	167	39
Patient care today compared with 3 years ago	Italy	–	–	–	–	21	36	30	53	6	11
		Strongly disagree		Disagree		Neither agree nor disagree		Agree		Strongly agree
Question 3	Country	n	%	n	%	n	%	n	%	n	%
Comfortable having family member treated without being present	USA	13	3	63	14	67	15	197	43	121	26
	Italy	1	2	1	2	15	26	30	53	10	18
Hospital focus on improving patient care has resulted in major performance gains	USA	4	1	21	5	63	14	247	54	124	27
	Italy	–	–	1	2	7	12	40	70	9	16
Patient satisfaction is where it should be	USA	33	7	200	43	99	21	110	24	19	4
Patient satisfaction is where it should be	Italy	1	2	2	3	5	8	45	75	7	12

the quality of patient care provided at their hospital; and

the impact of the different QI initiatives put in place.

CQOs were asked to rate patient care today at their hospitals compared with what they thought it should be, using a 5-point scale ranging from well below expectations to well above expectations. In the US, 36% of quality managers said that patient care was either above or well above their expectations, while in Italy only 3% responded that way, with 80% responding that patient care met their expectations.

When asked to rate patient care today compared with three years ago, using a 5-point scale ranging from “much worse” to “much better,” 87% of American CQOs and 64% of Italian CQOs, respectively, said that care was better or much better compared to the past. Only 10% of American managers versus 36% of Italian managers reported that patient care today is the same compared to three years ago.

About the same proportion of managers in the US and Italy (68% vs. 71%) agreed or strongly agreed that they would feel comfortable having a family member treated at the hospital without being present to monitor the care. However, only 28% in the US agreed or strongly agreed that patient satisfaction was at the level where it should be compared to 87% of respondents in Italy who felt this way. Quality managers in both countries perceived QI activities as having produced positive or very positive impacts on several organizational areas in their hospitals:

patient care outcomes (97% in US; 98% in Italy);

staff motivation and skills (86% in US; 100% in Italy);

productivity (70% in US; 93% in Italy); and

physician–nurse relationships (68% in US; 73% in Italy).

Discussion

As mentioned in the introduction, Italian and American hospitals, although in two different time frames, have both been urged by external forces to extensively redesign their QI strategies in order to keep up with a general expectation (from policy makers and citizens) to improve the overall quality of care.

In both nations quality issues have gained traction among hospital managers such that: (1) QI is explicitly mentioned in hospital strategic plans; (2) quality-related goals are systematically included in the budgeting process; (3) clinicians are becoming more aware of the necessity to change processes and behaviors to improve the safety and quality of healthcare delivery; and (4) there is shared agreement on the critical importance of standardizing clinical decision making and patient care processes through the adoption of clinical pathways.

However, the study shows differences between the two systems that offer interesting material to draw some conclusions on which models and tools are effective in improving quality of care. As explained in detail in the methodology session, we support these conclusions in two ways: we have compared the features of the QI systems adopted in the two countries with (i) the standards and best practices present in the scientific literature and; (ii) the evaluations expressed by CQOs.

First, on average, American hospitals have a less fragmented and more system-wide approach to QI, specifically:

the scope of responsibilities for quality managers is wider (Cfr. Table 1);

quality metrics refer to different dimensions (such as patient safety and patient flows logistics) and are diffused and communicated hospital-wide (Cfr. Table 4); and

QI training and programs target a wider spectrum of categories, including, for example, the administrative staff (Cfr. Figure 1).

We conclude that American QI systems are, in this perspective, better off since there is a strong body of literature^43–49 providing evidence that a system-wide and integrated approach to quality has a significant impact on overall organizational performance.

It is worth noting, in this benchmarking analysis of the two systems, that the siloed approach found in some Italian hospitals might be explained by the mainly public nature of the Italian healthcare system; in fact, many features of Italian healthcare management practices are inspired by a bureaucratic approach to dealing with new issues and problems through interventions in the formal organizational structure. For example, when the problem of medical litigation arose, the Minister of Health recommended that organizations set up a specific organizational unit dedicated to dealing with this area (see, for example, the National Health Plan 2009–2011). The same is true for operations management, which is an area of growing importance.⁵⁰ Some Italian regions (e.g. Tuscany and Lombardy) have required hospitals to set up offices dedicated to patient flow management. This bureaucratic thinking (“we have a problem, let’s set up an office to deal with it”) introduces the risk of creating silos in the overall organization resulting in poor communication among different units and adoption of only a partial approach to solving problems.

On the other hand, Italian hospital organizational structure designed around the so-called clinical directorates (CDs) seems, based on our findings, to work well. Italian CQOs state, on average, that they not experience problems of coordination of QI efforts across departments and workgroups and they do not perceive that the current hospital’s structure and work processes impede coordination across departments and workgroups (Table 2). Italian CDs are “intermediate organizational arrangements through which defined parts of larger hospitals’ health services are managed” and have been part of the Italian National Health Service (INHS) by law since the major reform of, 1992.^51,52 Despite initial resistance to change, the new organizational model, when actually applied,^c has been effective in: (1) improving hospital clinical governance through better integration of clinical pathways; and (2) exploiting economies of scale and scope, by sharing resources (beds, OR, ambulatories, etc.), technologies, and ancillary staff.⁵¹ Thus, CDs appear to mitigate some of the limitations of the bureaucratic approach.

Finally, in both countries the shortage of nurses is considered a major impediment to the improvement of patient safety and quality. Data from the surveys clearly show a significant positive correlation between nurse-to-patient ratios and quality managers’ perceptions about the levels of safety and quality.

Policy and managerial implications

In both countries the concept of QI is now well accepted and both surveys show a cohesive organizational culture that embraced QI as a strategic priority. For health care managers and policy makers, it is important to understand which organizational characteristics actually favor QI and safety. In this sense the comparative analysis of the two countries provides useful insights.

Our study, as well as the literature on QI, suggests several criteria for policymakers and executives to use in selecting improvement strategies. First, given that quality is a multi-dimensional concept,^3,4 it is essential that a QI system, if it is to be comprehensive, include strategies that address all dimensions of quality. The most widely recognized dimensions of quality are represented by the IOM’s six aims: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity.¹³ Furthermore, we believe that a healthcare organization should develop a strategic plan specifying important dimensions of quality and design improvement strategies that are aligned with these priorities.⁵³ A second criterion to consider in selecting improvement strategies is the impact the strategy is likely to have on quality of care. For example, the impact of nursing on quality was highly ranked in our study, thus calling for strategies aiming to achieve better patient-to-nurse ratios and for innovations in health care supply chain⁵⁴ management that reduce nursing time dedicated to logistical activities. Third, policymakers and executives should consider implementing strategies that address areas in which performance is lacking. In our study, these included extended responsibilities such as clinical risk management for QI offices and widespread dissemination of performance feedback among Italian hospitals, organizational structures to facilitate coordination among US hospitals, and involvement of patients in QI strategy development in both countries.

A final relevant message of our study is that, in order to be successful, any QI methodology or tool must be highly integrated with the other organizational components. For example, the Italian CDs model (indicated in the survey as an effective organizational design to favor clinical governance and multidisciplinary collaboration) must be managerially sustained through, for example, the appointment of administrators and operations managers supporting the role of the Chair, and developing grouping of clinical units consistent with these goals. The hospital information architecture (more advanced and complete in the case of American hospitals) must be aligned to the new managerial models, such as the patient-centered healthcare model in Italy and the accountable care model in the US, that call for a more integrated approach to care where assets and resources are shared by the different hospital clinical specialties. A limitation of the study is that the data are self-reported, which is inherent in surveys conducted by questionnaire. A second limitation is the low response rates in both surveys. However, while a survey with a low response rate will have less precision but not necessarily greater bias, it still can yield informative findings; this is particularly true if there are large numbers of respondents, as is the case in these two surveys (see Meterko et al.²⁹). A third limitation is that the questionnaires were sent only to top hospital managers responsible for QI and it is unknown whether their perceptions and opinions were shared by clinicians and patients in their institutions. Nevertheless, the approach and methods used in this study are sufficiently robust to offer a broad picture of the QI strategies actually used in Italian and American hospitals and the similarities and differences that exist between the two systems, thus providing useful insights to managers and policy makers.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Stefano Villa has been recipient of a Fulbright Research Scholarship 2017-2018.

Notes

References

Mattke

Epstein

Leatherman

The OECD health care quality indicators project: history and background. Int J Qual Health Care 2006; 18: 1–4

Blaise

Kegels

A realistic approach to the evaluation of the quality management movement in health care systems: a comparison between European and African contexts based on Mintzberg's organizational models. Int J Health Plann Manag 2004; 19: 337–364

Kunkel

Westerling

Different types and aspects of quality systems and their implications: a thematic comparison of seven quality systems at.

Univ Hosp Health Policy 2006; 76: 125–133

Macinati

MS.

The relationship between quality management systems and organizational performance in the Italian National Health Service. Health Policy 2008; 85: 228–241

Makai

Klazinga

Wagner

et al . Quality management and patient safety: survey results from 102 Hungarian hospitals. Health Policy 2009; 90: 175–180

Pejović

Filipović

Tasić

et al . Towards medicines regulatory authorities' quality performance improvement: value for public health. Int J Health Plann Manag 2014; 4: 44–30

Simou

Koutsogeorgou

Effects of the economic crisis on health and healthcare in Greece in the literature from 2009 to 2013: a systematic review. Health Policy 2014; 115: 111–119

Cohen

Restuccia

Shwartz

et al . A survey of hospital QI activities. Med Care Res Rev 2008; 65: 571–595

McGlynn

Asch

Adams

et al . The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348: 2635–2645

10.

Wachter

RM.

The end of the beginning: patient safety five years after “To Err is Human. Health Affairs 2004; 23: 534–545

11.

Leape

Berwick

DM.

Five years after to err is human: what have we learned?.

JAMA 2005; 293: 2384–2390

12.

Ross

Cha

Epstein

et al . Quality of care for acute myocardial infarction at urban safety-net hospitals. Health Affairs 2007; 26: 238–248

13.

Institute of Medicine 2000. To err is human: Building a safer health system. In: Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press, 2001.

14.

Galvin

Milstein

Large employers' new strategies in health care. N Engl J Med 2002; 347: 939–942

15.

Heget

Bagian

Lee

et al . System innovation: veterans health administration national center for patient safety. Jt Comm J Qual Patient Saf 2002; 28: 660–665

16.

Kizer

KW.

Establishing health care performance standards in an era of consumerism. JAMA 2001; 286: 1213–1217

17.

Shortell

SM.

National survey of hospital efforts to improve quality. Evanston, IL: Northwestern University, 1991.

18.

Cicchetti

Coretti

Iacopino

Italian chapter . In: Braithwaite,et al. (eds) Healthcare reform, quality and safety. Burlington, VT: Ashgate, 2015.

19.

Walshe

Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. Int J Qual Health Care 2009; 21: 153–159

20.

Cacace

Ettelt

Mays

et al . Assessing quality in cross-country comparisons of health systems and policies: towards a set of generic quality criteria. Health Policy 2013; 112: 156–162

21.

Ferlie

Shortell

SM.

Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q 2001; 79: 281.

22.

Sluijs

Outinen

Wagner

et al . The impact of legislative versus non-legislative quality policy in health care: a comparison between two countries. Health Policy 2001; 58: 99–119

23.

Groene

Klazinga

Wagner

et al . Investigating organizational QI systems, patient empowerment, organizational culture, professional involvement and the quality of care in European hospitals: the “Deepening our Understanding of QI in Europe (DUQuE)” project. BMC Health Serv Res 2010; 10: 281.

24.

Lee

Komatsu

Zhang

et al . Comparison of the hospice systems in the United States, Japan and Taiwan. Asian Nurs Res2010; 4: 163–173

25.

Legido-Quigley

Panteli

Brusamento

et al . Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health Policy 2012; 107: 146–115

26.

Wagner

Gulácsi

Takacs

et al . The implementation of quality management systems in hospitals: a comparison between three countries. BMC Health Serv Res 2006; 6: 50.

27.

Aiken

Sloane

Sochalski

Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002; 16: 1987–1993

28.

Hussey

Anderson

Osborn

et al . How does the quality of care compare in five countries? Health Affairs 2004; 23: 89–99

29.

Meterko

Restuccia

Stolzmann

et al . Response rates, non-response bias, and data quality: results from a national survey of senior healthcare leaders. Public Opin Q 2015; 79: 130–144

30.

Oppenheim

AN.

Questionnaire design, interviewing and attitude measurement. London: Pinter Publishers, 1992.

31.

Restuccia

Mohr

Meterko

et al . The association of hospital characteristics and QI activities in inpatient medical services. J Gen Intern Med 2014; 29: 715–722

32.

Leapfrog Group. Hospital quality and safety survey, 2005. Available at: http://leapfrog.medstat.com/(aud0023jrc3sa245wvvd3vrp)/index.aspx#refmaterials (accessed September 2016)

33.

Dillman

DA.

Mail and internet surveys. New York, NY: JohnWiley and Sons, 2000.

34.

Yammarino

Skinner

Childers

TL.

Understanding mail surveyresponse behavior. Public Opin Q 1991; 55: 613–639

35.

Podsakoff

Organ

DW.

Self-reports in organizationalresearch: problems and prospects. J Manag 1986; 12: 531–544

36.

De Pietro

La carenza di infermieri: condizioni del mercato del lavoro e risposte di management, 565-90. In: L’aziendalizzazione della sanità in Italia, Rapporto OASI 2006. Egea: Milano, 2006.

37.

Duggirala

Rajendran

Anantharaman

RN.

Provider-perceived dimensions of total quality management in healthcare. Benchmarking 2008; 15: 693–722

38.

Ep Koubaa Eleuch

Healthcare service quality perception in Japan. Int J Health Care Qual Assur 2011; 24: 417–429

39.

Atinga

Abekah-Nkrumah

Domfeh

KA.

Managing healthcare quality in Ghana: a necessity of patient satisfaction. Int J Health Care Qual Assur 2001; 24: 548–563

40.

Litvak

Buerhaus

Davidoff

et al . Managing unnecessary variability in patient demand to reduce nursing stress and improve patient safety. J Qual Patient Saf 2005; 31: 330–338

41.

Tafreshi

Pazargadi

Abed Saeedi

Nurses' perspectives on quality of nursing care: a qualitative study in Iran. Int J Health Care Qual Assur 2007; 20: 320–328

42.

Henry

Pylypchuk

Searcy

et al . Adoption of electronic health record systems among US non-federal acute care hospitals: 2008–2015. The Office of National Coordinator for HIT, 2016. healthit.gov

43.

Radnor

Implementing lean in health care: making the link between the approach, readiness and sustainability. Int J Ind Eng Manag 2011; 2: 1–12

44.

Mazzocato

Thor

Bäckman

et al . Complexity complicates lean: lessons from seven emergency services. J Health Organ Manag 2014; 28: 266–288

45.

Marsilio

Torbica

Villa

Healthcare multidisciplinary teams: the sociotechnical approach for an integrated system-wide perspective. Health Care Manag Rev 2017; 42: 315–327

46.

Pakdil

Harwood

TN.

Factors that influence efficiency in performing ENT cases: a qualitative and quantitative analysis. J Med Syst 2005; 29: 285–301

47.

Alesani

Lega

Villa

La reingegnerizzazione dei processi nelle aziende pubbliche nell’ottica del “valore”. Il caso dei processi amministrativi di una ASL. Azienda Pubblica 2007; 3: 431–464

48.

Duggirala

Rajendran

Anantharaman

RN.

Patient-perceived dimensions of total quality service in healthcare. Benchmarking 2008; 15: 560–583

49.

Bakan

Buyukbese

Ersahan

The impact of total quality service (TQS) on healthcare and patient satisfaction: an empirical study of Turkish private and public hospitals. Int J Health Plann Manag 2014; 29: 292–315

50.

Villa

Prenestini

Giusepi

A framework to analyze hospital-wide patient flow logistics: evidence from an Italian comparative study. Health Policy 2014; 115: 196–205

51.

Lega

The rise and fall(acy) of clinical directorates in Italy. Health Policy 2008; 85: 252–226

52.

Cicchetti

Baraldi

La diffusione del modello dipartimentale nel Ssn: solo un fatto formale?

Organizzazione Sanitaria 2001; 1: 71–81

53.

Donabedian

The criteria and standard of quality. Ann Arbor, MI: Health Administration Press, 1983.

54.

VanDeusen

Holmes

Cohen

et al . Transformational change in health care: an organizational model. Health Care Manag Rev 2007; 32: 4–14