Defining,Measuring,and Improving Surgical Quality: Beyond Teamwork and Checklists to Systems Redesign and Transformation

Abstract

Background:

Surgical complications are multifactorial but often are attributable to deficiencies in the quality of care. This review examines how quality is defined in surgery, the modalities employed to measure quality, and the approaches to improving the quality of surgical care. Beyond developing a hospital environment supportive of organizational learning, the next generation of surgical performance improvement will include broader, more innovative approaches. These ideas will create partnerships among patients, clinicians, industry, the arts, hospital leaders, and other sectors to look for ways to reinvent the system rather than simply to make a better hospital.

Methods:

Review of pertinent English-language literature on surgical quality, definitions of quality, quality measures, performance improvement, and organizational learning in health care.

Results:

Medical care should be safe, effective, patient-centered, timely, efficient, and equitable, as defined by the Institute of Medicine core values for health care quality. There is substantive lack of agreement as to how to measure the quality of care. Although the goal of each measurement system is to give patients the ability to compare hospitals nationally, most of the methodologies measure widely different aspects of hospital care, resulting in conflicting illustrations of institutional performance and confounded decision making for patients and for purchasers of healthcare services and insurance.

Conclusions:

The best pathway for surgical quality and performance improvement includes the application of systems engineering and innovation to determine ways to do better what we do currently, and to improve the present system while developing ideas for better delivery of high-quality care in the future.

Thirty percent of all direct health care costs are caused by defects in care—misuse, underuse, overuse, and waste. Low-quality care costs $1,900–$2,250 per covered employee-year [1]. Research funding and efforts have gone toward understanding disease mechanisms and identifying therapies. Until recently, however, little attention has been given to determining ways to deliver the best therapy effectively and efficiently [2].

Much of the excess length of stay, cost, and death caused by patient safety events in the hospital can be attributed to post-operative complications [3]. In a recent analysis of data from the Veterans Health Administration (VHA) National Surgical Quality Improvement Project (NSQIP), the occurrence of a complication 30 days in duration reduced the median patient survival rate by 69%, independent of preoperative patient risk [4,5]. Among the 45 million hospital-based surgical cases performed annually in the United States, an estimated 1.5 million serious surgical complications occur, resulting in 200,000 surgery-related deaths. There are multiple contributors to poor outcomes, including surgeon inexperience, excessive workload, poor communication among staff, inadequate hospital systems, and insufficient technology [6,7]. Surgical complications are multifactorial but often are attributed to deficiencies in quality. How is quality defined in surgery, and how can it be measured? How can we manage risk, decrease variability, create consistency, and improve the delivery of care?

One Definition of High-Quality Health Care

The Institute of Medicine (IOM) has provided six core values for health care that could be used to define quality broadly. Medical care should be safe, effective, patient-centered, timely, efficient, and equitable [8]. The IOM identifies the gap between best clinical evidence and actual implementation of the evidence as the Quality Chasm. Chassin [9] describes the forces preventing implementation of best practices as overuse—providing a health service when its risk of harm exceeds its potential benefit; under-use—failing to provide an effective service when it would have produced favorable outcomes; and misuse—avoidable complications of appropriate health care.

How Should Quality Be Measured?

There is little agreement about how to measure the quality of care. The American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) and the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) are two nationally recognized methods for determination of clinical outcomes. The ACS-NSQIP uses a validated, risk-adjusted methodology to compare observed with expected outcomes of a variety of general and vascular operations [10]. The AHRQ-PSI are a set of computer algorithms designed to identify potential adverse events during the hospital stay using secondary diagnosis and procedure codes from discharge data [11]. In a compelling study, Cima et al. compared these two processes for their ability to identify the same postoperative complications [12]. The authors, citing a similar comparative study published several years prior [13], in this instance used a single institution's ACS-NSQIP data set encompassing 7,606 hospital in-patients who had undergone general surgical or vascular operations between 2006 and 2009. Comparing AHRQ-PSI and ACS-NSQIP definitions, they were able to link seven types of inpatient adverse events: Infections, sepsis, bleeding, pulmonary embolism/deep vein thrombosis, respiratory failure, wound disruption, and acute kidney injury (Table 1). At the conclusion of the review, 564 patients (7.4%) had an ACS-NSQIP-adverse event identified during the hospitalization. An AHRQ-PSI was identified in 268 patients (3.5%). Only 159 patients had postoperative complications identified by both methods. The authors cited a number of reasons the two systems, both with inherent imperfections, differed in their ability to detect post-operative complications. Probably the most important reason ACS-NSQIP and AHRQ-PSI differ so greatly as quality assessment tools lies in their vastly different data collection methodologies. The ACS-NSQIP uses nurses trained to perform manual chart abstraction based on clinical documentation and objective data in the medical record, with defined criteria for complications. The AHRQ-PSI uses International Classification of Diseases—Ninth Revision, Clinical Modification (ICD9-CM) administrative billing data to associate diagnosis or procedure codes with adverse events. This study found that, although AHRQ-PSI is used frequently for public reporting of hospital quality, it failed to detect a number of clinically meaningful complications while simultaneously identifying some post-operative events that had little clinical relevance.

Table 1.

Definition of Groups for Comparing Specific Agency for Healthcare Research and Quality Patient Safety Indicators with American College of Surgeons National Surgical Quality Improvement Program Events

Event	AHRQ-PSI	ACS-NSQIP event
Infections	7: Selected infections due to medical care	Deep incisional surgical site infection, organ space surgical site infection, urinary tract infection
Sepsis	13: Postop sepsis	Sepsis or septic shock
Bleeding	9: Postop hemorrhage or hematoma	Bleeding with transfusion ≥4 units
Pulmonary embolism or deep vein thrombosis	12: Postop pulmonary embolus or deep vein thrombosis	Deep venous thrombosis requiring therapy or pulmonary embolism
Respiratory failure	11: Postop respiratory failure	Ventilator >48 h
Wound disruption	14: Postop wound dehiscence	Wound disruption
Renal failure	10: Postop physiologic and metabolic derangement	Acute kidney injury or progressive renal insufficiency

Adapted from Cima RR, Lacklore KA, Nehring SA, et al. How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) postoperative adverse events at a single institution. Surgery 2011;150:943–949. For definitions, see text.

Patients now have a wide variety of resources in print and online to compare hospital quality. The challenge for the patient/consumer is to determine which of the existing hospital quality ratings systems paints the most accurate picture of their hospital's complication rates and of the ability of the institution to treat specific diseases. The Centers for Medicare and Medicaid Services (CMS) introduced the first federally sponsored hospital quality scorecard—Hospital Compare—in 2005 [14]. Halasyamani and Davis evaluated Hospital Compare scorecards for 2004 and 2005 and contrasted them with the performance ratings of the widely recognized U.S. News and World Report's “Best Hospitals” ranking [15,16]. Those investigators calculated composite scores for core measures related to acute myocardial infarction (AMI), congestive heart failure (CHF), and community-acquired pneumonia (CAP) among 2,165, 3,130, and 3,462 hospitals, respectively. Fewer than 50% of the Best Hospitals for cardiac care rated in the top quartile of Hospital Compare scores for AMI and CHF. Fewer than 15% of the Best Hospitals for respiratory care ranked in the top quartile of Hospital Compare for treatment of CAP. The lack of agreement between the ratings systems was attributed to the marked differences in methodologies the systems employ. Hospital Compare scorecards focus on the delivery of disease-specific, evidence-based practices for acute medical conditions from admission to discharge. In contrast, the Best Hospitals system focuses on mortality data and the reputation of hospitals across a variety of clinical settings, including a number of clinical conditions in which core quality measures have not yet been developed. Best Hospitals also includes factors related to nursing and technology capability that may be appealing to consumers. Although the goal of each system is to give patients the ability to compare hospitals nationally, Hospital Compare and Best Hospitals measure widely different aspects of hospital care. Prominent, publicly reported hospital scorecards can paint a conflicting picture of institutional performance and confound decision making for patients and for purchasers of healthcare services and insurance.

How Can Quality Be Improved?

In order to improve, clinicians need to have fundamental, current knowledge of the clinical conditions they are treating. They also need feedback to know how they are doing, with avenues to initiate improvements once a need has been identified. Lack of agreement as to how to measure quality of care remains one of the primary barriers to performance improvement [8]. There are many approaches to improving the quality of surgical care, each of which possesses important positive and negative characteristics. Berenholtz and Pronovost [2] outlined several of the better-known recommendations, including evidence-based medicine (EBM) and evidence-based clinical practice guidelines, professional education and development, assessment and accountability, patient-centered care, and total quality management. The benefits and barriers the authors list for each approach are summarized in Table 2.

Table 2.

Approaches to Improving Quality of Care

Method	Premise/benefits	Barriers
Evidence-based medicine (EBM) and evidence-based clinical practice guidelines	Propose to improve performance by incorporating best available evidence with patients' values and providers' preferences. Best examples are the ventilator wean and sedation titration protocols adopted by many intensive care units (ICUs)	Not all providers know the guidelines/protocols exist; guidelines often occur with specialized areas of care; provider disagreement; inability of providers to implement the guidelines; lack of clear, consistent data demonstrating the impact of EBM on outcomes
Professional education and development	Depends largely on continuing medical education (CME)	CME approach alone is not effective; simulation, small groups, and interactive learning are the next phases and are more effective; professional education can be costly and resource intensive
Assessment and accountability	Assumes providers will change their activity when performance is compared and publicly reported; insurers, regulators, and accreditors like this approach	Providers have concerns about validity of the data and accuracy; overall performance has changed little as a result
Patient-centered care	Empowers patients to participate in medical decision making; patient and family satisfaction surveys, needs assessments, risk tables	Patients differ in their desire to participate; complexity of care can be overwhelming to decision makers; not clear as to whether satisfaction surveys result in performance changes
Total quality management and continuous quality improvement	Focuses on the health system's organizational characteristics to create changes in quality; measure performance, make changes, evaluate impact of the change	Depends on cooperative effort by administrators and providers for an integrated approach to performance measurement and improvement

Adapted from Berenholtz S, Pronovost PJ. Barriers to translating evidence into practice. Curr Opin Crit Care 2003;9:321–325.

How do clinicians and hospital leaders sift through myriad information aimed at improving clinical care? Which method is ideal, and how should the decision makers utilize scarce resources for the most valuable return? The authors assert that quality improvement efforts will likely require incorporation of several approaches concurrently in order to be successful. The approach to quality improvement must be multifaceted, because quality as a construct can be interpreted in many ways. A working partnership among clinicians and hospital leaders is vital to determine which combination of approaches would be appropriate for the patient population served and the available resources of personnel and technology.

There are fundamental characteristics that every system has to possess in order to implement quality improvement initiatives: A culture that supports teamwork, the ability to simplify processes (overcoming complexity), and independent redundancy [2]. Redundancy includes utilization of checklists and validated protocols plus incorporation of independent caregivers. A potential benefit of incorporation of independent caregivers or rotation of caregivers is a decrease in the incidence of “escalation of commitment”—the human tendency to continue to expend resources in pursuit of a failing course of action [17 –19]. Escalation of commitment, a concept well developed in the business literature, can affect the quality of decision making at every level within a health care institution, from the hospital leadership to bedside clinicians [20]. Particularly in a critical care setting, escalation behavior by clinicians can lead to significant cost increases with marginal or no clear benefit to patients (Awad, data not yet published). Redundancy may help to avoid over-commitment and increase the quality of decision making by looking at the strategy from the perspective of an outsider.

Key to improvement of surgical quality is involvement of the surgeons themselves as part of an integrated approach that includes an interdisciplinary team. That team focuses on specific processes of care associated with improved outcomes. They review and apply the available evidence, and then they measure performance and give feedback [2]. These steps represent some of the tools of organizational learning, which should be in place for any organization that aims to improve quality. Other tools include processes for knowledge acquisition and sharing and clear understanding of the institution's core competencies and resources. A learning organization has the ability to utilize information technology, not only for data collection, but also for decision support, and has the ability to embed knowledge in everyday practices. Organizational learning in health care also includes measurement of a wide range of outcomes while making the commitment to act on the knowledge acquired [21,22].

What Is the Future of Surgical Quality and Performance Improvement?

Although surgical quality at the hospital level is vital, the future of surgical performance improvement will incorporate broader concepts of systems redesign, innovation, and transformative change. Fundamental to these models is integration of knowledge and resources from diverse backgrounds. This broad-based approach aims to create better-designed, more functional processes for delivering the highest quality of care possible. Clinicians and hospital leaders are partnering with innovators in technology, design, the arts, and other previously unconventional arenas to reach beyond the limited goal of just developing a better hospital. The future state of high-quality surgical care incorporates concepts that reinvent the system completely, creating an inclusive, comprehensive strategy by which public and private sectors, industry, and clinical providers all have a stake in the success of the system. The annual Technology, Entertainment and Design Medical Conference (TEDMED) [23] and the Mayo Clinic Center for Innovation's annual TRANSFORM Conference [24] are just two of a growing list of contemporary examples of how health care and innovation of forging new and necessary relationships. The best pathway for surgical quality and performance improvement includes the application of systems engineering and innovation to determine ways to do better what we do now, while developing more effective and efficient methods for delivering high-quality care tomorrow [25].

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Appendix. In Honor of Doctor Stephen F. Lowry

Words cannot express the debt of gratitude I owe Stephen F. Lowry. He was a natural mentor, effortless in his ability to garner respect and admiration. He learned about my career aspirations, and twice presented opportunities for me to reach my career goals-first as a research fellow in his Trauma and Injury Biology Laboratory at The New York Hospital-Cornell Medical Center, then after residency and critical care fellowship as a member of his clinical faculty at the University of Medicine and Dentistry of New Jersey-Robert Wood Johnson Medical School, which allowed me to pursue concurrently my Master of Business Administration degree at Columbia University.

Stephen Lowry possessed a number of admirable leadership qualities, but I believe that his greatest gift as a leader was his ability to create an environment in which others could succeed around him. He was such a natural at mentoring that effortless is an apt description. Steve would facilitate everyone's achievements and then relish in your success as if it was his own. So few leaders possess such rare selflessness, but all of us who were mentored by Steve recognize that he had that gift. We are trying to repay it forward. Thank you, Steve, for creating an environment for so many of us to succeed.

References

Midwest Business Group on Health/Juran Institute. 2003.

Berenholtz

, Pronovost

. Barriers to translating evidence into practice. Curr Opin Crit Care, 2003; 9:321–325.

Khan

, Quan

, Bugar

et al. Association of postoperative complications with hospital costs and length of stay in a tertiary care center. J Gen Intern Med, 2006; 21:177–180.

Bratzler

, Hunt

. The Surgical Infection Prevention and Surgical Care Improvement Projects: National initiatives to improve outcomes for patients having surgery. Clin Infect Dis, 2006; 43:322–330.

Khuri

, Henderson

, DePalma

et al. Determinants of long-term survival after major surgery and the adverse effect of postoperative complications. Ann Surg, 2005; 242:326–343.

National Center for Health Statistics. Combined surgery data (NHDS and NSAS) data highlights. www.cdc.gov/nchs/fastats/insurg.htm. 2012 September 12.

Gawande

, Zinner

, Studdert

et al. Analysis of errors reported by surgeons at three teaching hospitals. Surgery, 2003; 133:614–621.

Institute of Medicine Committee on the Quality of Health Care in America 2001. Washington DC: National Academy Press www.nap.edu/openbook.php?isbn=0309072808Report Brief atwww.iom.edu/∼/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf. 2012 September 12.

Chassin

. Is health care ready for six sigma quality? Milbank Q, 1998; 76:565–591.

10.

Khuri

, Daley

, Henderson

et al. The Department of Veterans Affairs' NSQIP: The first national, validated, outcome-based, risk-adjusted and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg, 1998; 228:491–507.

11.

Patient Safety Indicators Technical Specifications Version 3.2 (Revised March 2008). Rockville, MD: Agency for Healthcare Research and Quality, 2007.

12.

Cima

, Lacklore

, Nehring

et al. How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) postoperative adverse events at a single institution. Surgery, 2011; 150:943–949.

13.

Romano

, Mull

, Rivard

et al. Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data. Health Serv Res, 2009; 44:182–204.

14.

U.S. Department of Health and Human Services. Hospital Compare. www.hospitalcompare.hhs.gov/. 2012 September 12.

15.

Halasyamani

, Davis

. Conflicting measures of hospital quality: ratings from “Hospital Compare” versus “Best Hospitals.” J Hosp Med, 2007; 2:128–134.

16.

US News and World Report. Best Hospitals. http://health.usnews.com/best-hospitals. 2012 September 12.

17.

Staw

. Knee deep in the Big Muddy: A study of escalating commitment to a chosen course of action. Org Beh Hum Perf, 1976; 16:27–44.

18.

Staw

. The escalation of commitment to a course of action. Acad Mgmt Rev, 1981; 6:577–587.

19.

Staw

, Ross

. Knowing when to pull the plug. Harvard Business Review, 1987; 65:68–74.

20.

Borkowski

. Decision making. Borkowski

. Organizational Behavior in Health Care. Boston: Jones & Bartlett Publishers, 2011; 275–276.

21.

Carroll

, Edmondson

. Leading organizational learning in health care. Qual Saf Health Care, 2002; 11:51–56.

22.

Nevis

, DiBella

, Gould

. Understanding organizations as learning systems. www.solonline.org/res/wp/learning_sys.html. 2012 April 17.

23.

TEDMED. www.tedmed.com/home. 2012 September 12.

24.

Mayo Clinic Center for Innovation, TRANSFORM. 2012. www.mayo.edu/transform. 2012 September 12.

25.

Chassin

, Galvin

. The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality. JAMA, 1998; 280:1000–1005.