Incision Classification Accuracy: Do Residents Know How to Classify Them?

Patients and Methods

This clinical study was a single-site, secondary data analysis at a 537-bed tertiary-care center. Patients undergoing surgery who were included in the NSQIP registry from April 1, 2015, through December 31, 2016, comprised the study sample. Data pertaining to patient demographics, case type, incision classification, post-graduate resident year (PGY1, 2 = junior; PGY 3 = midlevel; PGY 4 = senior, PGY 5–PGY7 = chief/fellow); and outcomes were obtained from the NSQIP registry and electronic medical records.

Resident classification accuracy was defined as either resident agreement with both the circulator and the surgical clinical reviewer (SCR) or, in the case of disagreement among the three raters, resident agreement with the SC. Therefore, case accuracy was a binary variable (0 = no agreement; 1 = agreement), and resident accuracy was the percent of cases classified correctly. The circulator is the nurse assigned to the specific case who participates in supporting the operative team, the SCR is a nurse specifically trained to abstract NSQIP data and recognize outcomes based on standard definitions, and the SC is the attending surgeon who is committed to supporting the efforts of NSQIP in the hospital. The ACS NSQIP policy mandates complete follow-up of patients 30 d after the principal operative procedure.

Categorical variables were analyzed with χ² tests, and continuous variables were compared through Student t-tests. Multivariable logistic regression analysis was performed with case accuracy (0, 1) as the dependent variable and PGY as the key independent variable, supplemented by covariates of age, sex, race/ethnicity, IC (1 [clean], 2 [clean-contaminated], 3 [contaminated], 4 [dirty/infected]), elective surgery (yes, no), and diabetes mellitus (yes, no). Pairwise comparisons among means were made through the Margin and Bonferroni options at α = 0.05. Statistical analysis was done with Stata/MP 14.2 for Windows software (StataCorp LLC, College Station, TX). The John Peter Smith Institutional Review Board approved this study.

Results

A total of 1,861 patients were included in this study. Table 1 outlines the patient count distributed among incision classes for baseline demographics and additional clinical variables. The patient sample is characterized by a 47.9 years average age, 56.1% female, and 69.2% white and 29.8% Hispanic origin. Surgical site infections occurred in 7.4% of cases, ranging from 2.4% in IC 1 to 15.7% in IC 4 cases. The odds of infection within IC 2, 3, and 4 were each significantly higher than in IC 1 (odds ratio [OR] 4.20, 3.14, and 7.41, respectively; p = 0.00 each). In addition, the difference between IC 4 and IC 2 infection rates (6.1%) approached significance (p = 0.10), whereas the difference between IC 4 and 3 (8.4%) was significant (p = 0.01).

Operations included general surgery (58.1%), followed by gynecology (17.6%) and orthopedics (13.7%). Combined PGY5–7 residents assisted in the highest percentage of operations (43.3%), followed by PGY4 residents (29.9%) and combined PGY1–2 (13.8%) (Table 2). Incisions were evaluated and classified by 115 residents, with the number of classifications per resident ranging from 1 to 76. A relatively large percentage of residents (54.8%) classified 10 or fewer patients each over the study period, accounting for only 14.5% of the total cases, whereas residents who classified more than 25 cases each (22.6%) accounted for 63.3% of the total cases.

Residents accurately identified 83.6% of incisions, with the highest mean accuracy achieved in class 1 (97.7%) and the lowest in class 3 (64.6%). Logistic regression analysis (Table 3) found male patients were 24% less likely to receive accurate classifications than were female patients (OR 0.76; p = 0.05), and African American patients were 30% less likely to receive accurate classifications. Incision classes 2–4 were less likely than IC 1 to be accurately classified (p = 0.00 for each class). Adjusted mean accuracies for PGY4 and PGY5–7 residents were significantly less than the PGY3 mean accuracy (Fig. 1). A disproportionate number of misclassifications by PGY4 and PGY5–7 residents occurred in IC 2 and 3 (Table 4). For example, 36.0% (123/342) of IC 3 incisions were misclassified compared with only 2.8% (12/431) of IC 1s.

OPEN IN VIEWER

FIG. 1.

Wound assignment accuracy adjusted for co-variates, by resident groups PGY1–5, N = 1844.

Discussion

Surgical site infections are a major component of hospital-acquired conditions (HACs) and as such are incorporated into federal quality improvement efforts. A key aspect of the HAC Reduction Program is reduced Medicare reimbursement to the worst-ranking hospitals [3]. In order to avoid these penalties, health systems have become more invested in opportunities to improve patient outcomes and minimize complications such as SSI.

In the setting of outcomes-based revenue loss, administrators are driven to demand better quality of care from attending staff physicians and residents alike. These efforts are supported by national organizations and governing bodies such as the Accreditation Council for Graduate Medical Education (ACGME), which implemented a formalized quality improvement (QI) component to surgical residency training in 2013 [7]. As QI gains significance in a surgical training programs, resident education and accountability in such matters is becoming increasingly important.

Teaching hospitals are promoting QI projects among residents in an effort to create awareness and acknowledge the pressure placed on outcomes-based reimbursement programs. Historically, teaching hospitals were perceived to foster higher-quality care, including the treatment of rare diseases and complex cases, the provision of specialized services and advanced technology, and the conduct of biomedical research [8]. More recent literature fails to identify differences in outcomes when comparing teaching and nonteaching hospitals, with the exception of treatment for some specific diseases and procedures [9]. One study from the Department of Veterans Affairs (VA) found that, compared with nonteaching hospitals, teaching hospitals in the VA perform the majority of complex and high-risk major procedures, with comparable risk-adjusted 30-d mortality rates. The study did, however, demonstrate a statistically significant difference in 30-d post-operative morbidity, of which incision complications and infections were the most frequent [10]. Other studies have reported similar results with regard to post-operative morbidity, including incision infections for cases in which residents were involved [11]. Findings such as these warrant further examination of efforts to improve outcomes, considering complex cases will continue to present at tertiary-care academic centers, in which residents are integral to patient care.

In our study, more than 50% of cases were IC 2 and 3 incisions. Not only are these classes at higher risk for infection than IC 1, they are also the least likely to be appropriately identified. These results clearly demonstrate opportunities for better education. However, balancing education with autonomy often can be challenging. The ACGME recognizes the value of education in resident autonomy, stating that residents must be granted leadership roles while clinical faculty observe resident behaviors directly, serve as bedside experts in criterion-based assessment and advancement, and optimize resident roles on interdisciplinary teams [12]. In the operating room, this often translates to attending surgeons assisting in the majority of operative cases in which residents perform the procedure independently. This practice is particularly true among PGY 4 and above residents, specifically at the time of incision closure. According to our findings, this may not be the best practice.

Our study investigated the accuracy with which residents classified surgical incisions at the time of closure, usually without the supervision of an attending surgeon. Residents were fairly accurate in classifying the cleanliness of incisions in the operating room, at a rate of 83.6%, yet those with five or more years of training had the lowest accuracy in classifying incisions intra-operatively, at 80.5%. This trend could be explained by the degree of complexity generally inherent in the cases covered by an upper-level resident and is supported by our data in which PGY 3 residents were more accurate at an overall rate of 91.6%. However, the majority of their cases were IC 1.

Despite the seemingly simplistic nature of using one of four choices to classify incisions, application of this system to all surgical sites is fraught with unreliability and subjective decision-making. In a review involving 11 institutions, in which more than 2,000 cases were examined, one study found only a 56% concordance rate for incision classification between the EMR recording by the circulating nurse and the operative note dictated by the surgeon. The study further substantiates the idea that SSI risk stratification by IC is an unreliable method to compare patients and institutions and should not be used for quality benchmarking [13]. This unreliability may call into question its role in determining incision management.

In general, incisions are managed and closed on the basis of the cleanliness classification. Such decisions have profound implications for the risk of subsequent SSIs. The U.S. Centers for Disease Control and Prevention (CDC) has published definitions regarding incision classification and has recommended closure management options accordingly [14]. Class 1 or a clean site in which no inflammation is encountered and the respiratory, alimentary, genital, or uninfected urinary tract has not been entered is the easiest to recognize, as demonstrated by our trial. These incisions have minimal risk of infection, 1%–5%, and can be closed routinely, which occurred almost 100% of the time in this study. Class IV, or a contaminated/dirty incision, suggests that the organisms causing post-operative infection are present in the operative field before the operation; such incisions tend to be more obvious and as a result were more accurately identified in our trial. These incisions confer a risk of SSI greater than 27%. Furthermore, almost half of these incisions had the skin left, open consistent with the CDC guidelines [15,16].

The struggle for the residents stems primarily from IC 2 and 3. In the current study, residents misclassified these incisions more frequently than those of IC 1 or 4. Distinguishing between IC 2 and 3 is challenging, as they are differentiated by the degree of contamination and demonstrate larger differences in infection risk at 3%–11% for clean–contaminated and 10%–17% for contaminated. Residents' poor performance with these classifications may have resulted in inappropriate incision closure, given that more than 95% of the time, all layers were closed in both of these groups. The CDC recommendations for incision management are to use delayed primary skin closure or leave an incision open to heal by second intention if the surgeon considers the surgical site to be heavily contaminated (e.g., IC 3 or 4) [16]. Appreciating the degree of contamination is gained through experience and education and may not be obvious to residents. However, our rates of SSIs were consistent with the expected rate of post-operative infections set forth by the CDC, even for the IC 3 incisions, which were the most misclassified, at a rate of 7.3%. This finding underscores the fact that many factors play a role in SSIs, and understanding incision classification and management is but one part of preventing them.

This study is not without limitations. Our sample was small and comprised secondary data from a single-site registry. Furthermore, as NSQIP is only a sample of collected data from contributing hospitals, the results may be skewed because of the inherent bias attributable to convenience sampling. Finally, this study was not designed to demonstrate cause and effect. Consequently, we cannot conclude that inaccurate incision classification leads directly to inappropriate management and SSI. The purpose of this study was simply to evaluate another component of care that may play a role in SSI and identify opportunities for improvement in education and patient care.

Conclusions

In an environment where outcomes-based compensation is becoming increasingly prominent, it is important to know how accurate resident surgeons are in classifying incisions. Additionally, it is important to understand discrepancies in classifications with possible roles for better education. Clarifying the link between patient outcomes and appropriate classification should be the goal of future research efforts.