Determining a Core Curriculum in Surgical Infections for Fellowship Training in Acute Care Surgery Using the Delphi Technique

Abstract

Background:

Recent data highlight the educational, financial, and healthcare benefits of acute care surgery (ACS). These data serve as the impetus to create ACS fellowships, which now are accredited by the American Association for the Surgery of Trauma. However, the core components of a curriculum fundamental for ACS training and that yield competence and proficiency have yet to be determined.

Methods:

Experts in ACS from the United States (n=86) were asked to propose topics in surgical infectious diseases of potential importance in developing a core curriculum for ACS fellowship training. They were then required to rank these topics in order of importance to identify those considered most fundamental.

Results:

Thirty-one filters ranking in the highest tertile are proposed as topics of surgical infectious diseases that are fundamental to any curriculum of ACS fellowship training. The majority pertains to aspects of thoracic infections (n=8), although topics of soft tissue infections (n=5) comprised four of the top 10 (40%) filters. Abdominal infections (n=6), the biology of sepsis (n=6), and risk, prevention, and prophylaxis (n=6) completed the list.

Conclusion:

This study identifies the most important topics of surgical infectious disease that merit consideration for incorporation into a core curriculum of ACS training. Hopefully, this information will assist in the development of ACS fellowships that optimize the training of future ACS surgeons.

Emergent care is at a crisis, with widespread problems of access, overcrowding, boarding, and delays [1 –4]. Marked increases in emergency department visits by older, sicker, and uninsured patients, many with surgical emergencies, are more common and require a continuous workforce of surgical specialists to respond, evaluate, operate, and deliver critical care [3, 5 –11]. Unfortunately, emergency departments are having substantial difficulties assuring specialist coverage [12 –16]. One recent report documents that more than two-thirds of emergency department directors report shortages of on-call specialists at their hospitals [12 –16]. Increasing sub-specialization within the field of general surgery and a shift in practice and training patterns to minimally invasive interventions on patients without acute illness has limited the availability of appropriately skilled general surgeons for the care of acute surgical conditions [17 –19].

In response to these issues, many hospitals have abandoned the traditional model of surgeon coverage for urgent care, which relied upon non-trauma surgeons taking home call, in favor of having their trauma programs absorb emergency surgery [20, 21]. This new model of acute care surgery (ACS), a combination of trauma surgery, broad-based emergency surgery, and surgical critical care, has been championed by the American Association for the Surgery of Trauma (AAST) and a number of other trauma and surgical societies [22, 23]. Data are accumulating that ACS services, particularly for time-dependent surgical pathology, streamline and improve the processes of delivering care and yield better outcomes [24 –29].

How to deliver this care most appropriately and train the future generations of ACS surgeons that will provide it have yet to be defined [30 –32]. Many of the current and established ACS programs developed out of a regional need for this specialized care [23]. Few studies have identified the characteristics of these programs that are deterministic of improved outcomes. Yet in the wake of optimistic data, particularly those suggesting financial benefits, many institutions are creating ACS divisions [33]. Similarly, recent studies highlight the additional volume and greater breadth of operative experience for surgical trainees when ACS is incorporated into a single discipline, simultaneously offsetting the progressively non-operative management in trauma [34 –36]. These data serve as the impetus to create ACS fellowships, which now are accredited by the AAST. However, the core components of a curriculum fundamental for ACS training and that yield competence and proficiency have yet to be determined.

In light of the importance of this field for the provision of care of complex, acutely ill, and resource-intensive patients, identifying a set of core topics to guide the development of and potentially standardize ACS fellowship training would be of considerable value. These initial topics would be invaluable in guiding developing fellowships and in establishing a standardized curriculum for all programs. The population of patients encompassed by ACS has a high proportion of diseases either initiated by or complicated by infectious processes that define the field of surgical infectious diseases. Examples of infectious processes would include secondary and tertiary peritonitis, appendicitis, diverticulitis, cholangitis, complicated and necrotizing soft tissue infections, necrotizing pancreatitis, intestinal ischemia, and hospital-acquired infections, including surgical site infections, pneumonia, and blood stream infections. The objective of this study was to determine by consensus of expert opinion, those areas of surgical infection considered to be fundamental to the core curriculum of ACS training.

Materials and Methods

Study design

This study used a modified Delphi method to collect experts' opinions and achieve consensus on topics considered fundamental in the training of ACS. The Delphi method, originally developed by the RAND Corporation, is a structured process that uses a series of questionnaires, each referred to as a round, to both gather and provide information [37 –40]. The process continues until “group” consensus is achieved. This approach has several distinct advantages. It enables the inclusion of a large number of individuals across diverse geographic locations with a broad range of expertise, and it affords anonymity to participants and privacy for iteration and the ability to change one's mind over several rounds. One of its key advantages is that unlike face-to-face meetings of experts, it eliminates the possibility that a specific expert might (advertently or inadvertently) dominate the consensus process. The Delphi method also helps to minimize the effects of group interactions and maximizes the ability to elicit expert knowledge. In this study, we used conventional mailings for all three rounds. This study was approved by the Institutional Review Board of the University of Pittsburgh.

Selection of experts

Experts were selected from among the rosters of the members of the Surgical Infection Society (SIS), the American Association for the Surgery of Trauma (AAST) Committee on Acute Care Surgery, and the Eastern Association for the Surgery of Trauma Acute Care Surgery Committee. Specific individuals were selected to represent both geographic diversity and a broad range of expertise in ACS on the basis of their prior published research in the area of ACS. There is no requisite number of experts in a Delphi survey; typically between 20 and 40 are included. We invited 86 experts from 86 hospitals, trauma centers, and research centers representing 26 states in the United States to participate. Each expert was contacted through an invitation letter included with round one. Surveys were to be completed within two weeks and returned. A reminder letter was sent to those experts who had not submitted responses within two weeks.

Methods and measurements

Delphi round one: Criterion generation

The first round provided an opportunity for experts to generate topics and provide subjects of potential utility in training in ACS. Specifically, each expert received an initial questionnaire that listed 80 questions divided into five categories: Biology of sepsis; risk, prevention, and prophylaxis; thorax; abdomen; and soft tissue infections. Each of these questions considered a particular topic in a specific field of ACS that the investigators considered important. For example, the participant was asked to indicate his or her level of agreement with the following: “Do you think that training in the biology of sepsis should be core to the curriculum of an ACS fellowship?” In addition to responding to each of these questions, each expert was requested to provide comments and suggestions pertinent to each. Further, experts were asked to provide any other aspects of ACS that they thought might function as an important topic.

Participants were given a total of 1 mo to formulate topics/subjects. Investigators collated and reviewed the list of audit filters in preparation for the second round. Criteria that were similar in nature were combined, and those that were not appropriate or were excessively general or vague were eliminated. The responses to these questionnaires were compiled for round two.

Delphi round two: Rating

Those experts responding to round one were included in the second round. In round two, each expert rated the relative merit of each topic on a five-point priority scale, which considered the importance of the topic. A score of 1 (not important) was defined as a subject not considered important or fundamental to the curriculum of ACS training. A score of 5 (most important) was defined as a topic fundamentally important to fellowship training in ACS. Experts were offered the opportunity to comment on each topic and to clarify or provide comments to support any of their ratings. The scores were collected, collated, and analyzed.

There are no established rules for determining consensus using this methodology. Our aim was to achieve consensus on the most important topics. The questions were ranked by their mean priority score, and those falling within the highest tertile or achieving a mean priority score greater than 4 were selected for final consensus scoring in round three. The proportion of participants rating each question as very important (score of 4) or most important (score of 5) was also reported.

Delphi round three: Final consensus rating

The topics falling into the highest tertile in the second round were presented to experts for final consensus rating in the third round. The questions were sent only to experts who responded to the second round. For each question, the mean priority score and proportion of experts ranking 4 or 5 was presented for them to consider in their final ratings. Experts again were asked to rate each question in terms of importance.

Primary data analysis

Statistical relationships between variables were assessed using Pearson's correlation coefficient. All statistical analyses were performed using Stata 12SE (Stata Corporation, College Station, TX).

Results

We invited 86 experts from 26 states to participate. Overall 46 of 86 (53%) invited experts participated in at least one round, and 22 of these 46 (48%) individuals completed all three rounds. The proportion of experts invited to participate in the first round, who generated topics fundamental for ACS training, was 53% (46 of 86). These 46 participants proposed 145 topics to be evaluated as potential core requirements for ACS fellowship training. These 145 suggestions were then collapsed by the investigators into 96 distinct questions in five categories and submitted to the experts for the second round of the survey. In the second round, 32 of 46 experts (70% of those originally participating in the first round) rated these topics on a scale of importance. Finally, 22 of the 32 experts (69%) who were sent the third round of the survey participated and contributed to the final consensus rating.

The 97 topics were grouped into five areas of ACS: 1) Biology of sepsis; 2) risk, prevention, and prophylaxis; 3) thorax; 4) abdomen; and 5) soft tissue infections. The topics with the most number of questions generated were those related to abdomen (n=31) and thorax (n=31), followed by the biology of sepsis (n=17) and risk, prevention, and prophylaxis (n=10). The fewest number of topics pertained to soft tissue infections (n=8).

In the second round there was a strong correlation between mean score and the proportion of experts who scored a question as important or very important. In the 40 topics constituting the top tertile, the mean priority scores ranged from 4.06 to 4.91, and the standard deviation was less than 1 for 35 (88%). Additionally, with increasing mean scores, the variability in scoring within each question decreased, suggesting increasing consensus for the highest ranked questions (round 2=-0.68; p<0.001; Fig. 1).

FIG. 1.

Relationship between the variation in priority across respondents (measured as standard deviation) and mean priority score.

Thirty-one of the final 40 criteria were considered important (mean priority score>4) by consensus in the final round. The mean priority scores ranged from 4.08 to 4.73 and the standard deviation was less than 1 in 30 (97%). The final rank order of these 31 most important topics is listed in Table 1 and collated by category in Table 2. Topics concerning soft tissue infections received the highest priority, comprising four of the top 10 (40%) topics. All the seven (100%) topics in risk, prevention, prophylaxis included in round two comprised 23% of the final 31 topics considered fundamental for ACS training (Table 1 and Table 2).

Table 1.

Priority Rankings of the Most Highly Rated Topics in Surgical Infection

Rank	Topic	Category	Mean priority score
1	It is important to ACS training that fellows acquire proficiency in bronchoscopy with aspiration/lavage.	Thoracic	4.73
2	It is important that ACS fellows acquire proficiency in open or percutaneous tracheostomy.	Thoracic	4.71
3	It is important that ACS fellows understand clinical criteria for the identification and management of necrotizing soft tissue infections.	Soft Tissue	4.64
4	It is important that ACS fellows understand principles of resection/debridement.	Soft Tissue	4.64
5	It is important that ACS fellows understand clinical criteria for identifying and management line sepsis/device sepsis.	Soft Tissue	4.64
6	It is important that fellows demonstrate competency in management strategies of the high-risk patient.	Abdomen	4.57
7	It is important that ACS fellows demonstrate expertise in the current evidence-based guidelines for minimizing surgical site infections.	Soft Tissue	4.55
8	It is important that fellows demonstrate competency in management strategies of the high-risk patient.	Thorax	4.48
9	It is important that fellows appreciate the risk factors for failed source control.	Abdomen	4.48
10	It is important to teach the distinguishing characteristics in the diagnosis and antimicrobial/operative management of primary/secondary, and tertiary peritonitis and abscesses.	Abdomen	4.48
11	It is important that fellows have to log a minimum number of cases in bronchoscopy with aspiration/lavage.	Thorax	4.45
12	It is important to ACS fellowship training to learn bedside ultrasound imaging.	Abdomen	4.43
13	It is important that expertise in evidence-based measures to prevent surgical infection be incorporated into ACS fellowship training.	Risk, prevention, prophylaxis	4.40
14	It is important that ACS fellows understand principles of reconstruction.	Soft Tissue	4.36
15	It is important for ACS fellows to receive training management strategies for the high-risk patient (e.g. percutaneous cholecystostomy tube).	Abdomen	4.33
16	It is important that fellows log cases for the management of shock.	Risk, prevention, prophylaxis	4.29
17	It is important that fellows log cases for the management of multiple organ dysfunction/failure syndrome (MODS/MOFS).	Risk, prevention, prophylaxis	4.29
18	Antimicrobials and antibiotic pharmacodynamics and pharmacokinetcs should be a fundamental topic of training.	Biology of Sepsis	4.27
19	Training in the biology of sepsis should be core to the curriculum of an ACS fellowship.	Biology of Sepsis	4.24
20	The basis for therapeutic interventions should be a fundamental topic of training.	Biology of Sepsis	4.23
21	Organ physiology and pathophysiology should be a fundamental topic of training.	Biology of Sepsis	4.23
22	It is important to teach the distinguishing characteristics in the diagnosis and antimicrobial/operative management of infectious versus non-infectious causes of pneumonitis/pulmonary infiltrates.	Thorax	4.22
23	It is important that training in bedside ultrasound imaging of the thorax be core to the curriculum of an ACS fellowship.	Thorax	4.19
24	It is important that fellows have core curriculum in enteral access.	Abdomen	4.19
25	It is important that ACS fellows understand the distinguishing characteristics of nosocomial versus community infection and the importance of ICU antibiograms.	Risk, prevention, prophylaxis	4.18
26	It is important that ACS fellows log cases for the management of acute kidney injury.	Risk, prevention, prophylaxis	4.10
27	Metabolism of sepsis should be a fundamental topic of training.	Biology of Sepsis	4.09
28	It is important that physiology be a class for the curriculum of ACS training.	Biology of Sepsis	4.09
29	It is important that antibiotic stewardship be incorporated into an ACS fellowship program.	Risk, prevention, prophylaxis	4.09
30	It is important to ACS training that fellows acquire proficiency in flexible esophagoscopy.	Thorax	4.09
31	It is important that fellows acquire an understanding of the spectra of pathogenic organisms to which various states predispose.	Risk, prevention, prophylaxis	4.08

ACS, acute care surgery.

Table 2.

Priority Rankings of the Most Highly Rated Topics in Surgical Infection by Category

Rank	Topic
	Abdomen
6	It is important that fellows demonstrate competency in management strategies of the high-risk patient.
9	It is important that fellows appreciate the risk factors for failed source control.
10	It is important to teach the distinguishing characteristics in the diagnosis and antimicrobial/operative management of primary/secondary, and tertiary peritonitis and abscesses.
15	It is important for ACS fellows to receive training management strategies for the high-risk patient (e.g. percutaneous cholecystostomy tube).
	Biology of Sepsis
18	Antimicrobials and antibiotic pharmacodynamics and pharmacokinetcs should be a fundamental topic of training.
19	Training in the biology of sepsis should be core to the curriculum of an ACS fellowship.
20	The basis for therapeutic interventions should be a fundamental topic of training.
21	Organ physiology and pathophysiology should be a fundamental topic of training.
27	Metabolism of sepsis should be a fundamental topic of training.
28	It is important that physiology be a class for the curriculum of ACS training.
	Risk, prevention, prophylaxis
13	It is important that expertise in evidence-based measures to prevent surgical infection be incorporated into ACS fellowship training.
16	It is important that fellows log cases for the management of shock.
17	It is important that fellows log cases for the management of multiple organ dysfunction/failure syndrome (MODS/MOFS).
25	It is important that ACS fellows understand the distinguishing characteristics of nosocomial versus community infection and the importance of ICU antibiograms.
26	It is important that ACS fellows log cases for the management of acute kidney injury.
29	It is important that antibiotic stewardship be incorporated into an ACS fellowship program.
31	It is important that fellows acquire an understanding of the spectra of pathogenic organisms to which various states predispose.
	Soft Tissue
3	It is important that ACS fellows understand clinical criteria for the identification and management of necrotizing soft tissue infections.
4	It is important that ACS fellows understand principles of resection/debridement.
5	It is important that ACS fellows understand clinical criteria for identifying and management line sepsis/device sepsis.
7	It is important that ACS fellows demonstrate expertise in the current evidence-based guidelines for minimizing surgical site infections.
14	It is important that ACS fellows understand principles of reconstruction.
	Thoracic
8	It is important that fellows demonstrate competency in management strategies of the high-risk patient.
22	It is important to teach the distinguishing characteristics in the diagnosis and antimicrobial/operative management of infectious versus non-infectious causes of pneumonitis/pulmonary infiltrates.
	Procedural/Diagnostic
1	It is important to ACS training that fellows acquire proficiency in bronchoscopy with aspiration/lavage.
2	It is important that ACS fellows acquire proficiency in open or percutaneous tracheostomy.
11	It is important that fellows have to log a minimum number of cases in bronchoscopy with aspiration/lavage.
12	It is important to ACS fellowship training to learn bedside ultrasound imaging of the abdomen.
23	It is important that training in bedside ultrasound imaging of the thorax be core to the curriculum of an ACS fellowship.
24	It is important that fellows have core curriculum in enteral access.
30	It is important to ACS training that fellows acquire proficiency in flexible esophagoscopy.

ACS, acute care surgery; ICU, intensive care unit.

Topics in the thoracic (n=3) and abdominal infection (n=3) completed the top 10 highest-ranking filters. They comprised seven (23%) and six (19%), respectively, of the final 31 important topics. Six (19%) topics in the biology of sepsis were included in the final 31 most important topics. However, five of eight (63%) in soft tissue infection included in the second round were in the final 31 most important topics.

Discussion

With the emergence of the field of ACS and the recent development of subspecialty ACS fellowship training, we considered it a priority to develop a core curriculum that standardizes ACS training. Evidence that a curriculum needs to be developed is highlighted by the variability in current ACS programs and the lack of data regarding those aspects of ACS training that optimize outcomes. Our focus was to identify topics of surgical infection that by expert consensus are considered fundamental to any ACS fellowship. Here we have identified 31 topics spanning five aspects of surgical training that represent a consensus of experts in ACS within the United States. The rapidly changing paradigm in surgical management and training with increasing sub-specialization, a shift towards minimally invasive techniques, a perceived decline in the general surgical training of basic surgical infectious diseases, all combined with advances and increased complexity of the management of acutely ill surgical patients make the establishment of an identified skill set in surgical infectious diseases of considerable importance.

Prior to this study, little evidence existed to suggest which topics of surgical infection should be incorporated into ACS training. This stems, in part, from institutional variability in the resources available and in the spectra of disease encountered. Many ACS fellowships have developed at institutions that have always functioned under an ACS paradigm. Thus, the development of ACS, and consequentially its training, developed out of an institution-specific need to address an infrastructural deficiency in patient care, rather than determining which characteristics, technical and knowledge-based, provide optimal patient outcome. An overarching theme of this study was to ensure that the teaching of each proposed topic be universally feasible and independent of institutional resources.

Our results emphasize the importance of knowledge, rather than procedural, training in surgical infections as 24 of 31 (77%) and eight of the top 10 (80%) topics focused upon knowledge (Table 2). Of these 20, six (30%) focused upon the basic biology of sepsis, including metabolism, physiology, pathophysiology, pharmacokinetics, and pharmacodynamics. Although these topics are fundamental to medical school education, they are not emphasized in subsequent residency and fellowship training. However, recent advances in the management of critically ill surgical patients necessitate that physicians possess a greater command of the basic biological sciences. Importantly, the majority of these topics can be taught universally at all institutions.

Five procedures were considered fundamental: Ultrasound, tracheostomy, bronchoscopy, esophagoscopy, and enteral access (Table 2). Although exposure to each procedure is acquired during surgical residency, it is clear that the acquisition of additional expertise, specific to surgical infections and ACS, is needed. For example, proficiency in focused abdominal sonography for trauma is acquired during surgical residency or fellowship. However, our experts concluded that additional ultrasound training in the context of acute surgical infections (e.g., drainage of pleural effusions, intra-abdominal abscesses, percutaneous cholecystostomy tube) is essential to any ACS curriculum. Similarly, exposure to bronchoscopy and esophagoscopy are acquired during surgical residency but specialized training is needed as it relates to the management of surgical infections (i.e., lavage, aspiration).

Although it was not our objective to obtain universal agreement on the topics proposed, there was general consensus as to which criteria were important. All 31 topics were considered very or most important (i.e., priority score>4). The standard deviation of the mean priority score was less than 1 in 30 (97%) of these final topics. We believe that the large number of topics developed by this panel reflects the lack of interaction between participants and an unbiased, consensus opinion of ACS experts.

Now that the important surgical infection topics of ACS have been defined, the next steps are implementing them into a curriculum and creating benchmarking systems to assess how well these topics are being taught, learned, and acquired. Many of the specific procedures included as important are routinely performed in most surgical critical care and or ACS fellowships at present. These procedures can be tracked through the AAST case log system that has been in existence for more than one year or through modifications of the American College of Surgeons case log system as described by Patel et al [41]. Currently, for certified programs of the AAST–ACS, all ACS fellows must enter their cases into this system, and data analysis is ongoing to determine adequate benchmarks for competency. The acquisition of technical skills in the use of ultrasound is currently not captured in the AAST–ACS log system, and guidelines have not been put forth to direct standards for its application in either trauma or critical care. This remains a difficult issue. The curriculum and procedural performance required to achieve appropriate skill for targeted use of this technology in ACS require further delineation.

For the cognitive and decision making topics included in the list of 31, each requires further delineation of the components of the topics that should be required and included within a curriculum. The authors recommend that a working group of the Surgical Infection Society Acute Care Surgery Committee develop an expanded topic list that defines further the components required in each individual topic. Whereas much of this content can be provided through didactic and written formats, applying concepts within the clinical setting is mandatory. Thus, documentation of management of appropriate patient populations through a case log system is also a component of verification of skills. The educational topics can be incorporated into ongoing didactics at the specific institutions. With the information provided by this study, programs will be better able to gear their education toward the core concepts identified. These concepts can then be emphasized in future iterations of the ACS examination, thereby benchmarking the success of the programs.

The Delphi survey design is considered one of the optimal methods available to elicit expert knowledge and come to a consensus, and at the same time minimize the effect of group interaction [42,43]. However, this method is not without limitations. For example, the Delphi technique has been criticized as lacking reliability. Specifically, if the same survey was provided to a different panel of experts or the same experts at some future point in time, there is no assurance that the same results will be obtained [44]. Further, the results of the survey are highly dependent on the composition of the panel. However, we believe the topics proposed reflect the broad range of experience and expertise of a heterogeneous panel of experts with experience in the establishment and maintenance of ACS training.

Within the constraints imposed by the limitations described above, this list of topics represents the most important aspects of surgical infections that merit consideration for incorporation into a core curriculum of ACS training. This information will assist our societal and institutional colleagues in the development of ACS fellowships that optimize the training of future ACS surgeons.

Footnotes

Acknowledgments

We thank Mrs. Arnold Glynis for her assistance in the preparation and distribution of each round.

Author Disclosure Statement

No competing financial interests exist.

References

Asplin

, Magid

, Rhodes

, et al. A conceptual model of emergency department crowding. Ann Emerg Med, 2003; 42:173–180.

Derlet

, Richards

. Overcrowding in the nation's emergency departments: complex causes and disturbing effects. Ann Emerg Med, 2000; 35:63–68.

Schafermeyer

, Asplin

. Hospital and emergency department crowding in the United States. Emerg Med, 2003; 15:22–27.

Schneider

, Gallery

, Schafermeyer

, Zwemer

. Emergency department crowding: A point in time. Ann Emerg Med, 2003; 42:167–172.

Moskop

, Sklar

, Geiderman

, et al. Emergency department crowding, part 1—concept, causes, and moral consequences. Ann Emerg Med, 2009; 53:605–611.

Ginde

, Lowe

, Wiler

. Health insurance status change and emergency department use among US adults. Arch Intern Med, 2012; 172642–647.

Cunningham

, J. M.

Mounting Pressures: Physicians Serving Medicaid Patient and the Uninsured, 1997–2001. Washington, DC: Center for Studying Health System Change, 2002.

Pisarek

, Auwera

, Smet

, et al. Insured versus uninsured patients in the emergency room: is there a difference?. Eur J Emerg Med, 2003; 10:314–317.

Hoot

, Aronsky

. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med, 2008; 52:126–136.

10.

Trzeciak

, Rivers

. Emergency department overcrowding in the United States: An emerging threat to patient safety and public health. Emerg Med J, 2003; 20:402–405.

11.

Andrulis

, Kellermann

, Hintz

, et al. Emergency departments and crowding in United States teaching hospitals. Ann Emerg Med, 1991; 20:980–986.

12.

Rao

, Lerro

, Gross

. The shortage of on-call surgical specialist coverage: A national survey of emergency department directors. Acad Emerg Med, 2010; 17:1374–1382.

13.

Sheldon

, Ricketts

, Charles

, et al. The global health workforce shortage: Role of surgeons and other providers. Adv Surg, 2008; 42:63–85.

14.

Group

. Emergency Department Overload: A Growing Crisis. The Results of the AHA Survey of Emergency Department (Ed) and Hospital Capacity. Available at: www.hospitalconnect.com/aha/press_room-info/content/EdoCrisisSlides.pdf. (Accessed April 1, 2013.)

15.

McConnell

, Newgard

, Lee

. Changes in the cost and management of emergency department on-call coverage: evidence from a longitudinal statewide survey. Ann Emerg Med, 2008; 52:635–642.

16.

Menchine

, Baraff

. On-call specialists and higher level of care transfers in California emergency departments. Acad Emerg Med, 2008; 15:329–336.

17.

Bell

Jr. , Biester

, Tabuenca

, et al. Operative experience of residents in US general surgery programs: A gap between expectation and experience. Ann Surg, 2009; 249:719–724.

18.

Fischer

. The impending disappearance of the general surgeon. JAMA, 2007; 298:2191–2193.

19.

Lynge

, Larson

, Thompson

, et al. A longitudinal analysis of the general surgery workforce in the United States, 1981–2005. Arch Surg, 2008; 143:345–350.

20.

Kim

, Dabrowski

, Reilly

, et al. Redefining the future of trauma surgery as a comprehensive trauma and emergency general surgery service. J Am Coll Surg, 2004; 199:96–101.

21.

Scherer

, Battistella

. Trauma and emergency surgery: An evolutionary direction for trauma surgeons. J Trauma, 2004; 56:7–12.

22.

Napolitano

, Fulda

, Davis

, et al. Challenging issues in surgical critical care, trauma, and acute care surgery: A report from the Critical Care Committee of the American Association for the Surgery of Trauma. J Trauma, 2010; 69:1619–1633.

23.

Kelly

, Rogers

Jr. , Graduate medical education in trauma/critical care and acute care surgery: Defining goals for a new workforce. Surg Clin North Am, 2012; 92:1055–1064.

24.

Carchman

, Peitzman

, Simmons

, Zuckerbraun

. The role of acute care surgery in the treatment of severe, complicated Clostridium difficile-associated disease. J Trauma Acute Care Surg, 2012; 73:789–800.

25.

Earley

, Pryor

, Kim

, et al. An acute care surgery model improves outcomes in patients with appendicitis. Ann Surg, 2006; 244:498–504.

26.

Cubas

, Gomez

, Rodriguez

, et al. Outcomes in the management of appendicitis and cholecystitis in the setting of a new acute care surgery service model: Impact on timing and cost. J Am Coll Surg, 2012; 215:715–721.

27.

Ball

, MacLean

, Dixon

, et al. Acute care surgery: the impact of an acute care surgery service on assessment, flow, and disposition in the emergency department. Am J Surg, 2012; 203:578–583.

28.

Lau

, Difronzo

. An acute care surgery model improves timeliness of care and reduces hospital stay for patients with acute cholecystitis. Am Surg, 2011; 77:1318–1321.

29.

Qureshi

, Smith

, Wright

, et al. The impact of an acute care emergency surgical service on timely surgical decision-making and emergency department overcrowding. J Am Coll Surg, 2011; 213:284–293.

30.

Kaplan

, Frankel

, Davis

, Barie

. Pitfalls of implementing acute care surgery. J Trauma, 2007; 62:1264–1270.

31.

Spain

, Miller

. Education and training of the future trauma surgeon in acute care surgery: Trauma, critical care, and emergency surgery. Am J Surg, 2005; 190:212–217.

32.

Hoyt

, Kim

, Barrios

. Acute care surgery: a new training and practice model in the United States. World J Surg, 2008; 32:1630–1635.

33.

Miller

, Wildman

, Chang

, Meredith

. Acute care surgery: Impact on practice and economics of elective surgeons. J Am Coll Surg, 2012; 214:531–535.

34.

Cherry-Bukowiec

, Miller

, Doherty

, et al. Nontrauma emergency surgery: optimal case mix for general surgery and acute care surgery training. J Trauma, 2011; 71:1422–1426.

35.

Pryor

, Reilly

, Schwab

, et al. Integrating emergency general surgery with a trauma service: impact on the care of injured patients. J Trauma, 2004; 57:467–471.

36.

Wood

, Buczkowski

, Panton

, et al. Effects of implementation of an urgent surgical care service on subspecialty general surgery training. Can J Surg, 2010; 53:119–125.

37.

Jairath

, Weinstein

. The Delphi methodology (Part one): A useful administrative approach. Can J Nurs Adm, 1994; 7:29–42.

38.

Jairath

, Weinstein

. The Delphi methodology (Part Two): A useful administrative approach. Can J Nurs Adm, 1994; 7:7–20.

39.

Fink

, Kosecoff

, Chassin

, Brook

. Consensus methods: Characteristics and guidelines for use. Am J Public Health, 1984; 74:979–983.

40.

Reid

. The Delphi Technique: Its Contribution to the Evaluation of Professional Practice. Ellis

, ed. New York. Chapman & Hall, 1988.

41.

Patel

, Guillamondegui

, Ott

, et al. O' surgery case log data, where art thou?. J Am Coll Surg, 2012; 215:427–431.

42.

Woudenberg

. An evaluation of Delphi. Technological Forecasting Soc Change, 1991; 40:131–150.

43.

Linstone

, Turoffi

. The Delphi Method: Techniques and Applications. London. Addision-Wesley, 1975.

44.

Walker

, Selfe

. The Delphi Method: a useful tool for the allied health researcher. Brit J Ther Rehab, 1996; 12:677–681.