The Role of Simulation in Teaching Sinus Surgery in Otolaryngology Residency: A Survey of Rhinologists

Abstract

Background

Simulation is currently recognized as an effective surgical training tool. However, no standardized curriculum exists for endoscopic sinus surgery (ESS) simulation training. The goal of this study was to obtain an understanding of current ESS simulation use to aid the future development of an ESS training curriculum.

Methods

A 14-question survey regarding sinus simulation in residency training was developed through the education committee of the American Rhinologic Society. The survey was administered to academic American Rhinologic Society members in the United States, Canada, and Puerto Rico. The participants provided information regarding the type, amount, and effectiveness of simulation use in their residency program.

Results

Responses were received from 67 training programs; 45% of the programs endorsed using simulation training, although only 23.9% used ESS simulation, and all the programs used cadavers. Only 12.5% of respondent programs required ESS simulation training before operating on live patients, and trainees had an average of <6 hours of simulation training before live operations. A majority of respondents observed subjective improvement in residents' endoscope handling, dexterity, and understanding of anatomy after ESS simulation. The greatest obstacles identified were associated cost and lack of realistic simulators.

Conclusion

A majority of responders observed improved surgical technique and knowledge in residents after simulation training. However, <25% of the survey responders used ESS simulation and cited cost and limited availability as the most common barriers. A curriculum of validated simulators has potential to improve the quality of ESS training during residency.

The purpose of graduate medical education is to develop residents into competent physicians and surgeons. Limitations in time, capital, and energy are barriers that all programs face in accomplishing this task. Due to a scarcity of critical resources, integrating methods of instruction that can accelerate competency development is valuable for any graduate medical education program. The Accreditation Council for Graduate Medical Education works to standardize understanding of competency as well as establish specialty-specific resident milestones toward competency attainment.¹

Developing competency in surgical residents involves many unique challenges. In the traditional intraoperative setting, sporadic opportunities to perform specific procedures combined with frequent mismatch between resident ability and procedure complexity hindered focused learning.^2,3 In addition, evaluating residents in this setting is often inconsistent.⁴ Over the past decade, simulation training has gained recognition as an effective surgical training tool. The use of simulation models allows deliberate practice of new skills, which can translate into intraoperative confidence and performance.^5–7 Simulation use also carries the promise of objective assessment of technical ability and a reduction of surgical complications.⁴ Despite the validation of new simulation models and increasing evidence in support of simulation use, no formalized, progressive curriculum currently exists for endoscopic sinus surgery (ESS) training. An important initial step would include determining the current state of and barriers to ESS simulation use in North American otolaryngology residency programs. A survey was distributed to academic members of the American Rhinologic Society (ARS). The goals of this survey were to provide an accurate understanding of current ESS simulation use and to aid in the future development and implementation of a curriculum for ESS training.

Methods

Institutional review board approval of this exempt protocol was obtained. A 14 -question survey was developed through the education committee of the ARS regarding the role of sinus simulation in residency training (Table 1). Content validity was established by using a modified subject matter expert rater method. This is a process by which a draft survey is initially developed by expert consensus. The survey is then iteratively refined by integrating the input of several experts until consensus is achieved. Information collected included demographics, residency size, role of simulation, role of sinus simulation, perceived benefits, and hindrances to implementation of ESS simulation. SurveyMonkey (surveymonkey.com) was used to administer surveys to academic ARS members in the United States, Canada, and Puerto Rico. Of note, if the respondent answered “no” to question no. 6 (regarding whether any type of simulation was used in the residency training program), then the survey respondent was routed to the final question regarding barriers to implementation of simulation. The survey was not sent specifically to program directors (unless they happen to be ARS members) or to residents.

Table 1

Survey of 14 questions sent to academic-affiliated rhinologists

1	In which otolaryngology training program are you faculty?
2	Where is your otolaryngology residency training program located (region)?
3	Please specify where your otolaryngology residency training program is located (state/territory).
4	Which of the following best describes your otolaryngology residency training. (private/community based, university-based civilian, military)
5	How many total residents (PGY 1–5) does your otolaryngology training program have?
6	Does your otolaryngology training program integrate surgical simulation training as part of the resident curriculum?
7	Does your otolaryngology training program integrate ESS simulation training into the resident curriculum?
8	Is completing an ESS simulation curriculum mandatory before participating in ESS on live patients?
9	Does the residency program provide simulation training “in-house” or is this “outsourced” (e.g., resident simulation course, sinus course)?
10	What type of simulation do the residents participate in?
11	What metrics are collected on residents to show they are progressing?
12	How long does the average resident spend working in simulation before his or her first live ESS case?
13	What benefits have you observed from ESS simulation?
14	What barriers exist to integrate ESS simulation into the resident curriculum?

PGY = Postgraduate year; ESS = endoscopic sinus surgery.

Results

Responses were received from 67 training programs (Table 2). Residents participated in some form of simulation training in 45% of the programs (30/67). However, only 23.9% of the programs (16/67) used ESS simulation. Only 2 of 16 of the programs (12.5%) required simulation before operating on live patients. On average, trainees spent <6 hours in ESS simulation before the first live ESS. The majority of programs performed ESS simulation in-house rather than sending residents to courses at other programs (13/16). All participating programs used cadavers, but 43.8% (7/16) also used manufactured simulators. Primarily, the residents were evaluated on their ability to complete a surgical task (13/16). Fluency and the time to task completion were not as often evaluated during simulation settings (4/16 and 5/16, respectively). The majority of respondents observed improved endoscope handling (15/16) and dexterity (14/16) after ESS simulation. The majority (10/16) also believed that the residents had a better understanding of anatomy, with a faster learning curve after simulation. The greatest obstacles to implementation of ESS simulation were the associated cost and the lack of realistic simulators.

Table 2

Responses to select survey questions

Question	Response	Percentage
Does your otolaryngology training program integrate FESS simulation training into the resident curriculum?	Yes	53.3
	No	46.7
Is completing a FESS simulation curriculum mandatory before participating in FESS on live patients?	Yes	12.5
	No	87.5
Does the residency program provide simulation training “in-house” or is this “outsourced” (e.g., resident sim course, sinus course)?	In-house	81.3
	Outsourced	18.8
What type of simulations do the residents participate in?	Cadaver	100
	Locally produced simulator	25
	Computer-based “virtual reality” simulation	6.3
	Commercial product	12.5
	Outside FESS training course	56.3
What metrics are collected on residents to show progression?	Ability to complete specified task	81.3
	Time to task completion	31.3
	Errors in task completion	43.8
	“Fluency” of motion	25
	Number of procedures performed	50
	Cumulative time using simulator	18.8
	Other	18.8
How long does the average resident spend working in a simulation	0–1 hr	37.5
before his or her first live FESS case?	1–3 hr	31.3
	4–6 hr	31.3
	6–10 hr	0
	>10 hr	0
What benefits have you observed from FESS simulation?	No benefit	6.25
	Improved endoscope handling skills	93.8
	Improved anatomic understanding	62.5
	Better able to negotiate endoscope and another instrument simultaneously	87.5
	Quicker learning curve once in operating room situation	62.5
	Fewer surgical complications	12.5
What barriers exist to integrating FESS simulation into the resident curriculum?	Cost restrictions	74.2
	Space restrictions	28.8
	Lack of available high-quality simulations	74.2
	Clinical duties	34.9
	Duty hours restrictions	25.8
	Lack of interest among residents	7.6
	Lack of interest among faculty	15
	Other	7.6

FESS = Functional endoscopic sinus surgery.

Discussion

Increased educational expectations in the context of reduced training hours encouraged many disciplines within medicine to use simulation to augment and accelerate skills attainment. Due to the changing educational environment, questions have been raised about the effectiveness of current teaching methods and how we might better accomplish educational missions.⁸ The Society of American Gastrointestinal and Endoscopic Surgeons has been at the forefront in establishing the need for curriculum development, surgical simulators, and assessment of proficiency.^9–12 Within otolaryngology, individual efforts have also been made, though no unified effort of this kind has taken place.¹³ Development of a formalized curriculum to standardize education and surgical training holds promise to improve ESS training. Surgical simulation has been used in many areas of otolaryngology and general surgery, and this method of learning is congruent with the attitudes of the current generation of residents.³

Simulation is only a part of resident education, and traditional teaching, such as didactics, bedside rounds, and textbooks, remain paramount.¹⁴ However, there is evidence that the current generation of students are open to technology and finding creative ways to learn.¹⁵ Further, they prefer hands-on experiences rather than lectures and enjoy learning by trial and error.¹⁶ One can learn much from errors, and, although certain mistakes are not permissible in the operating room, simulators allow residents a safe place to try new things and procedures. In accord with the learning methods of Generation Y, students generally showed a favorable attitude toward ESS simulation.¹⁷

Simulation is also beneficial because the skill set used for endoscopic procedures differs from open procedures.^18,19 Endoscopic procedures are performed in narrower confines than open procedures and, thereby, there is limited freedom of motion and accessibility. Also, the image on the monitor is displayed in two dimensions, whereas procedures are performed in three dimensions. Depth perception must be learned from other visual and tactile cues. Also, bleeding during live surgery compromises visualization, which adds considerable complexity (and risk) and can detract from learning. For these reasons, simulators offer residents safe alternatives to learn, practice, and improve their endoscopic skills.¹⁸ The ideal model would be inexpensive and readily available, and, in addition, possess face, content, and construct validity.²⁰ Face validity relates to how realistic the model looks. Content validity involves confirming that the skills required while practicing on the simulator translate to real-life situations. Construct validity aims to ensure that the test measures what it is intended to be measured and establishes differences among different skill levels.

Most respondents in this survey maintained that a lack of availability and the high cost of current models prevented their programs from implementing ESS simulation. However, several ESS simulators have been developed and validated for ESS training.²¹ These range from low to very high fidelity, but the ideal model does not yet seemingly exist. Leung et al.²² and Wais et al.²³ used common materials to create a low-fidelity simulator. They demonstrated improvements in performing surgical tasks but did not establish construct validity. Another low-fidelity task trainer was creatively (and inexpensively) assembled by using household goods.²³ Face, content, and construct validity have all been established by using this model, although it is moderately time intensive to build.^24,25

To provide a more-realistic endoscopic image, Burge et al.²⁶ designed a low-cost intermediate-fidelity trainer. The investigators demonstrated that participants learned to perform endoscopic tasks quicker and with fewer errors after using the trainer. Regarding high-fidelity simulators, Weghorst et al.,²⁷ Fried et al.,^28–31 and Arora et al.³² have done monumental work in ESS trainers, and published that their virtual reality simulator with haptic feedback demonstrated validity for teaching and measuring the skills needed in ESS. Another validated high-fidelity virtual reality model has also been validated but also has a high cost.³³ The biggest challenge to this simulator is that few, if any, otolaryngology programs have the funds to purchase such a simulator. The price associated with a system like this was considered by rhinologists in this survey to be one of the foremost obstacles to implementation of simulation. When the available simulators and studies were evaluated together, it seemed that there were multiple iterations of variable levels of fidelity and validity. Due to the disparate nature, a cohesive systematic review of simulators remains problematic.

Respondents in this study observed improved instrument handling and dexterity among residents who participated in ESS simulation. However, scope handling and dexterity may not be the educational objectives for a more-advanced resident. A wide range of fidelity is found among current ESS models, and it is important to consider which models are best suited for residents at different levels of training. For instance, a high-fidelity model that mirrors the complex anatomy of the sinus cavities may overwhelm the early learner whose initial learning objective is scope handling and three-dimensional conceptualization. Approximately 45% of respondents to this survey used some form of simulation in resident education. However, only 23.9% of the programs reported using ESS simulation. This result differed from a recent survey of program directors in the United States, which found that ~78% of the programs used some type of otolaryngologic simulation.³⁴ The difference in results may reflect a greater awareness or availability of simulation outside of rhinology. Whatever the cause, simulation is widely used among otolaryngology residency programs and sinus simulation lags behind.

There were several limitations to this survey. First, the survey method prevented establishing a specific survey response rate. Because the ARS does not have a separate membership category for academic members, the exact number of relevant recipients was unknown. The ARS does not have a current list of otolaryngology program directors, and, therefore, the survey could not be sent directly to this group. Although the survey was kept short to minimize the time that respondents had to commit, academic faculty may have had “survey fatigue.” In addition, because response was voluntary, it is possible that only those with a specific interest in simulation responded. It should also be noted that this survey did not acknowledge clinical intervention with endoscope and suction as an integral step in skill acquisition. Also, although respondents reported improvement in surgical skills after simulation usage, these results were often based on subjective assessments, which highlighted an important point: for simulation to be maximally effective, the component tasks required to be mastered and integrated to complete more complex tasks must be delineated. This has not been done in a systematic way for ESS, and, as a result, a curriculum of progressive skill attainment should be developed to build toward ESS competence.

The coordination of a viewing hand and an intervention hand while watching an unaligned monitor requires significant dexterity and control. The correct order of ESS learning environment remains undetermined and may be (1) endoscopic viewing (simulation and/or clinic), (2) endoscopic suctioning (clinic), (3) endoscopic intervention (simulation and/or operating room). Thus, the steps involved in learning how to perform ESS as well as the optimum environment from which to gain this experience is still open for debate. We believe that simulated environments may improve patient comfort and safety during this learning process.

Conclusion

Fewer than one-fourth of the survey respondents identified using ESS simulation in their respective programs. High cost and limited availability of ESS simulators are the most commonly reported barriers to implementing ESS simulation in residency training. A systematic effort to develop a progressive curriculum by using validated simulators will improve the quality of ESS training during residency.

Footnotes

The authors have no conflicts of interest to declare pertaining to this article

References

Philibret

The competencies: The ACGME and the community in 2008 and beyond. ACGME Bull 1–2, 2008.

Nogueria

J.F.

, Stamm

A.C.

, Lyra

Building a real endoscopic sinus and skull-base surgery simulator. Otolaryngol Head Neck Surg 139: 727–728, 2008.

Reznick

R.K.

, and MacRae

Teaching surgical skills—Changes in the wind. N Engl J Med 335: 2664–2669, 2006.

Darzi

, Smith

, and Taffinder

Assessing operative skill. BMJ 318: 887–888, 1999.

Bhatti

N.I.

, and Ahmed

Improving skills development in residency using a deliberate-practice and learner-centered model. Laryngoscope 125(Suppl. 8): S1–S14, 2015.

Seymour

N.E.

, Gallagher

A.G.

, Roman

S.A.

Virtual reality training improves operating room performance: Results of a randomized, double-blinded pilot study. Ann Surg 236: 458–463; discussion 463–464, 2002.

Sedlack

R.E.

, and Kolars

J.C.

Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: Results of a pilot study. Am J Gastroenterol 99: 33–37, 2004.

Stamm

A.C.

Is the training for paranasal sinus surgery really effective in our medical residencies?

Braz J Otorhinolaryngol 81: 231, 2015.

Wilcox

Jr. , Trus

, Salas

A proficiency-based skills training curriculum for the SAGES surgical training for endoscopic proficiency (STEP) program. J Surg Edu 7: 282–288, 2014.

10.

Soper

N.J.

SAGES and surgical education: Assuring that history does not repeat itself. Surg Endosc 15: 775–780, 2001.

11.

Sánchez-Margallo

F.M.

, Matos-Azevedo

A.M.

, Pérez-Duarte

F.J.

Performance analysis on physical simulator of four different instrument setups in laparo-endoscopic single-site (LESS) surgery. Surg Endosc 28: 1479–1488, 2014.

12.

Van Sickle

K.R.

, Ritter

E.M.

, McClusky

III Attempted establishment of proficiency levels for laparoscopic performance on a national scale using simulation: The results from the 2004 SAGES Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) learning center study. Surg Endosc 21: 5–10, 2007.

13.

Obeid

A.A.

, Al-Qahtani

K.H.

, Ashraf

Development and testing for an operative competency assessment tool for nasal septoplasty surgery. Am J Rhinol Allergy 28: e163–e167, 2014.

14.

Ooi

E.H.

, and Witterick

I.J.

Rhinologic surgical training. Otolaryngol Clin North Am 43: 673–689, xi, 2010.

15.

Lipkin

N.A.

, and Perrymore

A.J.

Y in the Workplace. Franklin Lakes, NJ: Career Press, 2009.

16.

Mangold

Educating a new generation Teaching baby boomer faculty about millennial students. Nurse Educ 32: 21–23, 2007

17.

Glaser

A.Y.

, Hall

C.B.

, Uribe

S J.I.

, and Fried

M.P.

Medical students' attitudes toward the use of an endoscopic sinus surgery simulator as a training tool. Am J Rhinol 20: 177–179, 2006.

18.

Dawson

S.L.

A critical approach to medical simulation. Bull Am Coll Surg 87: 12–18, 2002.

19.

Gallagher

A.G.

, McClure

, McGuigan

Virtual reality training in laparoscopic surgery: A preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy 31: 310–313, 1999.

20.

Gallagher

A.G.

, Ritter

E.M.

, and Satava

R.M.

Fundamental principles of validation and reliability: Rigorous science for the assessment of surgical education and training. Surg Endosc 17: 1525–1529, 2003.

21.

Javia

, and Deutsch

E.S.

A systematic review of simulators in otolaryngology. Otolaryngol Head Neck Surg 147: 999–1011, 2012.

22.

Leung

R.M.

, Leung

, Vescan

Construct validation of a low-fidelity endoscopic sinus surgery simulator. Am J Rhinol 22: 642–648, 2008.

23.

Wais

, Ooi

, Leung

R.M.

The effect of low-fidelity endoscopic sinus surgery simulators on surgical skill. Int Forum Allergy Rhinol 2: 20–26, 2012.

24.

Malekzadeh

, Pfisterer

M.J.

, Wilson

A novel low-cost sinus surgery task trainer. Otolaryngol Head Neck Surg 145: 530–533, 2011.

25.

Steehler

M.K.

, Pfisterer

M.J.

, Na

Face, content, and construct validity of a low-cost sinus surgery task trainer. Otolaryngol Head Neck Surg 146: 504–509, 2012.

26.

Burge

S.D.

, Bunegin

, Weitzel

E.K.

, and McMains

K.C.

The validation of an endoscopic sinus surgery skills training model: A pilot study. Am J Rhinol Allergy 26: 409–413, 2012.

27.

Weghorst

, Airola

, and Oppenheimer

Validation of the Madigan ESS simulator. Stud Health Technol Inform 50: 399–405, 1998.

28.

Fried

M.P.

, Satava

, Weghorst

Identifying and reducing errors with surgical simulation. Qual Saf Health Care 13(Suppl. 1): i19–i26, 2004.

29.

Fried

M.P.

, Sadoughi

, Weghorst

S.J.

Construct validity of the endoscopic sinus surgery simulator: II. Assessment of discriminant validity and expert benchmarking. Arch Otolaryngol Head Neck Surg 133: 350–357, 2007.

30.

Fried

M.P.

, Sadoughi

, Gibber

M.J.

From virtual reality to the operating room: The endoscopic sinus surgery simulator experiment. Otolaryngol Head Neck Surg 142: 202–207, 2010.

31.

Fried

M.P.

, Kaye

R.J.

, Gibber

M.J.

Criterion-based (proficiency), training to improve surgical performance. Arch Otolaryngol Head Neck Surg 138: 1024–1029, 2012.

32.

Arora

, Uribe

, Ralph

Assessment of construct validity of the endoscopic sinus surgery simulator. Arch Otolaryngol Head Neck Surg 131: 217–221, 2005.

33.

Varshney

, Frenkiel

, Nguyen

L.H.

Development of the McGill simulator for endoscopic sinus surgery: A new high-fidelity virtual reality simulator for endoscopic sinus surgery. Am J Rhinol Allergy 28: 330–344, 2014.

34.

Deutsch

E.S.

, Wiet

G.J.

, Seidman

Simulation activity in otolaryngology residencies. Otolaryngol Head Neck Surg 153: 193–201, 2015.