Impact of simulation on multidisciplinary NICU teamwork during delivery and transport of extremely preterm infants

Abstract

BACKGROUND AND OBJECTIVES:

Multidisciplinary teamwork during delivery room (DR) resuscitation and timely transport to the neonatal intensive care unit (NICU) can reduce morbidity and mortality for infants born Extremely Preterm (EP). We aimed to assess the impact of a multidisciplinary high-fidelity simulation curriculum on teamwork during resuscitation and transport of EP infants.

METHODS:

In a prospective study conducted at a Level III academic center, seven teams (each consisting of one NICU fellow, two NICU nurses, and one respiratory therapist) performed three high-fidelity simulation scenarios. Videotaped scenarios were graded by three independent raters using the Clinical Teamwork Scale (CTS). Times of completion of key resuscitation and transport tasks were recorded. Pre- and post- intervention surveys were obtained.

RESULTS:

Overall, time of completion of key resuscitation and transport tasks decreased, with significant decreases in the time to attach the pulse oximeter, transfer of the infant to the transport isolette, and exit the DR. There was no significant difference in CTS scores from Scenario 1 to 3. Scenarios led by first-year fellows showed a trend towards improvement in all CTS categories. A comparison of teamwork scores pre- and post-simulation curriculum during direct observation of high-risk deliveries in real time revealed a significant increase in each CTS category.

CONCLUSION:

A high-fidelity teamwork-based simulation curriculum decreased time to complete key clinical tasks in the resuscitation and transport of EP infants, with a trend towards increased teamwork in scenarios led by junior fellows. There was improvement of teamwork scores during high-risk deliveries on pre-post curriculum assessment.

Abbreviations

BPD:

Bronchopulmonary Dysplasia

CTS:

Clinical Teamwork Scale

DR:

Delivery Room

EP:

Extremely Preterm

IVH:

Intraventricular Hemorrhage

NICU:

Neonatal Intensive Care Uni

NRP:

Neonatal Resuscitation Program

PPV:

positive pressure ventilation

RT:

respiratory therapist

ROP:

Retinopathy of Prematurity

1 Introduction

Delivery room resuscitation and post-resuscitative care of an Extremely Preterm (EP) infant is a critical period during which the efficiency of medical interventions provided can have long-term implications on neonatal outcomes [1]. Effective, well-coordinated institution of neonatal resuscitative steps and shorter transport time of the EP infant to the Neonatal Intensive Care Unit (NICU), has been shown to improve survival and decrease the incidence of bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), and retinopathy of prematurity (ROP) [1, 2]. Major determinants of effective resuscitation and delivery of optimal care during the critical post delivery period include teamwork, communication, and collaboration between the multidisciplinary team members [3, 4]. The benefits of cohesive, well-coordinated multidisciplinary teamwork were first emphasized in the Joint Commission’s 2004 Sentinel Event Alert, which reported that poor communication was the most common root cause of infant death or permanent disability, and that failure to function as a team was a barrier to effective communication [5].

Additionally, transport of the EP infant from the delivery room to the NICU is often a stressful experience for providers due to space and resource constraints; this is especially true if interventions are required during the transport period. Multidisciplinary teamwork is a key aspect of this process. Currently, there is a paucity of literature examining the influence of teamwork on the time to transport an infant from the delivery room (DR) to the NICU, as well as on the timely identification of and appropriate response to the occurrence of adverse events during this transit phase.

Current data supports that high-fidelity simulations can play a role in improving provider comfort with neonatal resuscitation [6, 7] as well as increasing competence in neonatal procedural skills [8]. In addition, there is a strong correlation between the acquisition of technical skills and the demonstration of strong behavioral skills –for example, teamwork training in addition to traditional Neonatal Resuscitation Program (NRP) simulations can lead to faster and more comprehensive resuscitations [9 –11]. Thus, the acquisition of teamwork skills, particularly closed-loop and direct communication, situational awareness, and role assignment is now strongly emphasized in standard NRPtraining [12].

In our simulation cohort, we tested the following hypotheses: 1) a series of three high-fidelity simulation exercises will improve multidisciplinary teamwork during transport of an EP infant from the delivery room to the NICU and 2) administration of the sequential simulation scenarios will improve the time to perform key clinical tasks during resuscitation and transport of an EP infant from the delivery room to the NICU.

2 Methods

This IRB-approved prospective study was conducted from December 2017 to May 2019 at Tufts Medical Center (Boston, MA), an academic center with a Level III NICU. Informed consent was obtained from study members participating in the simulation scenarios.

2.1 Study population

All neonatology attendings, fellows, nurses, and respiratory therapists who attend deliveries were considered eligible for this study.

2.2 Clinical Teamwork Scale

The Clinical Teamwork Scale (CTS) was developed and validated in the delivery room setting as a tool to objectively evaluate multidisciplinary teamwork [13]. The scale consists of fifteen elements within five categories: communication, situational awareness, decision making, role responsibility, and overall teamwork. The CTS is graded on a Likert scale from 0 to 10 with 0 being unacceptable, 1 to 3 being poor, 4 to 6 being average, 7 to 9 being good, and 10 being a perfect score. This tool has been shown to demonstrate substantial agreement (Kappa 0.78) and interrater reliability (interclass correlation coefficient 0.98). This study utilized the CTS scoring sheet and definition of teamwork terms that were published with the original article in order to evaluate teamwork performance during real-time deliveries as well as simulations [13]. (Additional information about the CTS is published online by the State Obstetric & Pediatric Research Consortium[14]).

2.3 Pre-intervention: Survey and direct observations

The need for a teamwork-based simulation curriculum was ascertained by a comparison of a self-assessment of teamwork on a needs assessment survey and direct observations of teamwork during eight high risk deliveries. The study team distributed the anonymous teamwork needs assessment survey to all NICU fellows, attending physicians, nurses, and respiratory therapists (RTs). The survey included questions about participant demographics and current teamwork perceptions during resuscitation and transport of an EP infant. A designated study team member who did not participate in this study’s simulations then observed eight high-risk deliveries and graded them using the CTS.

2.4 Interventions

Three high-fidelity simulation scenarios were conducted over a five-month period, with a two-month interval between scenarios (Supplementary Material Figure 1). Each scenario involved the delivery room resuscitation and transport of an EP infant (all between 25–27 weeks gestation) to the NICU. Seven teams were recruited, each consisting of a neonatal fellow, two neonatal nurses, and a respiratory therapist, with the intent for all teams to perform the three scenarios sequentially. The second and third scenarios were designed to evaluate the NICU team’s response to an adverse event occurring during DR transport to the NICU, and involved a bradycardic event that requiring an immediate intervention by the multidisciplinary team. These scenarios were created to mimic two possible etiologies of an adverse event occurring during DR transport where multidisciplinary teamwork skills may impact resuscitation efforts. In Scenario 2, this event was secondary to endotracheal tube obstruction, and in Scenario 3, the event was secondary to respiratory equipment failure. This was an intentional increase in complexity of scenario 2 and 3, to test the team performance and assess effectiveness of the teamwork teachings as part of the simulation curriculum. All seven teams performed the first two scenarios, but due toparticipant availability, only four teams performed the final scenario. While all study participants participated in the scenarios in sequence, individual team members varied between scenarios due to scheduling conflicts.

All neonatal fellows (n = 7), nurses (n = 96), and respiratory therapists (n = 19) who regularly attend deliveries of preterm infants were considered eligible for this study. “Junior” fellows were defined as being in the first year and “senior” fellows as being in the second or third year of the neonatal-perinatal medicine fellowship training program at the start of the study.

All scenarios were initiated in Labor and Delivery, continued during transport in the isolette to the NICU, and ended once the infant was placed on telemetry in the NICU. A Premature Anne high fidelity simulator (Laerdal Medical Inc.) was used for all scenarios. All simulations were videotaped using a Samsung Gear 360 (2017) camera for future review. Participants consented to videotaping by signing an IRB approved informed consent form.

Following each scenario, the teams underwent a structured debriefing using key elements of the CTS including role clarity, situational awareness, and directed and closed-loop communication to help facilitate discussion. Each simulation scenario and subsequent debrief lasted one hour. All eighteen recordings were graded for teamwork using the CTS by three independent raters (two neonatologists and one neonatal nurse). The raters were blinded to the sequence of simulations and graded the scenarios individually. To standardize the grading process, raters were trained on usage of the CTS prior to reviewing the scenarios, and were refreshed after grading three simulation session videos.

2.5 Post-intervention: Survey and direct observations

The study team distributed a post-intervention teamwork survey to the same group of participants as the pre-intervention survey. A designated study team member who did not participate in this study’s simulations directly observed seven high risk deliveries and graded them using the CTS to assess change in teamwork post-intervention. There was no matching of participants between the simulation scenarios and the pre and post intervention direct observations of high risk deliveries.

2.6 Outcome measures

The impact of teamwork training through simulation was assessed using Kirkpatrick’s 4-level model of training evaluation [15]. Pre- and post-intervention surveys administered to NICU staff were used to evaluate learner reactions and learning (level 1 & 2) by assessing a) the need for teamwork training and b) self-perception of teamwork skills before and after intervention. Learner behavior (level 3) was evaluated using a) objective teamwork skills assessment pre- and post-intervention during real-time neonatal resuscitation in the delivery room and b) time of completion of key resuscitation tasks in each scenario, including time to attaching the pulse oximeter, initiation of positive pressure ventilation (PPV), transferring the infant to the transport isolette, departure time from delivery room, and arrival time in the NICU.

2.7 Statistical analysis

Descriptive statistics (means, medians and standard deviations) were used to summarize data obtained through surveys. A Student t-test (with an α level of 0.05) was used to analyze a) inter-scenario differences in CTS scores, b) time of completion of key resuscitation and transport tasks during the simulations, and c) pre- and post-intervention CTS scores obtained during direct observations of high-risk deliveries.

3 Results

3.1 Pre-intervention

A total of 143 eligible members of staff were contacted to complete the pre-intervention survey, with a response rate of 38% (54/143). The majority of respondents (30/54; 56%) were nurses, followed by neonatologists (13/54; 24%), respiratory therapists (6/54; 11%), and fellows (5/54; 9%). Notably on the pre-intervention survey, 100% of fellows and neonatologists felt there was a need for formal teamwork training in the NICU, versus only 70% of nurses and 50% of RTs.

Figure 1A demonstrates a self-assessment of teamwork skills derived from the pre-intervention survey in four areas: overall teamwork during resuscitation, teamwork during transport from the DR to the NICU, communication, and role clarity. Of the survey respondents, 83% (45/54) of participants felt current teamwork behaviors during DR transport were satisfactory to exceptional versus 17% (9/54) felt a need for improvement. There was no difference in self-assessment of teamwork based upon profession or years in practice. Figure 1B shows mean CTS scores obtained during direct observation of eight high-risk deliveries. The range of CTS scores was between 5.3 to 6.3 on a scale of 0 to 10, demonstrating an overall average performance. The areas needing most improvement during direct observation (mean scores) were direct communication (5.8), closed-loop communication (5.3), transparent thinking (5.5), and role clarity (5.9). The disparity between an overall positive self-assessment of teamwork skills versus the average real-time delivery room performance during objective evaluation of teamwork led us to identify a need for a teamwork-based multidisciplinary simulation curriculum.

Fig. 1

A. Comparison of self-perception of teamwork skills during resuscitation and transport pre- and post-intervention. Values expressed as percentage.

Fig. 1

B. Mean clinical teamwork scale scores obtained during direct observations of high-risk deliveries pre- and post-intervention. p < 0.01 for all categories.

3.2 Intervention –simulation curriculum

There was no significant difference between overall CTS and individual CTS category scores of Scenario 1 versus 2 (7.4 vs 6.8; p = 0.4), and of Scenario 1 versus 3 (7.4 vs 7.3; p = 0.9). However, when comparing mean difference in CTS scores of junior and senior fellows from the first scenario to last scenario, there was a trend towards improved teamwork in teams led by junior fellows (6.0 vs 6.8; p = 0.44) (Fig. 2).

Fig. 2

Mean difference in clinical teamwork scale scores from first scenario to last scenario – a comparison of junior versus senior fellow led scenarios.

There was a reduction in time to complete key resuscitation tasks as participants progressed from Scenario 1 to Scenario 3 (Fig. 3). This included a statistically significant reduction in the time to attach the pulse oximeter (0.8±0.3 min vs 0.6±0.1 min; p = 0.02), time to transfer infant to a transport isolette (11.8±3.5 min vs 8.7±1.7 min; p = 0.05), and the departure time from the DR (14.0±3.4 min vs 10.4±1.3 min; p = 0.02).

Fig. 3

Time to perform key resuscitation tasks for all scenarios. Values expressed as mean in minutes. * denotes p < 0.05.

3.3 Post-intervention

The post-intervention teamwork survey response rate was 39% (56/143), with a similar distribution of responses among the disciplines (64% nurses, 21% neonatologists, 7% respiratory therapists, and 7% fellows). A comparison of pre- and post-intervention surveys revealed more respondents reporting satisfactory-exceptional teamwork during delivery room resuscitation post-intervention (76% (41/54) versus 86% (48/56)).

A comparison of teamwork scores obtained during direct observation of high-risk deliveries revealed a significant increase in each category of the CTS, with p < 0.05 for all categories of CTS (Fig. 1B). The overall teamwork score improved from 6.3±1.3 pre implementation of simulation curriculum to 9.0±0 post implementation (p < 0.001).

4 Discussion

Teamwork training has been incorporated into nationally standard NRP simulations, resulting in faster and more comprehensive resuscitations as compared to a control group [3, 8]. Currently, there is a paucity of studies evaluating the impact of high-fidelity simulations on teamwork and other critical clinical measures in a neonatal environment outside of a delivery room setting. In our prospective study conducted at an academic Level III NICU, institution of a high-fidelity teamwork-based simulation intervention decreased time of completion of key clinical tasks in the resuscitation and transport of an EP infant, with a trend towards increased teamwork in scenarios led by junior fellows. While no significant increase in CTS scores was seen among the sequential simulation scenarios,there was a significant improvement in real-time teamwork scores during delivery room resuscitation from the average to good category (6.3 vs 9.0 pre-intervention versus post-intervention, p < 0.05). The latter two scenarios were more complex than the first scenario, and this may have attenuated the effect of our intervention upon overall CTS scores. Of note, teamwork scores did not decrease during Scenarios 2-3 despite an unexpected complication occurring en route to the NICU, a time during which there is often a communication breakdown leading to a decline in overall teamwork. This suggests a possible beneficial impact of the team-based intervention upon overall performance. In addition, despite the increasing complexity of the scenario, key resuscitation tasks were performed earlier during the latter two scenarios. For example, the pulse oximeter was attached approximately 17 seconds sooner in Scenario 3 compared to Scenario 1, and the time to exit the delivery room occurred 3.6 minutes earlier in Scenario 3 versus 1. It is possible that the other differences may have reached significance with a larger sample size. The decrease in time to resuscitate the infant and shortening the time to transport the infant to the NICU has the potential to help attain the golden hour targets for EP infants, which has been shown to improve both immediate and long term outcomes[16, 17].

In our study, while there was no significant improvement in teamwork scores from Scenario 1 to Scenario 3, there was a trend towards improved teamwork in teams led by junior fellows versus those led by senior fellows. While it is possible that this improvement may be secondary to the natural academic progression from first to second year, all scenarios occurred from May to September 2018. Therefore, the junior fellows were at the conclusion of their first year of fellowship, and presumably had better skills than had these scenarios been administered at the start of their fellowship. It is also possible that teamwork training may have more of a discernible impact upon less experienced clinicians (such as first year neonatology fellows, residents and/or medical students), and this would be an interesting area of future study.

While twenty-one study participants performed at least two scenarios, individual teams were not consistent between scenarios. The variability in team composition may have attenuated the effect of our intervention upon overall CTS scores. However, variation of individual team members is pragmatic and reflective of real-time delivery room resuscitation, as it is unlikely the same multidisciplinary team members consistently attend deliveries together. The tenets of teamwork training could ostensibly be viewed as not only a team-based endeavor, but also an acquisition of individual communication skills that can be extrapolated and applied to future delivery room scenarios with different team members.

Particularly striking about the pre-intervention survey responses was that 100% of all physicians felt a need for formal teamwork training within the NICU, whereas only 70% of nurses and 50% of RTs reported the same. This is consistent with prior teamwork studies [18 –20], which demonstrated that nurses’ perceptions of nurse-physician collaborations were demonstrably more positive in comparison to physicians’ perception of the interprofessional relationship.

Of note, a comparison of the pre-intervention teamwork survey and the direct observation scores revealed a disparity between self-assessment of teamwork skills and actual real-time delivery room performance. While most members of the multidisciplinary team felt teamwork in the NICU was satisfactory to exceptional, the direct observations demonstrated only an average performance as graded by the CTS (scores ranging 5.3–6.3). This disparity between self-perception and objective performance rating is also consistent with previously published data [21 –23], and led us to identify a need for a multidisciplinary teamwork-based simulation curriculum within our NICU.

Post-intervention survey responses demonstrated a trend towards an overall improvement in provider attitudes regarding teamwork within the NICU. Objective teamwork assessment during real-time DR resuscitation and transport of high-risk infants also showed a significant increase in CTS scores post-intervention. We cannot definitively attribute this increase due to our intervention alone, as it could be secondary to increased provider experience and different providers involved during the real-time deliveries versus the simulations. However, there is a paradigm of educational thought known as “Train the Trainer” which involves performing an intervention with a small subset of people who then can disseminate any concepts or skills learned to the general population by adapting and teaching it themselves. Therefore, while teamwork scores may not have increased during the study period, there is potential for long-term impact upon multidisciplinary teamwork within the NICU, which is demonstrated by the increased CTS scores post-intervention.

There were several limitations to our study. From its inception, our study was limited by the number of fellows enrolled in the fellowship program. Multicenter expansion of this study would be helpful to more clearly discern any significant differences in CTS scores and time to complete resuscitation tasks by level of training. In addition, our institution underwent mandatory Team STEPPS training, which is a formal teamwork training course within healthcare. This training occurred in April 2018, so did not impact the results of the pre-intervention teamwork surveys; however, part of the training did occur prior to conducting the simulation scenarios, which may have lessened the effect of our intervention upon CTS scores. Additionally, attending neonatologists were unable to participate in our simulations. Further studies involving attending neonatologists would be helpful to more accurately reflect the complete multidisciplinary team.

5 Conclusion

Institution of a high-fidelity teamwork-based simulation curriculum has the potential to decrease time to complete key clinical tasks in the resuscitation and transport of an EP infant. There was an improvement in real-time teamwork during high-risk delivery room resuscitation and transport. There was a trend towards improved teamwork with scenarios involving junior versus senior fellows; thus, teamwork training has the potential to improve teamwork in scenarios led by less experienced clinicians.

Footnotes

Acknowledgments

We would like to acknowledge Jaclyn Boulais, MD, Jennifer McGuirl, DO, and Joy Hilliard, RN, for their assistance in data review for this project. We would like to acknowledge Rachana Singh, MD, MS for her assistance in reviewing and revising this manuscript.

Additional information

Conflict of interest

The authors have no conflicts of interest to disclose.

Funding source

No funding was obtained or utilized for this study.

Ethics approval and consent to participate

This study was reviewed and approved by the Tufts Health Sciences Institutional Review Board (IRB), #12669. All study team members consented to videotaping by signing an IRB-approved Informed Consent Form (ICF). The study was performed in accordance with the Declaration of Helsinki.

The supplementary material is available in the electronic version of this article: .

References

Sharma

Golden hour of neonatal life: Need of the hour. Matern Health Neonatol Perinatol 2017;19:16.

Lapcharoensap

, Bennett

, Powers

, Finer

, Halamek

, Gould

, et al. Effects of delivery room quality improvement on premature infant outcomes. J Perinatol 2017;37(4):349–54.

Baker

, Gustafson

, Beabuien

, Salas

, Barach

, Battles

Medical teamwork and patient safety: The Evidence-based relation. 2005.

Marsch

, Müller

, Marquardt

, Conrad

, Tschan

, Hunziker

Human factors affect the quality of cardiopulmonary resuscitation in simulated cardiac arrests. Resuscitation 2004;60:51–6.

Joint Commission on Accreditation of Healthcare Organizations Preventing infant death and injury during delivery. Adv Neonatal Care 2004;4(4):180–1.

Lizotte

, Janvier

, Latraverse

, Lachance

, Walker

, Barrington

, et al. The impact of neonatal simulations on trainees’ stress and performance: A parallel-group randomized trial. Pediatr Crit Care Med. 2017;18(5):434–41.

Surcouf

, Chauvin

, Ferry

, Yang

, Barkemeyer

Enhancing residents’ neonatal resuscitation competency through unannounced simulation-based training. Med Educ Online 2013;21(18):1–7.

Bender

, Kennally

, Shields

, Overly

Does simulation booster impact retention of resuscitation procedural skills and teamwork? J Perinatol

2014;34(9):664–8.

Gilfoyle

, Koot

, Annear

, Bhanji

, Cheng

, et al. Improved clinical performance and teamwork of pediatric interprofessional resuscitation teams with a simulation-based educational intervention. Pediatr Crit Care Med 2017;18:62–9.

10.

Reed

, Hermelin

, Kennedy

, Sharma

Interdisciplinary onsite team-based simulation training in the neonatal intensive care unit: A pilot report. J Perinatol 2017;37(4):461–4.

11.

Sawyer

, Leonard

, Sierocka-Castaneda

, Chan

, Thompson

Correlations between technical skills and behavioral skills in simulated neonatal resuscitations. J Perinatol 2014;34:781–6.

12.

Kattwinkel

, Weiner

, Zaichkin

Textbook of neonatal resuscitation (NRP). 7th ed: American Academy of Pediatrics, issuing body; 2016.

13.

Guise

, Deering

, Kanki

, Osterweil

, Li

, Mori

, et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc 2008;3(4):217–23.

14.

Clinical Teamwork Scale (CTS™) The State Obstetric & Pediatric Research Consortium [Available from: https://storc.org/cts.

15.

Kirkpatrick

, Kirkpatrick

Kirkpatrick’s four levels of training evaluation. : ATD Press; 2016.

16.

Croop

SEW

, Thoyre

, Aliaga

, McCaffrey

, Peter-Wohl

The golden hour: A quality improvement initiative for extremely premature infants in the neonatal intensive care unit. J Perinatol 2020;40(3):530–9.

17.

Peleg

, Globus

, Granot

, Leibovitch

, Mazkereth

, Eisen

, et al. “Golden hour” quality improvement intervention and short-term outcome among preterm infants. J Perinatol 2019;39(3):387–92.

18.

Yildirim

, Ates

, Akinci

, Ross

, Selimen

, Issever

, et al. Physician-nurse attitudes toward collaboration in Istanbul’s public hospitals. Int J Nurs Stud 2005;42(2).

19.

Sterchi

Perceptions that affect physician-nurse collaboration in the perioperative setting. AORN J. 2007;86(1):45–57.

20.

Thomson

Nurse-physician collaboration: A comparison of the attitudes of nurses and physicians in the medical-surgical patient care setting. Medsurg Nurs. 2007;16(2):87–91.

21.

Davis

, Mazmanian

, Fordis

, Van Harrison

, Thorpe

, Perrier

Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA 2006;296(9):1094–102.

22.

Barnsley

, Lyon

, Ralston

, Hibbert

, Cunningham

, Gordon

, et al. Clinical skills in junior medical officers: A comparison of self-reported confidence and observed competence. Med Educ 2004;38(4):358–67.

23.

Coberly

, Goldenhar

Ready or not, here they come: Acting interns’ experience and perceived competency performing basic medical procedures. J Gen Intern Med 2007;22(4):491–4.