Abstract
Introduction. Healthcare certification standards do not mention
Methods. Senior-level nursing students (N=120) were randomized into groups to complete one pediatric critical care simulation. Simulation-based education (SBE) was delivered across five weeks. Two experienced facilitators, one with pediatric expertise and one without, used the PEARLS method to debrief each simulation. Following each debrief, participants completed a demographic survey and a SET-M. Results were analyzed using SPSS Version 24.
Results. Eighty-four students completed the survey and SET-M instrument. Independent samples t-tests showed no significant difference between facilitators based on the SET-M total scores (p=.516), or mean scores (p=.492). Debriefing-specific items also showed no difference (p=.983).
Conclusion. Subject matter expertise did not change the perception of simulation effectiveness for undergraduate nursing students during a critical care scenario. While SMEs are valuable for scenario creation and coaching non-SME facilitators on key elements for guided reflection, they may not be required to facilitate the debrief. Further research in this area is needed.
Introduction
Healthcare simulation-based education is guided by international standards. Within the International Nursing Association for Clinical Simulation and Learning (INACSL) Standards of Best Practice: SimulationSM Simulation Design (INACSL Standards Committee, 2016c) and the INACSL Standards of Best Practice: SimulationSM Facilitation (INACSL Standards Committee, 2016b), it is suggested that a trained facilitator is needed to conduct a simulation and debrief. However, the issue of being a subject matter expert or content expert, herein referred to as SME, as a prerequisite to guiding the learning experience or post-simulation discussion is not addressed. Similarly, the INACSL Standards of Best Practice: SimulationSM Debriefing (INACSL Standards Committee, 2016a) includes a criterion about the competence of the facilitator in debriefing. Subject matter expertise related to facilitating the simulation is not mentioned. Instead, the criterion stipulates that the facilitator must be educated in debriefing; an additional criterion is that the debriefer is able to assist with learner success in meeting the objectives and outcomes set for the experience as a whole (INACSL Standards Committee, 2016a).
Healthcare simulation educators universally support collaborating with SMEs when considering simulation creation, design, validation, implementation, and/or integration into the curriculum (Cheney & Josey, 2014; May & Edgar, 2016; Waxman, 2010). Support for SME use beyond the design phase of SBE is inconsistent. Some simulation practitioners emphasize it is not necessary to possess clinical expertise in a field to effectively teach the subject (May & Edgar, 2016). Often, those with clinical expertise are asked to facilitate simulations, although they may not have the training or ability to do so (LaFond & Blood, 2016). In some cases, the faulty assumption that an SME makes for a good simulation facilitator can lead to an ineffective debrief session. It has been emphasized that clinical expertise (Koh, 2017) is not sufficient for a simulation instructor to be effective (Ahmed et al., 2016; LaFond & Blood, 2016; McGaghie et al., 2010). Both INACSL (INACSL Standards Committee, 2016a) and the Society for Simulation in Healthcare (n.d.-a) stress the need for simulation debrief training but do not mention any required subject matter expertise to improve student outcomes. While some authors emphasize the importance of having a debriefer (Ahmed et al., 2016; Oikawa et al., 2016) or co-debriefer (Ahmed et al., 2016) who is an SME; a critical review of SBE reveals that subject matter and clinical expertise are not proxies for specific simulation facilitation training and in fact, may be detrimental to the reflection process (McGaghie et al., 2010).
Simulation-based education within healthcare education is on the rise (Seaton et al., 2019) necessitating more simulation faculty. SBE has replaced hands-on clinical hours in various proportions (Breymier et al., 2015). However, for many schools of nursing, hiring and training educators with specific subject matter expertise for all content areas throughout the curriculum could be a daunting task and further, be cost-prohibitive. Simulation-based education (SBE) requires both a financial and human resource investment (Isaranuwatchai et al., 2016). This has left researchers looking for alternatives, such as studying self-debriefing versus instructor-facilitated debriefing (Isaranuwatchai et al., 2016).
Some authors distinguish between subject matter expertise, simulation, and debriefing expertise (Díaz et al., 2016) while recommendations from others seem to confuse the issue. For example, Oikawa et al. (2016) asserts that an SME is a critical characteristic for a quality debrief facilitator and emphasizes that without it, experiential learning cannot occur. However, the critical characteristics are described as skills of guidance, observation, and assessment (Oikawa et al., 2016). These qualities align more with the definition of a debriefer or facilitator rather than an SME. A dearth of literature is noted on the requirement of an SME without providing evidentiary support or clarifying what defines a subject matter expert. This ambiguity in terminology suggests a need for clarification. This study uses the terms expert debriefer and subject matter expert, both individuals are certified as healthcare simulation educators.
The primary aim of this research was to determine if a difference exists in participant outcomes of SBEs facilitated by an expert debriefer, with or without subject matter expertise, using a consistent theory-based debrief during a pediatric critical care scenario.
Methods
This study used a two-group, single-blinded randomized design. Institutional Review Board approval was obtained. All students enrolled in a senior practicum course were eligible to participate (N=120). Members of the research team informed the students of the study protocols and provided a copy through the Learning Management System and in person. All students were randomly assigned to a group regardless of participation in the study. The control group was facilitated and debriefed by a Certified Healthcare Simulation Educator- Advanced (CHSE-A®) (Society for Simulation in Healthcare [SSH], n.d.-b). Another CHSE-A-Advanced with subject matter expertise debriefed the intervention group. Expertise was defined as having over 20 years as an active clinical pediatric nurse, Pediatric Advanced Life Support (PALS) Certification, and certification in a clinical area. The SBE was a mandatory component of the curriculum; however, students were not required to complete the instruments or participate in the study. Verbal consent was obtained from all individuals. It should be noted the SBE used for the study was previously piloted with similar learners in a previous semester by the facilitators.
The facilitators were randomly assigned to the debrief groups by session. During the discussion of the SBE, standard debriefing procedures were followed to ensure consistency, such as addressing the affective, cognitive, and kinesthetic domains. A graduate research assistant (GRA) with CHSE® designation (SSH, n.d.-a), monitored all sessions and took field notes to ensure consistency and accuracy.
Participants and Setting
A convenience sample of 120 eligible undergraduate nursing students was recruited; 84 students voluntarily participated, however; all required demographics and instruments were completed by (n = 83). The SBE took place in an established simulation learning center within the university. A pediatric hospital room was created with high realism and fidelity. Further information is provided under “Simulation.”
Procedure
Members of the research team, provided the Explanation of Research, administered, and collected the surveys to eliminate personal bias in an attempt to remove any suggestion of coercion or pressure to participate. The participants received IRB information sheets through the Learning Management System, as well as in-person, and were advised of the ability to discontinue involvement at any point without consequences.
Pre-simulation activities were provided to prepare students for the upcoming SBE. Pre-simulation activities included content from the classroom didactic, and associated articles relevant to the simulation (Leigh & Steuben, 2018). Debriefers delivered a prebrief and specific information regarding the scenario to the students as a group, and a patient hand-off at the simulated bedside. Prebriefing was defined as the precursor to the SBE, including information about equipment and the environment, schedule, expectations, etc. to facilitate participants meeting the SBE objectives (Lioce et al., 2020). The prebrief facilitator was also the debriefer. The pre-written, scripted content of the prebrief was consistent between all groups. Roles for participants were primary nurse, secondary nurse, member of the code blue team, or participant observer.
After the SBE, a structured debrief was facilitated using the Promoting Excellence and Reflective Learning in Simulation (PEARLS) method (Cheng et al., 2016; Eppich & Cheng, 2015) by one of two experienced debriefers (one with subject matter expertise, the other without). See further information under “Debrief.” After the debrief, each student was given an electronic tablet with a link to an anonymous survey.
Simulation
The simulation was based on a pediatric respiratory crisis and subsequent cardiopulmonary arrest. Within the pediatric curriculum, students were exposed to several key characters during SBE. Hence, the students were familiar with the pediatric patient, presenting with an asthma exacerbation, from a previous SBE (Díaz et al., 2020). For this hospital admission, the patient’s respiratory symptoms worsened, and he presented acutely ill with respiratory distress. The scenario objectives included: recognition of the decline in the patient’s status and initiation of cardiopulmonary resuscitation skills (CPR). All students were required to have Basic Life Support (BLS) before enrolling in the program. The simulation template was laminated and remained consistent among groups. The SBE lasted approximately 15-17 minutes. Didactic educators leveled the objectives of the SBE for the end of the semester students completing the Pediatric/Families didactic course.
Debrief
The PEARLS debriefing approach (Cheng et al., 2016; Eppich & Cheng, 2015) was utilized by both debriefers. According to the creators of this debriefing method, this blended approach has four phases: reactions, description, analysis, and summary (Cheng et al., 2016; Eppich & Cheng, 2015). In the reaction phase, emotions/feelings are shared by each participant. During the description phase, the participants discuss the understanding of the different portions of the case (Cheng et al., 2016). Self-assessment and framed discussions are promoted in the analysis phase, followed by key learning points in the summary phase (Cheng et al., 2016; Eppich & Cheng, 2015). Scripting, a written plan for debrief, is used (Eppich & Cheng, 2015).
Information on the PEARLS debriefing approach (Cheng et al., 2016; Eppich & Cheng, 2015) is widely available. The Debrief2Learn (n.d.) website offers tools for facilitators to include a useful guide of common student errors and solutions to implement when using PEARLS. There are recommendations for common pitfalls and ways to mitigate them along with a script (Cheng et al., 2016; Eppich & Cheng, 2015). The debriefers in this study had been previously exposed to this technique at conferences and during web-based training. It is noted that neither debriefer utilized PEARLS as their preferred method to debrief. To ensure consistency during the debriefs, the PEARLS pocket card was placed on the table to serve as a visual reminder and guide (Bajaj et al., 2018). At the beginning of the debrief, to elicit the affective domain including reactions, the debriefers used the Rapid-Fire Huddle (Díaz et al., 2019). As part of the Rapid-Fire Huddle, participants offered one word on how they felt, followed by an uninterrupted five minutes where the group processed the simulation without debriefer input. This was followed by structured debrief, using the PEARLS approach (Cheng et al., 2016; Eppich & Cheng, 2015). All the debriefs lasted approximately 30 minutes.
Instruments
Demographics
Demographics of the sample included ethnicity, age, grade expected, and certification in Pediatric Advanced Life Support (PALS).
SET-M
The Simulation Effectiveness Tool – Modified (SET-M) is a 19-item instrument scored on a 3-point Likert scale (Leighton et al., 2015). The instrument explores student perceptions on four domains. The domains are prebriefing, learning, confidence, and debriefing. The debriefing subscale contains five items that elicit feedback about the participant’s perception of how the debrief contributes to learning. Previous psychometrics for the four domains within the instrument were reported with the debriefing subscale demonstrating good internal consistency reliability (α=.908). All other domains had a Cronbach’s alpha of .833 or higher with the overall instrument of α=0.936 (Leighton et al., 2015). The SET-M instrument was used to evaluate perception of effectiveness by students.
Open-ended questions/affective words
Students were asked questions within the electronic survey following the debrief. The students were asked to write answers to the affective questions (see Table 1). This report will not explore the qualitative aspect of this study. Open-ended questions pertaining to the Rapid-Fire Huddle (Díaz et al., 2017, 2019) and other aspects of this SBE will be reported in generality rather than specifics for this manuscript. The focus of this manuscript is the quantitative analysis and implications of the SET-M (Leighton et al., 2015) in relationship to SME and debriefing.
Open-Ended Questions.
Statistical Analyses
Descriptive statistics were analyzed using Statistical Package for Social Science (SPSS) Version 24 (IBM, n.d.). Data analyzed were from the 83 student participants who completed the study. Members of the research team performed pretest checks to determine the appropriate statistical tests (see Table 2). Parametric testing was conducted using analysis of variance (ANOVA) to determine differences between the groups.
Pretest Check.
Results
Demographics
The sample included n=83 undergraduate nursing students (see Table 3). The sample was predominately White (64%), female (83%) students between the age of 18-24 years (90%). Ninety-three percent of the respondents did not have PALS certification.
Demographics.
SET-M (Leighton et al., 2015)
The debrief subscale was reported due to the nature of the study. A Kolmogorov-Smirnov test revealed a non-normal distribution of the data; therefore, a Mann-Whitney U test was used for comparisons of group scores from the SET-M instrument (Leighton et al., 2015). Results from the analysis demonstrated no significant difference (p = 0.44) between the SME group (µ = 2.51) and the non-SME group (µ = 2.45) for SET-M mean scores (see Table 4). Total SET-M scores also lacked a significant difference (p = 0.47) for those in the SME group (µ = 47.7) and non-SME group (µ = 46.6).
SET-M Total.
Confidence and debriefing
No significant differences in confidence and debriefing subscales were noted between debriefers. Those in the control group and intervention group had similar outcomes of SET-M scores, p = 0.24.
Prebriefing and learning
One item in the subscales for prebriefing and learning was found to be significantly different between debriefers (see Table 5). Mean SME group scores for providing support to learning were significantly higher than non-SME scores (2.58 vs. 2.27, p=0.01). Those in the non-SME group agreed more strongly at being better prepared to manage a change in their patient’s condition as compared to those in the SME group (2.85 vs. 2.65, p=0.03).
ANOVA SET-M Total Table.
Discussion
Similar to nursing education’s struggle with faculty shortages, simulation-based education is equally challenged by limited faculty resources. In 2018-2019, The American Association of Colleges of Nursing reported that of the respondents 488/871 schools or 56% of member schools had full-time vacancies (Li et al., n.d.). There are a finite number of trained simulation educators, even fewer with certification or advanced certification. This motivation, along with an interest in clarifying terminology and identifying evidence to support a recommendation on requisite skills and background for individuals who debrief simulations, was the catalyst behind this study.
A trained debriefer could learn content and use the experiential attributes from their peers in an effort to increase comfort by offering suggestions for interventions and patient care. It should be noted that the SME developed a comprehensive binder with information on the SBE to ensure students had an equitable experience. The trained debriefer, who was not an SME, relied heavily on the binder given the individual’s lack of specific expertise. Despite this obvious limitation, there was no difference in reported outcomes.
Anecdotally, the SME was more adept at providing real-world examples related to nursing diagnosis and patient safety. For example, a debriefer may be more willing to recommend an intervention in which they are familiar with and have knowledge of the positive outcomes. This is believed to be the difference between the two expert debriefers. The SME had experiences to draw upon and offer an extension of knowledge as applied to the specific population. These findings can be likened to an iceberg where the SME can do the deep dive, while the expert debriefer may need to stay above water and more superficial when responding to technical questions.
It is believed with appropriate guided reflection questions and written expanded rationales, such as included in the binder, trained or expert debriefers without subject matter expertise could extend that knowledge as well. The most important aspect of this experience was the role scripting played as mentioned above (Eppich & Cheng, 2015). The SME can provide examples within the written materials for consistency. The examples of different interventions that promote patient safety and improve care could be beneficial to the non-SME. The education principles of a good debriefer do not include being a subject matter expert but someone principled, dynamic, creative, versed in good debriefing pedagogy technique and willing to evolve and increased frequency. This is a call to action for organizations to invest in faculty development and to encourage debriefers to create scenarios that include debriefing questions, rationales, and patient examples, along with specific references for further information/reading. Educators should be taught that these items are crucial when developing simulated scenarios.
From an administrative lens, these findings are encouraging. Deans and Chairs should use their certified simulationist in conjunction with the content SME as “train-the-trainer” (Ahmed et al., 2016, p. 337). This would increase the pool of qualified simulation faculty. With advanced planning, the SME can prepare the trained debriefer. What is inherent in this model is the need to have expertise in debriefing. Resources may be better spent developing expert debriefers. An expert debriefer should be able to guide the learner to self-reflection and revelation. A noted requisite to be an effective debriefer is proper training, enthusiasm, active listening and professional integrity (INACSL Standards Committee, 2016a, 2016b).
Debriefers come from different backgrounds, have different personal and professional experiences, as well as varied beliefs. Expert debriefers approach each SBE slightly different; variations and nuances exist. The comfort level related to the content/specialty may be the attributing factor. The learners, in this experience, may have sensed a difference in the learning environment if the debriefer was tentative teaching the subject matter. A thorough run-through, i.e. pilot, and dialogue may mitigate these findings. Shadowing an SME for one or two scenarios and/or having a SME offer suggestions to a non-subject matter expert may be helpful as well.
Limitations
Several limitations in this study were noted. Scheduling of numerous simulations during the same timeframe created some initial confusion with randomizing participants and debriefers; however, randomization did occur. The participants were all learners having some pediatric content creating comfort in the information presented.
The experienced debriefers involved in the study were purposefully limited to two which from a pedagogical standpoint makes sense. It allowed for consistent practices during the debrief. However, we recognize this same consistency is also a flaw in the design and does limit the generalizability of our findings. Using only two debriefers resulted in fatigue and may have resulted in inconsistencies towards the end of the day. Additionally, the instruments were self-report. Although there were some differences in self-reported simulation effectiveness between the debriefers, this may not translate to actual differences in learning or other student outcomes.
Implications for Nursing Education
Loss of clinical placements and the need for increased simulation activities can be daunting and expensive if specialized faculty are required for every experience. Simulation can be utilized to decrease the burden of clinical placements and the need for increased clinical faculty, thus, impacting the strain on programs. Emphasis should be noted on the collaboration between the SME and expert debriefer. Not only a scenario outline or checklist, but an exploration of common learner questions and appropriate answers must be discussed and occur to ensure consistency among debriefers. Reflective practice for the debriefer and learner is essential in promoting optimal outcomes (Schön, 1987). A set of examples pertaining to the content within the clinical setting could be helpful for the expert debriefer. Shadowing of either the SME in the same simulation or shadowing of the expert debriefer by the SME with feedback may prove beneficial. It is also of importance that imparting a colleague with lived examples allows the learners to apply that to their frame of reference.
Future Studies
Given this study included only two expert debriefers, it begs the question would the findings have been different if another SME and non-SME were to conduct the debriefings? With this in mind, future work will include a greater pool of debriefers from different content areas and with further outcomes, beyond self-report.
Conclusion
In this study, there was no statistical difference between the SME and debrief expert’s simulation effectiveness responses from learners. The SME informed the debrief expert related to the specific content of the SBE. The need for the application of the lived experience of the debriefer was evident in the individual item referencing interventions. A close relationship within the SBE team needs to be the foundation of all experiences, reinforcing content and exploring thought processes while changing future practice. Lastly, this study has implications on simulation template formation and with educators who teach others how to write and implement simulations. Future work exploring the differences between expert debriefers and the subject matter expert in this area is recommended.
Footnotes
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Drs Anderson and Díaz are Associate Editors for Simulation and Gaming. Dr. Anderson is also the Vice President of Operations for INACSL.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Author Biographies
Contact:
Contact:
Contact:
Contact:
Contact:
Peggy P. Hill, MSN, RN, CHSE, prepared associate instructor for pre-licensure and RN to BSN students. She is currently a PhD candidate researching the use of simulation-based education to teach interruption management skills.
Contact: