A Pilot Trial of Online Simulation Training for Ebola Response Education

Abstract

This article describes a pilot trial of an internet-distributable online software package that provides course materials and built-in evaluation tools to train healthcare workers in high-risk infectious disease response. It includes (1) an online self-study component, (2) a “hands-on” simulation workshop, and (3) a data-driven performance assessment toolset to support debriefing and course reporting. This study describes a pilot trial of the software package using a course designed to provide education in Ebola response to prepare healthcare workers to safely function as a measurable, high-reliability team in an Ebola simulated environment. Eighteen adult volunteer healthcare workers, including 9 novices and 9 experienced participants, completed an online curriculum with pre- and posttest, 13 programmed simulation training scenarios with a companion assessment tool, and a confidence survey. Both groups increased their knowledge test scores after completing the online curriculum. Simulation scenario outcomes were similar between groups. The confidence survey revealed participants had a high degree of confidence after the course, with a median confidence level of 4.5 out of 5.0 (IQR = 0.5). This study demonstrated the feasibility of using the online software package for the creation and application of an Ebola response course. Future studies could advance knowledge gained from this pilot trial by assessing timely distribution and multi-site effectiveness with standard education.

The 2014-2016 Ebola outbreak began in West Africa and spread to 10 countries, infecting upwards of 28,000 people and causing more than 11,000 deaths.¹ Ebola virus has been isolated from nearly every type of body fluid,² with as few as 10 viral particles sufficient to cause infection and a case-fatality rate as high as 78%.³ Healthcare workers are at particularly high risk of acquiring the virus, with reports in Africa of infection up to 21 to 32 times higher than the general population.⁴

The index case in the United States during the 2014-2016 epidemic was identified in Dallas, Texas, on September 30, 2014, in a patient visiting from Liberia. Within 15 days, the Centers for Disease Control and Prevention (CDC) had confirmed 2 secondary cases in healthcare workers caring for this patient.^5,6 Additional cases treated in the United States included 7 American citizens who were medically evacuated to the United States, as well as a New York physician diagnosed after returning from Africa.⁵ This experience represents a rise in emerging or reemerging diseases and pandemics resulting from urbanization, globalization, and increased international travel and trade.^7,8 Recurring outbreaks this year in the Democratic Republic of the Congo testify to the ongoing threat Ebola presents to the world.^9,10

The Ebola epidemic raised international awareness of the need to create training resources that can be rapidly disseminated to healthcare providers. In addition to managing personal risk, personnel must be effective in infection control practices and must be prepared to respond to situations in which safety is critical. During the 2014 Ebola outbreak, the CDC created an Ebola virus disease (EVD) safety training course for US healthcare workers planning to work in Ebola treatment units in West Africa.¹¹ Similar Ebola training programs for healthcare workers in Africa were also created.^12,13 Subsequently, the US Department of Health and Human Services (HHS) developed Ebola preparedness and response activities to ensure that the “nation's health care system is ready to safely and successfully identify, isolate, assess, transport, and treat patients with Ebola.”^14(p00) Furthermore, in 2015 the US National Ebola Training and Education Center (NETEC) was created to provide education to healthcare workers treating patients with Ebola and other high-risk pathogens.¹⁵ According to CDC designation, frontline healthcare facilities must be able to identify and isolate potential Ebola patients and to transfer them to a higher level of care, while Ebola assessment hospitals that receive such patients are required to be capable of isolating the patient, performing lab evaluations, and caring for the patient for up to 5 days. They must then have the capability to transfer the patient to an Ebola treatment center that can care for the patient for the duration of his or her illness. NETEC and the CDC offer excellent training materials for healthcare personnel at all levels of healthcare facilities. However, 2 challenging aspects of implementation of these national training programs are: (1) standardizing training while still allowing for individualization based on differences between institutions, and (2) ensuring objectivity of performance feedback, as most institutional training programs rely heavily on subjective performance feedback from expert instructors.

This study reports the development and feasibility testing of an online-administered course offering background knowledge and simulation-based practice through an online software platform. The course is simultaneously generalizable to multiple types of healthcare workers and institutions, but it is also adaptable to individual institutional needs. We tested this software and training design using an Ebola response training program. The goal of the program was to achieve high-reliability hospital team performance in using personal protective equipment (PPE), infection control, and safety critical response in Ebola patient care practices. Key components are broadly and quickly disseminated materials; a standardized education process, to support mastery-based practice; a “high reliability” team approach; and course quality assurance, allowing comparison of training outcomes across sites. This provides a flexible, data-driven, rapidly disseminated course. This type of training program is vital to rapid, broad, just-in-time training of healthcare workers regardless of their location when facing high-risk infectious pathogen outbreaks like Ebola. It also allows standardized, objective gathering of performance data and storage of data that facilitates analysis and quality improvement.

Methods

Course Design

A 2-part course offered through a digital learning suite available over the internet was created by the small business SimTunes LLC for specific course demands (see Figure 1). The design of the process incorporated: (1) the provision of critical foundational knowledge to work safely in an Ebola patient care environment; (2) use of detailed, step-by-step team checklists (to support high-reliability performance); (3) a high-reliability team code of mutual responsibility for team member safety, including reinforced closed loop communication; (4) use of a digital assessment tool; and (5) a data-driven team report. Learning occurs in 2 settings: background learning using the internet, and the practical practice in a simulated standard hospital environment, configured to care for Ebola-infected patients, which included green (clean), yellow (transition), and red (contaminated) zones, with a simple mannequin used to represent the patient. A train-the-trainer module is provided to assist in the configuration and evaluation of the simulation using the automated assessment tool. Content was created and approved by a 4-person scientific advisory board consisting of a high-risk disease training expert (KLH), an investigator with extensive experience working with Ebola (WF), a physician medical simulation expert (JS), and a mastery education expert scientist (WM).

Figure 1.

Study Design Flow-chart

Part 1, the knowledge-based training, consisted of a 24-page, internet-housed curriculum hosted in a Moodle-based learning system,¹⁶ allowing study participants online log-in access. Topics covered included low-, moderate-, and high-risk exposure definitions, high-reliability team principles,¹⁷ operational awareness, and PPE background and use. Course content covered an introduction to EVD modeled after NETEC,¹⁵ the European network of Infectious Disease curriculum,¹⁸ and CDC¹⁹ and Occupational Safety and Health Administration (OSHA) guidelines.²⁰

Part 2, the practical training, consisted of 13 programmed, simulation-based, “hands-on” team training scenarios based on procedural checklists for the team (online material), with an assessment tool that guided an educator to digitally capture trainees' simulation-based performance. Simulation scenarios and corresponding task-based procedural team checklists were created by the content experts using preexisting checklists from the home institution specialized medical unit for high-risk infectious disease, NETEC, CDC, and World Health Organization (WHO) recommendations for safe practice of common clinical tasks performed while caring for an Ebola patient.^21,22 Specific simulation scenarios were grouped into 3 content domains: PPE use, infection control practices, and safety-critical responses (Table 1). Training was designed to be conducted by a team of 3, using a closed-loop, “read, repeat, complete” communication format. Each of the trainees fulfilled the role of “supervisor” (who read each step of the checklist and observed performance), “provider” (who repeated back each step and then performed the task described), and “buddy” (who assisted, observed, and verbally confirmed completion of each task). During simulation, training members of the team took turns in different roles but did not participate in all 3 roles for every task because of time constraints.

Table 1.

Description of Scenarios

Domain	Scenario Name	Steps	Steps in Domain
Infection Control (IC)			111
	Waste bagging	25
	Lab specimen bagging	17
	Bedside commode setup	9
	Bedside commode cleanup	41
	Spill cleanup	19
Personal Protective Equipment (PPE)			126
	Donning gown	21
	Doffing gown	42
	Donning biosuit	22
	Doffing biosuit	41
Safety			104
	Man down in gown	28
	Man down in biosuit	25
	Needlestick	7
	Body bagging	44

Simulation scenario team performance was recorded in real time by study educators using a team assessment toolset.²³ In the course “train the trainer” component, study educators received training in how to run the course and went through a practice training process with standardized videos on how to apply the team assessment toolset. The training process included standardized, automated, mastery-based practice simulation scenarios that provided accuracy feedback with each trial and were progressively more difficult until real-time operation speed and accuracy were achieved. Educators were able to grade each step in each scenario using an assessment toolset on a laptop programmed with Laerdal learning application (LLEAP) software.²⁴

Additionally, a novel software plug-in was incorporated to capture and transmit the data to a central database. At the beginning of the simulation training scenario, each specific protocol was loaded. Once loaded, the menu screen could be advanced (forward or back) for each step in the checklist. Using the performance assessment tool, the educator observed and recorded performance actions as “negative” (errors) or “positive” actions in 3 domains: technical, communication, and behavioral actions. Errors or positive actions could be logged for every team role on each step in the checklist (see Figure 2). The number of key steps per scenario varied between 7 and 44.

Figure 2.

Example of educator error selection screen

Evaluation Metrics

Knowledge was measured using a 25-question multiple-choice or true-false knowledge examination covering PPE, infection control, hazmat protocol, and safety. To provide a suitable challenge, some questions required the selection of multiple correct answers and were counted incorrect if any one of the correct answers was not selected. Trainees completed this successfully before proceeding to the simulation educational component.

An error grading scheme for the simulation scenarios was defined by the scientific advisory board. Critical errors were defined as errors that would put the provider at risk for infection or lead to the contamination of the clean zone, with any one critical error resulting in the team's failing the scenario. Risky actions, such as rushing, were defined as actions that could increase the chance of making a critical error. The accumulation of 4 risky actions during a scenario constituted a critical error.

At the end of each scenario, the software immediately generated a report of individual and team performance feedback, including time taken to complete the task, overall scores, and a breakdown of performance categorized by positive and negative technical, communication, and behavioral performance criteria (Figure 3). These reports can be used to facilitate specific discussion during the debriefing period to identify areas for performance improvement and reinforce positive performance.

Figure 3.

Educator display of software-generated report of individual and team performance

Feasibility Pilot Study

The study was conducted in May and June 2017 over 2 3-day periods and was determined to be exempt by the Medical University of South Carolina (MUSC) institutional review board.²⁵ Members of the preexisting MUSC specialized medical unit training team volunteered as study educators. They received training in course administration and practiced use of the assessment tool while watching standardized videos of scenario performance via the “train the trainer” course component.

Eighteen adult volunteer healthcare workers received the training. Participants received monetary compensation for their time. They were divided into 3 teams of 3 “novice” participants with no prior experience or training to work in a high-risk-pathogen unit, and 3 teams of 3 “experienced” participants who had completed training to work in the MUSC tertiary-care academic teaching hospital state treatment center unit. All 18 participants completed demographic data (Table 2) followed by parts 1 and 2 of the digital learning suite training program. Following completion of the course, the comfort and confidence of all study participants to care for a patient with Ebola was assessed using a 10-question Likert scale survey.²⁶

Table 2.

Demographic characteristics of cohorts

	Experienced		Novice
	N = 9		N = 9
	n	(%)	n	(%)	p-Value
Gender
Female	5	(55.6)	6	(66.7)	0.629
Male	4	(44.4)	3	(33.3)
Education
Baccalaureate	5	(55.6)	7	(77.8)	0.096
MD	0	(0.0)	2	(22.2)
Master's	1	(11.1)	0	(0.0)
Other	3	(33.3)	0	(0.0)
Specialized education
Medical fellowship	0	(0.0)	0	(0.0)	0.208
Medical residency	0	(0.0)	2	(22.2)
Nursing specialty training	6	(66.7)	3	(33.3)
Other	3	(33.3)	4	(44.4)
None	0	(0.0)	0	(0.0)
Current employment
Hospital staff	7	(77.8)	4	(44.4)	0.282
Medical staff	2	(22.2)	2	(22.2)
Resident	0	(0.0)	2	(22.2)
Student	0	(0.0)	1	(11.1)

Statistical Analysis

All statistical analyses were conducted using STATA, version 13.²⁷ Demographic characteristics such as gender, education, and employment type were described by novice or experienced status, and the 2 groups compared using a Chi-square test of homogeneity (Table 2). The differences in knowledge-based pre- and posttest mean scores were compared with paired t-test methods, while differences in knowledge-based scores were compared with t-tests for equality of means (Table 3). The mean time to complete each scenario was described by novice or experienced status, and when possible, t-tests for equality of means were performed (Table 4). Critical errors, risky actions, and positive actions were enumerated by domain (PPE, infection control, and safety), dichotomized into “none” and ≥1, and further described as the proportion of total steps. Differences in mean proportions were evaluated with t-tests for equality of means. Median (with interquartile range) total confidence score was described and compared using the 2-sample Wilcoxon rank-sum test (Table 5).

Table 3.

Didactic learning scores for novice versus experienced cohorts

						Absolute
	Pretest Mean ±Std	P-Value N vs E	Posttest Mean ±Std	P-Value N vs E	P-Value Pre and Post	Change in Score Mean ±Std	P-Value N vs E
Novice	78.9±12.0	P-Value N vs E	0.7204	P-Value N vs E	P-Value Pre and Post	97.1±4.1	P-Value N vs E	0.0679	0.0041	18.3±13.8	0.4079
Experienced	76.8±12.1		89.4±11.1		0.0318	12.6±14.5

Note: N = novice; E = experienced.

Table 4.

Average time to complete task

		Time to Complete Task^a (minutes)
		Experienced		Novice
Domain Task	Steps	# Teams	Mean ± Std	# Teams	Mean ± Std	P-Value
Infection control (IC)
Waste bagging	25	3	24.4 ± 11.2	3	24.4 ± 3.5	0.9982
Lab specimen bagging	17	3	8.4 ± 2.7	3	10.5 ± 1.4	0.2940
Bedside commode setup	9	2	10.1 ± 1.9	1	6.7
Bedside commode cleanup	41	3	15.3 ± 9.1	3	16.9 ± 10.8	0.8539
Spill cleanup	19	3	10.1 ± 3.1	2	14.7 ± 0.5	0.1365
Personal protective equipment (PPE)
Donning gown	21	3	12.5 ± 2.8	3	15.0 ± 3.2	0.3662
Doffing gown	42	3	15.2 ± 2.2	3	19.4 ± 4.8	0.2383
Donning biosuit	22	3	12.6 ± 2.2	3	12.1 ± 6.8	0.9062
Doffing biosuit	41	3	18.5 ± 4.0	3	14.5 ± 11.4	0.6023
Safety (S)
Man down in gown	28	0		2	8.9 ± 2.7
Man down in biosuit	25	3	11.1 ± 0.4	3	8.8 ± 2.3	0.1696
Needlestick	7	3	1.8 ± 1.4	3	4.7 ± 3.8	0.2807
Body bagging	44	2	28.1 ± 2.3	3	33.2 ± 6.2	0.3589

If task was done more than once, used average for each team.

Table 5.

Critical errors and risky actions

	Steps with
	Critical Errors							Risky Actions
	Total		Experienced		Novice			Total		Experienced		Novice
	≥1^a	Mean^b ± Std	≥1^a	Mean^b ± Std	≥1^a	Mean^b ± Std	P-Value	≥1^a	Mean^b ± Std	≥1^a	Mean^b ± Std	≥1^a	Mean^b ± Std	P-Value
Overall	5	2.4 ± 4.7	2	2.2 ± 0.6	3	2.6 ± 0.9	0.6971	4	1.7 ± 5.4	3	1.8 ± 0.9	1	1.6 ± 0.8	0.8190
PPE	4	2.5 ± 5.4	2	1.9 ± 3.2	2	3.1 ± 6.9	0.4791	2	3.4 ± 7.6	1	3.5 ± 8.1	1	3.2 ± 7.3	0.9306
Infection control	3	1.9 ± 3.5	2	2.8 ± 4.2	1	0.9 ± 2.2	0.1824	2	0.4 ± 1.5	2	0.8 ± 2.0	0	0.0 ± 0.0	0.1876
Safety	4	2.8 ± 4.8	2	1.9 ± 3.4	2	3.4 ± 5.7	0.5029	1	0.2 ± 0.8	0	0.0 ± 0.0	1	0.3 ± 1.1	0.4093

Number of teams with at least 1 step flagged for critical errors or risky actions.

Takes into account number of steps per task and number of tasks the teams performed. For example, team 1 performed 4 of the 5 infection control tasks, so the denominator for the proportion of steps is 102 vs 111.

Results

The 2 groups did not differ with regard to demographic characteristics (Table 2). The pre- and post- online curriculum testing (Table 3) demonstrate that both novices and experienced participants had a significant increase in their knowledge test scores after completing the online curriculum, with average scores for novices increasing from 19.7 to 24.3 (n = 9, t = −3.9804, df = 8, p = 0.0041) and average score in experienced participants increasing from 19.2 to 22.3 (n = 9, t = −2.5973, df = 8, p = 0.0318). There was no significant difference between novices and experienced participants in either pretest or posttest scores.

Simulation training results are summarized in Table 4 and Figure 4. The time required to complete each simulation was not significantly different for novices or experienced groups in 11 of the 13 scenarios (Table 4). Figure 4 shows the average proportion of steps flagged for critical errors and risky actions by the 2 groups. Error bars denote the confidence interval.

Figure 4.

Mean proportion of steps flagged for critical errors or risky actions by team experience. Error bars represent the confidence interval of the mean.

Overall, participants had an average of 2.3% of steps with critical errors (2.7% in PPE, 1.2% in infection control, and 3.5% in safety), and 1.7% of steps had risky actions noted. There was no significant difference in the average proportion of critical errors or risky actions performed between novice groups and experienced groups. We also identified the total number of teams that made errors of any sort. Five of 6 teams made some form of critical error, while 4 of 6 made a risky action. All the novice teams made critical errors, though whether this was with PPE, infection control, or safety varied. In contrast, 1 experienced team had no critical errors, while 2 made critical errors across all 3 domains. Risky actions were considerably more prevalent across the experienced teams.

Confidence survey results demonstrated that participants felt an overall sense of confidence in their ability to safely care for an Ebola patient (Table 5). The median total confidence score was 4.5 (IQR = 0.5). Experienced participants reported higher levels of confidence overall (median = 4.7; IQR = 0.3) compared with novices (median = 4.3; IQR = −0.5), which was a statistically significant difference (p = 0.0078).

Discussion

We found it feasible to create an internet-distributable software package to provide an Ebola response training course. This pilot study demonstrated a number of positive features of the course design using the online software package. First, both novices and experienced trainees improved their knowledge, demonstrating that both groups benefited from initial internet-based background study. Online material content is editable and can be adjusted or adapted for future outbreaks based on new knowledge or alternative infectious diseases with new programming. While this pilot study only evaluated online-content effect on Ebola background knowledge, it is reasonable to think that a similar construct with background content on other disease processes would also improve trainee knowledge, and this should be further evaluated in future studies.

Second, simulation scenarios were effectively performed, and the assessment tool was successfully used to collect real-time team performance data. Error rates in the simulation team assessments across both groups were generally low, though most teams experienced some critical errors and risky actions.

We believe the overall high performance of both groups in the simulation scenarios can be attributed to the use of checklists and adherence to high-reliability team functioning. The course was designed to follow practices of high-reliability team function.^28,29 The key elements in a high-reliability team as applied in this approach are: (1) highly proscriptive checklists for each team role in a protocol step, (2) mutual responsibility for safety such that each team member is responsible for the safety of every member of the team, and (3) the use of closed-loop team communication for clarity and error prevention. This requires constant vigilance and attention to detail regarding the performance, behavior, and communication of each individual and all teammates. Thus, all team members are not only trying to do things correctly; they are also observing and communicating to other team members about their actions. Closed-loop communication is important in this process, and the course insists that each step in a protocol be read to the team by a member, repeated by the person to do the action, and reported by another observer as being correctly performed. This high-reliability team process embedded in the course and the grading of performance contributed to the high performance of both the inexperienced group and the experienced group on the 13 clinical scenarios.

Checklists have demonstrated efficacy as an aid to the completion of procedural tasks in a range of safety critical industries, and more recently they have been employed across a wide variety of clinical settings.^30,31 We found that the highly prescriptive nature of each team checklist step produced similar performance, regardless of experience. Well-defined checklists with prescriptive steps for each clinical task provide standardization, visibility, reliability, and clear assessment criteria.

Interestingly, experienced participants had a higher number of risky behaviors. Thus, experience may not always be a reliable surrogate for performance in the management of high-risk infectious diseases. One hypothesis for this phenomenon is that, having performed Ebola response training in the past, experienced trainees may have been more prone to errors resulting from excessive confidence or rushing or from previously developed “bad habits.” This study also used a relatively small sample size, and future larger-scale studies are needed to provide further insight on these findings. Regardless, it reinforces the need for continual training with collection and aggregation of data to help identify individual, organizational, and systemic weaknesses.

Ultimately, we recommend a mastery-based team training approach, in which the trainees repeat the steps they fail until they achieve total effectiveness.^32,33 Though it may require a number of repeats to achieve such a demanding level of performance, this is advisable for highly virulent and fatal pathogens such as Ebola. Compared to more traditional ad hoc feedback based on the assessor's expert opinion, mastery-based reliability is only possible with a highly prescriptive process and clearly defined evaluation mechanisms. Our pilot study demonstrated that this level of reliability in both assessment and performance was feasible.

The lowest level of performance for any one team was 96.5%, suggesting an already high standard, and that remediation of the erroneous steps, rather than the whole procedure, could be enough to achieve mastery. Future studies should implement a fully mastery-based approach. Time in this pilot trial did not permit the repetition of steps until perfect performance was achieved.

As mentioned, our simulation scenarios were created using preexisting specialized medical unit training materials from our institution along with national guidelines from NETEC, CDC, and WHO. Due to the nature of the software package, additional or alternative simulation scenarios can be created by individual institutions using national guidelines and be programmed into the course software, using the pilot study course design incorporating the high-reliability team format and evaluation using the assessment tool. This flexibility means different simulations could be provided for different institutions, depending on their national designation (frontline center, assessment center, state or regional treatment center). It also means that varying simulations could be provided to healthcare team members to fit their specific role in the team. This pilot trial demonstrated effectiveness on a small scale that warrants further evaluation in future multi-site trials.

The assessment process automated the collection of participant performance outcomes, which were gathered and aggregated into a database for immediate feedback and future record. In the debriefing process, it allowed for detailed discussion of positive and negative actions of individuals and teams and thus a clear, detailed, and defensible critique for high-reliability team development and process mastery. These data also support detailed record keeping surrounding readiness of individuals and organizations and eventually would allow systemic comparative data analysis across units and institutions, both nationally and internationally. Most institutions still use paper-based assessments, which are neither easily amenable to aggregation nor for mastery-based practice, while for those using electronic education and assessment tools, there are few measurement and data standards, with limited capability to share. The ability to store, access, and track performance over time and geographical locations would be particularly powerful for systems-based quality improvement to defend against highly virulent pathogens that can quickly spread between institutions and across borders. The online software model used in our study allows for improved standardization of assessment, better data archiving, and vulnerability assessments. The assessment can detect causes for errors—for example, inadequate communication or lapse in other highly reliable team functional characteristics. It can also be used to support quality assurance across instructors, groups, or sites and might even be used to identify particular recurrent weaknesses that would be amenable to redesign—for example, error-predisposing PPE designs.

During this work, we noted limitations in the grading software that resulted in a failure to save some participant performance data. This affected all 6 teams in 6 of the scenarios: bedside commode set up, man down in gown, and body bagging (Table 4). This has since been addressed by an autosave feature. Also, assessors found it difficult to keep up with the pace of the simulation and collect data on all 3 participants simultaneously. This has informed software and training modifications to improve usability, and we accept that there may be further iterations to find the balance between data collection and data recording.

Evaluators were selected from members of the established MUSC specialized medical unit training team. It would be beneficial to assess the functionality and usability of the software for other educators and to conduct further studies in multiple healthcare settings, in addition to our own large tertiary care center. There is also an opportunity to assess performance and knowledge decay over time, in order to define the optimal recurrent training requirements to maintain readiness. The fact that experienced workers made errors suggests that refresher training is important at intervals yet to be defined. This has been found in other mastery-based training.

Finally, both groups rated high confidence in their ability to care for Ebola patients following completion of the course. Overall, this demonstrates that this course has educational value and is feasible to run successfully, and that knowledge, practice, and self-report evaluations provide multiple complementary appraisal dimensions. This system also provides a data-driven record of objective training outcomes as well as healthcare workers' confidence levels for institutions to use in workforce readiness analysis.

Inevitably (with subsequent high-risk infectious disease outbreaks⁹), there will be opportunities to validate our approach in an operational context, rather than a simulated setting, and we should carefully plan the best approaches to do so, in order to best protect our patients and professionals from these dangerous pathogens. Comparing this course with conventional educational training methods will ultimately test its value.

Since the 1940s more than 400 emerging infectious diseases have been identified with a corresponding increase in human infectious disease outbreaks.^8,34 The 2014-2016 Ebola outbreak sharply focused the need for improving preparedness for and response to the sudden emergence of high-risk pathogens. In prior training environments, evaluation and feedback were provided via a quantifiable but limited knowledge-based test or through qualitative, unstructured, and potentially ad hoc practical assessment. We have demonstrated the feasibility of a course that offers rapid, scalable online delivery of both knowledge and practical, hands-on skills, through the incorporation of prescriptive standardized team-based safety processes, which provides standardized data driven performance assessment. This successful pilot study suggests that this course can effectively increase healthcare worker knowledge, skills, and comfort with managing Ebola, while identifying weaknesses in performance for both experts and novices.

Our pilot study offers one example of how a prefabricated, online, simulation-based training program can be delivered to frontline healthcare workers using a novel assessment tool software that could permit institutional and national preparedness databases. The US Department of Health and Human Services recommends that Ebola treatment centers train all rostered staff quarterly in specialist care for Ebola or other high-risk pathogens. Treatment center staff must have the ability to receive training within 72 hours of notification of a patient with a special pathogen. We have demonstrated that an internet-delivered, standardized, predeveloped training and assessment tool with built-in simulation scenarios, educator grading tool, and facilitator training support has the potential to improve workforce readiness; support mastery-based training; provide specific diagnostic feedback; aggregate performance for quality assurance, risk mitigation, and electronic tracking; and provide broad, fast, and scalable deployability.

The Ebola pandemic offered us a glimpse into the devastation a highly pathogenic pandemic can cause. International infectious disease outbreaks are on the rise. As a healthcare community, we cannot afford to be unprepared to respond.

Footnotes

Acknowledgments

Funding: Kathy Lehman-Huskamp, J. G. Reves, John Schaefer, and Brandy Pockrus received grant support for this project from CDC SBIR Grant Number 1R43CK000444-01A1 REVISED via the awardee, SimTunes. Conflicts of Interest: Drs. Schaefer, Lehman-Huskamp, and Reves contributed mentorship, guidance concerning experimental design, and manuscript review for this study and were compliant with the Medical University of South Carolina conflict of interest (COI) policy. Data collection and analysis were consistent with institutional policy. Dr. Schaefer did not participate in data collection, reduction, or analysis related to this study because of his potential COI, which includes simulator patent and copyright royalties. Dr. Schaefer receives patent royalties from Laerdal Medical Corporation (SimMan/Baby/3G/SimMom), and he is a nonmajority owner of SimTunes, which is a commercial outlet for Medical University of South Carolina licensed, copyrightable simulation training products. These amount to less than 35% of Dr. Schaefer's annual income. Dr. Reves is scientific advisor to SimTunes and receives less than 35% of his income from SimTunes.

References

US Centers for Disease Control and Prevention. Ebola (Ebola Virus Disease). Case counts. CDC website. Updated December 27, 2017. https://www.cdc.gov/vhf/ebola/outbreaks/2014-west-africa/case-counts.html. Accessed October 30, 2018.

Osterholm

, Moore

, Kelley

, et al. Transmission of Ebola viruses: what we know and what we do not know. mBio 2015;6(2):e00137. Erratum: mBio, 2015; 6(4):e01154.

Wong

, Qiu

, Olinger

, Kobinger

. Post-exposure therapy of filovirus infections. Trends Microbiol, 2014; 22(8):456-463.

WHO. Health Worker Ebola Infections in Guinea, Liberia and Sierra Leone: A Preliminary Report. May 21, 2015. https://www.who.int/hrh/documents/21may2015_web_final.pdf. Accessed October 30, 2018.

US Centers for Disease Control and Prevention. Ebola (Ebola Virus Disease). 2014-2016 Ebola Outbreak in West Africa. CDC website. Updated December 27, 2017. https://www.cdc.gov/vhf/ebola/outbreaks/2014-west-africa/united-states-imported-case.html. Accessed October 30, 2018.

Liddell

, Davey

Jr , Mehta

, et al. Characteristics and clinical management of a cluster of 3 patients with Ebola virus disease, including the first domestically acquired cases in the United States. Ann Intern Med, 2015; 163(2):81-90.

Lashley

. Emerging infectious diseases: vulnerabilities, contributing factors and approaches. Expert Rev Anti Infect Ther, 2004; 2(2):299-316.

Morse

, Mazet

, Woolhouse

, et al. Prediction and prevention of the next pandemic zoonosis. Lancet, 2012; 380(9857):1956-1965.

Ebola is back. It's an emergency. Washington Post May 18, 2018. https://www.washingtonpost.com/opinions/ebola-is-back-its-an-emergency/2018/05/18/f87bc17c-59f5-11e8-8836-a4a123c359ab_story.html?noredirect=on&utm_term=.f36a98de511c. Accessed October 30, 2018.

10.

World Health Organization. Ebola virus disease – Democratic Republic of the Congo, disease outbreak news. September 14, 2018. http://www.who.int/csr/don/14-september-2018-ebola-drc/en/. Accessed October 30, 2018.

11.

Narra

, Sobel

, Piper

, et al. CDC safety training course for Ebola virus disease healthcare workers. Emerg Infect Dis, 2017; 23(13).

12.

Médecins sans Frontières. Operational Centre Brussels. OCB medical activity report; 2014. https://www.doctorswithoutborders.org/what-we-do/news-stories/news/activity-report-2014.

13.

Jones-Konneh

TEC

, Murakami

, Sasaki

, Egawa

. Intensive education of health care workers improves the outcome of Ebola virus disease: lessons learned from the 2014 outbreak in Sierra Leone. Tohoku J Exp Med, 2017; 243(2):101-105.

14.

US Department of Health and Human Services. Office of the Assistant Secretary for Preparedness and Response. Hospital Preparedness Program (HPP) Measure Manual: Implementation Guidance for Ebola Preparedness Measures. May 2017. https://www.phe.gov/Preparedness/planning/sharper/Documents/hpp-mmi-guide-ebola-508.pdf. Accessed October 30, 2018.

15.

Kratochvil

, Evans

, Ribner

, et al. The National Ebola Training and Education Center: preparing the United States for Ebola and other special pathogens. Health Secur, 2017; 15(3):253-260.

16.

Moodle. Moodle Learning System. https://moodle.org/. Accessed October 30, 2018.

17.

Baker

, Day

, Salas

. Teamwork as an essential component of high-reliability organizations. Health Serv Res, 2006; 41(4 Pt 2):1576-1598.

18.

Baka

, Fusco

, Puro

, et al. A curriculum for training healthcare workers in the management of highly infectious diseases. Euro Surveill, 2007; 12(6):E5-6.

19.

US Centers for Disease Control and Prevention. Guidance on personal protective equipment (PPE) to be used by healthcare workers during management of patients with confirmed Ebola or persons under investigation (PUIs) for Ebola who are clinically unstable or have bleeding, vomiting, or diarrhea in U.S. hospitals, including procedures for donning and doffing PPE. CDC website. Updated August 27, 2015. https://www.cdc.gov/vhf/ebola/healthcare-us/ppe/guidance.html. Accessed October 30, 2018.

20.

US Department of Labor. Occupational Safety and Health Administration. Personal protective equipment. https://www.osha.gov/SLTC/personalprotectiveequipment/standards.html. Accessed October 30, 2018.

21.

US Centers for Disease Control and Prevention. Ebola virus disease. Preparedness checklists. CDC website. Updated January 22, 2015. https://www.cdc.gov/vhf/ebola/healthcare-us/preparing/checklists.html. Accessed October 30, 2018.

22.

World Health Organization. Consolidated Ebola virus disease preparedness checklist. http://www.who.int/csr/resources/publications/ebola/ebola-preparedness-checklist/en/. Accessed October 30, 2018.

23.

Alfred

, Schaefer

, Catchpole

, Reves

. A novel approach to evaluating simulation scenarios [abstract]. Anesth Analg Abstracts Supplement 2018;S-394:719. https://iars.app.box.com/v/AM18AbstractSupplement/127file/305554658309719.

24.

Laerdal. LLEAP information and implementation. http://www.laerdal.com/us/LLEAPImplementation. Accessed October 30, 2018.

25.

Howe

, Dougherty

. Ethics, institutional review boards, and the changing face of educational research. Educ Res, 2016; 22(9):16-21.

26.

Warmbrod

. Reporting and interpreting scores derived from Likert-type scales. J Agric Educ, 2014; 55(5):30-47.

27.

StataCorp. Stata Statistical Software: Release 13.1, StataCorp LP: College Station, TX.

28.

Gamble

5 Traits of high reliability organizations: how to hardwire each in your organization. Becker's Hospital Review April 29, 2013. http://www.beckershospitalreview.com/hospital-management-administration/5-traits-of-high-reliability-organizations-how-to-hardwire-each-in-your-organization.html. Accessed October 30, 2018.

29.

Hines

, Luna

, Lofthus

, Marquardt

, Stelmokas

. Becoming a High Reliability Organization: Operational Advice for Hospital Leaders. Rockville, MD: Agency for Healthcare Research and Quality; February 2008. https://archive.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/hroadvice/hroadviceref.html. Accessed October 30, 2018.

30.

Degani

, Wierner

. Human Factors of Flight-Deck Checklists: The Normal Checklist. Moffett Field, CA: National Aeronautics and Space Administration; 1990.

31.

Verdaasdonk

, Stassen

, Widhiasmara

, Dankelman

. Requirements for the design and implementation of checklists for surgical processes. Surg Endosc, 2009; 23(4):715-726.

32.

McGaghie

. Mastery learning: it is time for medical education to join the 21st century. Acad Med, 2015; 90(11):1438-1441.

33.

Barsuk

, Cohen

, Caprio

, McGaghie

, Simuni

, Wayne

. Simulation-based education with mastery learning improves residents' lumbar puncture skills. Neurology, 2012; 79(2):132-137.

34.

Smith

, Goldberg

, Rosenthal

, et al. Global rise in human infectious disease outbreaks. J R Soc Interface, 2014; 11(101):20140950.