Asynchronous Web-Based Learning,a Practical Method to Enhance Teaching in Emergency Medicine

Abstract

Objective: To compare medical knowledge acquisition among emergency medicine (EM) residents who attend weekly core content lectures with those absent but asynchronously viewing the same lectures in a Web-based electronic platform. Subjects and Methods: During the study period all EM residents attending or absent from weekly educational conferences were given a quiz on the covered material. During Phase 1, absentees were not given supplemental educational content for missed lectures. During Phase 2, absentees were sent a link to an online multimedia module containing an audiovisual recording of the actual missed lecture with presentation slides. Scores between attendees and absentees during both phases were compared using a repeated-measures analysis to evaluate the effect of the supplemental online module on knowledge acquisition. Results: Thirty-nine EM residents (equally distributed in postgraduate years 1–4) were studied during a 15-week period. Overall and after adjusting for sex and postgraduate year level, both lecture attendance (b =27; 95% confidence interval, 22–32; p<0.0001) and Web-based learning (b=32; 95% confidence interval, 26–37; p<0.0001) were associated with significant increases in test scores compared with residents who were absent and not receiving supplemental Web-based learning. Neither the self-perceived level of mastery with the lecture topic nor the amount of reported reading was found to be a predictor of test scores. Conclusions: In an EM residency program, asynchronous Web-based learning may result in medical knowledge acquisition similar to or better than attending traditional core content lectures. The percentage of curriculum delivery by asynchronous learning that may be used to achieve overall terminal learning objectives in medical knowledge acquisition requires further study.

Introduction

Didactic learning is an important component of postgraduate training in emergency medicine (EM). It can both reinforce and prepare trainees for clinical learning in the emergency department (ED).^1
–3 EM educators have a variety of tools to provide excellent didactic curricula such as problem-based learning, small group sessions, simulation, skills-based learning, and others.^4,5 However, using an Internet-based platform to deliver content is becoming an increasingly attractive way to teach.^4,6

–9

Because the clinical experience of any single trainee will vary, didactic curricula are an important way to bring about some uniformity and standardization of learning within programs. However, the logistics of providing a uniform didactic curriculum to each trainee is challenging because of the need for continuous physician coverage in the emergency department, rotating shift schedules for trainees and faculty, duty hour requirements, and having trainees at geographically disparate hospitals within a single program. Although not a panacea, Web-based learning (WBL) modules may help overcome some of the logistical challenges of delivering didactic content. Learners may log in and complete an educational activity asynchronously and from any place they have an Internet connection.

Within graduate EM education, published data on WBL have largely been related to focused modules on certain topics within the larger EM curriculum, such as pediatric EM,¹⁰ emergency ultrasound,^11,12 bioterrorism,¹³ pediatric medication dosing,¹⁴ and others. However, WBL could be used more widely across the spectrum of the graduate EM curricula and has the potential to be routinely used within a program to ensure core content delivery.

The purpose of this study was to evaluate WBL across a variety of topics in a general EM curriculum by comparing knowledge acquisition by learners who received training in a traditional lecture format with those who received training on the same topics via Internet-based learning modules.

Subjects and Methods

Study Design

This is a prospective study involving EM residents at a single residency program from December 2009 until April 2010. The institutional review board at The George Washington University approved the study, and the study was exempt from review and informed consent requirements.

Study Setting and Population

Didactic conferences in EM training programs are an important part of the educational curriculum. Delivery of some core content lectures via WBL modules is a potentially an attractive way to deliver a portion of this content. We combined our data from 15 weeks of EM weekly conferences and included all residents postgraduate years 1–4 in the department regardless of their presence or absence from weekly conferences.

Study Protocol

During the study period all EM residents, whether present or absent from the weekly educational conference, were given a quiz on the covered material. During Phase 1 (observational phase), absentees were not given supplemental educational content for missed lectures. During Phase 2, absentees were sent a link to an Internet-based learning module containing a multimedia file with an audiovisual recording and the presentation slides of the actual missed lecture. During both phases, attendees and absentees were asked to take a quiz related to covered material each week. The quiz was e-mailed to any resident who did not complete it in person on the day of conference. The lecturer was asked to develop a quiz of multiple choice questions covering key concepts from his or her lecture. In some cases, if the lecturer did not provide a quiz, one of the study authors reviewed the content and created the quiz. Both attendees and absentees were given the same quiz. In addition to knowledge questions, residents completed questions asking about perceived level of subject mastery and self-reported assessment of their independent reading of the subject matter prior to the quiz. The core content topics covered during the study are listed in Table 1.

Table 1.

Lecture Topics Covered in Weekly Conferences

Traumatic brain injury

Diabetic ketoacidosis

Rhabdomyolysis

Hemolytic uremic syndrome

Urinary tract infection

Aggressive behavior

Anaphylaxis

Palliative care in ED

Sickle cell disease

Pediatric rash

Vasculitis

Upper extremity injuries

Barotitis media

Procedural sedation

COPD/bronchitis/asthma

COPD, chronic obstructive pulmonary disease; ED, emergency department.

Data Analysis

Scores between attendees and absentees during both phases were compared using repeated-measures analysis to evaluate the effect of the supplemental online module on knowledge acquisition. Different test scores of each person were assumed to be correlated to each other; thus the data were considered as panel (longitudinal) data. We used two different approaches: a fixed effect model (in which we assumed the format of lectures was the only predictor of score) and a random effect model (in which we assumed lecture format and some of the person-related variables such as gender, residency year of training, self-perceived level of mastery, and reported reading were the predictors of score). Further statistical analysis using the Durbin–Wu–Hausman test confirmed that the two approaches were similar, so we report the results of the fixed effect model here.¹⁵ Further testing in the final model showed clear indication of heteroskedasticity. We used a robust sandwich estimator of variance for the final model. All analyses were performed by Stata version 10.1 (StataCorp, College Station, TX).¹⁵

Results

Thirty-nine EM residents were studied during a 15-week period. They were equally distributed among postgraduate years 1–4, and 62% were female. From 585 expected completed quizzes 444 (76%) quizzes were completed. Among them, 247 (56%) attended the classes, 138 (31%) were absent from the lecture but had supplemental WBL, and 59 (13%) were absent with no supplemental WBL (Fig. 1). Scores for attendees and absentees were plotted at baseline and after the study intervention (Fig. 2). During Phase 1 (observational phase) lecture attendance was associated with significantly higher quiz scores compared with those absent (p<0.0001). In Phase 2, lecture attendance was associated with a 27% higher score (95% confidence interval, 22–32%; p<0.0001) and WBL with a 32% increase in test score (95% confidence interval, 26–37%; p<0.0001) compared with residents who were absent and not receiving supplemental WBL. This group had a mean test score of 61% (Table 2). Neither the perceived level of mastery with the lecture topic nor the amount of reported reading was found to be a predictor of test scores.

Table 2.

Effect of Lecture Type on Test Score from a Fixed Effect Model with Robust Sandwich Estimator

	MEAN % CHANGE IN SCORE (95% CI)	P VALUE
Absentees	Baseline
Lecture attendance	27 (22–32)	<0.0001
Web-based attendance	32 (26–37)	<0.0001
Intercept	61 (57–66)	<0.0001

The model is adjusted for the gender and residency year. R ²=0.39.

CI, confidence interval.

Fig. 1.

CONSORT diagram.

Fig. 2.

Mean (standard error) quiz scores for attendees (dashed line) and absentees (solid line) during the study period. Triangles indicate the participants who were absent and have not participated in Web-based learning courses. *After week 7 (Phase 2) absentees were sent supplemental Web-based learning on the lecture material they missed.

Limitations

This study involved a single residency program with internally developed content and quizzes, which may limit some generalizability. It is possible that there were confounders that modified the effect of adding WBL as an option for our program, such as an increase in the desire to submit correct answers causing either increased use of book resources or increased use of other online resources or search engines to find the correct answer, but the reported amount of topical reading did not increase across the 15 weeks of the study. During the study, we used submission of a completed quiz as a surrogate that the trainee viewed the WBL module. However, we could not control for their using other resources to answer asynchronous quizzes. However, if a WBL module and quiz influenced them to use additional resources, we believe that overall WBL had a positive impact on learning.

Finally, this was not a blinded study. Residents knew they were being offered a novel educational strategy, and the effects may not persist over time or across lectures varying in quality.

Discussion

Within an EM training program, it is nearly impossible for every learner to attend every scheduled didactic session across the entire curriculum. Educators rely on strategies such as independent reading and redundancy in the didactic schedule to help give maximum exposure to every resident on core topics. In this study, quiz scores significantly improved when residents who missed a didactic conference were given supplemental leaning via the Internet, compared with when they were not given any supplemental learning and were left to read (or not) on their own. And, when given WBL modules, trainees who missed traditional conferences scored equally well or slightly higher than those who attended the traditional lecture format.

These findings are consistent with what has been reported elsewhere. In a meta-analysis of WBL in health professions, Cook et al.⁷ reported that when compared with no intervention, WBL consistently had a positive effect, and when compared with non–Web-based methods there were generally small positive effects or no difference in effect at all.

For EM residency programs, this may have several implications. When residents must miss educational conferences because of clinical responsibilities, vacation, or illness, recording and archiving of educational conferences are feasible.⁹ Recording and archiving in a digital format may have several advantages. Digital audio and/or video files can be distributed via a link from a common Web site for download by students and residents. Website distribution can be password-protected, and oftentimes file access can be tracked, thus giving credit to residents for “on-line” attendance. The digital files can be sent via e-mail in a targeted fashion to those who actually missed conference or those need remediation in certain topics.

This study suggests that for core content material, WBL can be equally as effective as attendance at a lecture. Thus training programs may choose to deliberately deliver a portion of their educational content electronically for everyone, not just those who miss conference. The Accreditation Council for Graduate Medical Education common program requirements for EM mandate a minimum of 5 h of educational conferences per week. However, they allow up to 20% to be “individualized interactive instruction” and allow programs to apply for innovation and experimentation projects in their program.¹⁶ Thus, there is room for program directors to be innovative not just in what they teach, but how they deliver it.

Additionally, many programs at academic centers have learners beyond residents such as medical students, allied health students, international programs, and continuing medical education courses. Some modules may be suitable for many levels of learners (just as various learners often attend residency grand rounds in addition to residents) or may be altered and adapted using a single media file for many audiences. Over time we anticipate these strategies will provide economies of scale and increase the efficiency of faculty teaching time. It may even be feasible to share digital educational file across programs, allowing regional or national standardization of teaching certain topics or helping programs with less educational resources to implement WBL.

As with any delivery of educational content, planning and preparation are keys to quality. For programs to use WBL they must make an investment in some equipment, software programs, and server space. Faculty and staff expertise must be developed in this area as well.⁸ With time and internal evaluation, programs must learn what works effectively for them. Current research suggests that learning outcomes with WBL in the health professions are improved with factors such as interactivity, practice exercises, and feedback systems.¹⁷ Such research is still new, and there exists a large opportunity to study how best to use electronic formats in health education and how to best teach the next generation of emergency physicians.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

Hebert

, Wright

. Re-examining the value of medical grand rounds. Acad Med, 2003; 78:1248–1252.

Lewkonia

, Murray

. Grand rounds: A paradox in medical education. CMAJ, 1995; 152:371–376.

Croley

, Rothenberg

. Education of trainees in the intensive care unit. Crit Care Med, 2007; 35,2 Suppl:S117–S121.

Cook

. Learning and cognitive styles in web-based learning: Theory, evidence, and application. Acad Med, 2005; 80:266–278.

Mackway-Jones

, Carley

, Kilroy

. Advanced training in emergency medicine: A pedagogical journey from didactic teachers to virtual problems. Emerg Med J, 2007; 24:696–698.

Roe

, Carley

, Sherratt

. Potential and limitations of e-learning in emergency medicine. Emerg Med J, 2010; 27:100–104.

Cook

, Levinson

, Garside

et al. Internet-based learning in the health professions: A meta-analysis. JAMA, 2008; 300:1181–1196.

Robin

, McNeil

, Cook

et al. Preparing for the changing role of instructional technologies in medical education. Acad Med, 2011; 86:435–439.

Clinton

, Joing

. Computer-based solutions for archiving and viewing educational conferences at emergency medicine residency programs. Acad Emerg Med, 2004; 11:1371.

10.

Burnette

, Ramundo

, Stevenson

et al. Evaluation of a web-based asynchronous pediatric emergency medicine learning tool for residents and medical students. Acad Emerg Med, 2009; 16,Suppl 2:S46–S50.

11.

Chenkin

, Lee

, Huynh

, Bandiera

. Procedures can be learned on the Web: A randomized study of ultrasound-guided vascular access training. Acad Emerg Med, 2008; 15:949–954.

12.

Platz

, Goldflam

, Mennicke

et al. Comparison of Web-versus classroom-based basic ultrasonographic and EFAST training in 2 European hospitals. Ann Emerg Med, 2010; 56:660–607.

13.

Chung

, Mandl

, Shannon

, Fleisher

. Efficacy of an educational Web site for educating physicians about bioterrorism. Acad Emerg Med, 2004; 11:143–148.

14.

Frush

, Hohenhaus

, Luo

et al. Evaluation of a Web-based education program on reducing medication dosing error: A multicenter, randomized controlled trial. Pediatr Emerg Care, 2006; 22:62–70.

15.

Rabe-Hesketh

, Skrondal

. Multilevel and longitudinal modeling using Stata, 2nd. College Station, TX: Stata Press, 2008.

16.

Planned educational experiences for resident programs. www.acgme.org/acWebsite/RRC_110/110_guidelines.asp. 2011 December 1.

17.

Cook

, Levinson

, Garside

et al. Instructional design variations in Internet-based learning for health professions education: A systematic review and meta-analysis. Acad Med, 2010; 85:909–922.