COVID-19 pandemic: Early effects on pediatric rehabilitation medicine training

1. Introduction

The COVID-19 pandemic has created an unprecedented disruption in graduate medical education (GME). Despite the absence of traditional learning mechanisms and restrictions on group gatherings, it has remained the responsibility of individual program leadership to ensure trainee competence upon graduation [1]. As faculty have been tasked with additional care responsibilities including possible redeployments and increased telehealth, innovation in education has had its challenges during this time. Further, this educational disruption started at a time when many trainees were nearing their entrance to autonomous practice or nearing the advancement to their next training year with added clinical responsibilities. As such, it was prudent to ensure trainees continued to develop their knowledge and skills during this time. Many specialties have leaned on larger networks to supplement education through e-learning, defined as educational activities performed electronically over the internet, which has been met with early favorable perceptions by learners [2, 3, 4, 5]. In the past, hesitation to adopt e-learning was related to institutional support [6]. However, as others have noted, the urgency of developing accessible education for trainees during the COVID-19 pandemic has hastened innovation [4].

To become board certified in pediatric rehabilitation medicine (PRM), one must complete a four year physical medicine and rehabilitation (PM&R) residency followed by a two year PRM fellowship, a 5 year combined pediatrics-physical medicine and rehabilitation residency, or separate residencies in PM&R and pediatrics, with the inclusion of six months of PRM specific rotations [7]. During the pandemic, PRM educators and trainees recognized the challenges of medical education universal to all training programs and had particular concerns about challenges faced for PRM, given that it is a very small sub-specialty. In light of this, multiple virtual pediatric rehabilitation medicine (PRM) initiatives were created to support the education of PRM trainees, decrease the workload of program directors, and join forces across the country to provide expert education to all trainees. In March 2020, a national PRM virtual lecture series was created and advertised through a private PRM Facebook group comprised of confirmed PRM physicians and trainees and through an email chain of PRM program directors. This lecture series was in some ways a re-visitation of a previous national PRM trainee webinar series that was started approximately ten to twelve years ago and lasted for five to six years. This previous national PRM trainee webinar series was created through conversations between a small group of PRM program directors who felt that national PRM e-learning would allow trainees to learn from leaders in the field of PRM from across the country. When the prior national PRM trainee webinar series was active, webinars occurred every one to two months with about 20–25 participants. This prior series stopped years ago due to cost and scheduling constraints.

The re-visited national PRM virtual lecture series was developed in March 2020 for PRM fellows (PGY-5 through PGY-6 fellows after completion of a PM&R residency) and combined PM&R-Pediatrics residents (combined residents; PGY-1 through PGY-5 dual specialty residency program), but was also open to PRM attending physicians, trainees interested in pursuing PRM, trainees interested in learning more about PRM topics, and other rehabilitation medicine providers. Due to an overwhelming number of multidisciplinary educator volunteers, the series was offered approximately twice weekly starting March 27, 2020 through the end of the study period and the end of the academic year. The lecture series initially occurred weekly via the Zoom video conference platform. The American Academy of Physical Medicine & Rehabilitation (AAPM&R) became involved as the host in April 2020. The AAPM&R-hosted lectures were recorded with professional software and uploaded to the AAPM&R website [8]. There were thirteen lectures from March 27, 2020 until May 19, 2020, prior to the initiation of our research study. Lectures during this time frame were on the topics of neonatal abstinence syndrome, fluid management in pediatrics, pediatric burn injuries, anti-NMDA receptor encephalitis, acute flaccid myelitis, pediatric limb deficiency/amputation, developmental milestones, activity-based rehabilitation fundamentals, pediatric wheelchair prescriptions, rare conditions in PRM with case examples, quality improvement in pediatric rehabilitation medicine, how to approach rare conditions in PRM including an overview of the undiagnosed disease network, and emerging treatments for Duchenne Muscular Dystrophy. There were three lectures during the data collection period of our research study from May 20, 2020 until May 30, 2020. These lectures were on the topics of early diagnosis of cerebral palsy, normal nerve conduction value cutoffs in pediatric electromyography, and the use of the Hammersmith Infant Neurologic Exam for the early diagnosis of cerebral palsy. To our knowledge, lectures were mostly attended by PRM trainees, PRM attending physicians, and trainees interested in PRM. Live virtual attendance during the March 27, 2020 through May 30, 2020 time period ranged from 25 to 46 individuals with a mean of 67. Breakdown of attendee roles (i.e. PRM trainee, attending physicians, trainees interested in PRM, and other individuals) is unknown. Attendance numbers were based on log in information for the AAPM&R webinars and organizer collected data of attendee numbers during the Zoom lectures. The number of individuals who watched the recorded AAPM&R lectures after the live lectures and before closure of the survey is unknown.

In addition to the national PRM virtual lecture series, in April 2020, a national PRM journal club series was created. National advertisement of the journal club occurred through the PRM private Facebook group and a group email of interested parties that was subsequently developed. The series was offered approximately twice monthly from April 8, 2020 through the end of our research study period and the end of the academic year, June 30, 2020. Two or three articles, which were provided in advance of the session, were reviewed at each attending physician-led journal club session, followed by an interactive discussion. Similar to the national PRM lecture series, to our knowledge, the journal club sessions were mostly attended by PRM trainees, PRM attending physicians, and trainees interested in PRM. There were three journal club sessions from April 8, 2020 until May 19, 2020, prior to the initiation of our research study. Journal club sessions during this time frame were on the topics of the use of ankle foot orthoses (AFOs) in cerebral palsy/the use of botulinum toxin injections in cerebral palsy, outcome in brachial plexus injuries, and treatment of dystonia. There was one journal club session between May 20, 2020 and May 30, 2020, the time frame of data collection of our study, and it was on the prevalence, natural history and treatment of idiopathic toe walking. Live virtual attendance during the April 8, 2020 through May 30, 2020 time period ranged from 44 to 51 individuals with a mean of 48 individuals. Breakdown of attendee roles (i.e. PRM trainee, attending physicians, trainees interested in PRM, and other individuals) is unknown. Attendance numbers were based on organizer collected data of attendee numbers during the Zoom lectures.

In May 2020, it was recognized that while there were national efforts for scientific training for PRM trainees, the art of medicine was missing in the national efforts. With this in mind, a national arts in PRM series was created. The only event of this series prior to the closure of our research study occurred on May 8, 2020 and was a screening of the disability rights film, Crip Camp (2020) via a Netflix Watch Party, followed by a Zoom virtual video discussion led by a PRM attending physician. Prior to the event, questions were sent to the group to lay the framework for the discussion. As was true for other efforts, the event was primarily attended by PRM trainees, PRM attending physicians, and trainees interested in PRM. While approximately 46 individuals requested the event information, it is estimated that between 10 and 20 individuals attended the post-movie watch Zoom virtual video discussion. It is unclear how many individuals participated in the movie watch itself. Breakdown of PRM trainees, attending physicians, trainees interested in PRM, and other individuals is unknown.

Throughout GME, the trainee experience has been dramatically altered during the time of the COVID-19 pandemic. At times, residents have lost specialty-specific clinical learning opportunities and prepared for the possibility of unfamiliar clinical demands [9, 10]. Personal stress has been heightened, especially for those separated from their families during training [11]. Early literature included commentary on effects on trainees and best approaches to optimize education during the pandemic from the perspective of leadership [12, 13], but there was an apparent lack of the trainee voice in early publications. This study sought to assess the experience of PRM trainees during the early stages of the COVID-19 pandemic.

This study presented and evaluated the aforementioned national PRM e-learning curricula. Through a survey questionnaire, this study sought to determine trainees’ perceptions of national e-learning and to assess the overall early impact of the COVID-19 pandemic on PRM training.

Guided by Association for Medical Education in Europe (AMEE) guidelines, a questionnaire was developed and approved by the Northwestern University Institutional Review Board (IRB) [14]. By the time of the questionnaire release, there had been 13 lectures, 3 journal clubs, and 1 arts in PRM event. By the time of questionnaire closure, there had been 16 lectures, 4 journal clubs, and 1 arts in PRM lecture. Study data were collected from May 20 th 2020 to May 30 th 2020 and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at Northwestern University Feinberg School of Medicine [15]. REDCap is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources. Current PRM fellows and combined residents were recruited through announcements via an email listserv and the private PRM Facebook group. Participants received no compensation. To maintain anonymity, minimal demographic data was collected, and included the type of trainee (PRM fellow or combined resident) and whether the trainee was in the graduating class of 2020. To determine perceived impact on sponsoring institution (SI), participants were asked to define their SI to one of the three stages provided by the Accreditation Council for Graduate Medical Education (ACGME) at that time: 1) Stage 1: “business as usual;” 2) Stage 2: “increased but manageable clinical demand;” 3) Stage 3: “crossing a threshold beyond which the increase in volume and/or severity of illness creates an extraordinary circumstance where routine care education and delivery must be reconfigured to focus only on patient care” [16]. Note that since completion of this survey, the ACGME has updated their SI COVID-19 emergency categorization into a binary approach with emergency and non-emergency status designations [17].

The authors believed that at the time of the study, there were a total of 53 current PRM trainees (40 fellows; 13 combined residents) in the United States, according to National Resident Matching Program (NRMP) Match Data for PRM fellows with 2018 and 2019 appointment years, NRMP Match Data for combined residents with 2016–2019 appointment years, and discussion with/review of websites of combined residency programs as 2015 NRMP Match Data was not available online. Given the small sample size, only descriptive analysis of questionnaire items was performed.

3. Results

Thirty out of fifty-three PRM trainees responded to the survey (56.6%). This included 30.8% (4/13) of combined residents and 65.0% (26/40) of PRM fellows. Of those 30 responders, 43.3% (13/30) of PRM trainees were anticipated to graduate in the class of 2020. All trainees planned to incorporate procedures, and the majority of respondents planned to incorporate inpatient care (25/30, 83.3%) and outpatient care (29/30, 96.7%) into their future practice. As of survey closure in May 2020, the majority of respondents reported being in ACGME Stage 1 or ACGME Stage 2 during the majority of the March-May 2020 timeframe. During the majority of the March-May 2020 timeframe, 13.8% (4/29) reported being in ACGME Stage 3 status. At any point during the March-May 2020 timeframe, 24.1% (7/29) reported reaching ACGME Stage 3 status. Demographic data are further detailed in Table 1.

Most respondents reported attending the virtual lecture series (29/30, 96.7%) with one respondent leaving this question unanswered. Satisfaction with the educational value of the virtual lecture series was reported by (79.3%, 23/29) of respondents. Almost all respondents, (96.6%, 28/29), felt it would be valuable to continue the lecture series after the COVID-19 pandemic with respondent-suggested frequency ranging from once weekly to once monthly. All virtual lecture attendees (100%, 29/29) reported that access to subject experts from different institutions was a benefit of the series. The next most commonly cited benefits were recording of the lectures (89.7%, 26/29), flexibility of location (75.9%, 22/29), and networking (58.6%, 17/29). The most common perceived barrier was that lecture series time was not considered protected time (82.1%, 23/28). Virtual platform fatigue (64.3%, 18/28) was also a commonly cited concern. There was one free text comment noting that the new format, which we interpret to mean the AAPM&R webinar-style format versus the Zoom format, allowed for less ability to see other individuals attending the lecture and engage in conversations with the presenter or others. Although all respondents who attended the lecture series answered the question regarding benefits, one respondent who participated in the lecture series did not answer the question regarding barriers.

Nineteen out of thirty PRM trainee respondents (63.3%) reported attending the PRM national virtual journal club. Similar to the lecture series, the majority were satisfied with the educational value of the journal club (78.9%, 15/19). All virtual journal club attendees (100%, 19/19) thought it would be valuable to continue the virtual journal club, with the majority preferring a frequency of once monthly (78.9%, 15/19). The most common benefit noted was access to subject experts from different institutions (94.7%, 18/19), followed by networking with other PRM physicians (84.2%, 16/19) and flexibility of location (78.9%, 15/19). One respondent commented that it was valuable to have a place to discuss PRM topics. Lack of protected time was the most common perceived barrier (56.3%, 9/16). Noted in the free text section, there were concerns about participant engagement, difficulty with finding an optimal time, and lack of session recording. Although all respondents who attended the journal club answered the question regarding benefits, three respondents who participated in the journal club did not answer the question regarding barriers.

Four respondents (13.3%, 4/30) attended the national arts in PRM event. Similar to the other initiatives, the majority were satisfied with the educational value of the national arts in PRM event (75.0%, 3/4). All of the virtual arts attendees (100%, 4/4) felt it would be valuable to continue this series after the COVID-19 pandemic with respondent-suggested frequency ranging from once monthly to biannually. All of the virtual arts attendees noted networking, flexibility of location, and ability to discuss humanities/arts as it related to PRM as perceived benefits. Perceived barriers included technological difficulty and late-night timing. Although all respondents who attended the national arts in PRM event answered the question regarding benefits, only half (2/4) of the respondents who participated in the national arts in PRM event answered the question regarding barriers. Detailed responses on benefits and barriers of each of the three virtual initiatives are listed in Table 2.

OPEN IN VIEWER

Most trainees reported less satisfaction with their clinical education (70.0%, 21/30), but more satisfaction with their non-clinical education compared to pre-COVID-19 pandemic (60.0%. 18/30) (Fig. 1). Many of the PRM trainees, (85.7%, 6/7) who reported that their SI reached ACGME Stage 3 during the March-May 2020 timeframe reported less satisfaction with their clinical education during the pandemic compared to 63.6% (14/22) of the PRM trainees who reported that their SI did not reach ACGME Stage 3 during the March-May 2020 timeframe. All (100%, 4/4) of PRM trainees who reported that their SI was in ACGME Stage 3 for the majority of the March-May 2020 timeframe reported less satisfaction with their clinical education during the pandemic compared to 64.0% (16/25) of the PRM trainees who reported that their SI was not in ACGME Stage 3 during the majority of the March-May 2020 timeframe. Some (42.9%, 3/7) of PRM trainees who reported that their SI reached ACGME Stage 3 during the March-May 2020 timeframe reported less satisfaction with their non-clinical education during the pandemic compared to 9.1% (2/22) of the PRM trainees who reported that their SI did not reach ACGME Stage 3 during the March-May 2020 timeframe. Half (50.0%, 2/4) of PRM trainees who reported that their SI was in ACGME Stage 3 for the majority of the March-May 2020 timeframe reported less satisfaction with their non-clinical education during the pandemic compared to 12.0% (3/25) of the PRM trainees who reported that their SI was not in ACGME Stage 3 during the majority of the March-May 2020 timeframe.

Table 3 details respondents’ roles during the pandemic, including redeployments. Of the two reported redeployments, both reported reaching ACGME Stage 3 during the March-May 2020 timeframe; one respondent who was redeployed reported being in ACGME Stage 2 for the majority of the March-May 2020 timeframe, and one reported being in ACGME Stage 3 for the majority of the March-May 2020 timeframe. In addition to the reported roles in Table 3, one combined resident reported being on inpatient and outpatient pediatrics rotations and another combined resident reported being on a planned (not redeployed) adult outpatient PM&R rotation. These are typical rotations in combined residency training.

Participation in telehealth in some form was reported by 76.7% (23/30) of respondents. For respondents who reported being at ACGME Stage 3 for the majority of the March-May 2020 timeframe, 50% (2/4) participated in telehealth in some capacity. For those who reported being at ACGME Stage 2 for the majority of the March-May 2020 timeframe, this number was 61.5% (8/13), and for those who reported being at ACGME Stage 1 for the majority of the March-May 2020 timeframe, this number was 100% (12/12). The greatest concern with PRM training during the March-May 2020 timeframe was decreased procedure time (60.0%, 18/30), followed by decreased outpatient opportunities (43.3%, 13/30). Only 16.7% of respondents (5/30) listed decreased inpatient opportunities as a concern. Respondents also cited safety, stress, decreased PRM exposure, decreased clinical education from PRM attending physicians, and increased non-educational demands as concerns during the March-May 2020 timeframe. However, 26.7% of respondents (8/30) reported no concerns during the March-May 2020 timeframe.

The majority of graduating PRM trainees (83.3%, 10/12) felt confident to enter autonomous practice despite the altered learning environment due to COVID-19 during the March-May 2020 timeframe. A feeling of neither confident nor unconfident was reported by 16.7% (2/12). Note the n is only 12 as one graduating trainee’s response had to be removed from analysis due to an error on the REDCap survey questionnaire. The initial questionnaire erroneously listed choices for “How confident are you to enter autonomous practice upon graduation, given altered learning during the COVID-19 pandemic?” as completely unconfident, moderately confident, slightly unconfident, neither confident not unconfident, slightly confident, moderately confident, completely confident. Moderately confident was inadvertently listed twice and the individual chose the option listed in between completely unconfident and slightly unconfident, so it was difficult to interpret if the individual had meant to choose moderately confident or moderately unconfident. Once this was discovered, the survey was revised during the data collection period. We removed the one unclear response from data analysis.

The majority of respondents perceived that the PRM virtual education initiatives were valuable to their education. Although this study did not assess knowledge acquisition, prior studies have shown that e-learning can be an effective tool for health care education, though not superior to traditional in-person learning [18]. Access to subject experts was the most frequently marked benefit for both the virtual lecture series and the journal club. This was not surprising, as the virtual initiatives afforded direct, accessible education by field leaders without the personal and financial strains of out-of-town conferences.

On a similar note, networking was one of the top cited benefits to e-learning. With the COVID-19 pandemic, there has been concern about the loss of community that typically comes with GME training [19]. In a recent review article prompted by the COVID-19 pandemic, formation of virtual communities of practice were noted as an opportunity for distance learning in academic medicine [12]. Virtual communities of practice have allowed collaboration and learning between groups with common interests despite physical distance, similar to the multi-institution collaboration with the PRM national e-learning avenues. In e-learning, increased sense of community has been associated with increased perceived cognitive learning [20]. Still, educators must be cognizant to optimize engagement, as some e-learning settings have contributed to isolation related to decreased peer interaction and instructor feedback. With encouraged participation and increased sharing of ideas, a greater sense of community can occur. Additionally, online tools, such as discussion forums, have been effective in fostering a feeling of connectivism between instructors and learners [21]. Perhaps, using these strategies in an effort to increase the sense of community in future PRM e-learning sessions may help improve engagement.

The recording of lectures was the second most cited benefit of the virtual lecture series. Watching pre-recorded lectures and curriculum at one’s own pace, or asynchronous learning, has been increasingly utilized as a solution to the logistical challenges of in-person learning in GME [22, 23]. A survey of internal medicine program directors showed most programs were utilizing e-learning, but more commonly asynchronous [24]. Utilizing asynchronous learning also circumvents the difficulty of national initiatives across several time zones.

Lack of available recordings was noted by one respondent as a barrier to the virtual journal clubs, which were intentionally not recorded. The intent behind the journal club’s lack of recording was to foster a sense of psychological safety amongst trainees to facilitate open discussion [25]. Respective leaders of the virtual lecture series and virtual journal club series felt that it was appropriate to record lectures, but not record journal clubs where discussion and opinions could be raised without fear of judgment. In the future, benefits and threats of recording e-learning should be further investigated.

With all medical education innovation, opportunities as well as barriers must be considered. Virtual platform fatigue (64.3% of lecture series participants, 62.5% of journal club participants) and decreased engagement (37.5% of lecture series participants, 43.8% of journal club participants) were reported barriers of the virtual PRM lecture series and virtual PRM journal club participants. In the setting of COVID-19, videoconferencing users have noted what has been termed as “Zoom fatigue” [26, 27, 28].

Reported contributors to “Zoom fatigue” have included the demand to process nonverbal cues, anxiety related to technology malfunction, and lack of separation between the different contexts of life (e.g. work versus home life), all compounded by the increased personal stressors during the pandemic [26]. Virtual platforms have created an atmosphere in which users visualize other participants directly with more eye contact for longer periods of time which innately requires different types of attention skills [27]. Although online video platforms have connected users during the pandemic, difficulty interpreting nonverbal cues, video stalls, and delayed response to questions on chat functions all have been cited as barriers to engagement among participants [28].

Training programs must consider duty hours when applying these findings to educational curriculum. Most respondents (82.1%) noted a concern that the lecture series did not count as protected time. It is anticipated that e-learning will remain beyond the period of the COVID-19 pandemic, though likely supplemental to in-person educational time. As PRM shifts towards e-learning, ways for trainees to appropriately incorporate this added time within the current ACGME regulations should be addressed. While cost was not a frequent concern noted by trainees in the present study, programs must ensure equal opportunity access with regard to e-learning. Potential barriers could be reliability of home internet, access to computer or smart phone, availability of virtual platform, and institutional coverage of professional membership fees.

This study has demonstrated that the early impact of COVID-19 has dramatically affected the satisfaction rates of clinical and non-clinical PRM education. The results indicated that there was an overall increased satisfaction with non-clinical education when compared to the pre-pandemic environment. Many departments encouraged trainees to leverage downtime from clinical rotations during the early stages of the COVID-19 pandemic towards non-clinical education and research. Almost half of PRM trainee respondents spent some time on research (46.7%), and another 40.0% spent time on independent study. The shift in roles away from inperson clinical duties that occurred for some trainees could have theoretically presented a unique opportunity for research, self-directed learning, academic writing, and grant application, including the reported higher receptivity of IRBs to COVID-19 research and data collection for retrospective analyses [29, 30, 31]. Of course, others in more impacted areas may not have had the increased availability for academic work.

While nonclinical educational satisfaction rates during the early stages of the COVID-19 pandemic increased, clinical education satisfaction rates decreased, with particular concerns about the reduction of procedures (60.0%) and outpatient time (43.3%). With Centers for Disease Control and Prevention (CDC) guidelines requiring preservation of personal protective equipment, physical distancing, and establishment of a triaging framework for non-essential procedures, there has been a redistribution of personnel and reduction in specialty-specific outpatient clinical and procedural volumes, thereby reducing work-place based learning and apprenticeship [9, 31, 32]. Studies on the early stages of the COVID-19 pandemic that were outside of PRM have indicated that many trainees were redeployed to a clinical service different from their specialty; however, fortunately only two (6.7%) respondents reported redeployment away from PRM [31] Literature from other specialties has shown that even when clinical services were available during the early stages of the COVID-19 pandemic, there were reduced case volumes, and the required minimal contact environment may have caused the exclusion of trainees over attendings [9, 31]. Delay of PM&R procedures was common during the early stages of the COVID-19 pandemic as many procedures in PM&R are considered elective or non-urgent. A recent manuscript has recommended procedural stratification towards function-specificity and time-sensitivity [32]. This could result in less delayed functional care and subsequently additional PRM trainee opportunities during the remainder of the COVID-19 pandemic or during another similar crisis

This study showed that a greater proportion of PRM trainees at SIs that reached ACGME Stage 3 during the early stages of the COVID-19 pandemic reported less satisfaction with clinical and non-clinical education compared to pre-pandemic when compared to PRM trainees whose SI did not reach ACGME Stage 3. This is not unexpected as these trainees likely had added stress and additional responsibilities that could have pulled them away from clinical and non-clinical opportunities for education [33]. Additionally, while this is unknown as we did not collect this on the survey, it is also possible that trainees may have had less access to their attending physicians as their attending physicians may have also had additional responsibilities pulling them away from their clinical and non-clinical educator roles. Interestingly, those who reported being at a higher stage during the majority of the March-May 2020 timeframe participated in telehealth less, possibly because they were redeployed outside their specialty. One of the goals of the virtual PRM lecture series’ recording of lectures was to provide asynchronous educational opportunities for all trainees, including those pulled to pandemic-related care. This certainly begs the question of how else the playing field can be leveled for trainees who had a disproportionate amount of stress and additional responsibilities during the pandemic

The COVID-19 pandemic has resulted in a disruption of education for many graduates, including GME graduates in all fields of medicine. With this, there was concern among GME educators that this would lead to decreased confidence for graduation and autonomous practice. Fortunately, the upcoming PRM graduates studied showed a high confidence rate, with 83.3% of graduating PRM trainees reported feeling confident and 16.7% reported feeling neutral on the topic of readiness to graduate and proceed with autonomous practice Without data from years prior to 2020, it is unclear how this compares to pre-pandemic years. For reference, over a quarter of surgical residents have reported concern that they would not feel confident to perform procedures independently prior to training completion [34].

Limitations to this study included the lack of representation of all PRM trainees. Given the small number of national PRM trainees, the authors elected to have the three PRM trainee co-authors on this paper remain eligible to participate in the study. To prevent threats to internal validity, these co-authors were asked not to review preliminary data until after completion of the survey, should they decide to participate as voluntary survey participants. Given that the total number of national PRM trainees was an estimate, it is recognized that this estimate may be missing additional information on trainees who left training, took time off, or scrambled into the field. With regard to roles during the COVID-19 pandemic, the survey failed to ask about general pediatric roles or scheduled adult PM&R roles, which may apply to combined residents (and in fact did apply to at least two of the combined resident respondents who free texted these responses). ACGME Stages were self-reported by PRM trainees and not confirmed with the SI, which could have led to some inaccuracies in the estimation of the ACGME Stages. The authors also acknowledge that there was one error in the REDCap questionnaire regarding confidence to graduate, with two choices for moderately confident being available; one was in linear placement between completely unconfident and slightly unconfident and the other in the intended placement. While the survey was corrected as soon as it was noticed, it was difficult to interpret the intent behind the one respondent who chose the improperly placed moderately confident response. Given this error and the fact that this questionnaire was not validated, there was potential for measurement error.

Ultimately, this study demonstrated a favorable perception of e-learning tools for PRM trainees. Dissemination of this type of data can be impactful, as program directors with positive perceptions of e-learning have been more likely to integrate its use [24]. Organizers of the PRM e-learning initiative described in this manuscript (national PRM virtual lecture series, national PRM journal club, and the national arts in PRM series) plan to continue these initiatives for the foreseeable future. In some ways, the COVID-19 pandemic brought about e-learning initiatives for PRM that will benefit PRM trainees beyond the timeframe of the COVID-19 pandemic. In the future, as utility of e-learning continues to grow, it may be valuable to repeat this survey to evaluate changing perceptions in this new world of frequent e-learning. With the awareness of the barriers and threats presented here, PRM programs can more confidently proceed with the creation of optimal e-learning platforms as an adjunct to traditional in person education, even after the COVID-19 pandemic.

5. Conclusions

PRM trainees perceived national e-learning during the early stages of the COVID-19 pandemic as valuable to their education. Noted benefits included access to subject experts, networking, lecture recording, and location flexibility. Common concerns included lack of protected time, virtual platform fatigue, and decreased engagement. Relative to before the COVID-19 pandemic, less satisfaction with clinical education and greater satisfaction with non-clinical education was reported. A higher proportion of PRM trainees who reported that their SI reached ACGME Stage 3 during the March-May 2020 timeframe reported less satisfaction with both clinical and non-clinical education compared to PRM trainees who did not report that their SI reached ACGME Stage 3 during the same time period. Overall, PRM trainees felt confident for graduation and autonomous practice despite altered training during the early stages of the COVID-19 pandemic. Future work should explore ways to overcome barriers and optimize benefits of GME and PRM e-learning.