Primary Palliative Care Education for Trainees in U.S. Medical Residencies and Fellowships: A Scoping Review

Abstract

Background:

The medical profession increasingly recognizes the growing need to educate nonpalliative physicians in palliative care.

Objective:

This study aims to provide a scoping review of the primary palliative care (PPC) education currently available to graduate medical trainees in primary and specialty tracks.

Design:

Studies of PPC interventions in U.S. residency or fellowship programs of all subspecialties published in English and listed on MEDLINE, CINAHL, and EMBASE through January 2020 were included. To meet admission criteria, studies had to describe the content, delivery methods, and evaluation instruments of a PPC educational intervention.

Results:

Of 233 eligible full texts, 85 studies were included for assessment, of which 66 were novel PPC educational interventions and 19 were standard education. Total number of publications evaluating PPC education increased from 8 (2000–2004) to 36 (2015–2019), across 11 residency and 10 fellowship specialties. Residency specialties representing the majority of publications were emergency medicine, general surgery, internal medicine, and pediatric/medicine-pediatrics. PPC content domains most taught in residencies were communication and symptom management; the primary delivery method was didactics, and the outcome assessed was attitudes. Fellowship specialties representing the majority of publications were pediatric subspecialties, nephrology, and oncology. The PPC content domain most taught in fellowships was communication; the primary delivery method was didactics and the outcome evaluated was attitudes.

Conclusions:

While PPC education has increased, it remains varied in content, delivery method, and intervention evaluations. Future studies should include more widespread evaluation of behavioral outcomes, longitudinal persistence of use, and clinical impact.

Key Message

This scoping review describes the current state of primary palliative care education available to physician trainees in U.S. graduate medical education programs. It finds multiple medical specialties teach primary palliative care and the number of studies has increased. This study maps the current landscape by educational domains, methods, and intervention evaluation.

Introduction

The need for effective palliative care (PC) grows with the aging population and increasing number of patients with chronic conditions.^1,2 In addition to an increase in the number of trained PC specialists, non-PC providers in all specialties must learn PC knowledge and skills, known as primary palliative care (PPC), to meet the growing clinical demand.^3–5 PPC education as early as undergraduate medical education builds a foundation of understanding and acceptance. The importance of this by education is reflected in an increasing demand for early and widespread PPC education.⁶ Medical education researchers and training programs have been studying whether the current educational interventions to teach PPC knowledge and skills to their trainees are successful; however, the definition of success and how to achieve this vary.

Prior systematic reviews of PPC education provide a glimpse of the educational landscape.^1–4,7–10 Turrillas et al. conducted a systematic review of symptom management in graduate medical curricula that included six studies. They found pain to be the most common symptom taught, delivered through varied educational methods and evaluated by self-reported evaluations.¹⁰ Shaw et al. conducted a systematic review of PC curricula in graduate medical education (GME) across all medical subspecialties. They found 28 English-language studies that evaluated PC education interventions in any residency specialty. They concluded that an impactful PC curriculum should include a myriad of teaching strategies and address multiple competencies and called for further studies to be conducted. This study focused only on end-of-life (EOL) curricula, did not include studies that solely used trainee satisfaction for program evaluation, and did not evaluate curricula by medical specialty.¹

No systematic review of GME PPC curricula has been published in the last 10 years, and this is the first scoping review of this topic. This study offers an expanded search of PPC educational interventions for residents and fellows categorized by medical specialty. It includes additional characteristics of PPC curricula, specifically educational content, delivery methods, and outcomes. We hypothesize that PPC curricula will be varied across specialties in educational content and outcomes, but with similar delivery techniques.

Methods

Protocol and registration

This is a scoping review designed to “examine the extent, range, and nature of the evidence on a topic or question” and to “summarize findings from a body of knowledge that is heterogeneous in methods or discipline.”¹¹ The Preferred Reporting Items for Systematic Reviews and Meta-analysis extension for scoping reviews (PRISMA-ScR) guidelines informed the development of this study's protocol.¹¹ The authors searched for scoping reviews similar to this study on Open Science Framework (OSF) (www.osf.io) to ensure the absence of prior studies and confirmed this again before publication submission. The protocol was registered on OSF (osf.io/r9au4).

Data sources and searches

The literature was searched from inception through January 17, 2020, with the databases MEDLINE, Cumulative Index to Nursing and Allied Health (CINAHL), and Exerpta Medica (EMBASE). The following CINAHL search was used: ((((“Palliative Care”[Mesh] OR “Palliative Medicine”[Mesh] OR “Terminal Care”[Mesh:NoExp] OR “Hospice Care”[Mesh] OR “Death”[Mesh] OR Palliative[tw] OR Hospice[tw] OR “end-of-life care”[tw] OR “comfort care”[tw] OR “terminal care”[tw]))) AND ((“Education, Medical, Graduate”[Mesh] OR residency[tw] OR resident[tw] OR residents[tw] OR “graduate medical education”[tw] OR fellow*[tw]))) AND ((“Education”[Mesh:NoExp] OR “Curriculum”[Mesh] OR “Knowledge”[Mesh] OR “Clinical Competence”[Mesh] OR “Program Evaluation”[Mesh] OR “Surveys and Questionnaires”[Mesh] OR “Attitude”[Mesh] OR Education*[tw] OR Knowledg*[tw] OR “clinical competence”[tw] OR evaluat*[tw] OR survey*[tw] OR “program director”[tw] OR “program directors”[tw] OR opinion[tw])). A medical informationist assisted with all literature searches.

Study selection

Studies written in English and conducted in the United States graduate medical training programs (residencies and fellowships) in any primary or specialty medical track, except hospice or palliative medicine, were included. The educational intervention described could be novel or already incorporated into standard teaching, and content and delivery, as well as trainee evaluation, needed to be described. Studies were excluded if they were not in English, included GME programs outside the United States, or were review articles or meta-analyses. In addition, we excluded studies that included nongraduate medical trainees (i.e., medical students, nurses, and faculty) and did not differentiate between the results of these trainees such that upon review, we could not analyze the graduate medical trainees' responses. We also excluded studies that did not explicitly describe the educational intervention's content or delivery methods.

Two authors (K.P. and A.K.M.) divided abstracts to review. Full-text review followed if either the study met full inclusion criteria or if uncertainty existed. The same two authors resolved discrepancies through discussion until reaching consensus.

Data extraction and categorization

The above two authors read every included study. Each author evaluated half of the studies to assess for year of publication, trainee level (residents or fellows), medical specialty, intervention novelty, content domains, delivery methods, and evaluation instruments used.

The educational intervention was described as novel if it was developed for the purposes of the study. If it had been incorporated into the existing training before the study, the intervention was described as standard. As we reviewed the literature, we identified educational content domains and compared these to the Accreditation Council for Graduate Medical Education (ACGME) requirements for hospice and palliative medicine.¹² We also identified delivery methods and compared them to those referenced by Thomas et al.¹³ Our groupings aligned with each. If an intervention used multiple domains or methods, all were included in the final assessment for that study. Content domains were communication, symptom management, EOL care, psychosocial support, spiritual support, transitions of care/resources (TOC/RE), ethics, and bereavement. Categories of educational delivery methods were didactics/lecture, discussion, rotation, workshop/retreat, role-play, standardized patients (SP), independent learning, web-based learning, and journal club. Workshops and retreats were considered one intervention even if they included additional methods of delivery to emphasize a one-time intervention versus ongoing education.

Our characterization of evaluation instruments was consistent with established medical education terminology.¹³ If multiple instruments were utilized in a study, each one was counted and characterized in our assessment. Types of outcomes measured included subjective knowledge, objective knowledge, attitudes, skills through direct observations (including evaluations of SP or real patient examinations), and rate of PC specialty consultations.

Results

Included studies

An initial search identified 3761 studies. After removing duplicates, non-English studies, and screening of titles and abstracts, 233 full-text articles were assessed for eligibility. Eighty-five studies met selection criteria for final descriptive analysis (Fig. 1). The first study was published in 1983 and 54 studies (64%) were published in the last decade (Fig. 2). Appendix Tables A1 and A2 contain full descriptions of included studies.

FIG. 1.

Flow diagram of study selection process. From: Moher et al.¹⁰⁶

FIG. 2.

Number of published studies from 1980 through 2019 related to primary palliative care education in residency and fellowship programs in the United States.

Program characteristics

Fifty-nine studies included residents only, 20 studies included fellows only, 2 studies included residents and fellows, 3 studies included residents and program directors, and 1 study included fellows and program directors. Sixty-six studies evaluated novel educational interventions, while 19 evaluated standard education. The majority of studies evaluating residents were in internal medicine, emergency medicine, family medicine, pediatric/medicine-pediatrics, and general obstetrics and gynecology specialties (n = 52, 82%). Residency specialties represented by three or more studies were emergency medicine (n = 4), general surgery (n = 5), internal medicine (n = 28), and pediatrics and medicine-pediatrics (n = 14) (Fig. 3a). Fellowship specialties represented by three or more studies were pediatric subspecialties (n = 4), nephrology (n = 3), oncology (n = 4), and pulmonology/critical care (n = 3) (Fig. 3b).

FIG. 3.

Medical residency (a) and fellowship (b) program specialties represented by published studies included in this scoping review.

Residency programs: Overall

Sixty-three studies included residents in some capacity.^14–77 Studies of novel interventions ranged from years 1983 to 2019. Studies of standard education ranged from the years 2003 to 2019. Communication and symptom management were the content domains emphasized across most interventions for residents. Every residency specialty, except family medicine, used didactics to deliver education. Other delivery methods were utilized equally across programs, except for journal club, which no program utilized. Older studies include rotations, which have been used equally throughout the last two decades. In the last decade, programs have increasingly utilized didactics, workshops, and retreats. Every residency specialty utilized knowledge assessments (subjective or objective) as part of their intervention.^{14–41,43–77} Six studies included residents from multiple specialties or mixed residents and fellows. Outcomes of these studies could not be attributed to a specific specialty or training level, and were therefore not included in the following summaries of PPC by specialty.^71–76 Figure 4 and Appendix Tables A3–A5 provide a comprehensive breakdown of content domains, delivery methods, and evaluation instruments, respectively.

FIG. 4.

Content (a), delivery methods (b), and evaluation instruments (c) of primary palliative care educational interventions used in U.S. residency and fellowship training programs.

Residency programs: Emergency medicine

Four studies evaluated PPC education in emergency medicine residency programs, of which three evaluated novel interventions^14–16 and one evaluated standard training.¹⁷ Content domains taught were communication (n = 4), symptom management (n = 3), EOL (n = 1), psychosocial support (n = 1), TOC/RE (n = 1), ethics (n = 1), and bereavement (n = 1). No study included spiritual support. Delivery methods used were didactics/lectures (n = 3), discussion (n = 2), role-play (n = 1), SP (n = 1), and independent learning (n = 1). Outcomes measured were objective knowledge (n = 2) and attitudes (n = 3).

Trainees reported increased comfort with communication such as giving bad news and prognosis discussion as a result of the interventions.^14,16,17 Two studies evaluated trainee competence using objective measures.^14,15 Gisondi et al. studied the impact of using the comprehensive curriculum, the Education in Palliative and End-of-Life Care for Emergency Medicine (EPEC-EM), alone or in conjunction with other methods of learning. They found that with both methods, trainees demonstrated an objective knowledge improvement.¹⁵ Devader and Jeanmonod employed EPEC-EM to develop their resident curriculum, which led to improved knowledge of hospice admission criteria, translating over time into more hospice admissions. Resident comfort with symptom management did not improve.¹⁴

Residency programs: General surgery

Six studies evaluated PPC educational interventions in surgery programs. All incorporated novel interventions.^19–24 PC domains taught were communication (n = 6), symptom management (n = 4), EOL (n = 1), TOC/RE (n = 3), and ethics (n = 2). Delivery methods used were didactics/lecture (n = 3), discussion (n = 2), workshop/retreats (n = 2), role-play (n = 3), SP (n = 1), independent learning (n = 2), and web based (n = 1). Outcomes measured were subjective knowledge (n = 3), objective knowledge (n = 3), and attitudes (n = 4).

Objective scores of trainee knowledge significantly increased in three studies.^19,22,23 Interventions utilized objective structured clinical examination scores to measure effects of interventions on trainee competence and found no significant improvement.^19,24 Trainees showed an increase in confidence in communication skills such as speaking about EOL and PC issues with families after the intervention and at three months.^20,24 They demonstrated an interest in learning about discussing prognosis and the role of PC in surgery.²³

Residency programs: Internal medicine

Twenty-eight articles studied internal medicine residents, of which 24 used novel interventions. PC domains were communication (n = 25), symptom management (n = 11), EOL (n = 9), psychosocial support (n = 3), TOC/RE (n = 6), ethics (n = 7), and bereavement (n = 2). No intervention included spiritual support. Delivery methods included didactics/lectures (n = 11), discussion (n = 7), rotations (n = 10), workshop/retreat (n = 5), role-play (n = 6), SP (n = 3), independent learning (n = 4), and web based (n = 2). Journal club was not used in any study. Outcomes measured were subjective knowledge (n = 13), objective knowledge (n = 4), attitudes (n = 16), and skills through direct observation (n = 5).

With novel interventions, objective knowledge of communication and nonpain management increased.^{25,27,29,34,39,40,47–51} Some trainees' confidence decreased with higher objective competence scores.^26,40,50 One study found no correlation between attitude scores and knowledge or training experience, despite significantly increased knowledge scores.⁴⁰ One study reported a significant improvement in confidence and competence.⁵¹

Residency: Pediatrics and medicine-pediatrics

Thirteen studies included pediatrics or medicine-pediatrics residents, of which 10 were novel interventions.^{56–59,61,63,65–68} Every PC education domain was taught in at least one study: communication (n = 11), symptom management (n = 10), EOL (n = 3), psychosocial support (n = 2), spiritual support (n = 3), TOC/RE (n = 3), ethics (n = 2), and bereavement (n = 4). Methods of delivery utilized were didactics/lectures (n = 8), discussion (n = 4), rotation (n = 2), workshop/retreat (n = 3), role-play (n = 1), SP (n = 1), and independent learning (n = 3). No program employed web-based methods or journal clubs. Outcomes measured were subjective knowledge (n = 5) and attitudes (n = 13). One study objectively measured trainee competence and found no significant change.⁵⁸

Fellowship programs: Overall

In total, 22 studies included fellows.^{62,72,78–98} Seven studies evaluated standard training from 2005 to 2019.^{62,72,81,82,87,89,91,98} Fifteen evaluated novel interventions from 2003 to 2019.^{78–80,83–86,88,90—97} Didactics/lectures was the most common method of teaching. One study included fellows and residents in multiple training specialties.⁷² Since outcomes could not be attributed to one specialty, this study was not included in the following specialty descriptions.

Fellowship programs: Pediatrics

Pediatric fellowships included hematology/oncology, oncology, mixed (cardiology, critical care, hematology/oncology, and neonatology), and critical care subspecialties. Four focused on novel interventions,^62,92–94 and one on standard interventions.⁵⁹ PC domains included communication (n = 5), symptom management (n = 3), spiritual support (n = 1), ethics (n = 2), and bereavement (n = 2). Delivery methods used were didactics/lectures (n = 2), discussion (n = 1), workshops/retreats (n = 2), SP (n = 2), independent learning (n = 1), and web based (n = 1). Outcomes measured were subjective knowledge (n = 4), attitudes (n = 5), and consultation rate (n = 1).

Knowledge significantly increased in grief/bereavement, GOC, and symptom management.^92,94 Self-reported comfort and confidence increased significantly.^93,94

Fellowship programs: Nephrology

Three studies evaluated PPC education interventions in nephrology fellowships, of which two were novel^85,86 and one was standard.⁸⁷ Content domains taught were communication (n = 2), symptom management (n = 1), EOL (n = 1), psychosocial support (n = 1), TOC/RE (n = 1), and ethics (n = 1). Delivery methods utilized were didactics/lectures (n = 1), rotation (n = 1), and workshop/retreat (n = 2). Outcomes measured were attitudes (n = 3) and skills through direct observation (n = 1). All studies asked trainees their perspectives regarding comfort levels after their interventions. They found a significant improvement in preparedness, discussing dialysis, and not offering dialysis.⁸⁷

Fellowship programs: Oncology

Four studies evaluated PPC education interventions for fellows in oncology programs of which two were novel^88,90 and two were standard.^89,91 Content domains taught were communication (n = 4), symptom management (n = 3), EOL (n = 1), TOC/RE (n = 1), and bereavement (n = 1). Delivery methods used were didactics/lecture, discussion (n = 2), rotation (n = 1), workshop/retreat (n = 2), role-play (n = 1), SP (n = 1), and journal club (n = 1). Outcomes measured were subjective knowledge (n = 3), attitudes (n = 3), skills through direct observation (n = 2), and consultation rate (n = 1).

Interventions focused on improvement in trainee skills, commonly communication.^88–91 Rotations were recommended by trainees. Improvement of symptom management skills, especially with opioids, was limited.

Discussion

This study is a scoping review of all published studies of PPC education interventions in U.S. GME programs. Overall, the number of studies published evaluating PPC education increased over the last 2 decades, representing a myriad of medical specialties and subspecialties. Curricula chiefly taught communication and symptom management using didactics. Programs relied on trainee self-report to evaluate program quality and effectiveness. The literature included in this review emphasizes PPC education in emergency medicine, general surgery, internal medicine, pediatric/medicine-pediatric residencies and pediatric subspecialties, nephrology, and oncology fellowships. These programs are creating novel curricula to address gaps in primary palliative education.

In previous studies, multifaceted educational innovations helped improve trainee knowledge and skills in PC.^1,2,10 Our findings confirm heterogeneity of content, delivery, and outcomes in PPC education innovations across programs. Multiple reasons may account for this diversity. First, some medical educators may find a one-size-fits-all approach to PPC education does not meet the needs of their trainees, educators, program leadership, or patients. Second, limited defined national standards for PPC education broadly or by medical specialty leave programs without guidance. Third, PPC education may need to be tailored to meet the needs of different medical specialties.⁹⁹ Last, PPC in and of itself is an early and evolving area of education.

We did find communication to be the educational content domain included by every specialty in their PPC curriculum, which previous reviews also identified as the most common domain.^1,2 This fact may reflect recognition of the need for, and importance of, teaching communication skills.^99–102 However, comprehensive education in PC may require a multifaceted approach, including spiritual support, which we found to be included in more fellowship curricula than residency curricula. The absence of studies of education on spiritual support for internal medicine residents was surprising and points to an opportunity. Internal medicine residents have been found to have positive attitudes toward, but low knowledge levels about, spiritual care.¹⁰³

Regarding intervention delivery, as expected, there were some methods more commonly utilized across program specialties. Delivery methods should match the curriculum's educational objectives and be feasible to implement.¹³ Lectures, readings, and discussions are often less resource-intense, likely contributing to their widespread use across residency and fellowship specialties. The educational outcome best achieved by lectures and readings is cognitive knowledge gain. Discussions may be better at impacting an affective outcome such as changing students' attitudes and can be especially effective when combined with real-life experiences such as role-play, simulated experiences, and rotation.¹³ We found that educational interventions using discussion also employed at least one of these real-life experience methods as well. Educational methods utilized in curricula should reflect the method that best achieves the intended objectives.

Self-assessments of perception/attitudes and assessments of knowledge were the most common forms of evaluation utilized across all specialties. In the development of educational curricula, methods of measuring outcomes should be as reliable and valid as possible. According to Kirkpatrick's Model of Evaluation, patient and health care outcomes are the “highest” level of measurement outcomes. After these, behaviors/performances are more important than skills, which are more important than knowledge or attitudes. These are more important than satisfaction or perception. Outcomes measured using objective ratings are preferred to subjective ratings.¹⁰⁴ If measurements are not the “highest” according to this model, but match the curriculum's learning objectives, they are even more desirable.^13,105

While more challenging to implement, more educational research measuring patient or system outcomes are needed. Also, few studies reported whether educational interventions had longitudinal impacts on behaviors and results.^20,49,93,94 Future educational interventions should study long-term outcomes.

This study has several inherent limitations to be considered, given the study design. First, some medical subspecialties may not have representation in our study because they lack studies that meet our inclusion criteria. Some programs may utilize interventions without reporting them in the literature. Second, selection criteria excluded training programs outside the United States, given the large differences between the U.S. medical education training system and the international training system. Finally, studies published in non-English were excluded, given translation limitations, which may have excluded studies about U.S. medical education written in other languages.

Conclusions

For educators working on PPC education, this review may serve as a guide for current curricular resources as per medical specialty and opportunities for future educational research. Educational content, methods, and outcomes are varied depending on the specialty. National standards that define excellence in PPC education are necessary to ensure quality and meaningful outcomes.

We suggest the need for future research on: subspecialty programs underrepresented, or not represented, in this review; the clinical impact of PPC education; and the long-term impact on trainees. Fostering existing platforms for the immediate broad delivery of PPC curriculum education can be done through partnerships between professional societies of PC and non-PC specialties. National legislation such as the Palliative Care and Hospice Education and Training Act in conjunction with national organizations such as the Center to Advance Palliative Care has the potential to highlight and actuate growth of curricula. Universal access to excellent PPC education across all U.S. GME programs would provide a crucial step toward increasing access to PC in the United States.

Footnotes

Acknowledgments

The authors would like to acknowledge Bethany Myers, MS (UCLA), for her invaluable assistance with the literature searches.

Funding Information

Dr. Wu is supported by the Cambia Health Foundation Sojourns Scholar Leadership Award and the Hearst American Academy of Hospice and Palliative Medicine Leadership Scholars Fund.

Author Disclosure Statement

No competing financial interests exist.

Appendix Table A5.

Primary Palliative Care Educational Evaluation Instruments Reported in Residency and Fellowship Programs

Type of program	Program specialty (n) (Reference)	Outcomes
Type of program	Program specialty (n) (Reference)	Subjective knowledge	Objective knowledge	Attitudes	Direct observation	Consultation rate
Residency	N = 63
	Emergency medicine (n = 4)^14–17	—	14, 15	14, 16, 17	—	—
	Family medicine (n = 1)¹⁸	18	—	—	—	—
	Surgery (General and orthopedics) (n = 6)^19–24	19, 21, 23	22–24	19, 20, 22, 24	—	—
	Internal medicine (n = 28)^{25–41,43–47,49–52}	27–31, 36–40, 47, 49, 52	39, 41, 43, 52	26, 27, 29, 30, 32–35, 40, 42, 44–47, 50, 51	25, 48, 50	—
	Neurology (n = 1)⁵³	—	53	53	—	—
	Neurosurgery (n = 1)⁵⁴	—	54	54	—	—
	Obstetrics/Gynecology (n = 1)⁵⁵	55	—	55	—	—
	Pediatrics, Medicine pediatrics (n = 13)^{56–61,63–68,77}	57, 59, 61, 64, 77	—	56–61, 63–68, 77	—	—
	Psychiatry (n = 1)⁶⁹	69	—	69	—	—
	Radiation oncology (n = 1)⁷⁰	—	—	70	—	—
	Not specific/Mixed (n = 6)^71–76	72, 74, 76	71, 72	73, 75	76	—
Fellowship	N = 22
	Cardiology (n = 1)⁷⁸	—	—	78	—	—
	Critical care (n = 2)^79,80	79	—	79, 80	80	—
	Geriatrics (n = 2)^81,82	—	—	81, 82	—	—
	Gynecology oncology (n = 1)⁸³	—	—	83	—	—
	Neonatal-Perinatal (n = 1)⁸⁴	84	—	84	—	—
	Nephrology (n = 3)^85–87	—	—	85–87	86	—
	Oncology (n = 4)^88–91	89–91	—	89–91	88	91
	Pediatrics (n = 5)^62,92–95	62, 92–94	—	62, 92–94	—	93
	Pulmonary critical care (n = 2)^96,97	—	—	96, 97	—	—
	Surgical oncology, HPB (n = 1)⁹⁸	—	—	98	—	—

References

Shaw

, Marshall

, Howard

, et al.: A systematic review of postgraduate palliative care curricula. J Palliat Med, 2010; 13:1091–1108.

Carroll

, Weisbrod

, O'Connor

, et al.: Primary palliative care education: A pilot survey. Am J Hosp Palliat Care, 2018; 35:565–569.

Bickel-Swenson

: End-of-life training in U.S. Medical schools: A systematic literature review. J Palliat Med, 2007; 10:229–235.

DeCoste-Lopez

, Madhok

, Harman

: Curricular innovations for medical students in palliative and end-of-life care: A systematic review and assessment of study quality. J Palliat Med, 2015; 18:338–349.

Quill

, Abernethy

: Generalist plus specialist palliative care—Creating a more sustainable model. N Engl J Med, 2013; 368:1173–1175.

Fitzpatrick

, Heah

, Patten

, et al.: Palliative care in undergraduate medical education—How far have we come?. Am J Hosp Palliat Care, 2017; 34:762–773.

Kottewar

, Bearelly

, et al.: Residents' end-of-life training experience: A literature review of interventions. J Palliat Med, 2014; 17:725–732.

Lloyd-Williams

, Macleod

RDM

: A systematic review of teaching and learning in palliative care within the medical undergraduate curriculum. Med Teach, 2004; 26:683–690.

Donne

, Odrowaz

, Pike

, et al.: Teaching palliative care to health professional students: A systematic review and meta-analysis of randomized controlled trials. Am J Hosp Palliat Care, 2019; 36:1026–1041.

10.

Turrillas

, Teixeira

, Maddocks

: A systematic review of training in symptom management in palliative care within postgraduate medical curriculums. J Pain Symptom Manage, 2019; 57:156–170.

11.

Tricco

, Lillie

, Zarin

, et al.: PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med, 2018; 169:467–473.

12.

ACGME:. Hospice and Palliative Medicine Milestones. Chicago, IL: ACGME: The Accreditation Council for Graduate Medical Education, 2019.

13.

Thomas

, Kern

, Hughes

, et al.: Curriculum Development for Medical Education: A Six-Step Approach. Baltimore, MD: JHU Press, 2016.

14.

DeVader

, Jeanmonod

: The effect of education in hospice and palliative care on emergency medicine residents' knowledge and referral patterns. J Palliat Med, 2012; 15:510–515.

15.

Gisondi

, Lu

, Yen

, et al.: Adaptation of EPEC-EM curriculum in a residency with asynchronous learning. West J Emerg Med, 2010; 11:491–499.

16.

Goldonowicz

, Runyon

, Bullard

: Palliative care in the emergency department: An educational investigation and intervention. BMC Palliat Care, 2018; 17:43.

17.

Meo

, Hwang

, Morrison

: Resident perceptions of palliative care training in the emergency department. J Palliat Med, 2011; 14:548–555.

18.

Simmons

, Petrie

, Gracely

: Impact and reception of a required 4-week palliative medicine rotation. Fam Med, 2018; 50:287–290.

19.

Bradley

, Webb

, Schmitz

, et al.: Structured teaching versus experiential learning of palliative care for surgical residents. Am J Surg, 2010; 200:542–547.

20.

Klaristenfeld

, Harrington

, Miner

: Teaching palliative care and end-of-life issues: A core curriculum for surgical residents. Ann Surg Oncol, 2007; 14:1801–1806.

21.

Nakagawa

, Fischkoff

, Berlin

, et al.: Communication skills training for general surgery residents. J Surg Educ, 2019; 76:1223–1230.

22.

Pernar

, Peyre

, Smink

, et al.: Feasibility and impact of a case-based palliative care workshop for general surgery residents. J Am Coll Surg, 2012; 214:231–236.

23.

Raoof

, O'Neill

, Neumayer

, et al.: Prospective evaluation of surgical palliative care immersion training for general surgery residents. Am J Surg, 2017; 214:378–383.

24.

Schmitz

, Braman

, Turner

, et al.: Learning by (video) example: A randomized study of communication skills training for end-of-life and error disclosure family care conferences. Am J Surg, 2016; 212:996–1004.

25.

Alexander

, Keitz

, Sloane

, et al.: A controlled trial of a short course to improve residents' communication with patients at the end of life. Acad Med, 2006; 81:1008–1012.

26.

Buss

, Alexander

, Switzer

, et al.: Assessing competence of residents to discuss end-of-life issues. J Palliat Med, 2005; 8:363–371.

27.

, Hughes

, Weiss

, et al.: Evaluation of a Web-based palliative care pain management module for housestaff. J Pain Symptom Manage, 2008; 36:596–603.

28.

El-Sourady

, Chen

, Martin

, et al.: Effects of a primary palliative care educational system for teaching learners at different levels of training. Am J Hosp Palliat Care, 2019:36:675–681.

29.

Fischer

, Arnold

: Feasibility of a brief workshop on palliative care communication skills for medical interns. J Palliat Med, 2007; 10:19–23.

30.

Hallenbeck

, Bergen

: A medical resident inpatient hospice rotation: Experiences with dying and subsequent changes in attitudes and knowledge. J Palliat Med, 1999; 2:197–208.

31.

Han

, Keranen

, Lescisin

, et al.: The palliative care clinical evaluation exercise (CEX): An experience-based intervention for teaching end-of-life communication skills. Acad Med, 2005; 80:669–676.

32.

Karlen

, Cruz

, Leigh

: Resident-led palliative care education project. J Palliat Med, 2016; 19:428–436.

33.

Kerai

, Wheeler

: The forgotten educational needs of the house staff: Training internal medicine residents to address end-of-life issues. Omega, 2013; 67:147–153.

34.

Liao

, Amin

, Rucker

: An innovative, longitudinal program to teach residents about end-of-life care. Acad Med, 2004; 79:752–757.

35.

Mathew

, Jahnigen

, Scully

, et al.: Attitudes of house officers toward a hospice on a medical service. J Med Educ, 1983; 58:772–777.

36.

McFarland

, Rhoades

: End-of-life care: A retreat format for residents. J Palliat Med, 2006; 9:82–89.

37.

Miller

, Sullivan

, Soffler

, et al.: Teaching residents how to talk about death and dying: A mixed-methods analysis of barriers and randomized educational intervention. Am J Hosp Palliat Care, 2018; 35:1221–1226.

38.

Montagnini

, Varkey

, Duthie

Jr .: Palliative care education integrated into a geriatrics rotation for resident physicians. J Palliat Med, 2004; 7:652–659.

39.

Mullan

, Weissman

, Ambuel

, et al.: End-of-life care education in internal medicine residency programs: An interinstitutional study. J Palliat Med, 2002; 5:487–496.

40.

Okon

, Evans

, Gomez

, et al.: Palliative educational outcome with implementation of PEACE Tool integrated clinical pathway. J Palliat Med, 2004; 7:279–295.

41.

Olden

, Quill

, Bordley

, et al.: Evaluation of a required palliative care rotation for internal medicine residents. J Palliat Med, 2009; 12:150–154.

42.

Rhodes

, Tindall

, Xuan

, et al.: Communication about advance directives and end-of-life care options among internal medicine residents. Am J Hosp Palliat Care, 2015; 32:262–268.

43.

Fischer

, Gozansky

, Kutner

, et al.: Palliative care education: An intervention to improve medical residents' knowledge and attitudes. J Palliat Med, 2003; 6:391–399.

44.

Smith

, Hough

: Using death rounds to improve end-of-life education for internal medicine residents. J Palliat Med, 2011; 14:55–58.

45.

Smith

, O'Sullivan

, Lo

, et al.: An educational intervention to improve resident comfort with communication at the end of life. J Palliat Med, 2013; 16:54–59.

46.

Szmuilowicz

, el-Jawahri

, Chiappetta

, et al.: Improving residents' end-of-life communication skills with a short retreat: A randomized controlled trial. J Palliat Med, 2010; 13:439–452.

47.

Tung

, Wieland

, Verdoorn

, et al.: Improved resident physician confidence with advance care planning after an ambulatory clinic intervention. Am J Hosp Palliat Care, 2014; 31:275–280.

48.

Vergo

, Sachs

, MacMartin

, et al.: Acceptability and impact of a required palliative care rotation with prerotation and postrotation observed simulated clinical experience during internal medicine residency training on primary palliative communication skills. J Palliat Med, 2017; 20:542–547.

49.

von Gunten

, Twaddle

, Preodor

, et al.: Evidence of improved knowledge and skills after an elective rotation in a hospice and palliative care program for internal medicine residents. Am J Hosp Palliat Care, 2005; 22:195–203.

50.

Wayne

, Moazed

, Cohen

, et al.: Code status discussion skill retention in internal medicine residents: One-year follow-up. J Palliat Med, 2012; 15:1325–1328.

51.

, Kern

, Dy

, et al.: Narrative approach to goals of care discussions: A novel curriculum. J Pain Symptom Manage, 2019; 58:1033–1039.e1031.

52.

Yacht

, Suglia

, Orlander

: Evaluating an end-of-life curriculum in a medical residency program. Am J Hosp Palliat Care, 2006; 23:439–446.

53.

Schuh

, Biondo

, An

, et al.: Neurology resident learning in an end-of-life/palliative care course. J Palliat Med, 2007; 10:178–181.

54.

Miranda

, Schaefer

, Vates

, et al.: Palliative care and communication training in neurosurgery residency: Results of a trainee survey. J Surg Educ, 2019; 76:1691–1702.

55.

Margolis

, Blinderman

, de Meritens

, et al.: Educational intervention to improve code status discussion proficiency among obstetrics and gynecology residents. Am J Hosp Palliat Care, 2018; 35:724–730.

56.

Bagatell

, Meyer

, Herron

, et al.: When children die: A seminar series for pediatric residents. Pediatrics, 2002; 110(Pt 1):348–353.

57.

Balkin

, Ort

, Goldsby

, et al.: Pocket reference card improves pediatric resident comfort in caring for children at end of life. J Palliat Med, 2017; 20:409–414.

58.

Barnett

, Maurer

, Wood

: Pediatric palliative care pilot curriculum. Am J Hosp Palliat Care, 2016; 33:829–833.

59.

Flint

, Meyer

, Hossain

, et al.: Discussing serious news. Am J Hosp Palliat Care, 2017; 34:254–257.

60.

Kolarik

, Walker

, Arnold

: Pediatric resident education in palliative care: A needs assessment. Pediatrics, 2006; 117:1949–1954.

61.

McCabe

, Hunt

, Serwint

: Pediatric residents' clinical and educational experiences with end-of-life care. Pediatrics, 2008; 121:e731–e737.

62.

Michelson

, Ryan

, Jovanovic

, et al.: Pediatric residents' and fellows' perspectives on palliative care education. J Palliat Med, 2009; 12:451–457.

63.

Ross

, Doshi

, Carrasca

, et al.: Interactive palliative and end-of-life care modules for pediatric residents. Int J Pediatr, 2017; 2017:7568091.

64.

Rossfeld

, Tumin

, Humphrey

: Self-assessment of skills and competencies among residents participating in a pediatric hospice and palliative medicine elective rotation. J Palliat Med, 2018; 21:229–235.

65.

Schiffman

, Chamberlain

, Palmer

, et al.: Introduction of a pediatric palliative care curriculum for pediatric residents. J Palliat Med, 2008; 11:164–170.

66.

Weaver

, Wichman

: Implementation of a competency-based, interdisciplinary pediatric palliative care curriculum using content and format preferred by pediatric residents. Children (Basel), 2018; 5:156.

67.

Wilson

, Herbst

, Gonzalez-Del-Rey

: Development and implementation of an end-of-life curriculum for pediatric residents. Am J Hosp Palliat Care, 2018; 35:1439–1445.

68.

Yazdani

, Evan

, Roubinov

, et al.: A longitudinal method of teaching pediatric palliative care to interns: Preliminary findings regarding changes in interns' comfort level. Palliat Support Care, 2010; 8:35–40.

69.

Irwin

, Montross

, Bhat

, et al.: Psychiatry resident education in palliative care: Opportunities, desired training, and outcomes of a targeted educational intervention. Psychosomatics, 2011; 52:530–536.

70.

Garcia

, Braunstein

, Anderson

: Palliative care didactic course for radiation oncology residents. Int J Radiat Oncol Biol Phys, 2017; 97:884–885.

71.

Harrison

, Boyden

, Kalish

, et al.: A hospice rotation for military medical residents: A mixed methods, multi-perspective program evaluation. J Palliat Med, 2016; 19:542–548.

72.

Lester

, Daroowalla

, Harisingani

, et al.: Evaluation of housestaff knowledge and perception of competence in palliative symptom management. J Palliat Med, 2011; 14:139–145.

73.

Patell

, Gutierrez

, Lee

, et al.: Practicing communication skills for responding to emotionally charged questions. J Palliat Care, 2018; 33:209–214.

74.

Seoane

, Bourgeois

, Blais

, et al.: Teaching palliative care in the intensive care unit: How to break the news. Ochsner J, 2012; 12:312–317.

75.

von Gunten

, Mullan

, Harrity

, et al.: Residents from five training programs report improvements in knowledge, attitudes and skills after a rotation with a hospice program. J Cancer Educ, 2003; 18:68–72.

76.

von Gunten

, Mullan

, Nelesen

, et al.: Primary care residents improve knowledge, skills, attitudes, and practice after a clinical curriculum with a hospice. Am J Hosp Palliat Care, 2017; 34:713–720.

77.

Baker

, Torkildson

, Baillargeon

, et al.: National survey of pediatric residency program directors and residents regarding education in palliative medicine and end-of-life care. J Palliat Med, 2007; 10:420–429.

78.

Berlacher

, Arnold

, Reitschuler-Cross

, et al.: The impact of communication skills training on cardiology fellows' and attending physicians' perceived comfort with difficult conversations. J Palliat Med, 2017; 20:767–769.

79.

DeVita

, Arnold

, Barnard

: Teaching palliative care to critical care medicine trainees. Crit Care Med, 2003; 31:1257–1262.

80.

Hope

, Hsieh

, Howes

, et al.: Let's talk critical. Development and evaluation of a communication skills training program for critical care fellows. Ann Am Thorac Soc, 2015; 12:505–511.

81.

Cao

, Lee

, Hayes

, et al.: Are geriatric medicine fellows prepared for the important skills of hospice and palliative care?. Am J Hosp Palliat Care, 2015; 32:322–328.

82.

Pan

, Carmody

, Leipzig

, et al.: There is hope for the future: National survey results reveal that geriatric medicine fellows are well-educated in end-of-life care. J Am Geriatr Soc, 2005; 53:705–710.

83.

Urban

, Fay

, Podgurski

, et al.: Pilot study of a condensed communication skills workshop for gynecologic oncology fellows. Gynecol Oncol Rep, 2019; 30:100492.

84.

Harris

, Placencia

, Arnold

, et al.: A structured end-of-life curriculum for neonatal-perinatal postdoctoral fellows. Am J Hosp Palliat Care, 2015; 32:253–261.

85.

Schell

, Green

, Tulsky

, et al.: Communication skills training for dialysis decision-making and end-of-life care in nephrology. Clin J Am Soc Nephrol, 2013; 8:675–680.

86.

Schell

, Cohen

, Green

, et al.: NephroTalk: Evaluation of a palliative care communication curriculum for nephrology fellows. J Pain Sympt Manage, 2018; 56:767–767.

87.

Shah

, Monga

, Caperna

, et al.: Palliative care experience of US adult nephrology fellows: A national survey. Ren Fail, 2014; 36:39–45.

88.

Back

, Arnold

, Baile

, et al.: Efficacy of communication skills training for giving bad news and discussing transitions to palliative care. Arch Intern Med, 2007; 167:453–460.

89.

Buss

, Lessen

, Sullivan

, et al.: Hematology/oncology fellows' training in palliative care: Results of a national survey. Cancer, 2011; 117:4304–4311.

90.

Lenzi

, Baile

, Berek

, et al.: Design, conduct and evaluation of a communication course for oncology fellows. J Cancer Educ, 2005; 20:143–149.

91.

Reddy

, Tanco

, Yennu

, et al.: Integration of a mandatory palliative care education into hematology-oncology fellowship training in a comprehensive cancer center: A survey of hematology oncology fellows. J Oncol Pract, 2019; 15:e934–e941.

92.

Baughcum

, Gerhardt

, Young-Saleme

, et al.: Evaluation of a pediatric palliative care educational workshop for oncology fellows. Pediatr Blood Cancer, 2007; 49:154–159.

93.

Brock

, Cohen

, Sourkes

, et al.: Training pediatric fellows in palliative care: A pilot comparison of simulation training and didactic education. J Palliat Med, 2017; 20:1074–1084.

94.

Gerhardt

, Grollman

, Baughcum

, et al.: Longitudinal evaluation of a pediatric palliative care educational workshop for oncology fellows. J Palliat Med, 2009; 12:323–328.

95.

Johnson

, Hamilton

, Watson

, et al.: An intensive, simulation-based communication course for pediatric critical care medicine fellows. Pediatr Crit Care Med, 2017; 18:e348–e355.

96.

Arnold

, Back

, Barnato

, et al.: The critical care communication project: improving fellows' communication skills. J Crit Care, 2015; 30:250–254.

97.

McCallister

, Gustin

, Wells-Di Gregorio

, et al.: Communication skills training curriculum for pulmonary and critical care fellows. Ann Am Thorac Soc, 2015; 12:520–525.

98.

Amini

, Miura

, Larrieux

, et al.: Palliative care training in surgical oncology and hepatobiliary fellowships: A national survey of the fellows. Ann Surg Oncol, 2015; 22:1761–1767.

99.

Makoul

: Communication skills education in medical school and beyond. JAMA, 2003; 289:93–93.

100.

Selman

, Brighton

, Hawkins

, et al.: The effect of communication skills training for generalist palliative care providers on patient-reported outcomes and clinician behaviors: A systematic review and meta-analysis. J Pain Symptom Manage, 2017; 54:404–416.e405.

101.

Weissman

, Meier

: Identifying patients in need of a palliative care assessment in the hospital setting a consensus report from the Center to Advance Palliative Care. J Palliat Med, 2011; 14:17–23.

102.

Bernacki

, Block

: Communication about serious illness care goals: A review and synthesis of best practices. JAMA Intern Med, 2014; 174:1994–2003.

103.

Piscitello

, Martin

: Spirituality, religion, and medicine education for internal medicine residents. Am J Hosp Palliat Care, 2019:37:272–277.

104.

Kirkpatrick

, Kirkpatrick

: Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler Publishers, 2006.

105.

Yardley

, Dornan

: Kirkpatrick's levels and education ‘evidence’. Med Educ, 2012; 46:97–106.

106.

Moher

, Liberati

, Tetzlaff

, et al.: Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA statement. PLoS Med, 2009; 6:e1000097.