Comparison of International Medical Graduates with US Medical Students and Residents after a Four-Week Course in Palliative Medicine: A Pilot Study

Abstract

Background:

The need for doctors who have skills in pain management and palliative medicine is greatest in low and moderate resource countries where patients most frequently present to their health care system with advanced illness and greater than 80% of the global deaths occur. While medical students trained in the United States are required to have training in palliative medicine, international medical graduates (IMGs), who have completed medical school outside North America, may not have the same exposure to palliative medicine training as U.S. physicians. The goal of this study was to evaluate whether a four-week course in palliative medicine could bring IMG attitudes, concerns, competence, and knowledge to the level of U.S. trainees.

Methods:

As part of a prospective cohort study, 21 IMGs from 14 countries participated in a four-week course in palliative medicine. Attitudes, concerns, self-reported competence, and knowledge were assessed pre-course and post-course. The course was evaluated weekly and at the end of the four-week program. The data from the IMGs was compared to data from U.S. medical students and residents using the same assessment tools.

Results:

After the course, IMGs had significantly decreased concern about ethical and legal issues in palliative medicine to the level of U.S.-trained residents and a significant increase in knowledge and self-rated competence equivalent to the level of U.S. trainees.

Conclusions:

A four-week course in palliative medicine can improve the levels of concern, knowledge and self-assessed competence in IMGs to the level of US trainees.

Introduction

Training for physicians in the United States in palliative medicine and end-of-life care has historically been inadequate.^1–4 To ameliorate this issue, some states, like California, require training in palliative and end-of-life care for licensing.⁵ Knowledge can be improved through palliative medicine courses and rotations; for example a structured curriculum during a clinical rotation led to a significant improvement in medical students' and residents' knowledge and competence, and a significant decrease in concern related to palliative and end-of-life care issues.^6,7

A significant number of international medical graduates (IMGs), who have completed medical school outside North America, are training and practicing in the United States. In 2012, 4898 (20.4%) of 24,034 postgraduate year-one (PGY-1) residency positions in the National Resident Matching Program were filled by IMG physicians (Canadian, 5th Pathway, U.S. and non-U.S. IMGs).⁸ Typically, palliative medicine is not part of the curricula in foreign undergraduate or graduate medical training. Consequently, IMGs do not have the same exposure as U.S. medical students, who are required to include education in palliative care and end-of-life care as part of their curriculum.⁹ Our preliminary studies indicated that IMGs have lower scores in baseline attitudes, knowledge, and skills than U.S. physicians at comparable levels of training.¹⁰

As a pilot education project, the International Visiting Scholars Program (IVSP) aimed to determine whether a structured curriculum during a four-week course in palliative medicine would lead to improvements in attitudes, knowledge, and self-reported competence in IMGs similar to the improvements seen in U.S.-trained physicians.

Methods

Selection of participants

Potential participants for the IVSP were recruited from low and moderate income countries (World Bank classification)¹¹ through recommendations of our international partners based on the criteria listed in Table 1. Nominees who indicated that they were prepared to undertake this four-week course completed an online application and participated in a 30–60 minute telephone interview to review the details of the IVSP and assess their English skills. Finally, 21 participants were selected from 14 countries: Argentina, Brazil (2), Colombia, Egypt (2), Georgia (2), Guatemala, Jordan, Lebanon, the Palestinian Authority (2), Peru, Serbia, Turkey, Uganda (2), and Ukraine (3).

Table 1.

Inclusion and Exclusion Criteria

Inclusion criteria	Exclusion criteria
• Early-to midcareer IMGs from low to moderate income countries (World Bank classification) who expressed an interest in palliative medicine, but not necessarily a career commitment to the field	• High income countries (World Bank classification)
• Recommendation of the home organization, permission to be away for five weeks, and commitment to attend all four weeks of the IVSP	• Lack of support from their home organization
• Commitment to share new knowledge and skills with colleagues upon return home	• Application of poor quality or incomplete
• Sufficient fluency in medical and common English to fully participate in the IVSP	• Poor English skills • Unable to attend all activities of the IVSP

IMG, international medical graduates; IVSP, International Visiting Scholars Program.

Curriculum

The IVSP classroom and bedside curriculum was based on the resident physician curriculum developed in our institution. The classroom-based teaching was based on the modules in the Education on Palliative and End-of-Life Care for Oncology (EPEC-Oncology) Curriculum.¹²

The IVSP aimed to develop

1. Basic clinical palliative care knowledge and skills, including

a. Assessment of symptoms and disease processes

b. Pharmacology

c. Medical and nonpharmacologic management

d. Communication skills

2. Teaching and presentation skills; how to give effective feedback

3. An overview of a population-based model of palliative care, including

a. Conceptual framework

b. Bedside experience in multiple settings of patient care

c. Role and function of the interdisciplinary team

d. Planning, goals for implementation

4. How to set effective personal goals

5. Self-care, burnout avoidance, and reflective practice

Four cohorts

Study participants were invited in four cohorts of four to six people in each cohort. Cohorts were formed based on participant availability to travel and institutional capacity. During their four-week course, each participant spent one week in the classroom reviewing a structured curriculum based on the EPEC-Oncology Curriculum. Participants then spent three weeks doing bedside training in a number of clinical settings, including

• one week in acute inpatient palliative medicine in a 24-bed inpatient care center

• one week in home care

• one week in a palliative medicine consult service at an acute care hospital

• other clinical experiences as requested by the scholar (e.g., pediatric homecare)

Daily afternoon sessions were held to discuss the day's activities and provide further education based on their clinical experiences.

Evaluation

Prior to starting the IVSP, participants provided written consent to include their evaluation data in the anonymous collated data set. Participants completed online evaluations before and after the course measuring their attitudes, concerns, knowledge, and self-rated competence (see Table 2a). Each week and at the end of the program they completed evaluations about the educational interventions and the program as a whole (see Table 2b). The evaluation protocol used to evaluate the International Visiting Scholar Program was reviewed and deemed “exempt” by the Institutional Review Board at San Diego Hospice and The Institute for Palliative Medicine.

Table 2a.

Quantitative Evaluations

Tool	Number of questions	Content
Attitudes Survey	14 (Likert scale)	Personal reaction to death/dying, opiophobia, truth-telling/patient autonomy, suffering, beliefs about symptom management
		Example: I worry that patients who receive morphine die faster.
Concerns Survey	20 (Likert scale)	Legal, ethical, and malpractice concerns about: Pain relief, withdrawal of non-oral feedings, intravenous fluids, parenteral antibiotics, ventilatory support
		Example: Withdrawing IV hydration from a terminal cancer patient, who can no longer take oral fluid and who is clearly dying violates a) my country's law, b) medical practice standards and represents malpractice, c) accepted ethical norms, d) my personal religious or ethical beliefs
Knowledge Survey	34 (Multiple choice)	Pain and other symptom control, addiction, communication with patients and families, informed consent, prognosis, physician-assisted suicide, death pronouncement, grief
		Example case: A 67 y/o woman with pancreatic cancer metastatic to liver comes to your clinic together with her husband. Over the past 4 weeks she has lost her appetite and experienced steady weight loss. She spends more than 75% of the day in bed or lying on a couch because of fatigue. Her oncologist has indicated that there is no role for further chemotherapy.
Competence Survey	26 (Likert scale)	Communication with patients and families, prognostication, pain and other symptom assessment and management, whole person assessment, interdisciplinary teamwork, determining capacity, death pronouncement and certification
		Example: Assess and manage nausea and vomiting

Table 2b.

Qualitative Evaluations

Tool	Number of free-text questions	Content
Week 1 evaluation (classroom)	4	What was best, most surprising, most challenging?
		What should we change?
Weeks 2–4 evaluation (clinical experience)	4	What was best, most surprising, most challenging?
		What should we change?
		What was most & least helpful about the clinical site?
		What was the most helpful thing about the team or individuals?
		What could the team or individuals change to make this experience better?
Program evaluation	10	What was the best part of this experience, most challenging, most enjoyable or joyful experience, most memorable?
		What should we keep; what would you like to see improved? Why?
		What are you likely to do differently in your clinical practice?
		What would you tell others about this?

After the IVSP was complete, the data were compared with data collected from U.S. medical students and residents who used the same evaluation tools. The same tools were used to assess the concerns, knowledge, and competence of the IMGs, medical students, and residents. These tools were initially developed by Dr. David Weissman. The attitudes and course evaluation tools were developed at San Diego Hospice and The Institute for Palliative Medicine.

Data analysis

Attitudes, concerns, and knowledge were analyzed using the Wilcoxon Signed-Rank test to compare data provided post-course with the data obtained pre-course. A “repeated measures with grouping factor” analysis was performed to evaluate the individual knowledge score by group: IMGs, residents, and medical students trained in our institution.

Comments provided during the weekly and overall program evaluations were reviewed by the research team to identify common themes. The identified themes were reviewed by IVSP faculty to ensure consensus.

Results

Participants were surveyed in four areas: attitudes, concerns, knowledge, and self-reported competence. While attitudes did not change significantly after the course, there was a significant decrease in concern and a significant increase in both knowledge and self-perceived competence.

Attitudes

In the Attitudes Survey, participants ranked 14 statements about their attitudes towards palliative medicine. Twenty pre-course and twenty-one post-course responses were received. The course did not produce any statistically significant changes in the attitudes surveyed.

Concerns

The Concern Survey asked participants to indicate their concerns about 20 ethical and legal issues related to palliative medicine. Results showed that the participants' concern decreased significantly after training (Wilcoxon Signed-Rank Test rank difference=3, Z=3.215, p=.001; see Figure 1).

FIG. 1.

Trainee concern scores. Scale: 1=Not Concerned, 2=Somewhat Unconcerned, 3=Somewhat Concerned, 4=Very Concerned.

When comparing the concern scores of the IMGs, medical students, and residents before and after training, there were significant differences over time and between trainee groups, and significant training group × time interactions (F₂₃₉₁=13.25, p<.001). Before training, medical students had a significantly higher level of concern than IMGs (p=.027), and the IMGs were significantly more concerned about these issues than the residents (p<.001). All groups showed a significant decrease in concern scores (p<.001) after their respective educational interventions. Medical students (−0.6) and IMGs (−0.7) showed similar and significantly larger decreases (p<.001) in concern than residents (−0.22). After training, IMGs had the same concern scores as the residents despite having started with higher concern scores (see Figure 1).

Knowledge

The Knowledge Survey asked participants to answer 34 multiple-choice questions on a series of scenarios related to palliative medicine. Results showed that knowledge of the IMGs increased significantly after training (z=3.21, p<.001) (see Figure 2).

FIG. 2.

Trainee knowledge scores.

When the pre- and post-course knowledge scores of the IMGs, medical students, and residents were compared, there were significant differences post-course and by trainee group, and significant training group x time interactions (F₂₃₉₀=5.856, p=.003). Prior to the educational interventions, the medical students and IMGs were not significantly different in their knowledge scores, but both had significantly lower scores than the residents (p<.001). All groups improved their knowledge scores after training (p<.001). While the medical students and the scholars improved their scores to the same level, the residents improved their scores to a significantly higher level (p<.001) (see Figure 2).

Competence

The Competence Survey asked participants to self-report in their competence to perform 26 different patient/family interactions and management activities that are important skills in palliative medicine. Comparison was made between the self-reported competence score pre- and post-course of the IMGs, medical students, and residents. There was no significant group effect or interaction (p>.147), meaning that the change in perceived competence did not change based on the group to which an individual belonged. However, self-rated competence increased significantly in all groups after completing training (F₂₃₈₉=180.34, p<.001) (see Figure 3).

FIG. 3.

Trainee competence scores. Scale: 1=Need Further Basic Instruction, 2=Perform with Close Supervision, 3=Perform with Minimal Supervision, 4=Perform Independently.

Qualitative data

To assess their progress and challenges, as well as to improve the IVSP educational interventions both in the classroom and at the bedside, IMGs evaluated the course weekly and at the end of the four-week program.

During the first week of didactic and experiential learning, many of the comments focused on the interactive teaching style and components of the curriculum, like the simulated patient experience to teach communication skills. IMGs were surprised by the supportive learning environment, the inclusion of nonpharmacologic therapies like integrative medicine, as well as techniques for self-care. They found the interactive sessions, like presentation skills, to be challenging. This may be due to performance anxiety, especially because of concern over their English ability. It is also possible that the novel educational techniques proved to be challenging.

Weeks two to four included clinical rotations with bedside training and opportunities to review, discuss, and learn from the cases IMGs had seen each day. Feedback was focused on interactions with physician faculty, interdisciplinary team members, and patients, as well as how the interdisciplinary teams worked together. IMGs found that adapting to the U.S. medical system was challenging. They also noted that witnessing the suffering of the patients and families was difficult.

The final program evaluations echoed the assessments of the prior weeks, focusing on patient, faculty, and team interactions. The IMGs discussed the importance of communication, and reported that changes in their approach to communication with patients as one way they would improve their practice. As has been previously reported to us by other international trainees, the IMGs were challenged with the language and culture. The overall comments reflected a sense of personal and professional development, appreciation of the supportive environment, a sense of community and group vision, and a sense of being valued as individuals and for their clinical role and expertise.

“It is an excellent program designed to provide clinical care, communication, teaching, and self-care skills to leaders in other countries that are already working in the field. This period of education, as well as the full observation of a population-based model in palliative care, improve your status as a physician in Palliative Care in our own countries, and help us to expand Palliative Care.”

“I will say, That it was a great experience, full of inspiration. That this program makes you able to see the best part of the end of life and how you can contribute to that.”

Discussion

The results of the quantitative and qualitative evaluations suggest that the IVSP was both worthwhile and confirmed our hypothesis: This four-week course in palliative medicine demonstrated statistically significant improvements in IMGs' knowledge and self-reported competence, and reduction of concerns similar to the improvements seen in U.S.-trained physicians.

Attitudes

There was no change in the attitudes surveyed, possibly because the IMGs were selected for their interest in palliative medicine. A group of not-so-interested physicians might have a larger change in their attitudes.

Concerns

The IMGs had a statistically significant decrease in their level of concern, indicating they became more comfortable with palliative medicine issues. Before the course, the medical students were significantly more concerned than the IMGs, and IMGs were significantly more concerned than the residents. This pre-course difference may reflect the level of training of the three groups, exposure to palliative medicine, or more clarity about law and medical practice by U.S. residents. When teaching palliative care internationally, our faculty has found that laws are often unclear or nonexistent, and frequently very different from what the international medical professionals believe and use to guide their medical practice.

After training, the medical students and IMGs showed significantly larger decreases in concern scores than the residents, likely due to their significantly higher level of concern pre-course. Notably, the IMGs' post-course level of concern was the same as the U.S. residents, indicating that the course helped to resolve concern that may have come from lack of exposure to palliative medicine training and/or the U.S. medical system. These results indicate that a didactic and bedside training curriculum in palliative medicine can decrease physicians' concern surrounding palliative and end-of-life issues, in U.S. medical students and residents as well as in IMGs, with greatest change in the medical students and IMGs.

Knowledge

As anticipated, the IMGs' knowledge scores pre-course were the same as the U.S. medical students' (19.55, 19.03, respectively); both were significantly lower than the U.S. residents' pre-course scores. After the educational intervention, all three groups showed a statistically significant improvement in their knowledge scores. While the IMGs' scores improved more than the residents' scores (24.5% change versus 15.5%, p=0.02), the IMGs and medical students improved to the same level (mean scores of 23.14 and 23.58, respectively), equivalent to the residents' pre-course scores (22.99), but still lower than the residents' post-course scores (25.96).

The participants may not have gained as much knowledge as the residents because of differences in their medical training in countries outside the United States and the reality that English is not their first language. Of note, the 34-item knowledge assessment only examines selected content covered during the course. IMGs, medical students, and residents may have improved in other content areas covered during their respective courses but not included in the knowledge assessment, such as communication skills, interdisciplinary teamwork, presentation and facilitation skills, approaches to effective feedback, goal setting, self-care, and burnout avoidance.

Competence

The IMGs' pre-course self-reported competence scores and their retrospective reassessment of their pre-course competence were not significantly different. After the course, the three groups of trainees rated themselves as significantly more competent. There was no difference in the self-rated competence scores among the three groups. This suggests that didactic and bedside training in palliative medicine improves self-rated competence for all physician trainees.

Qualitative data

The week one, weeks two to four, and overall program evaluations helped the IVSP faculty understand the scholars' experience of the four-week course. Each cohort's evaluations were reviewed in aggregate and the curriculum was adjusted to improve the experience of each subsequent cohort. As a result of this feedback, the groupwork outside the classroom and the number of case examples were increased.

In follow-up, 8 of the 21 participants expressed interest in pursuing specialist training in palliative medicine through our International Palliative Medicine Fellowship program. Currently five of the eight participants have commenced training in this program. Additionally, two of the eight have opted to pursue leadership training in palliative medicine through our Leadership Development Initiative.

Limitations

For all of the participants, this course was their first educational opportunity in North America. From observing many international visitors, fellows, and leaders at our institution, the learning that occurs in the international trainees occurs at many different levels, with much of the initial learning taking place where the differences are most striking between the United States and their home countries. Many have commented on the vast differences in culture, language, and the health care systems, as well as the population-based model of palliative medicine. They are challenged during their first visit to simply get oriented to the significant differences between their home countries and the United States. When trainees return for follow-up training visits, their mastery of clinical concepts is much greater than during their first training visit as they become familiar with U.S. health care, U.S. culture, and medical English. Given this reality, it is impressive that the IMGs improved their knowledge and self-rated competence as much as they did during this first four-week course in the United States.

Footnotes

Acknowledgments

The authors wish to acknowledge the confidence and support provided to us by Joe Harford, Center for Global Health, National Cancer Institute, Bethesda, Maryland; the faculty at San Diego Hospice and The Institute for Palliative Medicine who helped train and support the International Visiting Scholars; and the staff of our International Programs who helped make the IVSP successful, especially Paula Brown, Melissa Poole, and Jane Trevelyan.

Author Disclosure Statement

Support for this study was provided by the National Cancer Institute through the administrative supplement to 5R25 CA104990. All authors report that no competing financial interests exist.

References

Billings

, Block

. Palliative care in undergraduate medical education. JAMA, 1997; 278:733–738.

Darer

, Hwang

, Pham

, Bass

, Anderson

. More training needed in chronic care: a survey of U.S. physicians. Acad Med, 2004; 79:541–548.

Fraser

, Kutner

, Pfeifer

. Sewor medical students' perceptions of the adequacy of education on end-of-life issues. J Palliat Med, 2001; 4,3:337–343.

Ogle

, McElroy

, Mavis

. No relief in sight: postgraduate training in pain medicine. Am J Hosp Palliat Care, 2008; 25,4:292–297.

California State Assembly 2001–2002 Regular Session: Bill 487. California Statute ch. 518. 2001. www.leginfo.ca.gov/pub/01-02/bill/asm/ab_0451-0500/ab_487_bill_20010531_amended_asm.pdf

von Gunten

, Mullan

, Nelesen

, Soskins

, Savoia

, Bucholz

, Weissman

. Development and evaluation of a palliative medicine curriculum for third-year medical students. J Palliat Med, 2012; 15,11:1198–1217.

von Gunten

, Mullan

, Harrity

, Diamant

, Heffernan

, Ikeda

, Roberts

. Residents from 5 training programs report improvements in knowledge, attitudes and skills after a rotation with a hospice program. J Cancer Ed, 2003; 18:68–72.

National Resident Matching Program. Results and Data: 2012 Main Residency Match, 2012. www.nrmp.org/data/resultsanddata2012.pdf. 2013 January 31.

Liaison Committee on Medical Education. Functions and Structure of a Medical School Standards for Accreditation of Medical Education Program Leading to the M.D. Degree May 2011. (For schools with full accreditation surveys in 2012–2013). http://www.lcme.org/functions2011may.pdf. 2013 January 31.

10.

Ferris

, Stjernswärd

, Wheeler

, Foley

, Callaway

. Assessing the effectiveness of international palliative care education interventions using standardized competence, knowledge evaluations. European Association for Palliative Care (EAPC) 5th Research Forum, Trondheim, Norway. European J Palliat Care, 2008; 15,3.

11.

The World Bank Country and Lending Groups: data. http://data.worldbank.org/about/country-classifications/country-and-lending-groups. 2013 January 31.

12.

Emanuel

, Ferris

, von Gunten

, Von Roenn

. EPEC-O: Education in Palliative and End-of-life Care for Oncology. Chicago, IL: The EPEC Project, 2005. www.ipcrc.net/who-education-epec-oncology.php. 2013 January 31.