Does Interdisciplinary and Multiprofessional Undergraduate Education Increase Students' Self-Confidence and Knowledge Toward Palliative Care? Evaluation of an Undergraduate Curriculum Design for Palliative Care at a German Academic Hospital

The Mainz teaching course

The study was performed at the University of Mainz, Germany. Here we report data from the first two terms introducing the subject of PC into UPCE in their fifth year of a six-year curriculum. All students were informed about the purpose of the questionnaires as a curriculum-evaluation project. According to local ethics standards, no formal ethical approval was requested.

The course started with an introductory lecture for all students embracing the “total pain” concept. Hospice home care was also demonstrated by a live interview of bereaved family members, a PC nurse, and a physician. For the following units, students were randomly assigned to eight groups, with a maximum of 25 students each. Each learning group passed five domains of PC: pain, symptom control, ethics, spirituality/grief, and breaking bad news. Two units were reserved for the latter with the intent of improving communication skills, and students were divided in small groups to provide safe environments as recommended by Macauley. ¹³ At the end of the term, students' knowledge was assessed using a tool of multiple choice items presented as an electronic exam (see Fig. 1).

OPEN IN VIEWER

FIG. 1.

Overview of course structure and points of measurement.

Study participants

Participants of this study were 5Y medical students of the first (Winter 2011/2012) or second (Summer 2012) term, when the mandatory PC course was taught.

Evaluation of students' self-confidence and self-assessment of knowledge

We used a pre-post survey design to evaluate students' attitude toward PC and moreover their estimation of self-confidence in various PC issues before and after the course. Therefore we adapted a questionnaire which we already successfully introduced in a previous study. ¹⁴

This questionnaire, which was originally developed by an interdisciplinary panel of experts in PC, ¹⁵ inquires about student's self-assessment of knowledge regarding palliative medicine and student's self-confidence in handling terminally ill and dying patients. Answers range from 1=very low or very unsure to 10=very high or very confident. Thereafter students are asked to estimate their self-confidence with 10 relevant aspects of PC: assessment and examination of patients, basic principles and contents of PC, integration of psychological aspects into treatment, medical treatment of cancer pain, integration of spiritual aspects into treatment, treatment of symptoms, communication, disclosure of palliative situation, explaining the change of therapeutic aim from tumor-specific treatment to PC, and treatment and accompaniment of terminally ill and dying patients, using a four-step Likert scale: confident/rather confident or not confident/rather not confident. Students are also asked about their general interest in PC, their perception of satisfaction working as a PC physician, and also their perceived reasonability of implementing the interdisciplinary subject of PC on a numeric 10-step rating scale.

The postquestionnaire is equivalent to the prequestionnaire but additionally asks for global assessment of the subject of PC and also to evaluate the provided syllabus. Students were also asked to fill in free-text comments to complement quantitative by qualitative data. Records of students at the two investigated points were matched using pseudonyms provided by the students themselves.

Evaluation of students' knowledge

Provided with intensive teaching of PC in small groups during the course and a detailed script, students had to pass an electronic exam at the end of the term. These 21-item single best answer of multiple choice questions referring to eight case examples were already in use for knowledge assessment for 5Y medical students without a PC course in Mainz in a previous study. ¹⁴ The exam, based on the validated tool of the National Residency End-of-Life Education Project by Weissmann,^16–18 was adjusted to local conditions, i.e. German health policy, health care organization, and medication labels.

Statistical methods

The results of the pre-post changes are presented as frequency tables. Self-assessment of confidence was at four levels (4=confident, 3=rather confident, 2=rather not confident, 1=not confident). Scores were obtained by adding the outcomes for all questions relating to one particular dimension. Somatic score was obtained from questions 1, 4, and 6; psychological score was obtained from questions 3, 7, 8, 9, and 10; knowledge score was assessed by question 2; and spiritual score consisted of question 5 only. Items not available were set to 2.5 for obtaining scores. For ease of presentation, scores were also dichotomized. Students were deemed confident in one dimension if the related score was greater than (maximum possible score+minimum possible score)/2. Otherwise they were deemed not confident in this dimension. Scores for the examination of students' knowledge were calculated as presented in Tables 3 and 4.

Data was analyzed using SPSS 20.0 (IBM, Armonk, NY) and statistical software SAS (SAS version 9.4; SAS Institute Inc., Cary, NC). The questionnaires were summarized by giving absolute and relative frequencies for answers to single items. Scores derived from more than one single item were summarized by giving minimum, maximum, median, and quartiles.

Change in self-assessed confidence was evaluated using the sign test for single items and for scores derived from more than one single item using the Wilcoxon signed-rank test. In order to assess the possibility of bias introduced by restricting the main analysis to those students with two matching questionnaires, the outcomes for the pre-course and postcourse surveys are compared between those students with only one questionnaire and those with two matching questionnaires, using the Wilcoxon Mann-Whitney U tests.

Self-confidence

Overall, confidence as measured by the total score was considerably higher after the course than before the course among the students who provided two questionnaires that could be matched (see Table 1). While only 9 (6%) of the students felt confident before the course, 107 (69%) felt confident after the course. This is reflected in the median total score of 20 (IQR 18–23) before the course and 27 (IQR 25–30) after the course; the median change in total score was 8 (IQR 4–10) (p<0.0001). The outcome of the dichotomized scores is shown in Table 2.

In all aspects of PC a large proportion of those students who supplied two matching forms felt more confident after the course than before (p<0.0001). The largest improvement in confidence concerned knowledge, with 126 (81%) feeling more confident than before. The smallest number of students with increased confidence was observed for communication; in this domain, even 14 (9%) felt less confident than before the course. A complete overview of changes is given in Table 2.

No pertinent differences were found when comparing the answers to the questions relevant to self-efficacy given by students who supplied matchable questionnaires to those who supplied unmatchable questionnaires. However, there was some indication that students whose prequestionnaire could not be matched to a postquestionnaire were more confident before the course with respect to treatment of pain (p=0.0146) and symptoms (p=0.0081). In contrast, postcourse survey analysis compared to matched survey data showed less confidence after the course with respect to knowledge (p=0.0188).

Knowledge

All students (n=329) passed the exam with average scores greater than 90% in all core dimensions (see Table 3). One question was answered poorly when students were asked to diagnose increased pain due to progressive disease (bone metastases) rather than habituation or addiction: only 46.5 % managed to differentiate (see Table 4).

Increased self-estimated confidence from PC training

In a historic cohort of 5Y medical students without PC training in Mainz in 2011, a lack of self-confidence was found ¹⁴ compared to only 6% of students of the present cohort, who felt confident at the beginning of the course. Overall, after the course, 69% of the students felt confident with their proficiency in caring for dying patients.

Our findings resemble results of a recent study evaluating a nonmandatory German undergraduate PC training at a private medical school as a complex intervention ¹⁰ based on the Harvard Program in Palliative Care Education and Practice (HMS PCEP). ¹⁹ This program of Schulz et al. refers to Bandura's idea of self-efficacy just as our course does. Although methodologically challenging, they found increased self-efficacy in communication, knowledge, and skills. ¹⁰

The self-confidence of medical students in communication is of special interest, as Schaefer reported communication to be one of the most important competences in PC assigned by 68 PC experts. ²⁰ This outcome acknowledges the fact that effective physician-patient communication improves patient health outcomes and may be considered as “central clinical function.”^21,22 Strikingly, in our cohort, communication was the topic with the smallest increase in self-efficacy. Moreover, even 14 students felt less confident in their communicative abilities after the course. Hypothesized that students before the course felt that training on communication is not necessary in medical education, it might be that they changed their attitude during the course while learning more about breaking bad news and reflecting on their initial self-confidence.

Additionally, the current training framework may not be effective for all students due to differences in personality traits, which were not assessed. Training in communication requires repetition to enhance self-confidence with breaking bad news. The Mainz medical curriculum includes training in communication only twice, of which UPCE is the second. Moreover, in our course, an active role-play is only warranted for about 25% of the students.

Finally, the postcourse survey did not assess students' experiences of active role-plays and feedback groups in the unit on communication. Changes in self-reported measures might reflect students' positive experiences during a role-play, or these changes might reflect a decrease in self-efficacy while watching an excellent simulation of doctor-patient interaction. ¹⁹ Nevertheless, 44% were found to feel more self-confident in communication after the course. Since patients with life-threatening illnesses frequently suffer from psychological distress, ²³ willingness, knowledge, and empowerment to use standard communication techniques in an empathic way are necessary. Hence, training in communication skills is essential and should be integrated early and continuously into medical education. As a consequence, in Germany, communication training has recently been integrated into undergraduate medical education by law. ²⁴

Interestingly, data analysis of matched surveys showed similar results to unmatched surveys. This phenomenon was already reported by Morrison, supporting the assumption that results are reflective of the whole sample. ²⁵ The response rate is remarkable: 95% of the students were willing to complete the postquestionnaire. And 87% were willing to complete the prequestionnaire within the introductory lecture. Hence, the points of measurement should be maintained. As the rate of paired surveys was less than 50%, further practice should try to increase students' willingness to provide the required code, e.g., repeated reminders to enter the code. Students' feedback revealed confusion with codes used in parallel classroom evaluations.

Knowledge improvement

The increased self-efficacy may be due to both decreased misled self-certainty as well as a gain in knowledge as shown by the higher scores in all four dimensions of the exam. Compared to marks of a historic cohort of students of the same university who got no PC training, ¹⁴ there was a gain of 40% of correct answers for knowledge about pain control in trained students, 68.7% gain in symptom control, and 32.8% for psychosocial knowledge. Students were provided with a detailed PC script. Every chapter outlined the learning targets in a concise way. The exam focused on main learning targets. One ethical knowledge question examining the definition of physician-assisted suicide, in contrast to active and passive euthanasia, deserves closer attention. The differences were correctly differentiated by 98% of trained students compared to 48% of students who left medical school without PC education. ¹⁴ In our opinion, this disparity might be of importance, since concise knowledge of definitions is required in an ongoing national and international discussion of death on demand—physicians are recognized as experts in the field, so professional expertise is prerequisite.^26,27

A high proportion of 5Y medical students believing in the myth of addiction in therapeutic use of opioids (66.5%, with additional 13.9% unsure) was observed in another study, too. ²⁸ Since the facts about opioid addiction could not be disclosed by the curriculum, further examination about the reason for the belief is required.

The external validity of the increase in knowledge levels is limited, because these are results from a single medical school. However, von Gunten et al. ⁹ found an improvement of knowledge from 52% to 62% correct answers compared to our data of 48.0% to 91.2%. The baseline knowledge score before training was reported, with 52% also for a U.S. sample of more than 10,000 1Y intern physicians. All three samples used the same core exam.^16,17 The difference is probably due to the differences in health care systems and medical education, but improvement was shown in both the Californian and the German sample. Further high commitment by the lecturers, both by high deployment—one palliative physician as first lecturer supported by a chaplain (spirituality/grief) and a psychologist (communication skills)—teaching in small groups and a detailed syllabus of the Mainz course may have contributed to the good marks. Each teaching unit was accompanied by at least one chapter introduced by a summary of the essential information.

Comparable to our historic cohort, untrained 5Y students of a Turkish cross-sectional study scored low for knowledge in pain and symptom management, although the concept of PC was understood. ²⁸ A limitation of the latter study was the use of a nonvalidated instrument. A research group from Edinburgh found physicians in their first year not well prepared to deliver PC, both as reported by themselves (67%) and assessed by their consultants, who rated their recently graduated doctors “not very prepared” for pain control in about 60%, and even worse for psychosocial issues. ²⁹ This study did not comprise exam scores.

One limitation of the exam is that the clinical meaning of this gain in knowledge remains unclear. Therefore this cohort will be followed up for the long-term effects of the palliative training. Further limitation is that an MC exam is not an appropriate measure of future doctors' attitudes toward care of patients at the end of life.

Whether the increased self-confidence is directly linked to improved knowledge is not proven by our study, since results of the exam could not be matched with pre- and posttraining surveys due to local policy of data privacy, which limited the rigor of the evaluation. As all students passed the exam with average scores around 90% a feeling of improved self-confidence in knowledge may have influenced the self-assessment of self-efficacy in other domains.

Generalizability of this monocentric study is limited. The results from Mainz cannot be considered to be representative for all medical students in Germany. However, there are different concepts realized for implementation of mandatory UPCE at German medical schools, ³⁰ but this is the first one that has been evaluated.

Implications for practice and further research

Although there are different national and international concepts realized for implementation of mandatory UPCE at German medical schools^8,30 no comparison of curricula effectiveness is made yet. A course evaluation after each semester in Mainz showed that students appreciated both the interdisciplinary team and lecturers' engagement. Further, they indicated high satisfaction with the content of every unit. The evaluation will continue to assure high quality for every semester.

Whether the success of the Mainz curriculum provided an instant or long-term effect is already the subject of further research. Maguire showed a lasting effect on communication skills in physicians four to six years after an appropriate undergraduate training. ³¹ Our students already passed a 90-minute bedside-teaching course in PC in their fourth year. ³² However, a more longitudinal approach is desired. Qualitative data of our study supports the approach of early integration of PC into undergraduate education. According to Bandura's model of self-efficacy, this could foster not only a gain of knowledge as shown, but also ease of communication skills and maturity in attitudes of future doctors. Prepared students will be able to face forthcoming challenges of care for an aging population by providing a confident relationship between the physician and the patient that is the base for every therapeutic effort. This assumption should be worth further research in UPCE regarding long-term effects as well as patient-related outcomes. ³³