Improving Residents' End-of-Life Communication Skills with a Short Retreat: A Randomized Controlled Trial

Abstract

Background:

Internal medicine residents are largely unprepared to carry out end-of-life (EOL) conversations. There is evidence that these skills can be taught, but data from randomized controlled trials are lacking.

Purpose:

We studied whether a day-long communication skills training retreat would lead to enhanced performance of and confidence with specific EOL conversations. We also studied the effect of the retreat on residents' ability to respond to patient emotions.

Methods:

PGY-2 resident volunteers were randomly assigned to a retreat group or a control group. The retreat involved a combination of teaching styles and skills practice with standardized patients. All participants completed questionnaires and were evaluated carrying out two types of conversations (breaking bad news or discussing direction of care) with a standardized patient before (T1) and after (T2) the intervention phase. Conversations were audio-taped and later rated by a researcher blinded to group assignment and time of assessment.

Results:

Forty-nine residents agreed to randomization (88%) with 23 residents randomized to the retreat group and 26 to the control group. Compared to controls, retreat participants demonstrated higher T2 scores for breaking bad news, discussing direction of care, and responding to emotion. Comparing T2 to T1, the retreat group's improvement in responding to emotion was statistically significant. The retreat group's confidence improved significantly only for the breaking bad news construct.

Conclusions:

A short course for residents can significantly improve specific elements of resident EOL conversation performance, including the ability to respond to emotional cues.

Introduction

Patients facing life-threatening illnesses value open and compassionate communication with their physicians,^1–3 but the literature suggests that physicians' communication skills are suboptimal.^4–6 In particular, physicians frequently miss patients' emotional cues,^7–13 and as a result, miss opportunities to respond to patients' most pressing feelings, hopes, and concerns. Inadequate training, particularly during residency, may be a major contributor to low confidence and poor performance.^14–18

Fortunately, there is evidence that communication skills can be taught. Retreats and workshop formats focused on communication training have been shown to improve physician and oncology fellow confidence in and objectively measured performance of key communication skills.^19–22 Data on communication skills training in residency, however, are limited. Alexander and colleagues²³ used a 16-hour retreat to help medical residents improve specific skills essential to end-of-life care; only 5 hours were devoted to communication skills, specifically breaking bad news and eliciting patient preferences for treatment/discussing do not resuscitate (DNR) orders. Compared to residents who did not participate in the retreat, participants demonstrated significant improvements in delivering bad news to standardized patients. The study was limited, however, by the fact that it was not randomized.

This article reports the results of a randomized, controlled-trial studying the effects of a day-long communication skills retreat on trainees' competence and confidence in carrying out end-of-life (EOL) conversations. We hypothesized that compared to feedback alone, the retreat would lead to greater competence in carrying out specific end-of-life communication tasks (i.e., breaking bad news and discussing the direction of care) and enhanced confidence. Our secondary outcomes included better general communication skills and improved responses to emotional cues.

Methods

Participants

After obtaining approval from our institution's Institutional Review Board, we invited all 56 second-year (PGY-2) internal medicine residents at Brigham and Women's Hospital (BWH) to participate in a study examining the effectiveness of a day-long communication skills-building retreat. We sent the invitation by e-mail and then sent a follow-up letter to those who did not respond within 2 weeks. A total of 49 PGY-2 residents consented to participate (response rate of 88%) and were randomized.

Randomization

We randomized participants by ambulatory blocks due to scheduling considerations. Each ambulatory block was assigned a number and half the blocks were randomly allocated to the retreat arm of the study using an online random number generator (www.random.org/integers). Twenty-three residents were randomized into the retreat group, and 26 into the control group. Two residents randomized to the intervention group did not participate in the retreat, but completed both preretreat and postretreat assessments; their data is included with the control group (n = 28). Seven residents completed the first assessment and questionnaire but later withdrew from the study during the year due to personal or family illness (4), scheduling conflicts (2), or change of heart (1). Of the 7, 5 were in the control group and 2 were in the intervention group. Baseline data includes responses from all residents randomized to one of the groups (control n = 28, retreat n = 21).

Intervention

Those who were randomized to the retreat group participated in a day-long skills building retreat consisting of a combination of teaching modalities over 5 hours (for retreat schedule see Appendix A). Two types of conversations—“Breaking Bad News” and “Discussing the Direction of Care”—created the framework and scenarios for the day. In addition to review of EOL communication skills, we also focused on skills related to responding to emotions. Every participant had the opportunity to interview a standardized patient and receive feedback from trained faculty at least once during the retreat.

Evaluation

All participants, regardless of randomization, were evaluated at two separate times, once in the fall of 2006 (preintervention), and once in the spring of 2007 (postintervention). Retreats for those randomized to the intervention group were scheduled between the first and second evaluations. The average amount of time between retreat and second evaluation was 14 weeks (range, 0–28 weeks; ± 11.7 weeks).

Each evaluation consisted of two 10-minute conversations with standardized patients. A trained member of the Department of Psychosocial Oncology and Palliative Care observed each conversation and provided 10 minutes of feedback, though the content of the feedback was not used for data analysis. Communication competence was calculated by analyzing the recorded conversations using rating tools derived from previous studies in this field²³ and expert opinion in palliative care.²⁴ For details of each rating tool, please see Appendices B and C. We looked for subjects to perform behaviors from each of three categories: general interviewing skills (e.g., explicitly elicits additional questions or concerns), task-related skills (for either breaking bad news [e.g., “Gives a clear and concise ‘warning shot’”] or discussing direction of care [e.g., “Explores patient's understanding of the prognosis”]), and responses to emotion (e.g., “explores the emotional reaction in greater detail”). Points were given for each behavior exhibited, allowing us to calculate scores for communication competence in each category. The sum of each category gave us a total score for each type of conversation.

Each participant also completed questionnaires before and after each skills evaluation (see Appendices D and E). The surveys inquired about personal experiences with EOL care, formal teaching and/or rotation in palliative care, and explored self-reported confidence to perform given communication tasks.

Analysis

Recordings were analyzed by two raters blinded to subject identity, time of year, and group assignment. Disagreements between the coders were discussed and resolved. Interrater reliability was excellent for both types of conversations: overall, the κ for the exercises ranged between 0.52 and 1.0 with an average κ if 0.87 or greater. Intrarater reliability was also excellent (minimum κ 0.94 ± 0.06).

We analyzed the differences between groups at baseline with respect to demographic characteristics and self-assessment of communication skills. For analysis of the communication competence scores, we compared the performance of the two groups using t tests to determine whether there was a significant difference between the control and retreat groups postintervention. For the communication competence scores, we adjusted for both gender (male versus female) and residency training track (primary care versus others) because both variables have been shown to influence communication style^25–27 and empathic responses.²⁸ We also used paired one-sample t tests to determine whether there was a significant difference in performance within groups comparing preintervention and postintervention assessments. Finally, we used two-way repeated measures analysis of variance (ANOVA) to analyze the relative effects of group assignment and time (preintervention versus postintervention) on both self-reported confidence and competence with both types of conversations. In the case that the sample sizes were small, we used nonparametric procedures (Fisher's exact test). Data were analyzed using SPSS (SPSS for Windows: Release 16.0, SPSS Inc., Chicago, IL).

Results

Overall, the groups were similar at baseline prior to any intervention (Table 1). Compared to the control group, a greater proportion of the retreat group was female and in the primary care track, although neither difference was statistically significant. The only differences in experiences between the groups were more frequent participation in “giving bad news” and “hospice” conversations in the control group. There was no significant difference between groups in terms of self-reported preparation to carry out, having received formal teaching about, or having received feedback on any type of end-of-life conversation. There was also no difference in terms of personal experience with illness or death (Table 2).

Table 1.

Baseline Demographic Characteristics by Group

	Control (n = 28^a)	Retreat (n = 21)	Stat, p value
Age	29.7 ± 2.1	29.2 ± 1.7	t, p = 0.44
Gender (% female)	28.6	47.6	χ², p = 0.17
Residency track (% primary care)	14.3	33.3	FET, p = 0.17
Formal EOL training (% 4 + hours)	39.3	47.6	χ², p = 0.56
Palliative/hospice rotation—medical school (% yes)	10.7	14.3	FET, p = 0.52
Palliative care/hospice rotation during residency (% yes)	0.0	9.5	FET, p = 0.18
Contact with palliative care clinicians during residency (% yes)	92.9	100.0	FET, p = 0.50

Includes two subjects originally randomized to the retreat group but who did not participate in any of the retreats.

EOL, end of life; FET, Fisher's exact test.

Table 2.

Baseline Self-Assessed Communication Training, Skills, and End-of-Life Experiences by Group

Times participated (% 4 or more times during residency)	Control (n = 28)	Retreat (n = 21)	Stat, p value
Code discussion	100.0	95.2	FET, p = 0.43
Giving bad news	89.3	52.4	χ², p = 0.004
Discussion about prognosis	85.7	66.7	FET, p = 0.11
Discussion about EOL	82.1	57.1	χ², p = 0.055
Hospice	78.6	52.4	χ², p = 0.05
Self-assessed preparation to … (% well-prepared^a)
… Tell about new, life-threatening diagnosis	42.9	45.0	χ², p = 0.88
… Talk about treatment options	32.1	15.0	χ², p = 0.18
… Talk about code status	57.1	68.4	χ², p = 0.44
… Talk about prognosis	28.6	25.0	χ², p = 0.78
… Talk about EOL care	18.5	15.0	FET, p = 1.00
… Talk about hospice	21.4	20.0	FET, p = 1.00
Explicitly been taught to … (% yes)
… Tell about new, life-threatening diagnosis	5.3	33.3	FET, p = 0.07
… Talk about treatment options	42.1	46.7	χ², p = 0.79
… Talk about code status	89.5	86.7	FET, p = 1.00
… Talk about prognosis	21.1	13.3	χ², p = 0.56
… Talk about EOL care	36.8	66.7	χ², p = 0.08
… Talk about hospice	36.8	13.3	FET, p = 0.24
Personal experiences
Experienced death of parent, sibling or child (% yes)	21.4	9.5	FET, p = 0.44
Experienced death of another relative or friend (% yes)	92.9	81.0	FET, p = 0.41
Personally faced life threatening illness (% yes)	7.1	0.0	FET, p = 0.13

“Well-prepared” is defined as the total percentage reporting being “somewhat well”, “well” and “very well-prepared.”

FET, Fisher's exact test; EOL, end of life.

Communication competence

At baseline, there was no significant difference between retreat and control groups, except for the discussing goals of care subtotal where we found the retreat group to be slightly better (Table 3; t = 2.2, p = 0.04). Comparing preintervention to postintervention results, the retreat group improved their average total score with marginal significance (p = 0.07) for the “breaking bad news” total, while the control group did not. When adjusting for gender (male versus female) and residency track (primary care versus other), the retreat group had significantly higher postintervention scores than the control group (F = 1.74, p = 0.046). For discussing direction of care, neither group improved its total score significantly, but the retreat group's postintervention total score was significantly higher than the control group's. In terms of competence subtotals, neither group improved significantly over time with respect to general interviewing skills or task-related skills.

Table 3.

Conversation Scores by Group (Control vs. Retreat) and Time (Pre- Vs. Post-) ^a

Item/measure	Group	Preretreat Mean ^b (SD)	Postretreat Mean (SD)	Change	Paired t test p value
	Breaking Bad News
Breaking Bad News total score	Control	9.2 (2.3)	9.4 (2.2)	+ 0.2	p = 0.73
	Retreat	9.4 (1.8)	10.6 (2.0)	+ 1.2	p = 0.07
General Interviewing Skills Subtotal	Control	2.8 (0.6)	3.0 (1.2)	+0.2	p = 0.59
	Retreat	2.6 (0.6)	2.7 (0.9)	+0.1	p = 0.83
Breaking Bad News Skills Subtotal^c	Control	4.6 (1.4)	5.1 (1.4)	+0.5	p = 0.12
	Retreat	5.1 (1.1)	5.9 (1.4)	+0.8	p = 0.12
Responding to emotion subtotal	Control	1.8 (1.4)	1.4 (1.4)	−0.4	p = 0.14
	Retreat	1.7 (1.1)	2.0 (1.2)	+0.3	p = 0.39
	Direction of Care
Direction of Care Total score	Control	8.1 (1.7)	8.1 (2.1)	+ 0.0	p = 0.92
	Retreat	8.9 (2.5)	9.4^d (2.0)	+ 0.5	p = 0.46
General interviewing skills subtotal	Control	3.3 (0.9)	2.8 (1.2)	−0.5	p = 0.09
	Retreat	3.3 (1.0)	3.4 (0.9)	+0.1	p = 0.65
Discussing goals of care skills subtotal	Control	3.2 (1.2)	3.6 (1.2)	+0.4	p = 0.48
	Retreat	3.9^d (1.4)	3.6 (1.5)	−0.3	p = 0.67
Responding to emotion subtotal	Control	1.5 (0.7)	1.7 (1.2)	+0.2	p = 0.28
	Retreat	1.7 (1.1)	2.4 (0.9)	+0.7	p = 0.007
	Responding to Emotion
Overall responding to emotion score	Control	3.1 (1.7)	2.9 (1.6)	− 0.2	p = 0.57
	Retreat	3.4 (1.9)	4.5 (1.4)	+ 1.1	p = 0.03

Table 3 represents data after adjusting for gender (male vs. female) and residency track (primary care vs. other).

Difference at baseline between control and retreat groups NS for all items except for “Discussing goals of care skills subtotal.”

Difference between groups, postintervention significant at p ≤ 0.05.

Difference at baseline between groups, preintervention, significant at p = 0.04.

Responding to emotions

Because the “responding to emotion” behaviors were the same for both types of conversations, we added the subtotals from both conversations to derive an “Overall Responding to Emotion Score.” The retreat group improved significantly (32.4% improvement) in responding to emotion while the control group exhibited a nonsignificant decline (Table 3). When entered into a model adjusting for gender and residency track, there was a significant group-by-time interaction (F = 4.14, p = 0.05) confirming that the groups changed over time and changed in different directions (as seen in the univariate analyses).

Self-assessed confidence

Self-assessed preparation for breaking bad news and direction of care questionnaire items is summarized in Table 4. There were no significant differences found between the groups at baseline. At the postretreat assessment, the retreat group showed significantly higher self-assessed preparation to tell about new, life threatening diagnoses (representing the breaking bad news construct) compared to both its preretreat assessment and to the control group. No other significant between-group differences were found. The only other marginally significant within-group improvement for the retreat group was an increase in confidence talking about EOL care. The control group reported increased preparation to perform the same communication tasks, but the degree of improvement was smaller in each case. Confidence did not significantly improve for other elements of the direction of care construct, and for some elements that were not explicitly covered during the retreat (i.e., talking about code status), confidence declined.

Table 4.

Self-Assessed Preparation to Breaking Bad News and Discuss Direction of Care, Pre- vs. Postretreat

Preparation to …	Group	Preretreat ^a	Postretreat	Diff.	Paired t test p value
… Tell about new, life-threatening diagnosis^b–d	Control	2.87 ± 0.87	3.26 ± 0.54	+0.39	p = 0.05
	Retreat	3.06 ± 0.90	3.71 ± 0.59	+ 0.65	p = 0.004
… Talk about treatment options^b	Control	3.52 ± 0.99	3.22 ± 0.60	−0.30	p = 0.13
	Retreat	3.59 ± 0.80	3.35 ± 0.61	−0.24	p = 0.36
… Talk about code status^b	Control	3.70 ± 0.93	3.39 ± 0.50	−0.30	p = 0.15
	Retreat	4.00 ± 0.73	3.50 ± 0.52	−0.50	p = 0.002
… Talk about prognosis^b	Control	3.09 ± 0.90	3.13 ± 0.46	+0.04	p = 0.80
	Retreat	2.88 ± 0.93	3.29 ± 0.59	+0.41	p = 0.11
… Talk about EOL care^b	Control	3.17 ± 0.89	3.26 ± 0.69	+0.09	p = 0.65
	Retreat	2.94 ± 0.83	3.35 ± 0.49	+0.41	p = 0.05
Discuss Direction of Care Subtotal^b,e	Control	12.83 ± 2.82	13.00 ± 1.48	+0.17	p = 0.77
	Retreat	13.06 ± 2.35	13.69 ± 1.35	+0.63	p = 0.27

Scores represent mean responses using a 1- to 5-point Likert scale where 1 = “not well at all” and 5 = “very well.”

Difference at baseline between control and retreat group NS.

Difference between groups, postretreat, significant at p ≤ 0.05.

This item represents the “Breaking Bad News (BBN)” construct.

Direction of care (DOC) subtotal is the sum of the following items: talk about treatment options, talk about code status, talk about prognosis, and talk about EOL care.

Participant satisfaction

The retreats were uniformly well received. Participants gave an average score of 4.95 (standard deviation [SD] 0.23) of 5 (Likert scale, 5 = strongly agree; 1 = strongly disagree) when asked to comment on the following statement: “I found the retreat helpful.” Eighty-seven percent of all participants (both retreat and control groups) endorsed the idea of making such training mandatory during residency.

Discussion

Our study has shown that a short, skills-building retreat can lead to modest improvements in internal medicine residents' communication skills. Specifically, residents exposed to the retreat showed improved ability to deliver bad news and respond to emotion and developed increased confidence to perform specific EOL conversations.

The results are consistent with the findings of other work in the literature. Alexander and colleagues²³ noted significant improvement in “responding to emotional cues,” and overall improvement in summary scores of the intervention group compared to the control group. Using a similar rating system, the improvements were also modest (1.77 points of 17). Our results are also consistent with other studies involving oncologists,²⁹ oncology fellows,²⁰ general practitioners,^30,31 nurses,^32,33 and medical students³⁴—although all of these studies involved much longer interventions or follow-up “consolidation” sessions. Not all communication skills changed for the better. General communication skills did not improve, potentially due to the fact that they were not an explicit focus of the retreats. The fact that residents displayed improvement in the breaking bad news task, but not in discussing direction of care is also not surprising. Back and colleagues²⁰ showed that oncology fellows acquired a higher number of breaking bad news skills than transitioning to palliative care (a context similar, although not identical, to our discussing direction of care) after a 4-day retreat. Giving bad news involves discrete steps that typically follow a linear progression. Other end-of-life conversations are more difficult to predict and require greater flexibility in responding to—and eliciting—information and emotions from the patient.

The fact that the most significant improvement was in responding to emotions is important. Educators have noted that trainee empathy may actually decrease in the course of medical training.^35,36 The reasons are not clear, but may involve insufficient training, inadequate feedback, a lack of effective modeling by attending physicians,^7,9,11,37,38 a fear of unleashing uncomfortable emotional responses (psychic defense), and a hidden curriculum valuing a objective findings over the subjective experience of the patient.^8,39 The intervention described here may provide a tool to not only prevent a decay in skills, but to also improve competence. Given that empathic responses are a central component of effective communication, physician professionalism, and patient satisfaction,^40,41 the results of this study are of great importance.

The data on confidence shed light on how accurately residents assess their own skills and their training. The retreat group's self-assessed confidence improved by the greatest extent in breaking bad news construct, paralleling the group's improved competence in that type of conversation. But the extent to which residents' self-assessed confidence predicts improved performance is limited. The control group reported improved self-assessed preparation to perform given tasks without commensurate improvement in objective performance. This finding confirms previous work showing that self-reported confidence may increase with years of training but is only marginally correlated with competence.¹⁴ It further supports the notion that improved confidence is not sufficient to judge the value of a given educational intervention.

This study has several limitations. First, although the control group did not receive formal training in a retreat, they did receive some feedback between the first and second assessments, and that training may have been sufficient to improve performance and confidence. It is possible, therefore, that our results underestimate the effect of the retreat. It would be useful to assess residents who are not exposed to any form of training or feedback in order to compare the effect of the retreat to usual training. Additional limitations include a small sample size from a single institution and absence of follow-up beyond a few months.

In summary, this study shows that a short, multimodal retreat with skills practice and feedback leads to improvements in residents' EOL conversation skills. That a limited, one-time retreat produced any results in the face of “usual” training—in which empathic responses are undervalued and communication skills are infrequently taught—is remarkable. The next question is how we can build on these small improvements in performance, not just for the sake of structured educational exercises, but for actual patient care. A single intervention will most certainly not be enough to change behavior in a meaningful way for most trainees. What effect would repeated evaluation and feedback have? Should we use different modalities to reinforce the skills in different settings?

Even with modest results and a number of unanswered questions, this type of program satisfies the Accreditation Council for Graduate Medical Education (ACGME) requirement for teaching 3 of the 6 General Competencies (professionalism, patient care, and interpersonal and communication skills) and is in line with the goals of the ACGME Outcomes project.⁴² It also uses a relatively simple rating tool that may be used in real time to provide feedback and instruction. We encourage other educators to incorporate such experiences into their training programs, even as we continue to search for the optimal approach to teaching—and sustaining—this challenging set of skills.

Footnotes

Acknowledgments

This work was supported by a Brigham and Women's Hospital (BWH) Department of Medicine Support for Excellence in Educational Development (S.E.E.D.) grant. The project would not have been possible without support from the Dana-Farber Cancer Institute Department of Psychosocial Oncology and Palliative Care and the BWH Department of Medicine and Medicine Residency Program.

Portions of this work were presented in abstract form at the American Academy of Hospice and Palliative Medicine Annual Assembly in Austin, Texas on March 26, 2009.

Author Disclosure Statement

No competing financial interests exist.

Appendix A.

Sample Retreat Schedule

Teaching End-of-Life Communication Skills to Internal Medicine Residents
Retreat Schedule
9:15 am	Breakfast
	Welcome/Introduction
9:30 am	Small-Group Discussion: “What's hard about having End-of-Life Conversations for you?”
10:00 am	Modeling a Conversation: Direction of Care
	(using one of the scenarios brought up during earlier discussion)
	Preparation: 5 min
	Role–play: 10 min
	Debrief with group: 30 min
10:45 am	Break
11:00 am	Large-Group Didactic: 2 Conversations
	Review of Key Communication Skills
12:00 pm	Lunch
1:45–2:00 pm	Role–Play Intro: “I Hate Role Plays”
2:00–2:40 pm	Skills Practice: participants split into groups of 2–3; groups switch stations at 20 minutes
	Skills Practice #1 (20 min)
	Skills Practice #2 (20 min)
2:40–2:50 pm	Break
2:50–3:30	Skills Practice #3 (20 min)
	Skills Practice #4 (20 min)
3:30 pm	Group Debrief/Review Key Concepts
4:00 pm	Session Concludes

Appendix B.

Conversation Rating Sheet for Breaking Bad News

Subject Code: _____________	CD # ___	Track # ____
Communication skill	Rating	Points
A. General Patient-Centered Interviewing Skills
1. Creates initial rapport (within first 30 seconds)	1 = Yes/0 = No
2. Explicitly elicits additional questions and/or concerns	1 = Yes/0 = No
3. Makes an explicit expression of nonabandonment	1 = yes/0 = No
4. Addresses patient's symptoms immediately when brought up by patient.	1 = Yes/0 = No
5. Uses summary statements to ensure understanding of patient's statements (e.g., “It sounds like …”)	1 = Yes/0 = No
	Subtotal	/5
B. Delivering Bad News
1. Assess patient's understanding of the illness	1 = Yes/0 = No
2. Assesses patient's preferences for receiving information	1 = Yes/0 = No
3. Gives a clear and concise “warning shot” with a pause prior to delivery	1 = Yes/0 = No
4. Delivers bad news early in the conversation (within first 3 minutes)	1 = Yes/0 = No
5. Delivers information clearly, concisely, and in plain language	1 = Yes/0 = No
6. Silence after delivery to allow patient to process	2 = > 10 sec
	1 = 3–10 sec
	0 = < 3 sec
7. Suggests a plan for the next step	1 = Yes/0 = No
8. Assesses patient's immediate safety and/or social support	1 = Yes/0 = No
	Subtotal	/9
C. Responding to Emotion
1. Validates or expresses understanding of the patient's emotional reaction	1 = Yes
	0 = No
2. Explores the emotional reaction in greater detail	0 = None
	1 = once
	2 = 2 + times
3. Names the patient's emotional reaction (when the patient does not name it)	1 = Yes
	0 = No
4. Immediately provides information in response to patient's expression of emotion	−1 = provides information
5. Immediately provides reassurance in response to a patient's expression of emotion	−1 = provides reassurance
	Subtotal	/4
	TOTAL	/18

Appendix C.

Conversation Rating Sheet for Discussing Direction of Care

Subject Code: ______________	CD # ______	Track # _____
Communication Skill	Rating	Points
A. General Patient-Centered Interviewing Skills
1. Establishes rapport (within first 30 seconds)	1 = Yes/0 = No
2. Describes the purpose of the encounter during the first third of the interview (i.e. discussing general/future care) in the first 1/3 of the conversation	1 = Yes/0 = No
3. Uses summary statements to ensure understanding of patient's statements (“It sounds like …”)	1 = Yes/0 = No
4. Explicitly elicits additional questions and/or concerns	1 = Yes/0 = No
5. Makes explicit statement of “partnership building”	1 = Yes/0 = No
	Subtotal	/5
B. Discussing Goals of Care
1. Asks patient about prior experiences with EOL decision-making or about assignment of Healthcare Proxy	1 = Yes/0 = No
2. Assessing patient's understanding of his/her current condition	1 = Yes/0 = No
3. Explores patient's understanding of the prognosis and/or concerns about the future	1 = Yes/0 = No
4. Explicitly asks about patient's concerns about the future	1 = Yes/0 = No
5. Explores the patient's general values and goals	1 = Yes/0 = No
6. Discusses probable clinical scenarios that the patient might face (“What if” questions)	1 = Yes/0 = No
7. Provides a “big picture” summary statement	1 = Yes/0 = No
8. Proposes a care plan that respects patient's goals, values, and concerns (i.e., recommendation for future care)	1 = Yes/0 = No
	Subtotal	/8
C. Responding to Emotion
1. Validates or expresses understanding of the patient's emotional reaction	1 = Yes
	0 = No
2. Explores patient's emotional reaction(s) in greater detail	0 = None
	1 = once
	2 = 2 + times
3. Names the patient's emotional reaction (when patient does not name it)	1 = Yes
	0 = No
4. Immediately provides information in response to expression of emotion	−1 = provides information
5. Immediately provides reassurance in response to expression of emotion	−1 = provides reassurance
	Subtotal	/4
	TOTAL	/17

Appendix D.

PreIntervention Questionnaire

End-of-Life Communication Observed Structured Clinical Exercise (OSCE)

Pre-Intervention Questionnaire

Thank you for participating in this OSCE. This exercise is part of a research study that will help us understand whether a day-long skills retreat improves residents' comfort and confidence with end-of-life communication compared to our usual training.

Every member of the junior (i.e., PGY2) class has been invited to participate. All participants will be asked to complete two (2) communication OSCEs—one in the fall and one in the spring. In addition, approximately half of the class will be randomized to participate in the retreat. You may or may not be randomized to the retreat group.

Please complete this preintervention questionnaire. You will be asked about your previous experience and comfort in dealing with end-of-life conversations.

Please fill out the questionnaire as completely as you can. All responses will be kept confidential; your name will not be associated with your answers in any way.

Please fill in the following so that we can compare your responses from before and after the session.

First three letters of your mother's maiden name: __________

Last 4 digits of you SSN: __________

Part I: Background
Residency Track:	[] Categorical	[] Primary Care	[] Med–Peds
	[] GHE	[] Genetics	[] Hemi–doc
Gender	[] Female	[] Male
Age: ______

Your plans after graduation from residency:

[] Primary care practice

[] Hospitalist

[] Subspecialty Fellowship Subspecialty: __________________

[] General Medicine Fellowship

[] Research Area of interest: _____________________________

[] Other ______________________________________________________

Part II: Professional Experience

Please estimate you prior formal training in end-of-life care (lectures, seminars, or rotations during residency or medical school):

[] 0 hours

[] 1–3 hours

[] 4–10 hours

[] 10–40 hours

[] > 40 hours

	Yes	No
Did you complete a palliative care and/or hospice rotation in medical school?	[]	[]
Did you complete a palliative care and/or hospice rotation in residency?	[]	[]
During your residency, have you had any contact with clinicians (i.e., in the context of caring for patients) who specialize in palliative care?	[]	[]

Please estimate how many times you have participated in the following activities with a patient during your residency thus far?
	0	1–3	4–9	10 or more
Code discussion (i.e., DNR discussion)	[]	[]	[]	[]
Giving bad news (i.e., new diagnosis of life-threatening illness)	[]	[]	[]	[]
Discussion about prognosis	[]	[]	[]	[]
Discussion about care at the end of life (i.e., location of care, important goals, quality of life vs. quantity)	[]	[]	[]	[]
Discussion about hospice	[]	[]	[]	[]

Part III: Self-Assessed Preparation
Please circle the number that best fits your answer.
Overall, how well has your internship/residency training prepared you to:
	Very well		Somewhat well		Not well At all
Tell patients about a new, life-threatening diagnosis (i.e., heart failure, or cancer)	5	4	3	2	1
Talk to patients about their treatment options	5	4	3	2	1
Talk to patients about their prognosis	5	4	3	2	1
Talk to patients about their code status	5	4	3	2	1
Talk to patients about the type of care they would like to receive at the end of life based on their values and goals	5	4	3	2	1
Talk to patients about hospice	5	4	3	2	1

During your residency, have you explicitly been taught to:
	Yes	No
Tell patients about a new, life-threatening diagnosis (i.e., heart failure, or advanced cancer)	[]	[]
Talk to patients about their treatment options	[]	[]
Talk to patients about their code status	[]	[]
Talk to patients about their prognosis	[]	[]
Talk to patients about the type of care they would like to receive at the end of life based on their values and goals	[]	[]
Talk to patients about hospice	[]	[]

During your residency, how many times were you given feedback from an attending or more advanced resident about the specific strengths and weaknesses of the following discussions?
	Yes	No	If yes, # of times feedback given
Tell patients about a new, life-threatening diagnosis (i.e., heart failure, or advanced cancer)	[]	[]
Talk to patients about their treatment options	[]	[]
Talk to patients about their code status	[]	[]
Talk to patients about their prognosis	[]	[]
Talk to patients about the type of care they would like to receive at the end of life based on their values and goals	[]	[]
Talk to patients about hospice	[]	[]

Overall, how well prepared do you feel to TEACH others (i.e., less advanced residents and medical students) to:
	Very well		Somewhat well		Not well At all
Tell patients about a new, life-threatening diagnosis	5	4	3	2	1
Talk to patients about their treatment options	5	4	3	2	1
Talk to patients about their prognosis	5	4	3	2	1
Talk to patients about their code status	5	4	3	2	1
Talk to patients about the type of care they would like to receive at the end of life based on their values and goals	5	4	3	2	1
Talk to patients about hospice	5	4	3	2	1
When discussing (or preparing to discuss) end-of-life care with patients, how well prepared do you feel to:
	Very well		Somewhat well		Not well At all
Establish what the patient understands about his/her illness?	5	4	3	2	1
Clarify how much the patient wants to know and how he/she wants to make decisions?	5	4	3	2	1
Inquire about the patient's values, goals, hopes, and fears?	5	4	3	2	1
Respond to the patient's emotions?	5	4	3	2	1
Summarize the conversation and establish a shared plan?	5	4	3	2	1
Support hope while disclosing information about a poor prognosis?	5	4	3	2	1

Part IV: Attitudes about End-of-Life communication
	Completely Agree	Agree	Disagree	Completely Disagree
It is important to learn how to talk with patients about the type of care they want at the end of life
Talking to patients about their goals of care is solely the attending's responsibility
Talking to patients about death is depressing for me
I dread having to talk to patients and families about their wishes for care at the end of life
Talking about dying tends to make patients with terminal illnesses more discouraged
It is up to the patient to bring up the conversation about they type of care that he/she wants at the end of life

Part V. Personal Experiences

Have you ever experienced the death of a parent, sibling, or child?

[] Yes

[] No

Have you ever experienced the death of another relative or friend?

[] Yes

[] No

Have you ever personally faced a life threatening illness?

[] Yes

[] No

THANK YOU FOR YOUR TIME!

DNR, do not resuscitate

Appendix E.

PostIntervention Questionnaire

End-of-Life Communication Observed Structured Clinical Exercise (OSCE)

Postintervention Questionnaire

Thank you for participating in this educational research study.

Please fill out the following. All responses will be kept confidential; your name will not be associated with your answers in any way. You may decline to answer any of the questions for any reason.

Please fill in the following so that we can compare your responses from before and after the session.

First three letters of your mother's maiden name: __________

Last 4 digits of you SSN: __________

Part I: Demographic Information and Personal Experience

Please indicate your stage of training:

[] PGY2

[] PGY3

[] PGY4

[] PGY5

Have you, a family member, or a close friend ever experienced a serious or life-threatening illness?

[] Yes

[] No

How long ago was the experience noted above?

[] It is ongoing

[] Less than 1 year ago

[] 1–4 years ago

[] More than 5 years ago

Part II: Self-Assessed Preparation

For each item below, please think back to the beginning of the year.

Please rate your level of preparation:

(a) before your first OSCE (Pre-program) and

(b) now (Post-program).

Circle the number on each line that best describes your answer.

5 = very well

1 = not well at all

Overall, how well prepared did you feel (and how well prepared do you feel now) to:
	Very well		Somewhat well		Not well at all
Tell patients about a new, life-threatening diagnosis (i.e., heart failure, or cancer)
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about their treatment Options
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about their prognosis
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about their code status
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about the type of care they would like to receive at the end of life based on their values and goals
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Overall, how well prepared did you feel (and how well prepared do you feel now) to TEACH others (i.e., interns and medical students) to:
	Very well		Somewhat well		Not well at all
Tell patients about a new, life-threatening diagnosis (i.e., heart failure, or cancer)
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about their treatment Options
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about their prognosis
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about their code status
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Talk to patients about the type of care they would like to receive at the end of life based on their values and goals
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
When discussing (or preparing to discuss) end-of-life care with patients, how well prepared did you feel (and how prepared do you now feel) to:
	Very well		Somewhat well		Not well at all
Establish what the patient understands about his/her illness?
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Clarify how much the patient wants to know and how he/she wants to make decisions?
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Inquire about the patient's values, goals, hopes, and fears?
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Respond to the patient's emotions?
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Summarize the conversation and establish a shared plan?
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1
Support hope while disclosing information about a poor prognosis?
a. Preprogram (before)	5	4	3	2	1
b. Postprogram (now)	5	4	3	2	1

Part II: Feedback
Please answer the following questions.
1. OSCEs	Strongly Agree	Agree	Neither agree nor disagree	Disagree	Strongly Disagree
a. I found the OSCE with feedback helpful
b. The OSCE with feedback changed the way I communicate with my patients
c. I feel more confident about discussing end-of-life care with my patients after participating in both OSCE's
If you participated in a day-long communication retreat, please answer question #2. If you did NOT participate in a day-long retreat, please skip to question #3.
2. Communication Skills Retreat	Strongly Agree	Agree	Neither agree nor disagree	Disagree	Strongly Disagree
a. I found the communication skills retreat helpful
b. The communication skills retreat changed the way I communicate with my patients
c. I feel more confident about discussing end-of-life care with my patients after participating in the communication skills retreat
3. Since your first OSCE in the fall, how many of the following conversations have you had with your patients?
Type of conversation	0	1–3	4–9	10 or more
Breaking bad news
Discussing direction of care at the end-of-life
Code Status (DNR/DNI)
Hospice care

4. Has participation in either the OSCEs or the Communication Skills Retreat (or both) changed the way you communicate with your patients in general?

Yes _______

No _______

5. If the answer to #6 is yes, please list the ways in which your approach to these conversations has changed?

i. _________________________________________________________________________________

ii. _________________________________________________________________________________

iii. _________________________________________________________________________________

iv. _________________________________________________________________________________

v. _________________________________________________________________________________

6. Future directions	Strongly Agree	Agree	Neither agree nor disagree	Disagree	Strongly Disagree
All junior residents should participate in a day-long communication skills retreat
Other comments:
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Thank you once again for all your time and your input!
Your efforts are much appreciated!

DNR, do not resuscitate; DNI, do not intubate

References

Hanson

, Danis

, Garrett

. What is wrong with end-of-life care? Opinions of bereaved family members. J Am Geriatr Soc, 1997; 45:1339–1344.

Steinhauser

, Christakis

, Clipp

, McNeilly

, McIntyre

, Tulsky

. Factors considered important at the end of life by patients, family, physicians, and other care providers. JAMA, 2000; 284:2476–2482.

Steinhauser

, Christakis

, Clipp

, McNeilly

, Grambow

, Parker

, Tulsky

. Preparing for the end of life: Preferences of patients, families, physicians, and other care providers. J Pain Symptom Manage, 2001; 22:727–737.

Marvel

, Epstein

, Flowers

, Beckman

. Soliciting the patient's agenda: have we improved? JAMA, 1999; 281:283–287.

Morita

, Akechi

, Ikenaga

, Kohara

, Mukaiyama

, Nakaho

, Nakashima

, Shima

, Matsubara

, Fujimori

, Uchitomi

. Communication about the ending of anticancer treatment and transition to palliative care. Ann Oncol, 2004; 15:1551–1557.

Tulsky

, Fischer

, Rose

, Arnold

. Opening the black box: How do physicians communicate about advance directives? Ann Intern Med, 1998; 129:441–449.

Butow

, Brown

, Cogar

, Tattersall

, Dunn

. Oncologists' reactions to cancer patients' verbal cues. Psychooncology, 2002; 11:47–58.

Suchman

, Markakis

, Beckman

, Frankel

. A model of empathic communication in the medical interview. JAMA, 1997; 277:678–682.

Levinson

, Gorawara-Bhat

, Lamb

. A study of patient clues and physician responses in primary care and surgical settings. JAMA, 2000; 284:1021–1027.

10.

Davenport

, Goldberg

, Millar

. How psychiatric disorders are missed during medical consultations. Lancet, 1987; 2:439–441.

11.

Pollak

, Arnold

, Jeffreys

et al. Oncologist communication about emotion during visits with patients with advanced cancer. J Clinical Oncol, 2007; 25:5748–5752.

12.

Anderson

, Alexander

, Rodriguez

, Alexander

, Olsen

, Abernethy

, Sugg Skinner

, Rodriguez

, Tulsky

. “What concerns me is …” Expression of emotion by advanced cancer patients during outpatient visits. Suppor Care Cancer, 2008; 16:803–811.

13.

Detmar

, Muller

, Wever

, Schornagel

, Aaronson

. The patient-physician relationship. Patient-physician communication during outpatient palliative treatment visits: An observational study. JAMA, 2001; 285:1351–1357.

14.

Buss

, Alexander

, Switzer

, Arnold

. Assessing competence of residents to discuss end-of-life issues. J Palliat Med, 2005; 8:363–371.

15.

Calam

, Far

, Andrew

. Discussions of “code status” on a family practice teaching ward: What barriers do family physicians face? CMAJ, 2000; 163:1255–1259.

16.

Teno

, Clarridge

, Casey

, Welch

, Wetle

, Shield

, Mor

. Family perspectives on end-of-life care at the last place of care. JAMA, 2004; 291:88–93.

17.

Smith

, Ries

, Zhang

, Tulsky

, Prigerson

, Block

. Resident approaches to advance care planning on the day of hospital admission. Arch Intern Med, 2006; 166:1597–1602.

18.

Sullivan

, Lakoma

, Block

. The status of medical education in end-of-life care: a national report. J Gen Intern Med, 2003; 18:685–695.

19.

Fallowfield

, Lipkin

, Hall

. Teaching senior oncologists communication skills: Results from phase I of a comprehensive longitudinal program in the United Kingdom. J Clin Oncol, 1998; 16:1961–1968.

20.

Back

, Arnold

, Baile

, Fryer-Edwards

, Alexander

, Barley

, Gooley

, Tulsky

. Efficacy of communication skills training for giving bad news and discussing transitions to palliative care. Arch Intern Med, 2007; 167:453–460.

21.

Sullivan

, Lakoma

, Billings

, Peters

, Block

, Faculty

. Teaching and learning end-of-life care: Evaluation of a faculty development program in palliative care. Acad Med, 2005; 80:657–668.

22.

Sullivan

, Lakoma

, Billings

, Peters

, Block

, Faculty

. Creating enduring change: Demonstrating the long-term impact of a faculty development program in palliative care. J Gen Intern Med, 2006; 21:907–914.

23.

Alexander

, Keitz

, Sloane

, Tulsky

. A controlled trial of a short course to improve residents' communication with patients at the end of life. Acad Med, 2006; 81:1008–1012.

24.

Roter

, Larson

, Fischer

, Arnold

, Tulsky

. Experts practice what they preach: A descriptive study of best and normative practices in end-of-life discussions. Arch Intern Med, 2000; 160:3477–3485.

25.

Kaplan

, Greenfield

, Gandek

, Rogers

, Ware

Jr . Characteristics of physicians with participatory decision-making styles. Ann Intern Med, 1996; 124:497–504.

26.

Roter

, Stewart

, Putnam

, Lipkin

Jr , Stiles

, Inui

. Communication patterns of primary care physicians. JAMA, 1997; 277:350–356.

27.

Roter

, Hall

, Aoki

. Physician gender effects in medical communication: A meta-analytic review. JAMA, 2002; 288:756–764.

28.

Bylund

, Makoul

. Empathic communication and gender in the physician-patient encounter. Patient Educ Couns, 2002; 48:207–216.

29.

Fallowfield

, Jenkins

, Farewell

, Saul

, Duffy

, Eves

. Efficacy of a Cancer Research UK communication skills training model for oncologists: A randomised controlled trial. Lancet, 2002; 359:650–656.

30.

Roter

, Hall

, Kern

, Barker

, Cole

, Roca

. Improving physicians' interviewing skills and reducing patients' emotional distress. A randomized clinical trial. Arch Intern Med, 1995; 155:1877–1884.

31.

Bonvicini

, Perlin

, Bylund

, Carroll

, Rouse

, Goldstein

. Impact of communication training on physician expression of empathy in patient encounters. Patient Educ Couns, 2009; 75:3–10.

32.

Razavi

, Delvaux

, Marchal

, Durieux

, Farvacques

, Dubus

, Hogenraad

. Does training increase the use of more emotionally laden words by nurses when talking with cancer patients? A randomised study. Br J Cancer, 2002; 87:1–7.

33.

Wilkinson

, Perry

, Blanchard

, Linsell

. Effectiveness of a three-day communication skills course in changing nurses' communication skills with cancer/palliative care patients: A randomised controlled trial. Palliat Med, 2008; 22:365–375.

34.

Marteau

, Humphrey

, Matoon

, Kidd

, Lloyd

, Horder

. Factors influencing the communication skills of first-year clinical medical students. Med Educ, 1991; 25:127–134.

35.

West

, Huntington

, Huschka

, Novotny

, Sloan

, Kolars

, Habermann

, Shanafelt

. A prospective study of the relationship between medical knowledge and professionalism among internal medicine residents. Acad Med, 2007; 82:587–592.

36.

Moore

, Block

, Style

, Mitchell

. The influence of the New Pathway curriculum on Harvard medical students. Acad Med, 1994; 69:983–989.

37.

Easter

, Beach

. Competent patient care is dependent upon attending to empathic opportunities presented during interview sessions. Curr Surg, 2004; 61:313–318.

38.

Beach

, Easter

, Good

, Pigeron

. Disclosing and responding to cancer “fears” during oncology interviews. Soc Sci Med, 2005; 60:893–910.

39.

Hojat

, Mangione

, Nasca

, Rattner

, Erdmann

, Gonnella

, Magee

. An empirical study of decline in empathy in medical school. Med Educ, 2004; 38:934–941.

40.

DiMatteo

, Hays

, Prince

. Relationship of physicians' nonverbal communication skill to patient satisfaction, appointment noncompliance, and physician workload. Health Psychol, 1986; 5:581–594.

41.

Matthews

, Feinstein

. A new instrument for patients' ratings of physician performance in the hospital setting. J Gen Intern Med, 1989; 4:14–22.

42.