Abstract
Background:
Serious illness conversations are part of advance care planning (ACP) and focus on prognosis, values, and goals in patients who are seriously ill. To be maximally effective, such conversations must be documented accurately and be easily accessible.
Objectives:
The two coprimary objectives of the study were to assess concordance between written documentation and recorded audiotaped conversations, and to evaluate adherence to the Serious Illness Conversation Guide questions.
Methods:
Data were obtained as part of a trial in patients with advanced cancer. Clinicians were trained to use a guide to conduct and document serious illness conversations. Conversations were audiotaped. Two researchers independently compared audiorecordings with the corresponding documentation in an electronic health record (EHR) template and free-text progress notes, and rated the degree of concordance and adherence.
Results:
We reviewed a total of 25 audiorecordings. Clinicians addressed 87% of the conversation guide elements. Prognosis was discussed least frequently, only in 55% of the patients who wanted that information. Documentation was fully concordant with the conversation 43% of the time. Concordance was best when documenting family matters and goals, and least frequently concordant when documenting prognostic communication. Most conversations (64%) were documented in the template, a minority (28%) only in progress notes and two conversations (8%) were not documented. Concordance was better when the template was used (62% vs. 28%).
Conclusion:
Clinicians adhered well to the conversation guide. However, key information elicited was documented and fully concordant less than half the time. Greater concordance was observed when clinicians used a prespecified template. The combined use of a guide and EHR template holds promise for ACP conversations.
Introduction
Serious illness care planning, a core element of advance care planning (ACP), can help patients with a serious illness to receive care consistent with their wishes and priorities.1–3 Recent research has demonstrated that timely, high-quality communication between patients and clinicians is associated with improved quality of life, earlier use of hospice services, and improved family bereavement outcomes.4–8 In our fragmented health care system, the electronic health record (EHR) is an essential communication vehicle for conveying patients' preferences to the many clinicians providing care across different settings. In an emergency, both clear and accurate, as well as accessible ACP documentation can make a critical difference in care quality. 9 However, research has shown that clinicians often struggle to find key components of advance care plans in the EHR. 10 A growing number of institutions have thus initiated quality-improvement projects to improve ACP documentation in their EHR. 11 These include automated prompts, electronic order sets, or templates specifically designed to document key elements of ACP, including patient values and individual goals. 11
The extent to which clinicians address the key components of high-quality ACP conversations and document them accurately largely remains unexplored. We used audiorecordings of clinician-led conversations using the Serious Illness Conversation Guide (SICG), 12 an evidence-based structure for conversations about goals, values, and preference, to evaluate adherence to recommended elements, and then compared documentation of conversations using the SICG with the corresponding clinician documentation in the EHR, both in a template and in free-text progress notes.12,13 The goals of this study were as follows: (1) to assess clinician adherence to the conversation guide, and (2) to compare the concordance between the substance of the audiorecorded conversation and the documentation in the EHR.
Methods
Trial design and setting
We conducted a secondary analysis of data obtained through a cluster-randomized controlled trial in oncology of a communication quality-improvement intervention6–8 : the Serious Illness Care Program. 12 The current analyses represent a set of intervention group conversations that were audiorecorded. In a subset of patients from the larger study, audiorecordings of conversations using the SICG, conducted by oncology clinicians, were compared with conversation documentation. All clinicians and patients included in this secondary data analysis were part of the intervention arm. In addition, two audiorecorded conversations from a pilot study conducted among patients with metastatic melanoma were included.
The trial was conducted in the outpatient oncology clinics at Dana-Farber Cancer Institute (DFCI) and approved by the Institutional Review Board. The goal of this intervention was to support the patient's primary oncology clinician in conducting ACP conversations with advanced cancer outpatients, using a set of tools, training, and systems change interventions. 12 Tools included the SICG (outlined in Supplementary Fig. S1) as well as supporting documents to help patients both prepare for and continue ACP conversations at home.
The SICG contains eight conversational components: illness understanding, information preferences, prognostic communication, goals, fears/worries, function, tradeoffs, and family. The version of the SICG used in this study included a prompt to share a time-based prognosis with patients as a range. The training itself included broader information regarding prognostic communication. All intervention-arm clinicians (physicians, nurse practitioners, or physician assistants) received a 2.5-hour skills-based training to use this conversation guide. Systems change interventions included systematic patient identification, reminders to conduct conversations, and an EHR template to support documentation of all elements of the SICG.
The EHR template
The EHR template consisted of dropdown menus and free-text options for each component to support and structure clinician documentation of responses to the SICG (Table 1). The template served as a mechanism to document patients' responses and was not a medical order. The dropdown options were selected through expert review of observed and audiotaped conversations, free-form documentation and patient input followed by an iterative revision.
Serious Illness Conversation Guide and Dropdowns in the Electronic Health Record Template
EHR, electronic health record; SICG, Serious Illness Conversation Guide.
Study sample
All serious illness conversations between clinicians (n = 48) and patients (n = 134) assigned to the intervention arm of the trial were eligible for audiorecording. Study staff initially approached one out of every four clinician–patient dyads. However, because the response rate was low using this approach, the protocol was changed to allow investigators to approach every dyad. Clinicians and patients were asked directly after randomization whether their conversation could be audiorecorded, and informed consent was obtained again before the conversation. Patients with metastatic melanoma and their clinicians were also eligible for audiorecording but were excluded from the larger trial because they served as a pilot site. Since these clinicians had received the same skills-based training as the remainder of the intervention group, we included them in this analysis.
Since clinicians assigned to the control arm did not receive specific instructions on how or where to document, and because their conversations were not audiorecorded, we did not include their EHR data in this study. Twenty-five analyzable ACP conversations, conducted by 16 clinicians, were obtained. Clinical and demographic characteristics of this census sample of clinicians and patients were compared with the rest of the clinicians in the intervention arm of the trial (Table 2). These conversations were transcribed verbatim and deidentified.
Clinician Sample Compared with the Remainder Overall Study Sample of Clinicians
The remainder overall study sample included both intervention and control clinicians, and does not include clinicians caring for patients with melanoma since these patients and clinicians served as a pilot site.
CI, confidence interval.
Analysis
To ensure trustworthiness of our categorizations, two researchers (D.J.L. and O.P.G.) independently read each transcript and subsequently all corresponding clinician documentation. The researchers resolved coding differences by consensus, or when necessary, with the assistance of a third investigator blinded to their responses (S.D.B.). Because the clinicians had been trained to follow a structured guide, we identified all conversation components in the SICG and systematically recorded which topics in the SICG were not discussed. The category “Not discussed” was used to identify elements of the conversation guide that were skipped entirely and provide an indicator of adherence to the conversation guide. Of those topics that were discussed, researchers then rated the concordance of the documentation compared with the audiorecorded conversation by answering the question: Does the documentation accurately reflect the information that is elicited during the conversation?
Using this prompt, researchers rated the documented response to each of the eight components as either one of the following four categories: Concordant (information is present and accurate); Partially concordant (information is present but incomplete); Not concordant (information is elicited but inaccurate information is documented, or information is documented but not discussed); and Not documented (information elicited, but no information documented). Conversational elements that arose in response to the SICG and were discussed but not documented were classified as Not documented.
These responses were used to categorize all available information and assess overall concordance across all SICG components. Data elements were determined to be accurate if they documented the themes and statements patients made in response to the related component of the SICG. Overall concordance was calculated using the total number of conversational components that were discussed across all conversations (e.g., had they all been discussed, the eight conversational components across 25 conversations would have led to 200 potential components across all conversations). Last, conversation-documentation concordance was examined separately in the template and the progress note.
Results
Clinician population and documentation
Table 2 displays the characteristics of the 16 clinicians included in our sample compared with clinicians in the remaining study sample. Of note, half of the clinicians were female, and their average number of years in clinical practice was 13.8 (95% confidence interval: 7.0–20.7). All characteristics were comparable with the exception of practice site since clinicians caring for patients with metastatic melanoma were also eligible for inclusion in this study. These clinicians were similar to the clinicians caring for patients from different practice sites (data not shown).
In the 25 sets of audiorecorded conversations and corresponding documentation, clinicians used the template across 16 conversations (64%) and documented using the progress note in 7 conversations (28%). We did not identify any documentation in either the progress note or the template for the remaining two conversations (8%). Thus, documentation was only available on 23 conversations. Examples of template and progress note documentation are presented in Table 3 along with the component of the ACP conversation and rated concordance.
Examples of Transcript and Progress Note Documentation Including Rating of Concordance
Adherence to the conversation guide
Overall, clinicians addressed 87% of the eight conversation guide components. Eleven clinicians discussed at least seven of eight components of the SICG. The most commonly addressed were illness understanding and information preferences, which were discussed in all conversations. Preferences for family involvement in decision making were discussed in all but one conversation. All other components—with the exception of prognosis—were discussed in 80–100% of conversations.
Thirteen percent of conversation components were not discussed. Sixty percent (15/25 conversations) of the conversations did not address prognosis. When asked about information preferences, 5 of the 25 patients did not want to know their prognosis, and two said they only wanted the information their clinician deemed necessary. For the remaining 18 who wanted all information, prognosis was discussed with 10 of them (55%). Rates of nondiscussion of other conversation elements varied between 0% and 20% (illness understanding and goals, respectively).
Overall concordance
Figure 1 graphically displays the overall level of concordance of documentation in relation to audiorecorded conversation content. Overall, 43% of the information in all ACP conversations was fully concordant as documented in either a progress note or the template, 10% of all information was partially concordant, and 11% was not concordant. The remaining information (36%) was not documented.
Graphical display of overall level of concordance for all conversations, conversations documented in the template, and conversations documented in the progress note.
Figure 2 provides a graphical display of each element of the SICG and the concordance rating. For clarity, concordance ratings for all conversations, conversations documented in the template (n = 16), and conversations documented in the progress note (n = 7) are displayed separately. Overall, components for which we observed the highest rates of concordance centered around family (15/24 conversations in which family was discussed; 60%) and goals (12/20 conversations; also 60%). We observed the highest partial concordance rate in relation to illness understanding (7/25 conversations; 28%) and information preferences (4/25 conversations; 16%).
Graphical display of conversation-documentation concordance for each of the elements of the Serious Illness Conversation Guide.
Nonconcordant documentation
The highest rate of nonconcordant documentation was found for prognostic communication (3/10 conversations; 30%) and illness understanding (4/25 conversations; 16%). Of the information rated as nonconcordant (11%) in all available documentation, we identified two conversations in which inaccurate documentation could potentially lead to patient harm. Both of these examples were in template notes and involved dropdown menus. In both of these instances, information not articulated during the conversation by the patient was included in the template documentation. First, based on the audiorecording, a patient had described his goals as follows: “To live as long as possible, as long as I am comfortable.” The clinician documentation, as selected from dropdown options, reflected the patient's goals as follows: “To be physically comfortable, not to be a burden, be independent.”
Second, one patient described that he would be “Willing to fight” as long as he was not in pain. The clinician documented, again from the dropdown, that the patient “Does not want to undergo aggressive tests and/or procedures.” In both of these cases, the documentation diverged significantly from what the patient had expressed, and could lead to a limitation on life-sustaining treatment not aligned with the patient's expressed wishes. We did not observe similar inaccuracies in the documentation using the free-text progress notes or any other instances in which erroneous (not concordant) information was included in the documentation.
Not documented
Of the conversational information rated as not documented (32%), conversation elements most frequently discussed but not documented primarily pertained to function (11/24 conversations; 46%), fears/worries, and tradeoffs (both 10/23 conversations; 43%).
Documentation in the template versus progress note
Overall, documentation in the progress note alone (n = 7) accorded with conversation content less frequently (13%) compared with documentation in the template (62%). Using the template, documentation regarding family was most commonly fully concordant (13/16 conversations; 81%) followed by documentation regarding information preferences and goals (each fully concordant in 11/16 conversations; 69%). In contrast, documentation about illness understanding was fully concordant in only 5/16 conversations (31%). Documentation about goals, tradeoffs, and prognostic communication was most frequently rated as not concordant. Clinician documentation when using a progress note was concordant much less frequently. In fact, documentation of elicited information was absent for a majority (64%) of key elements discussed.
Discussion
The use of an evidence-based guide and the associated training program led to a high level (87%) of adherence to the key elements of an ACP conversation in the setting of serious illness. This study, which represents the first reported evaluation of EHR documentation concordance with information elicited in audiorecorded ACP conversations, demonstrated full concordance of conversations and documentation in ∼43% of our sample. Significant, clinically meaningful information elicited by clinicians in these conversations was often not fully documented in the EHR, most commonly with regard to illness understanding, fears/worries, and tradeoffs. These findings demonstrate that the use of the SICG supports clinicians in eliciting a high proportion of critical information from patients; however, in spite of a streamlined system for recording information in the EHR, less than half of this important information was recorded anywhere in the EHR.
Although this study used a multicomponent intervention to improve performance, including a set of questions to structure conversations, a clinician training program that emphasized the importance of discussing all the elements of the SICG, information about the value of documenting conversations in the EHR, and a documentation template to structure and streamline the documentation process, 12 numerous limitations were found. Gaps were identified in both addressing key elements of a high-quality, serious illness conversation (e.g., prognostic disclosure) and documenting key information elicited during the conversation.
Overall, clinicians demonstrated a high degree of adherence to the SICG, with the majority including seven of the eight possible conversation elements in their serious illness conversations. By far the most frequently omitted element was prognostic disclosure; only 55% of conversations addressed prognosis, even among patients who explicitly expressed their desire for this information. Clinician discomfort with prognostic disclosures and concerns about “taking away hope” are likely to contribute to avoidance of disclosure.14–16 Although no data are currently available to show that prognostic disclosure improves outcomes, studies have shown that understanding the possibility of a limited prognosis is associated with patient preferences for less aggressive care.17,18 This suggests that enhanced training of clinicians to sensitively discuss prognosis in patients could result in clinician behavior changes that promote more patient-centered care.
The extent to which each conversation element contributes to patient outcomes is unknown, and more research is needed to answer this question. However, all components included in the SICG were developed and tested with input from patients and endorsed by the High Value Task Force of the American College of Physicians. 3 In addition, the SICG has been adopted by thousands of clinicians nationally and internationally, and has been used by large number of patients, with high levels of satisfaction, 8 demonstrating strong face validity.
In our study, high rates of concordance (meaning the topic was discussed and accurately documented) around family matters and goals of care were observed. In addition, high rates of concordance were observed when information detailed on illness understanding and information preferences was discussed. Prognosis was frequently not discussed and not documented in the observed conservations. Discussions revolving around function, fears/worries, and tradeoffs were frequently made but not always documented. Although some might question the importance of documenting all components of an ACP conversation, documenting a patient's (mis)understanding of prognosis or expected illness trajectory may lead clinicians to further explore these issues and implement changes in their care plan, including further information sharing (including addressing language and cognitive barriers) or psychological interventions to address distress that may be contributing to avoidance or denial of undesirable reality.19,20 The high frequency of missing or erroneous documentation raises significant questions about medical decisions made based on documented ACP preferences that are documented by clinicians and not confirmed in the moment. 21
Time pressures, documentation requirements, and negative attitudes regarding the EHR are widely recognized and likely contribute to poor documentation by clinicians.22,23 Since many other routine and important informational elements (e.g., documentation of allergies) are documented much more consistently and accurately, incomplete documentation of end-of-life preferences is likely to be a reflection of attitudes within the culture of medicine that devalue the importance of such planning and information sharing. 24 Attention to attitudes within medical culture toward serious illness care planning is needed, including interventions to address clinician discomfort with serious illness conversations. A template for taking notes throughout the conversation might also support easier and more accurate documentation. In addition, to enhance fidelity to patient values, we suggest that it could be useful to have the patient and physician contribute to the template note, and then, as desired by the patient, share information with the family. Research is needed to evaluate these strategies.
Even when clinicians did document in the EHR, only two thirds of the conversation documentation was in a structured, easily accessible source in the EHR, using a template, where it would be readily retrievable, and thus available during a potential medical crisis. Use of a “single source of truth” in the EHR, including a template, has been recommended as a way of making documentation more accessible at the point of care, and helping to assure inclusion of critical information in medical decision making. 25
Our study did show that the use of a template was associated with a higher rate of concordance than documentation in a progress note. Because a template has discrete fields for each question and answer, it might serve as a trigger for a clinician's memory about the specifics of a conversation or make the documentation demands easier. Indeed, such templates have proven useful when it comes to improving documentation in other areas of medicine, for example in the documentation of obesity in primary care clinics. 26 However, the template, which included free text and dropdowns, may also have drawbacks. First, the imprecision of prespecified dropdowns may lead clinicians to include erroneous information, which could pose a danger to patients.
In our small sample, we observed two instances in which inaccurate (not concordant) documentation in the template could potentially lead to patient harm. In both of these instances, the treating clinician selected a dropdown option that did not accurately represent what the patient said during the encounter. In addition, questions about nuanced issues such as illness understanding and tradeoffs are difficult to capture in a dropdown menu, and critical information may be lost with the use of such menus. For example, the question regarding how much a patient is willing to go through for the sake of more time has the potential to inform decision making about how the patient sees the risks and benefits of future care options. This often subtle and nuanced information about patient values, goals, and preferences can be critical to patient-centered decision making, yet is not amenable to the use of a dropdown. While inaccurate documentation represents one potential hazard, absent or incomplete documentation, which we observed more frequently than erroneous (not concordant) documentation, represents potential safety concerns as well, and could contribute to provision of goal-discordant care.
More studies are needed to better understand why the strategies used in this intervention were only partially effective. Such studies could also elucidate other barriers that led to incomplete use of the SICG and poor documentation of such conversations. If confirmed by larger studies, our findings add to serious concerns raised in multiple other studies about documenting ACP conversations and about the ACP process more broadly.6,22,23,25,27
We note multiple limitations in this study. This is a very small sample drawn from a single institution. Both patients and clinicians declined to participate in this part of the study at high rates. The sample thus represented a small subset of overall study participants, and was prone to self-selection bias among both patients and clinicians, as patients who participated may have been less anxious about this conversation, and clinicians who agreed to have their conversations audiorecorded might represent good communicators with the best documentation practices. This study is also limited to patients with cancer and thus cannot be generalized to other populations. Our comparison of template and free-text notes is very small, yet provides hypotheses for future study. Our analysis also has potential for analytical bias, with possible documentation misclassification. Yet, we worked to address this through the use of independent ratings and verification through a third, independent researcher when necessary. Larger studies, using carefully designed and well-validated measures of conversation and documentation quality, are clearly needed. Finally, although the number of conversation elements not discussed is briefly described in this analysis, it is worth noting that this study is not aimed to offer a rigorous analysis of adherence to the intervention but instead aimed to describe practices surrounding documentation.
Overall, while our study raises numerous caveats about the overall process of ACP, in terms of both quality of conversations and accurate documentation, our study offers positive findings about clinician adherence to the SICG, a structured approach that is designed to improve the quality of conversations. Our data also suggest that significant opportunities remain to improve the quality of ACP conversations and the concordance between conversations and documentation, even in a context in which tools, training, and systems support are in place. Further research is needed to better understand how to overcome clinician failure to initiate prognostic disclosures, and to enhance documentation about specific key areas such as illness understanding, tradeoffs, and fears/worries that provide key information for subsequent clinical decision making. Finally, and importantly, research is needed to evaluate whether improved conversations and documentation result in better care for seriously ill patients.
Footnotes
Acknowledgments
We acknowledge Grant Smith, Judith Vick, and Mathilde Hutchings for data collection. We are also deeply grateful to all patients, caregivers, and clinicians who provided their support to this study.
Author Disclosure Statement
No competing financial interests exist.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
