Benefits and Challenges in Use of a Standardized Symptom Assessment Instrument in Hospice

Abstract

Background:

Hospices are now mandated to perform routine quality assessment under the final Medicare Hospice Conditions of Participation, creating an opportunity to explore standardized approaches to monitoring hospice quality.

Objective:

We report hospice staff experiences using a standardized symptom assessment instrument, the Edmonton Symptom Assessment System (ESAS), in a pilot study designed to develop and test quality measures on symptom management. Use of the ESAS illustrates the benefits and challenges arising with standardized symptom assessment for quality monitoring in hospice.

Methods:

We interviewed 24 individuals representing 8 hospices involved with the National Association for Home Care & Hospice Quality Assessment Collaborative, which pilot tested the ESAS as a source of standardized data for quality assessment. Transcripts were analyzed using the constant comparative method.

Results:

Participants reported benefits and challenges with the ESAS. Benefits were that the ESAS was a brief and easy tool that identified areas of concern, engaged patients in symptom assessment, and monitored symptom changes over time. Additionally, the ESAS was viewed as a useful teaching tool for less experienced staff. Challenges included lack of clarity about inclusion rules and frequency of assessments; difficulty interpreting the numeric symptom rating scale, difficulty incorporating patient preferences with symptoms, and a sense that the use of standard assessment instruments was “unnatural.”

Discussion:

Recommendations to promote effective use of ESAS data for quality monitoring of hospice care include standardizing implementation procedures, adding patients' preferences to the ESAS form, and staff education to enhance comfort with the instrument before implementation.

Introduction

The Centers for Medicare & Medicaid Services (CMS) have mandated that hospices perform routine quality assessment as of December 2008 under the final Medicare Hospice Conditions of Participation (CoP),¹ creating an opportunity to explore standardized approaches to monitoring hospice quality. Although the CoP does not require the use of standardized quality measures, standardization would enable benchmarking and will likely be considered.

Despite the availability of several symptom assessment instruments validated for end-of-life use,^2–4 little is known about the benefits and challenges of implementing such instruments for standardized quality monitoring in hospice. Staff experiences in doing so are critical to understanding the validity and sustainability of collected data for quality monitoring. We sought to describe the experiences of hospice staff using a standardized symptom assessment instrument, the Edmonton Symptom Assessment System (ESAS),² as part of a pilot study to develop and test quality measures for symptom management. The ESAS is an exemplar of benefits and challenges that may arise with use of standardized symptom assessment in hospice. Increased understanding of the benefits and challenges experienced by staff can contribute to designing and implementing a valid and sustainable approach to standardized quality monitoring in hospice.

Methods

We conducted a qualitative study of hospice staff experiences with the ESAS using data from the National Association for Home Care & Hospice Quality Assessment Collaborative,⁵ the goal of which was to develop and pilot test performance measures and a data collection system that hospices could use in partial fulfillment of CMS requirements.

The ESAS was designed as a clinical tool to provide a profile of symptom severity over time as part of a clinical assessment. The ESAS has been validated for use in palliative and hospice settings⁶ and is suited for such use because it is brief, requires minimal effort and is feasible for daily use. Patients are asked to rate 9 symptoms (pain, fatigue, nausea, depression, anxiety, drowsiness, shortness of breath, appetite, and well-being) on a 0–10 scale (0 = no symptom, 10 = worst possible symptom, except for appetite and wellbeing where 0 = best symptom) for the worst it has been in the last 24 hours.^2,7 We used a modified version of the ESAS that uses the assessment time frame of “now” versus “over the past 24 hours,” and includes the symptom “fatigue” versus “tiredness.”

Sample and procedures

A convenience sample of 9 hospices was invited to participate in the Collaborative between November 2006 and August 2007 based on varying ownership type, size, experience, and location. Hospices engaged in small tests of change related to implementing the ESAS and to collecting data for quality measurement so that their experiences could aid in developing quality measures and tools. Ongoing feedback from staff was used to modify processes for further testing. None of the hospices was using the ESAS previously.

Each hospice appointed a team of 3–5 members to the roles of Team Leader (e.g., CEO, administrator), System Expert (e.g., nursing director, administrator), Clinical Expert (e.g., hospice nurse, other interdisciplinary team member), and Daily Manager (e.g., quality improvement manager, staff nurse). Hospices were given a manual explaining the purpose and procedures relating to the Collaborative, the ESAS, and the Excel-based electronic data collection tool.⁵ Teams were asked to use the ESAS at least weekly with 10–30 patients per month. Per ESAS instructions, staff encouraged patients to complete the ESAS independently; however, a family caregiver and/or a nurse would assist if necessary. Hospices incorporated the ESAS into their symptom assessment processes variously, such as by including the ESAS in admission packets, by testing the ESAS with one nurse and then expanding to additional staff, and by discussing use and results of the ESAS during clinical meetings. All hospices used the standardized electronic data collection tool to enable tracking and reporting of trends.

To facilitate sharing of experiences, conference calls of 1.5–2 hours with Collaborative organizers and hospice Team Leaders occurred every 2–3 weeks. Calls focused on obtaining feedback, troubleshooting, and refining procedures, measures, and the electronic data collection tool. At the end of the project, the Collaborative summarized data for individual hospices and in aggregate for each participating hospice.

Participant interviews

Of the 39 participants, 24 individuals across the 9 hospices were interviewed (response rate of 62%). Of the 15 nonparticipants, 4 reported inadequate involvement, and 1 was no longer at the hospice. Ten were unable to be reached. Among the sample, all but one hospice had at least one staff participating. The average number of staff interviewed per hospice was 2.7.

To learn about staff experiences, we conducted semistructured telephone interviews with participants at the end of the implementation period, during July to August 2007. Interview topics included: (1) how the ESAS was incorporated into the hospice's usual processes; (2) what worked/did not work in implementing the ESAS; and (3) strategies to overcome challenges. We also collected data on participants' hospice position and role in the Collaborative. This study was approved by the Yale School of Medicine Human Investigation Committee.

Data analysis

Transcripts were coded using Atlas.ti software (Scientific Software, Berlin, Germany). We performed line-by-line review of transcripts, first coding independently, then jointly by two investigators using the constant comparative method.^8–10 We reconciled coder discrepancies through review of coded material until consensus was reached. During this process, we identified main coding categories and divided them into subcodes as indicated. The code key was expanded as new themes were identified, and we reapplied the final code key to all transcripts. Data reports for each code were prepared and analyzed to extract main themes, which we then examined and summarized to describe staff experiences. Trustworthiness of findings was enhanced by conducting interviews immediately following the Collaborative, by using a professional transcriptionist, by corroborating findings with Collaborative organizers, and by keeping an audit trail.

Findings

Participant characteristics

Participants included 6 administrators, 6 quality assessment directors/nurses, 3 performance improvement coordinators, 4 team leaders/program supervisors, 2 staff nurses, 2 hospice presidents/chief executives, and 1 chaplain. The 15 nonparticipants did not differ from participants in terms of Collaborative role or hospice characteristics. Hospices assessed 348 patients using the ESAS (range, 3–118). The sample is described in Table 1.

Table 1.

Participant and Hospice Characteristics

Hospice characteristics (n = 9)
Ownership type	n	%
Nonprofit	6	67
For-profit	3	33
Structure
Home health agency-based	3	33
Free-standing	3	33
Hospital-based	1	11
Hospital affiliated	1	11
Corporate chain	1	11
Experience
New (2006)	1	11
Established 1974–2000	8	89
Location
Primarily rural area	2	22
Primarily urban area	3	33
Serve urban and rural areas	4	44
Participant Characteristics (n = 24)
Role in hospice
Administrator	6	25
Quality assessment director/quality assessment nurse	6	25
Team leader/program supervisor	4	17
Performance improvement coordinator	3	13
Staff nurse	2	8
Hospice president/chief executive	2	8
Chaplain	1	4

Benefits

Brief and easy

Participants found the ESAS to be a brief, easy-to-use tool that, when used with clinical notes, helped identify concerns:

… when we got the [ESAS] assessment forms back and then compared them with the notes and the charts, it brought to light some areas that maybe we could look at a little closer. [#13]

Engages patients

Because the ESAS is intended to be completed by the patient, the instrument encouraged nurses to involve patients in symptom assessment rather than focus on family caregiver reports:

[The ESAS] made me think about asking the patients to answer those questions first of all, rather than us always asking the caregiver. [I saw] how important it is for the patient to say, “This is how I feel. This is how I rate my pain.” [#4]

Participants also described how the patient-completed ESAS helped identify clinical issues that patients had difficulty reporting:

A patient that was having depression marked it on his [ESAS] sheet where he couldn't talk about it, and so the nurse said, “You know, it says here on your sheet that you are having some problems with your feelings.” He got treatment for that, but he wouldn't have otherwise because he couldn't verbalize [that he needed treatment]. [#12]

Enables symptom monitoring

Participants liked the ability to monitor patients' progress using ESAS ratings over time and used this information to address symptom distress:

The nurse brings back the sheet from the patients and I fill out the system assessment flow sheet. They are really easy to use and they give you a nice across time view … you can see the change in the symptoms, and if there is a big change in the total distress score, then I discuss it with the nurse and make sure she is aware. [#12]

Useful teaching tool

Staff felt the ESAS was a good teaching tool by providing novice nurses with a simple, standardized way to assess symptoms and by enabling experienced nurses to sharpen their assessment skills. A participant described how she felt the ESAS led to more precise patient assessments:

I think we got our staff used to trying to assess and quantify what they are doing, not subjectively reviewing someone's pain and saying, “Yeah, I think it is this,” and we got people trying to put a more definitive assessment into their visit. [#15]

Challenges

Inclusion criteria/frequency of assessment

Participants struggled with who should be assessed and how frequently, despite standardization of the ESAS and explicit training on these points. For instance, how should the ESAS be used and interpreted with patients who are nonresponsive, have severe dementia or are not well-known to staff? A participant said:

… you could only rate certain things that you were observing or you thought, so in some respects that was a struggle … [one patient was] just sleeping more of the time. Well, if they are comatose of course you are going to write that as a ten [worst symptom level]; that is the expected outcome, but looking at that it makes it look like you are not responding to it [or] like you are not treating it appropriately, so there was a struggle there because those would be expected outcomes. Just like with not eating … if they are comatose there is no way they could eat, yet you would have to rate that as high. [#7]

Participants also struggled with how often to assess patients' symptoms given frequent changes:

We really are often looking at moving targets with our patients … patients during the same visit would give us different answers … and I think that is true always–pain is a moving target. It is constantly changing. Respiratory distress is a moving target. It is always changing, even within an hour visit. So numbers are nice and somewhat useful, but … not as solid a foundation as you might think. [#10]

Numeric rating system

A prominent concern was the numeric rating system. A participant described patients' confusion with the numbers and how nurses sometimes substituted words to elicit ratings:

The numeric scale even for pain is a fairly recent innovation for most people, so we had been getting a lot of “unable to rate,” whereas the words seem to have more meaning to say, “Is it mild, moderate, or severe?” and they were more likely to admit to one of those and then the nurses were more likely to put that in their documentation … so the numbers they put on the [symptoms] were somewhat random. [#5]

Additionally, participants were uncomfortable rating symptoms as severe when symptoms were an expected part of the dying process, or when treating the symptom might exacerbate other symptoms:

… the drowsiness [that] we anticipate with hospice care was very confusing because … in our thinking, if a pain number is high, we have to treat it. Therefore, if a drowsiness number is high, do we have to treat that too? … Do you count normal drowsiness as a ten, or do you count it as a zero since we are not treating it? [#14]

Incorporating patient preferences

Another concern was incorporation of patients' preferences into quality measures on symptom management. Participants grappled with how to address patients' perspectives:

They're eating 25%, but their experience is, “No, my appetite is fine. I don't have a problem, I just have no appetite.” But you know they are not [fine], so the data you get for some of these … populations is not really that useful in terms of assessing the quality of the care and assessing the improvement in symptom management. [#8]

Participants worried that high symptom ratings could be interpreted as poor quality of care rather than as reflective of the patient's preference that the symptom not be addressed:

I think it has raised some concerns for us … in terms of how we measure quality and how quality is externally measured for us, because you have got people whose pain is not going to be managed because they won't take pain medication … [and there is a] concern that we are supposed to fix these things … when in fact it really is part of the patient's end-of-life journey and they are not looking to us to make it better; they are looking to us to support them in their process … [#5]

Some addressed this issue by enabling patients to express their preferences on each symptom rating. For example:

… the patient may be satisfied with a pain level of four because they don't want to have the other symptoms, the drowsiness, being out of it, and that is acceptable to the patient. We added that to our inpatient [form] too, if the patient has a pain level above [a certain number], we had them write in a number, are they satisfied with it, is this where they want their pain so we would be able to answer that question. [#4]

“Unnatural” to use

Participants reported that it felt unnatural to use a paper-and-pencil instrument because patient assessments are usually conducted less formally. Hence, use of the ESAS was seen as potentially interfering with the intimacy of the nurse-patient relationship because symptoms must be brought up and rated in a standard way rather than as part of a conversation. A participant explained:

… when they bring out the tool they are actually doing something formal and written with the patients, which is not their standard practice. Very often they are not even taking notes. Some of them depend on their memories, and some of them will just scribble notes on a pad during a visit, and some of them will do it after the visit … [Using the ESAS] would be a big culture shift for some nurses in terms of how they practice and getting past that discomfort, because if the nurse is uncomfortable, yes, it is going to impact that relationship. [#16]

Themes are summarized in Table 2.

Table 2.

Reported Benefits and Challenges in Use of the Edmonton Symptom Assessment System (ESAS) for Quality Monitoring in Hospice

Benefits	Challenges
Brief and easy-to-use means of identifying patient care issues	Determining inclusion criteria and frequency of assessment
Engages patients in symptom assessment	Interpreting the numeric rating scale
Allows for patient symptom monitoring over time	Incorporating patient preferences with symptoms into quality measures
A useful teaching tool for inexperienced staff	Feeling the ESAS was “unnatural” to use

Discussion

Previous studies of palliative staff members' perceptions of the ESAS have reported that the instrument is quick and easy to use, involves patients in symptom assessment, offers insight into patients' symptoms, and helps staff to care for patients.^11,12 The challenges of assessing certain symptoms among very ill or dying patients, and difficulty interpreting the ESAS instructions and numeric rating system have also been reported.^13,14 Our findings add depth to these results, which were mostly obtained using brief surveys, and identify additional benefits and challenges to use of the ESAS from a quality assessment perspective.

Standardization of implementation procedures is critical to the effective use of ESAS data for quality monitoring. Currently, ESAS instructions⁷ offer standard approaches as to who should complete the ESAS and how often. Despite these instructions, our participants expressed confusion about assessment of certain patients and about the frequency of assessments. As ESAS data may be considered for benchmarking of hospices nationally, consistency in implementation procedures is paramount.

Another important consideration is the integration of patients' preferences into quality measurement. Patient preferences are an important element of symptom management because tolerance of symptoms or of the side effects of medication to treat symptoms is an individual characteristic and choice requiring individualized treatment plans. Participants highlighted the importance of capturing these data. It may be useful to include a check box next to each symptom on the ESAS where patients can signal their concern about a symptom and if it should be addressed.

A major barrier to use of the ESAS for quality monitoring was the feeling that it was unnatural to use a standardized instrument within the nurse–patient relationship. If staff do not feel comfortable using a standardized instrument, they may not use it in a valid, ongoing way. Prior to widespread use of the ESAS, hospice staff should be educated to increase their comfort. Demonstrating how the ESAS can improve professional practice and patients' experience may enhance staff support. It should be emphasized that the ESAS need not be completed in an overly formal way, but can be used as a springboard to deeper discussion of symptoms and of sensitive clinical issues.

Hospice is a complex setting in which to assess symptoms. Patients' symptoms fluctuate over time, particularly in the last week of life,^15,16 and are expected as part of the dying process. The question was raised of what is a “normal” or “acceptable” level of symptom distress in dying patients, and how can quality be assessed? In all stages of hospice care, symptoms distressing to the patient should be treated according to patient preferences; however, in the final stages of life, symptom ratings are more likely to involve proxy assessment, which has been found to be less valid.¹⁷

Our findings should be considered with some limitations in mind. The Collaborative tested the ESAS as part of a pilot study. Therefore, use of the ESAS duplicated symptom assessment processes (e.g., use of the hospice's own symptom rating scale or a widely used industry tool), which may have affected participants' time and willingness to test the ESAS. In addition, not all participants directly administered the ESAS; Daily Managers gathered reports about implementation experiences from staff. Hospices also varied in how they provided ESAS training to fit their organizational needs. We were unable to assess the impact of this variation on staff experience, although we did try to represent recurrent themes across hospices. Finally, although nonparticipants may have added additional insight to our findings, at least 1 individual from 8 of the 9 hospices was interviewed.

As the final Medicare Hospice CoP go into effect, the benefits and challenges that arise with standardized symptom assessment in hospice will continue to be elucidated. This study identified benefits and challenges to use of the ESAS and offers some points for consideration in use of data from standard symptom assessment instruments for national quality monitoring in hospice. A certain amount of standardization in quality assessment is necessary and desirable; however, in the hospice setting, the need for standardization must be balanced with the priority of individualized care.

Footnotes

Acknowledgments

This research was supported by the John D. Thompson Hospice Institute for Education, Training, and Research. Dr. Bradley is supported by the Patrick and Catherine Weldon Donaghue Medical Research Foundation. At the time this research was conducted, Dr. Schulman-Green was supported by the National Cancer Institute (Grant #R01 CA116398, E.H. Bradley, Principal Investigator). She is currently supported by the American Cancer Society (Grant #MRSG-08-292-01-CPPB, D. Schulman-Green, Principal Investigator).

Author Disclosure Statement

No competing financial interests exist.

References

Centers for Medicare & Medicaid Services: Hospice Conditions of Participation Final Rule, 2008. www.cms.hhs.gov/CFCsAndCoPs/05_Hospice.asp. 2009 March 2.

Bruera

, Kuehn

, Miller

, Selmser

, Macmillan

. The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. J Palliat Care, 1991; 7:6–9.

Cleeland

, Mendoza

, Wang

, Chou

, Morrissey

, Engstrom

. Assessing symptom distress in cancer patients: The M.D. Anderson Symptom Inventory. Cancer, 2000; 89:1634–1646.

Portenoy

, Thaler

, Kornblith

, Lepore

, Friedlander-Klar

, Kiyasu

, Sobel

, Coyle

, Kemeny

, Norton

et al. The Memorial Symptom Assessment Scale: An instrument for the evaluation of symptom prevalence, characteristics and distress. Eur J Cancer, 1994; 30A:1326–1336.

Pace

. National Association for Home Care & Hospice Quality Assessment and Performance Improvement Program Development Collaborative, Manual on Implementing a Set of Hospice Quality Measures, 2007. www.nahc.org/facts/. 2009 May 11.

Nekolaichuk

, Watanabe

, Beaumont

. The Edmonton Symptom Assessment System: A 15-year retrospective review of validation studies (1991–2006) J Palliat Med, 2008; 22:111–122.

Regional Palliative Care Program in Edmonton, AC: Guidelines for using the Edmonton Symptom Assessment System (ESAS) Regional Palliative Care Program in Edmonton: Alberta, Canada www.palliative.org/PC/ClinicalInfo/AssessmentTools/AssessmentToolsIDX.html. 2009 May 11.

Bradley

, Curry

, Devers

. Qualitative data analysis for health services research: Developing taxonomy, themes and theory. Health Serv Res, 2007; 42:1758–1772.

Glaser

, Strauss

. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine, 1967.

10.

Miles

, Huberman

. Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: Sage Publications, 1994.

11.

Watanabe

, McKinnon

, MacMillan

, Hanson

. Palliative care nurses' perceptions of the Edmonton System Assessment Scale: A pilot survey. Int J Palliat Nurs, 2006; 12:111–114.

12.

Baba

, Fransson

, Lindh

. Use of a modified ESAS in cancer patients: A pilot study of patient and staff experiences. Int J Palliat Nurs, 2007; 13:610–616.

13.

Rees

, Hardy

, Ling

, Broadley

, A'Hern

. The use of the Edmonton Symptom Assessment Scale (ESAS) within a palliative care unit in the UK. J Palliat Med, 1998; 12:75–82.

14.

Watanabe

, Nekolaichuk

, Beaumont

, Mawani

. The Edmonton Symptom Assessment System: What do patients think? Support Care Cancer, 2009; 17:675–683.

15.

Kutner

, Bryant

, Beaty

, Fairclough

. Time course and characteristics of symptom distress and quality of life at the end of life. J Pain Symptom Manage, 2007; 34:227–236.

16.

Tranmer

, Heyland

, Dudgeon

, Groll

, Squires-Graham

, Coulson

. Measuring the symptom experience of seriously ill cancer and noncancer hospitalized patients near the end of life with the Memorial Symptom Assessment Scale. J Pain Symptom Manage, 2003; 25:420–429.

17.

Tang

, McCorkle

. Use of family proxies in quality of life research for cancer patients at the end of life: A literature review. Cancer Invest, 2002; 20:1086–1104.