The Disparity between Patient and Nurse Symptom Rating in a Hospice Population

Abstract

Background:

Meticulous assessment, monitoring, and treatment of symptoms are key components of palliative care. The Symptom Assessment Scale (SAS) is a 0–10 numerical rating scale (NRS) used for self-report of sleep, appetite, nausea, bowels, breathing, fatigue, and pain. Ideally, symptom rating should be by self-report; however, the ratings are often completed by nurses. The aim of this study was to examine the relationship between simultaneous symptom ratings of patients and nurses in an Australian inpatient palliative care unit.

Methods:

All inpatients and their treating nurse (blinded to the patient's report) scored the SAS weekly. Nurse symptom rating was considered accurate if the score for an individual symptom differed by ≤1. The relationship for each dyad was examined by comparing absolute values, mean differences, and Pearson correlations.

Results:

Sixty-four pairs of symptom ratings were obtained from 29 patients (mean 65 years, male 55%, malignancy 90%, mean Karnofsky 50). The absolute differences were closest for nausea, bowels, and breathing domains, and worst for appetite. Mean differences and Pearson correlations did not adequately reflect the disparity between patient and nurse symptom rating, and nurses systematically underreported symptoms. The total difference for the seven items ranged from nurses overreporting the patients' symptoms by 39 points to underreporting by 31 out of a possible 70 points.

Discussion:

Proxy symptom assessment only modestly correlates with patient symptom rating, can significantly misrepresent the patient's actual symptom burden, and should only be used when the patient is absolutely unable to contribute his or her own view.

Introduction

Across health care there is increasing emphasis on patient reported outcomes as a means to drive improved care.¹ Responsiveness of health care services to unmet needs will only occur with systematic and adequate appraisal of the issues from the perspective of the patient.

Palliative care patients have a high burden of symptoms, which increase as death approaches.^2–5 Underestimation of symptoms may be associated with poor symptom control, reduced function and quality-of-life, and impaired survival. Conversely, overestimation of symptoms may lead to inappropriate interventions that create iatrogenic morbidity or even premature mortality. Meticulous assessment, monitoring, and treatment of symptoms are central tenets of palliative care. It is only through careful assessment of symptoms that appropriate investigations and treatment can be instigated, response assessed, and modifications made as needed.

Barriers to accurate assessment of symptoms include performance of the tools used in assessing symptoms, expertise of the treating health professional in responding to uncontrolled symptoms, perceived burden of symptom assessment on the patient, and, especially in palliative care, the deteriorating patient who is unable to reliably rate or communicate his or her symptoms.

The SAS is a validated 0–10 NRS used for patients' self-report of symptoms in seven domains – sleep, appetite, nausea, bowels, breathing, fatigue, and pain – with good internal consistency and intra-rater reliability.^6,7 The SAS is used extensively across Australia as part of the national Palliative Care Outcomes Collaboration.⁸ In the SAS, symptom rating is intended to be by self-report; however, in practice, the ratings are often completed by a proxy, usually the treating nurses.

The aim of the study was to examine the correlation between simultaneous symptom ratings by patients and their treating specialist nurse in an Australian inpatient palliative care unit. The null hypothesis was that there was no difference in the contemporaneous symptoms ratings of patients and their specialist palliative care nurses.

Methods

Study setting

Australian health care entails provision of universal insurance that can be supplemented by private insurance and patient copayments. Southern Adelaide Palliative Services is a specialist metropolitan palliative care program that provides inpatient care, community and outpatient visits, nursing home and hospital consultations, complementary care, and bereavement services. It serves a population of 350,000 people over an area of more than 750 km² in both public and private sectors. The service receives more than 1300 referrals per year.

The inpatient palliative care unit provides direct specialist care and has approximately 330 admissions of 280 patients annually. Patients may be admitted to the unit for symptom management, terminal care, or caregiver respite. The unit has 15 beds and is staffed by 30 nurses (with a ratio of 7.22 nursing hours per patient day).

Study design and participants

All inpatients of the hospice were reviewed on a weekly basis over a three-month period from December 2009 and were considered eligible for the study. Therefore, 195 potential episodes (15 beds over 13 weeks) were available assuming 100% bed usage. Patients were excluded if they were unable to self-rate their symptoms through incapacity, or if they declined to participate.

Data collection

Patient demographics (age and gender); Australia-modified Karnofsky Performance Status;⁹ phase of illness (stable, unstable, deteriorating, and terminal);¹⁰ and diagnosis were obtained from routine documentation in the medical record. Patients were asked to self-rate their symptoms using the SAS. The patient's treating nurse, blinded to the response of the patient, was asked to rate the patient's symptoms simultaneously. All nurses on the unit had been formally educated in the administration and use of SAS and routinely record the SAS for each patient each shift, although this often means completing it on behalf of the patient with variable levels of consultation with the patient about specific ratings. Data from the first assessment and subsequent assessments are analyzed separately given that, theoretically, correlation may be expected to improve the longer a nurse has had an opportunity to care for a specific patient. All patients who were unable to participate were also noted, along with the reason for nonparticipation.

Data analysis

Differences in individual symptom rating and the sum total symptom rating in all seven symptom domains were calculated by subtracting nurse rated from patient rated scores. Absolute differences in individual symptom rating, or concordance, were sorted into three groups: (1) a difference of 0 to 1; (2) a difference of 2 to 5; and (3) a difference of greater than 5. Adequate concordance of the nurse rating of individual symptoms was defined to be in the first group. Mean difference in symptom rating was calculated by averaging the difference for an individual symptom.

Data were analyzed in PWAS 18.0 (SPSS Corp., Chicago, IL). Means were compared using independent t tests. Categorical variables were compared using Chi-square test or Fisher's exact test as appropriate. Pearson's correlation coefficient (r) was calculated between patient and nurse symptom rating.

Ethical permission for the study was granted by the Flinders Clinical Research Ethics Committee, Bedford Park, South Australia, a National Health and Medical Research compliant committee.

Results

One hundred and fifty-one episodes were documented for 55 patients; however only 64 paired observations were available from 29 patients, with 87 encounters not being assessed.

Twenty-one nurses were involved in the study. Of these, 83% had been nursing for over 10 years (range 4–34 years) with 37% in palliative care for over 10 years (range .5–26 years).

The characteristics of the patients who were and were not assessed, and reasons for each episode not being assessed, are outlined in Tables 1 and 2. The only significant difference identified between the two groups was a poorer performance status in the group that was not assessed.

Table 1.

Patient Characteristics

		Eligible (n=55)	Assessed patients (n=29)	Not assessed patients (n=45)
Age (range)		69.3 (44–91)	64.8 (44–88)	69.8 (45–91)	p=0.08
Male		53%	55%	51%	p=0.73
Mean AKPS^*		43%	49%	41%	p=0.04
Diagnosis of cancer		91%	90%	91%	p=0.81
Phase^*	1	45%	79%	37%
	2	22%	10%	26%
	3	18%	10%	21%
	4	14%	0%	16%	p=0.36
Median days since admission^*			6.0
Avg number of assessments			2.2

NB. There are a number of patients who were assessed at one point, but also declined on one or more other occasions. They are considered separately in the assessed and not assessed groups.

At first assessment.

AKPS, Australia-modified Karnofsky Performance Status.

Table 2.

Reasons for an Episode Not Being Assessed

Physical incapacity (unwell/lethargic/drowsy)	39	45%
Mental incapacity (confused/delirium)	12	14%
Declined/refused	10	12%
Not available (procedure/appointment/leave)	9	10%
Other	7	8%
Unknown	10	12%
TOTAL EPISODES	87	100%

The symptom ratings at the first and subsequent episodes, and the range of differences for each symptom, are outlined in Table 3. Of note, the symptoms with the smallest range of differences in rating between patients and nurses were bowel function and breathing.

Table 3.

Comparison Between Patient and Nurse Symptom Rating – First and Subsequent Episodes

					Correlation		Concordance
First episode (n=29)	Mean patient rating	Mean difference	Median difference	Range	r	p	+/− 1	+/− 2–5	+/− >5
Sleep	4.3	−0.03	0	−7 to 7	0.39	0.04	41%	48%	10%
Appetite	3.9	1.07	1.0	−6 to 9	0.03	0.87	14%	66%	21%
Nausea	2.0	0.66	0	−8 to 8	−0.05	0.81	38%	45%	17%
Bowel	3.3	0.93	0	−4 to 6	0.51	0.01	52%	38%	10%
Breathing	2.8	0.17	0	−5 to 5	0.68	<0.01	45%	55%	0%
Fatigue	5.1	−0.24	0	−7 to 8	0.10	0.62	45%	35%	21%
Pain	4.7	0.38	0	−7 to 6	0.44	0.02	38%	52%	10%
							+/− 7	+/− 8–14	+/− >14
Total	26.1	2.93	3.0	−39 to 28	0.25	0.19	45%	17%	38%
Subsequent episodes (n=35)
Sleep	4.2	1.23	1.0	−3 to 9	0.51	<0.01	46%	49%	6%
Appetite	2.9	0.77	1.0	−4 to8	0.29	0.09	40%	57%	3%
Nausea	1.5	0.51	0	−3 to 7	0.56	<0.01	69%	29%	3%
Bowel	2.5	0.89	0	−5 to 6	0.43	0.01	54%	40%	6%
Breathing	2.0	−0.03	0	−4 to 5	0.74	<0.01	60%	40%	0%
Fatigue	5.7	1.63	1.0	−7 to 9	0.18	0.30	43%	43%	14%
Pain	5.2	1.29	1.0	−4 to 8	0.41	0.02	43%	51%	6%
							+/− 7	+/− 8–14	+/− >14
Total	24.0	6.29	8.0	−21 to 31	0.47	<0.01	34%	37%	29%

Comparison on first episode

Adequate concordance between the patient and nurse rated symptoms ranged widely from appetite (14%) to bowel function (52%) (Table 3). Large differences (+/− >5) were seen in 21% of patients in the appetite and fatigue domains. The best correlations were for pain, bowel function, and breathing, and the worst for nausea, appetite, and fatigue. The correlation of the sum total of all seven symptom domains was poor (r=0.25). The mean difference for symptoms ranged from sleep (−0.03) to appetite (1.07); however, the range of differences was large for each symptom.

Comparison for subsequent episodes

Rates of adequate concordance for subsequent episodes were highest for nausea (56%) and breathing (74%), with the lowest rates of 40% for appetite and 43% for fatigue and pain (Table 3). The rates of large differences (+/− >5) were low, ranging from breathing (0%) to fatigue (14%). Correlations were highest for breathing and nausea, and worst for appetite. Correlation for the sum total of all seven symptom domains was moderate (r=0.47). The mean difference between patient and nurse rating of symptoms ranged from −0.03 to 1.63, however the range of differences remained high. The lower limit of the range of differences also increased, suggesting nurses were less likely to overestimate symptoms, but were still likely to underestimate.

First episode versus subsequent episodes

Generally, the rate of adequate concordance between patient and nurse rated symptoms improved with repeated assessments, particularly for appetite and nausea, with concomitant decrease in large differences (+/− >5) in symptom ratings (Table 3). Correlations for appetite and nausea were significantly improved with subsequent assessment (p=0.01 and 0.03, respectively). However the mean difference between patient and nurse rated symptoms did not change significantly, except for an increase in difference for fatigue (p=0.03).

All episodes

Considering all episodes, adequate concordance was highest for nausea, bowel function, and breathing, with the lowest rate for appetite (Table 4). The large differences (+/− >5) in symptom ratings were in fatigue and appetite. Correlation was strongest for breathing and worst for fatigue. While the mean differences between patient and nurse rated symptoms remained small, the range of differences remained large. Overall, the difference in sum total of all seven symptom domains ranged from nurses overreporting the patients' symptoms by 39 points to underreporting by 31 points.

Table 4.

Comparison Between Patient and Nurse Symptom Rating – All Episodes

					Correlation		Concordance
All episodes (n=64)	Mean patient rating	Mean difference	Median difference	Range	r	p	+/− 1	+/− 2–5	+/− >5
Sleep	4.2	0.66	0	−7 to 9	0.43	<0.01	44%	48%	8%
Appetite	3.3	0.91	1.0	−6 to 9	0.17	0.18	28%	61%	11%
Nausea	1.8	0.58	0	−8 to 8	0.23	0.07	55%	36%	9%
Bowel	2.8	0.91	0	−5 to 6	0.48	<0.01	53%	39%	8%
Breathing	2.4	0.06	0	−5 to 5	0.71	<0.01	53%	47%	0%
Fatigue	5.4	0.78	1.0	−7 to 9	0.11	0.40	44%	39%	17%
Pain	5.0	0.88	1.0	−7 to 8	0.42	<0.01	41%	52%	8%
							+/− 7	+/− 8–14	+/− >14
Total	24.9	4.77	6.0	−39 to 31	0.37	<0.01	39%	28%	33%

Discussion

This study shows significant discrepancy between patient and nurse rated symptoms in routine data collection for frequently encountered symptoms in palliative care. There were poor levels of adequate concordance if clinical decisions were to be based on these ratings. While the mean differences between patient and nurse rated symptoms appear small, the range of differences and poor correlation mean that on the individual level there is a significant proportion of proxy assessments that poorly reflect patients' symptom severity. With repeated assessment, the rates of adequate concordance improved marginally, but to levels that are still not adequate to substitute proxy assessment in the majority of domains.

While this is not the first study to examine the role of proxy assessment in the rating of patient symptoms, no other study has examined this with the SAS, which is increasingly used in Australia as part of the national outcomes collaborative dataset (NOCD).⁸ Therefore it is imperative to understand the implications of using the SAS by proxy. Inaccurate symptom assessment has the potential to lead to inappropriate interventions, which may be associated with significant morbidity, or even mortality. For instance, poor pain assessment may lead to overtreatment with inappropriately escalating doses of analgesia with subsequent sedation, immobility, and dependency related complications. Similarly, undertreatment of pain may lead to decreased mobility, function, and dependency related complications.

The symptom domains of appetite, nausea, and fatigue were the least accurate for proxy rating, in contrast to bowel function and breathing. This may reflect the difficulty in assessing the severity and associated distress of subjective and nonvisible symptoms of appetite, nausea, and fatigue. This is in contrast to bowel function which generally has an objective endpoint such as bowel opening, and breathing which often has visible signs of labored breathing or requirement for supplemental oxygen; still, nurse observation may not reflect the full subjective experience of the patient.

Symptom assessment tools are useful to systematically assess patient symptoms, but they are not without their limitations as demonstrated in this study. For example, when considering a particular symptom, it is often unclear to the patient and staff whether to report the severity of the symptom and the distress the symptom is causing the patient; and they often have difficulty with timeframes (current symptom compared with best, worst, or average in last 24 hours). Symptoms are rarely unidimensional, and thus can be poorly translated to an 11-point NRS – for example, pain characteristics, intensity, and meaning to the patient are not reflected in an NRS. Breathlessness has both severity (intensity) and affective (unpleasantness) components that change at different rates under induced breathlessness.¹¹ Therefore symptom assessment tools are likely more useful in screening for symptoms and assessment of response to interventions. Furthermore, patients have had difficulties accurately completing a similar multidimensional NRS,¹² and in this study, only 42% of patients were able to rate their own symptoms.

The SAS itself has limitations in that it only examines seven domains, yet it has been demonstrated that patients in a palliative care service have a median of eleven symptoms, and often do not volunteer symptoms without direct inquiry.^3,13 Furthermore, the SAS sets out to examine only physical domains, not accounting for psychological, existential, or social distresses.⁷

The study used a number of methods to compare patient and nurse rating of symptoms, including absolute and mean difference, correlation, and adequate concordance. From the patients' perspective, adequate concordance, here defined as a nurse symptom rating +/− 1 from the patient symptom rating, is the most important to reflect accurately the patients' views of their symptoms. While this may be considered a high standard to meet, in palliative care it is critical to understand the symptom burden of a patient to try to tailor investigations and management to maximize patient comfort and function. Furthermore, if adequate concordance was set as +/− 2 from the patient symptom rating, by random number generation alone, the chance of getting adequate concordance would approach 5/11 (45%) (i.e., of the 11-point NRS, there would be up to five values that would meet the criteria for adequate concordance – the patient rated number and the two above, and two below. This estimate is decreased because of scores of 0, 1, 9, and 10). Importantly, the study demonstrated a modest proportion of wildly inaccurate proxy assessments with large discrepancies (+/− >5). Even ignoring these vastly differing assessments, there are still a considerable proportion differing from patient rating by 2–5. While these assessments are modestly inaccurate, they are not adequate for sound clinical decision making, As demonstrated here, small mean differences or high levels of correlation do not reflect adequate concordance, highlighting the difficulties of staff providing responses to questions that are intrinsically subjective to the patient alone.

Other studies have examined the role of proxy assessment for symptom rating. Overall, proxy assessments of patients with chronic disease, including cancer, tend to underestimate quality of life and overestimate performance status and psychological symptoms such as depression and anxiety when compared to patients' own views.^9,14 In palliative care patients, concordance between patient and proxy rating is generally poor,^15,16 though one study showed 78% complete agreement in symptom assessment between doctor and patient using a four-point categorical severity scale.¹⁷ Physicians and nurses consistently underrate patient physical symptoms,^15,17–19 though nurses tend to more closely approximate patient ratings.¹⁸ Depression and anxiety in palliative care patients may be overestimated by physician and nurses.¹⁵ Family caregivers of people with advanced cancer tend to overestimate some key symptoms.^16,19,20 Proxy assessment does not always improve over time, though may be consistently inaccurate.^15,17,18

Generalizability

This study was performed using multiple proxy raters in one acute palliative medicine facility. The proxy assessors were trained specialist palliative care nurses who had undergone education in the use of the SAS. The patient population had predominantly malignant disease, and the characteristics of those who were not assessed were similar to those who were. The study of 29 patient/nurse dyads in one center limits generalizability to other staff, sites, or countries.

Limitations

As noted, any single site study may have local factors that limit the ability of the findings to be applied in other settings – services or health systems. While the use of multiple proxy symptom assessors potentially increases the risk of error and inter-rater variability in the administration of the SAS, this also mirrors real-world practice. Intra-rater variability for patients has been examined previously,⁷ and variation across time for patients may be a significant factor, particularly as cognition may be impaired with disease progression and increasing polypharmacy. In this inpatient unit, symptom data are routinely collected and, over the duration of the study, staff may already have increasingly been asking the patients to rate their symptoms, potentially lessening the total variation between patient and nurse symptom rating. Despite this, the study shows poor correlation, again confirming that patients should be asked to rate the intensity of symptoms themselves.

Implications for practice

Symptom assessment should ideally involve the patient. Proxy assessment has its place only for patients who are unable to report their symptoms. If any proxy is used, interpretation of their assessment must be carefully weighed against other available information. Medical decision making rarely considers one piece of information in isolation, but must take into account the nature and stage of the disease, comorbidities, and patient and family preferences.

Implications for research

Whilst proxy symptom assessment has consistently demonstrated limitations, the widespread use of proxy assessments in palliative care means that understanding the implications of proxy assessment is important and similar work should be performed at multiple sites to understand the limitations and advantages of proxy ratings. Further work is required to understand what different symptom scores mean to staff and the thresholds for triggering interventions. Symptom assessment needs also to appreciate not only the severity of the symptom, but the associated distress and effects on functional status.^11,21 Only a few of the current symptom assessment tools make this distinction.

Conclusion

In this study, proxy nursing symptom assessment, even by experienced specialist palliative care nurses, was a poor surrogate for patient symptom assessment for those able to reliably rate their symptoms. While the accuracy of proxy rating may be fair with a number of symptoms, there was a significant proportion that was wildly inaccurate. Ensuring all health care professionals directly ask patients at each time point is crucial if care is to be responsive to patients' needs.

Footnotes

Acknowledgment

Thanks to Aileen Mowat who assisted the study team in administration of the surveys.

Author Disclosure Statement

No conflicting financial interests exist.

References

Basch

, Abernethy

. Supporting clinical practice decisions with real-time patient-reported outcomes. J Clin Oncol, 2011; 29:954–956.

Teunissen

, Wesker

, Kruitwagen

, de Haes

, Voest

, de Graeff

. Symptom prevalence in patients with incurable cancer: A systematic review. J Pain Symptom Manage, 2007; 34:94–104.

Donnelly

, Walsh

. The symptoms of advanced cancer. Semin Oncol, 1995; 22:67–72.

Currow

, Plummer

, Crockett

, Abernethy

. A community population survey of prevalence and severity of dyspnea in adults. J Pain Symptom Manage, 2009; 38:533–545.

Clark

, Currow

. Plain abdominal radiographs to diagnose constipation patients with advanced progressive illness? J Pain Symptom Manage, 2011; 41:e2–3.

Kristjanson

, Pickstock

, Yuen

, Davis

, Blight

, Cummins

, Dean

, Tinnelly

, Cousins

. Development, testing of the revised Symptom Assessment Scale (SAS): Final report. Perth, Western Australia: Edith Cowan University, 1999.

Aoun

, Monterosso

, Kristjanson

, McConigley

. Measuring symptom distress in palliative care: Psychometric properties of the Symptom Assessment Scale (SAS) J Palliat Med, 2011; 14:315–321.

Currow

, Eagar

, Aoun

, Fildes

, Yates

, Kristjanson

. Is it feasible and desirable to collect voluntarily quality and outcome data nationally in palliative oncology care? J Clin Oncol, 2008; 26:3853–3859.

Abernethy

, Shelby-James

, Fazekas

, Woods

, Currow

. The Australia-modified Karnofsky Performance Status (AKPS) scale: A revised scale for contemporary palliative care clinical practice [ISRCTN81117481] BMC Palliat Care, 2005; 4:7.

10.

Eagar

, Green

, Gordon

. An Australian casemix classification for palliative care: Technical development and results. Palliat Med, 2004; 18:217–226.

11.

von Leupoldt

, Bradley

, Lang

, Davenport

. Neural processing of respiratory sensations when breathing becomes more difficult and unpleasant. Front Physio, 2010; 1:1–5.

12.

Garyali

, Palmer

, Yennurajalingam

, Zhang

, Pace

, Bruera

. Errors in symptom intensity self-assessment by patients receiving outpatient palliative care. J Palliat Med, 2006; 9:1059–1065.

13.

Homsi

, Walsh

, Rivera

, Rybicki

, Nelson

, LeGrand

, Davis

, Naughton

, Gvozdjan

, Pham

. Symptom evaluation in palliative medicine: Patient report vs systematic assessment. Support Care Cancer, 2006; 14:444–453.

14.

Sprangers

, Aaronson

. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: A review. J Clin Epidemiol, 1992; 45:743–760.

15.

Pautex

, Berger

, Chatelain

, Herrmann

, Zulian

. Symptom assessment in elderly cancer patients receiving palliative care. Crit Rev Oncol Hematol, 2003; 47:281–286.

16.

McMillan

, Moody

. Hospice patient and caregiver congruence in reporting patients' symptom intensity. Cancer Nurs, 2003; 26:113–118.

17.

Stephens

, Hopwood

, Girling

, Machin

. Randomized trials with quality of life endpoints: Are doctors' ratings of patients' physical symptoms interchangeable with patients' self-ratings? Qual Life Res, 1997; 6:225–236.

18.

Nekolaichuk

, Bruera

, Spachynski

, MacEachern

, Hanson

, Maguire

. A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat Med, 1999; 13:311–323.

19.

Oi-Ling

, Man-Wah

, Kam-Hung

. Symptom distress as rated by advanced cancer patients, caregivers and physicians in the last week of life. Palliat Med, 2005; 19:228–233.

20.

Lobchuk

, Degner

. Symptom experiences: Perceptual accuracy between advanced-stage cancer patients and family caregivers in the home care setting. J Clin Oncol, 2002; 20:3495–3507.

21.

Dorman

, Jolley

, Abernethy

, Currow

, Johnson

, Farquhar

, Griffiths

, Peel

, Moosavi

, Byrne

, Wilcock

, Alloway

, Baudsewein

, Bausewein

, Higginson

, Booth

. Researching breathlessness in palliative care: Consensus statement of the National Cancer Research Institute Palliative Care Breathlessness Subgroup. Palliat Med, 2009; 23:213–227.