High Scores on the Edmonton Symptom Assessment Scale Identify Patients with Self-Defined High Symptom Burden

Abstract

The terms symptom burden and symptom distress are frequently seen in palliative care literature yet are used in multiple ways ranging from addition of symptom scores to more in-depth assessments of interference in function. Patient input to date has had little role in these varied definitions yet previous reviews have suggested the importance of such input. This mixed qualitative and quantitative prospective study was conducted to seek patient input into the definition of ‘symptom burden’ and to correlate burden self-ranked scores with symptom scores on the Edmonton Symptom Assessment Scale (ESAS). Results suggest that patients with advanced disease followed by palliative care providers are a group with a high level of self-defined burden. Further, any patient rating one or more symptoms on the ESAS ≥7 is at high risk of self-defined burden and likely to be experiencing significant impact on physical, emotional and social functioning. Further work should look at which aspects of symptom management best promote a lessening of perceived burden.

Introduction

Patients with advanced illness experience multiple symptoms, which have been shown to impact overall quality of life (QOL) as well as physical, emotional, and social domains of function. A basic tenet of palliative care is to address such symptoms with the goal of improving QOL. In a recent systematic review, Kirkova et al.¹ identified multiple instruments used to identify symptoms experienced by patients and to quantify both severity and perceived impact. Across these instruments varying definitions exist for the term “burden” or “distress.” For example, Bruera and colleagues² defined the Symptom Distress Score (SDS) as the sum of the nine symptom items on the Edmonton Symptom Assessment Scale (ESAS). Cleland et al.³ incorporated functional interference items in the MD Anderson Symptom Inventory (MDASI), as did the European Organization for the Research and Treatment of Cancer QLQ-30,⁴ whereas Portenoy and coworkers⁵ with the Memorial Symptom Assessment Scale (MSAS) included specific rankings of distress caused by each symptom. In using such symptom scales, several studies refer to “symptom burden,” but again, substantial variation exists in how the term is defined, used, and measured. For many, it is simply the number of symptoms noted or the sum of individual symptom severity ratings,^6–12 whereas others specifically include interference in function as a measure of “burden.”^13–16 Adding to this difficulty, the term “symptom distress” is often used interchangeably with “symptom burden” though it is also used to indicate “symptom intensity.”¹⁷

Previous reviews by Gapstur¹⁸ and by Goodell and Nail¹⁹ focusing on “symptom burden” and “symptom distress,” respectively, confirmed the use of multiple definitions and both indicated the need for a unified construct. Goodell and Nail further highlighted the importance of including patient input in defining such concepts, a contribution that to date has been missing as essentially all definitions in the literature are health care provider derived. The goal of this study was to seek patient perspectives on the meaning of the term “symptom burden” and to look for associations between patient-ranked “burden” and symptom scores on the ESAS. The ESAS is a simple, well-validated tool in the palliative care population,²⁰ and was used in this study as it has recently been promoted in Ontario cancer centers as part of the standard of care for patient management.⁹

Methods

This mixed qualitative and quantitative prospective study included patients followed by the Palliative Care Consult Team (PCCT) at Sunnybrook Health Sciences Centre (SHSC), a regional cancer center and academic tertiary care hospital in Toronto, Canada. The PCCT provides specialist inpatient palliative care consultation and runs an outpatient palliative care clinic through the ambulatory cancer center. Eligibility criteria included adult patients, fluent in English, with advanced cancer (i.e., metastatic disease) referred for palliative care consultation and management. At the planned time of the interview the patient's primary care provider determined if the patient was well enough to participate.

Recruited patients, after providing informed consent, were first asked to reflect on and define “symptom burden” in their own words, the complete results of which are reported elsewhere. After defining “symptom burden,” participants completed the quantitative arm of the study (Appendix 1). Using an 11-point Likert scale, participants rated their “total symptom burden during the prior week” with a rating of 0 representing “no symptom burden” and 10 representing “worst symptom burden.” Using a similar question format, participants rated the perceived impact of this “burden” on three functional domains (physical, emotional, and social impact) with the intent being their definition of “symptom burden” would directly inform responses.

QOL was assessed using a 7-point Likert scale with each numerical value assigned a descriptive phrase similar to the Rotterdam Symptom Checklist.²¹ Functional status was assessed with the Palliative Performance Scale (PPS), an 11-point scale, ranging in increments of 10% from 0% to 100% (where 0%=dead and 100%=normal) with each score evaluated according to five observable parameters: ambulation, activity and evidence of disease, self-care, intake, and level of consciousness.²² All patients completed an ESAS and demographic information collected included age, gender, care setting (inpatient versus outpatient) and underlying malignancy.

The study was approved by the hospital Research Ethics Board.

Statistical analysis

Demographics were analyzed with descriptive statistics. For the purposes of analysis both Total Symptom Burden (TSB) and ESAS item ratings were divided into the binary categories of “severe” (scored 7–10) or “not severe” (scored 0–6) based on previous work with ESAS cutpoints.²³ Correspondingly, responses for all three functional domains were categorized as “high” or “severe” (rating 7–10) versus “not high” (rating 0–6). For the overall QOL item, binary categories representing “poor” (score 5–7) versus “not poor” (score 1–4) were used, with the division based on the language used to describe each numeric possibility (see Appendix 1). The “symptom distress score” (SDS), defined as the sum of nine ESAS items, was analyzed using 10-point increments. Scores were analyzed for the full cohort as well as comparing select inpatient and outpatient subgroup responses.

The association between binary variables was analyzed using a χ² test or Fisher's exact test. Odds ratios (Ors) were computed using a logistic regression analysis adjusting for patient's gender and PPS.

Three authors (DS, AC, AG) independently reviewed the first 25 patient definitions of “symptom burden” to identify themes. Common themes were agreed upon with no new themes emerging after the first 19 respondents. At study completion using NVivo9 (QSR International, Doncaster, Victoria, Australia)²⁴ software the authors coded each patient response and intercoder reliability was assessed using Cohen's kappa.

Results

Of the 139 patients initially considered for recruitment between November 2009 and September 2010, 35 were ultimately too ill to be approached as determined by their primary physician (reasons ranging from emotional distress to severe physical symptoms precluding participation), 10 had a language barrier precluding the interview, 12 were discharged from the hospital before completing the questionnaires, and 8 had only symptoms unrelated to an active cancer (e.g., treatment-related or chronic nonmalignant pain), leaving 74 eligible to participate. Of these, 13 declined and 3 had incomplete data, for a final cohort of 58 patients. Patient demographics are summarized in Table 1. The median age was 60 and 40% of patients were male. Median PPS for the cohort was 70% and was significantly lower for inpatients (median=60%) versus outpatients (median=80%, p<0.001).

Table 1.

Demographics

Gender (n=58)
Female	35 (60%)
Male	23 (40%)
Age (years)
Median	60
Mean	60.4
Range	34–85
PPS
Median	70
Mean	69.3
Range	30–100
Malignancy
Breast	9
Hematologic	9
Colorectal	7
Gynecologic	6
Pancreas/biliary	5
GI other^a	5
Lung	4
Prostate	3
GU other^b	3
Other^c	7

Includes appendix, anus, stomach.

Includes penile, bladder.

Includes melanoma, renal cell, primary brain, thyroid.

GI, gastrointestinal; GU, genitourinary; PPS, Palliative Performance Scale.

Based on care setting, no significant differences existed between individual mean/median ESAS ratings, SDS, or TSB (Tables 2 and 3). Median TSB was “severe” for both inpatients and outpatients. Among inpatients, median ratings for impact of symptom burden were in the “severe” range for all three functional domains, whereas for outpatients, median ratings tended to be slightly lower (Table 3). Patients rating TSB as “severe” were significantly more likely than those rating TSB ≤6 to also rate the impact on physical (p<0.006), emotional (p=0.0007), and social (p<0.001) function as being “high” and QOL as being “poor” (p=0.02) (Table 4).

Table 2.

ESAS Scores

ESAS Items	Entire sample (n=58)	Inpatients (n=19)	Outpatients (n=39)	P-value
Pain	4.38^a	4.53	4.31	0.781
	5 (0–10)	5	4
Tiredness	4.60	5.16	4.33	0.305
	5 (0–10)	5	3
Nausea	1.09	1.16	1.05	0.852
	0 (0–7)	0	0
Depression	2.22	1.79	2.44	0.433
	1 (0–9)	1	1
Anxiety	2.12	2.00	2.18	0.818
	1 (0–10)	1	1
Drowsiness	3.26	3.32	3.23	0.919
	3 (0–10)	2	3
Loss of appetite	3.22	3.21	3.23	0.981
	3 (0–10)	3	3
Well-being	4.31	4.84	4.05	0.242
	5 (0–8)	5	4
Shortness of breath	1.90	1.53	2.08	0.470
	0 (0–8)	0	0
Symptom Distress Score	27.10	27.53	26.90	0.888
	24 (4–68)	25	23

Mean, Median (range) provided for all ESAS items.

Table 3.

Total Symptom Burden, Functional Impact and Quality-of-Life Ratings across Care Settings

	Total (n=58)	Inpatients (n=19)	Outpatients (n=39)	P-value
Total symptom burden	6.21^a	6.95	5.85	0.141
	7	7	7	0.145
Physical impact	7.56	8.72	7.03	0.017
	8	10	7	0.004
Emotional impact	6.28	7.17	5.87	0.109
	7	8	6	0.068
Social impact	5.53	6.61	5.03	0.061
	6	8	5	0.051
Quality of life^b	4.05	5.11	3.56	0.002
	4	6	3	0.005

Mean, Median provided for all items.

Measured on a 7-point scale where 1=excellent and 7=extremely poor.

Table 4.

Association between Total Symptom Burden and Functional Impact/Quality of Life

	High physical impact		High social impact		High emotional impact		Quality of life
Severe burden	No	Yes	No	Yes	No	Yes	No	Yes	Total
No	13	13	23	3	19	7	20	6	26
Yes	3	29	10	22	9	23	15	17	32
Total	16	42	33	25	28	30	35	23	58
P-value	0.006		0.0001		0.0007		0.02

There was a strong association between ranking one or more ESAS items 7 or higher and ranking all other burden measures negatively, including TSB (OR 10.64, p=0.0006) (Tables 5 and 6), QOL (OR for “poor”=5.4, p=0.0042), and all functional interference items (Table 6). Regression analysis as shown in Table 6 showed no significant association with PPS, however males were consistently 4 to 5 times less likely to report “severe” rankings for TSB (OR 0.23), and impact on both physical (OR 0.23) and emotional function (OR 0.25).

Table 5.

Association between Total Symptom Burden and Number of “Severe” ESAS Ratings

	Number of severe ESAS items^a
High burden	0	1+	Total
No	19	7	26
Yes	7	25	32
Total	26	32	58
Odds Ratio (95% CI)	Odds Ratio 10.64 (95% CI=2.77–40.77)
P-value	0.0006

Severe ESAS item defined as any item scored 7 or higher.

CI, confidence interval.

Table 6.

Regression Analysis: ESAS Ratings 7–10 versus Symptom Burden Severity and Functional Impact

Severe Total Symptom Burden (scored 7+)
Variable		Odds Ratio	95% Confidence interval	P-value
Number of scores 7+	0	1.00
	1+	10.64	2.77–40.77	0.0006
Gender	F	1.00
	M	0.23	0.06–0.93	0.0385
PPS		0.96	0.92–1.01	0.1518

High Physical Impact (scored 7+)
Variable		Odds Ratio	95% Confidence interval	P-value
Number of scores 7+	0	1.00
	1+	5.70	1.38–23.62	0.0164
Gender	F	1.0
	M	0.23	0.06–0.92	0.0371
PPS		0.95	0.89–1.01	0.0776

High Emotional Impact (scored 7+)
Variable		Odds Ratio	95% Confidence interval	P-value
Number of scores 7+	0	1.0
	1+	7.31	2.07–25.77	0.002
Gender	F	1.0
	M	0.25	0.07–0.90	0.0346
PPS		0.98	0.94–1.03	0.3733

High Social Impact (scored 7+)
Variable		Odds Ratio	95% Confidence interval	P-value
Number of scores 7+	0	1.00
	1+	9.91	2.65–37.08	0.0007
Gender	F	1.00
	M	0.51	0.14–1.84	0.3045
PPS		0.98	0.94–1.03	0.3918

CL, confidence level; PPS, Palliative Performance Scale.

On the qualitative analysis, six themes emerged (Table 7) of which the most prevalent were “can't do usual activities,” “psychological suffering,” and “specific severe symptoms.” Overall, the 58 patient definitions yielded 109 thematic responses. The intercoder reliability (kappa) was 0.7494 (1=perfect agreement, 0.80=nearly perfect, and 0.60=satisfactory agreement).

Table 7.

Thematic Analysis of Patient Definitions for “Symptom Burden”

Theme	Number of Responses (n=58)^a
Can't do usual activities	34 (59%)
Psychological suffering	28 (48%)
Specific severe symptoms	25 (43%)
Worry/anxiety specifically about death and disease	11 (19%)
Not my old self	8 (14%)
Drug issues	3 (5%)

Numbers add to >100% as many patients included more than one theme in their definition.

Representative patient definitions included:

1. It just means that I can't do what I normally do. That's all.

2. Symptom burden is how my symptoms negatively affect my day-to-day life. That is what symptom burden means to me—how it affects what I do on a daily basis.

3. It means a lot of stress daily, which I find very difficult to manage.

4. It makes me feel worried and anxious a lot of the time.

5. It is when I wake up and have a lot of pain and I am so depressed I don't want to get up.

Discussion

“Symptom burden” is referenced frequently in palliative care literature, but it is inconsistently defined. Most authors use the term to represent the prevalence of symptoms, severity of symptoms, or the sum of symptom scores,^{6–10,25–27} which in turn are used to look at wide-ranging aspects of patient experience. Recently, symptom burden has been specifically linked to prognosis,^8,28,29 inflammatory cytokine levels,³⁰ and gene polymorphisms³¹; however, interpreting clinical meaning is difficult given the differing concepts and definitions of the term. For example, one study linking symptom burden to prognosis in lung cancer²⁹ found specifically that high scores for “cough” was linked to poor prognosis, as opposed to a more global construct of the term “burden.”

In Goodell and Nail's literature review,¹⁹ the authors reported seven operationalizations of the term “symptom distress” aside from “the many sources in which symptom distress simply was equated with symptom intensity or severity.” These included the introduction of alternative terms such as “bother” or “discomfort,” physical and emotional aspects of symptoms, as well as interference in activities. Further, in looking at the impact of different symptoms on “distress,” Goodell and Nail noted that some symptoms might well contribute more than others to distress, a concept echoed by Cleeland et al.¹³ Similarly, in an attempt to unify a definition encompassing these aspects of symptom experience, Gapstur¹⁸ suggested a definition of burden as “the subjective, quantifiable prevalence, frequency and severity of symptoms that place a physiologic burden on patients and may produce multiple negative physical, psychological and emotional patient responses.” To date, health care providers have largely derived these definitions with little or no input by patients, and easily measuring such a construct has remained problematic.

We believe this is the first study to directly ask patients to reflect on and define “symptom burden,” then to rate both their perceived burden and its impact on functional domains. Our population identified themselves as highly burdened, with a median score of 7. “Burden” was linked to impact on all spheres of functioning, as those with TSB scores ≥7 had a significantly increased likelihood of also rating social, emotional, and physical interference as severe. Consistent with this, the qualitative portion of the study revealed that the most common themes were interference in usual activities (34 of 58) and psychological suffering (28 of 58). Symptom severity was also a prevalent theme in the qualitative analysis but often in the context of its subsequent impact on functioning.

In terms of relating burden and impact on function with commonly used symptom scales, one key finding in our study was that having one or more ESAS symptoms rated ≥7 was strongly associated with a “severe” rating for “burden” and all functional domain scores. In fact, those having at least one symptom scored ≥7on the ESAS had a roughly 10-fold likelihood of also identifying their total symptom burden as “severe” (OR 10.64). Equally, those with one or more ESAS scores ≥7 clearly indicated high impact on all of physical, emotional, and social functioning (Tables 5 and 6). As the number of severe symptoms increased, in general the OR for severe burden/interference also increased, a finding corroborated by Shih and colleagues,¹⁴ who reported that the total number of severe symptoms ranked on a linear analog scale was a strong predictor of patient-identified disutility. Although incremental rises in the sum of ESAS items were statistically significantly related to higher burden, from the point of clinical utility, adding the scores does not seem to matter; rather, having any ESAS item rated ≥7 identifies a patient highly likely to feel burdened.

Although the primary goal of the study was not to compare inpatient and outpatient populations, some interesting similarities and differences were evident. Inpatients may intuitively be thought of as “sicker” and, indeed, their PPS was significantly lower. Further, inpatients had a median “physical impact of burden” score of 10 compared with 7 for outpatients and in general tended to report higher impact scores. Interestingly though, the ESAS scores were not significantly different, suggesting that burden and impact on function are not being captured by simple numeric symptom scores. Similarly, on the qualitative analysis patients often defined burden in terms of psychological distress, and qualitatively emotional impact was ranked high, yet the median scores on the ESAS for both depression and anxiety were only 1. In general, our ESAS scores were low, which may reflect symptom management prior to enrollment, and yet with the high burden scores these numbers were not reflecting the patients' lived experience. With the differential impact scores in the three domains, inpatients would appear to have a more accentuated experience of the symptoms, warranting further investigation.

Looking further at subgroups of patients, it is important to note that on multivariate regression PPS did not seem to have an important role in the rating of burden. Gender, however, was a significant factor, with women rating burden, physical, and emotional interference higher than men (Table 6). This is a finding in keeping with earlier work by Bernhard and coworkers,³² who similarly asked patients to score burden on a linear analog scale. Patients were asked to assess the overall burden of their chemotherapy treatment with the question “Overall, how much are you bothered by any treatment-related difficulties?” on a 0–100 scale. Women reported greater burden with a median score of 22 versus a median score of 3 for men. Equally, in a study on cancer survivors Mao et al.¹² found that women reported greater prevalence and severity of symptoms. Our study does not answer the question of why women would report more “burden,” and it raises the question of whether, for example, this represents a greater comfort with reporting high burden or whether there is a true gender difference in the physiologic/psychological experience of symptoms.

Another finding of note in our study in contrast with previous work, is that median ESAS scores were lower than the median score for TSB. Both Portenoy et al.⁵ and Tranmer and colleagues³³ used the MSAS to examine symptom experience in patients with advanced disease. The MSAS, unlike scales such as the ESAS, has specific rankings for severity, frequency, and distress caused by specific symptoms. Both authors found that the distress component was consistently ranked lower than the severity score. In contrast, in our study, no ESAS score had a mean value above 5 (highest was tiredness, 4.60) and yet overall burden and impact on function (whether physical, emotional, or social) was rated quite high with median scores of 6–8. This may be explained by work reported by Johnson et al.,³⁴ who asked 375 patients receiving chemotherapy to complete a questionnaire asking about preferences around theoretic treatments that carried differing symptom severity and effects on functioning. They reported that patients were “likely to be less concerned about specific symptoms than the impact of those symptoms on their ability to function physically, socially and in their daily roles,” and, indeed, it was the improvement in physical functioning from “need help” to “can care for self” that was among the most strongly rated. Cleeland³⁵ similarly identified the need to address impact of symptoms on daily living in addition to raw severity in order to more fully understand the burden imposed by a symptom or group of symptoms, findings strongly in keeping with the qualitative definitions our patients provided.

Overall, although symptom scores clearly give important information, their relation to symptom burden or distress is undoubtedly more complicated than the raw number or sum of scores. Patients expressed significant concern about impact on function in addition to the severity of symptoms in both the qualitative and quantitative sections of the study, and it is too simplistic to equate single or summative scores with burden. However, from the perspective of providing clinical care in busy clinic settings, long questionnaires that may more specifically explore these realms are not feasible on a recurring basis (for either patient or provider). Our study suggests two practical and simple approaches. First, one or more ESAS scores of ≥7 acts as a surrogate identifier of high burden, and second, like Bernhard and coworkers³² simply asking patients to rank their overall burden, or how symptoms are impacting their life, may capture the concept of burden as described by Gapstur's definition.

Our study has a number of limitations, of which the most significant is the relatively small numbers. Although this limited analysis of subgroups and required separating data into the binary categories of “severe” versus “not severe,” the intent was that the initial consideration of the term would lead to more thoughtful responses to the Likert scale items. Further limitations include that this study was from a single center, and included only English-speaking patients. Finally, 35 patients who met our initial requirements for entry were felt to be too unwell by their primary provider to be approached. As a result it is important to note that the final population therefore does not represent the more seriously ill cohort.

Conclusions

Multiple uses of the terms “burden” and “distress” appear in the literature, but few if any include patient definitions of the terms. This study, which directly asked patients to define and then rate “burden,” suggests that patients with advanced disease followed by palliative care providers are a group with a high level of self-defined burden. One or more scores ≥7 on an ESAS identify patients at high risk of feeling burdened and of feeling their symptoms have adversely affected physical, emotional, and social functioning. Future work should look at which aspects of symptom management best promote a lessening of perceived burden, and also explore the apparent discrepant results between males and females.

Footnotes

Author Disclosure Statement

No conflicting financial interests exist.

Appendix 1

References

Kirkova

, Davis

, Walsh

, Tiernan

, O'Leary

, LeGrand

, Lagman

, Russell

. Cancer symptom assessment instruments: A systemic review. J Clin Onc, 2006; 24:1459–1473.

Bruera

, Kuehn

, Miller

, Selmser

, Macmillan

. The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. J Palliat Care, 1991; 7:6–9.

Cleeland

, Mendoza

, Wang

, Chou

, Harle

, Morrissey

, Engstrom

. Assessing symptom distress in cancer patients: The MD Anderson Symptom Inventory. Cancer, 2000; 89:1634–1646.

Aaronson

, Ahmedzai

, Bergman

, Bullinger

, Cull

, Duez

, Filiberti

, Flechtner

, Fleishman

, de Haes

JCJM

, Kaasa

, Klee

, Osoba

, Razavi

, Rofe

, Schraub

, Sneeuw

KCA

, Sullivan

, Takeda

. The European Organisation for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst, 1993; 85:365–376.

Portenoy

, Thaler

, Kornblith

, Lepore

, Friedlander-Klar

, Kiyasu

, Sobel

, Coyle

, Kemeny

, Norton

, Scher

. The Memorial Symptom Assessment Scale: An instrument for the evaluation of symptom prevalence, characteristics and distress. Eur J Cancer, 1994; 30A:1326–1336.

Modonesi

, Scarpi

, Maltoni

, Derni

, Fabbri

, Martini

, Sansoni

, Amadori

. Impact of palliative care unit admission on symptom control evaluated by the Edmonton Symptom Assessment System. J Pain Symptom Manage, 2005; 30:367–373.

Fadul

, El Osta

, Dalal

, Poulter

, Bruera

. Comparison of symptom burden among patients referred to palliative care with hematologic malignancies versus those with solid tumors. J. Palliat Med, 2008; 11:422–427.

Cheung

, Barmala

, Zaarinehbaf

, Rodin

, Le

, Zimmerman

. The association of physical and psychological symptom burden with time to death among palliative cancer outpatients. J Pain Symptom Manage, 2009; 37:297–304.

Barbera

, Seow

, Howell

, Sutradhar

, Earle

, Liu

, Stitt

, Husain

, Sussman

, Dudgeon

. Symptom burden and performance status in a population-based cohort of ambulatory cancer patients. Cancer, 2010; 116:5767–5776.

10.

Sawicki

, Sellers

, Robinson

. Self-reported physical and psychological symptom burden in adults with cystic fibrosis. J Pain Symptom Manage, 2008; 35:372–380.

11.

Silveira

, Kabeto

, Langa

. Net worth predicts symptom burden at end of life. J Palliat Med, 2005; 8:827–837.

12.

Mao

, Armstrong

, Bowman

, Xie

, Kadakia

, Farrar

. Symptom burden among cancer survivors: Impact of age and comorbidity. J Am Board Fam Med, 2007; 20:434–443.

13.

Cleeland

. Symptom burden: Multiple symptoms and their impact as patient-reported outcomes. J Natl Cancer Inst Monogr, 2007; 37:16–21.

14.

Shih

, Wang

, Cantor

, Cleeland

. The association between symptom burden and utility in Chinese cancer patients. Qual Life Res, 2006; 15:1527–1438.

15.

Burkett

, Cleeland

. Symptom burden in cancer survivorship. J Cancer Surviv, 2007; 1:167–175.

16.

Rosenthal

, Mendoza

, Chambers

, Asper

, Gning

, Kies

, Weber

, Lewin

, Garden

, Ang

, S Wang

, Cleeland

. Measuring head and neck cancer symptom burden: The development and validation of the M.D. Anderson symptom inventory, head and neck module. Head Neck, 2007; 29:923–931.

17.

Bradley

, Davis

, Chow

. Symptom distress in patients attending an outpatient palliative radiotherapy clinic. J Pain Symptom Manage, 2005; 30:123–131.

18.

Gapstur

. Symptom burden: A concept analysis and implications for oncology nurses. Onc Nurs Forum, 2007; 34:673–680.

19.

Goodell

, Nail

. Operationalizing symptom distress in adults with cancer: A literature synthesis. Onc Nurs Forum, 2005; 32:E42–E47.

20.

Chang

, Hwang

, Feuerman

. Validation of the Edmonton Symptom Assessment Scale. Cancer, 2000; 88:2164–2171.

21.

de Haes

, van Knippenberg

, Neijt

. Measuring psychological and physical distress in cancer patients: Structure and application of the Rotterdam Symptom Checklist. Br J Cancer, 1990; 62:1034–1038.

22.

Anderson

, Downing

, Hill

, Casorso

, Lerch

. Palliative performance scale (PPS): A new tool. J Palliat Care, 1996; 12:5–11.

23.

Selby

, Cascella

, Gardiner

, Do

, Moravan

, Myers

, Chow

. A single set of numerical cutpoints to define moderate and severe symptoms for the Edmonton Symptom Assessment System. J Pain Symptom Manage, 2010; 39:241–249.

24.

QSR International. 2010. NVivo (version 9.0) [Software] www.qsrinternational.com/products_nvivo.aspx. 2011 January 21.

25.

Blinderman

, Homel

, Billings

, Portenoy

, Tennstedt

. Symptom distress and quality of life in patients with advanced congestive heart failure. J Pain Symptom Manage, 2008; 35:594–603.

26.

Hui

, Parsons

, Nguyen

, Palla

, Yennurajalingam

, Kurzrock

, Bruera

. Timing of palliative care referral and symptom burden in phase 1 cancer patients: A retrospective cohort study. Cancer, 2010; 116:4402–4409.

27.

Janssen

, Spruit

, Wouters

, Schols

. Daily symptom burden in end-stage chronic organ failure: A systemic review. Palliat Med, 2008; 22:938–948.

28.

Palmer

, Fisch

. Association between symptom distress and survival in outpatients seen in a palliative care cancer center. J Pain Symptom Manage, 2005; 29:565–571.

29.

Wang

, Shi

, Basch

, Johnson

, Mendoza

, Mobley

, Cleeland

. Prognostic value of symptom burden for overall survival in patients receiving chemotherapy for advanced nonsmall cell lung cancer. Cancer, 2010; 116:137–145.

30.

Wang

, Shi

, Williams

, Mao

, Cleeland

, Komaki

, Mobley

, Liao

. Inflammatory cytokines are associated with the development of symptom burden in patients with NSCLC undergoing concurrent chemoradiation therapy. Brain Behav Immun, 2010; 24:968–974.

31.

Rausch

, Clark

, Patten

, Liu

, Felten

, Li

, Sloan

, Yang

. Relationship between cytokine gene single nucleotide polymorphisms and symptom burden and quality of life in lung cancer survivors. Cancer, 2010; 116:4103–4113.

32.

Bernhard

, Maibach

, Thurlimann

, Sessa

, Aapro

. Patients' estimation of overall treatment burden: Why not ask the obvious? J Clin Oncol, 2002; 20:65–72.

33.

Tranmer

, Helyand

, Dudgeon

, Groll

, Squires-Grahm

, Coulson

. Measuring the symptom experience of seriously ill cancer and noncancer hospitalized patients near the end of life with the Memorial Symptom Assessment Scale. J Pain Symptom Manage, 2003; 25:420–429.

34.

Johnson

, Hauber

, Osoba

, Hsu

, Coombs

, Copley-Merriman

. Are chemotherapy patients' HRQoL importance weights consistent with linear scoring rules? A stated-choice approach. Qual Life Res, 2006; 15:285–298.

35.

Cleeland, CS:. When is it justified to treat symptoms? Measuring symptom burden. Oncology, 2002; 16,Suppl 10:64–70.