Abstract
Abstract
Background:
Proxy reporting is frequently used to assess symptom distress of patients with advanced chronic organ failure. The aim of the present cross-sectional study was to examine agreement in severity of symptom distress, presence of symptom-related interventions, and satisfaction with medical treatment among patients with advanced chronic obstructive pulmonary disease (COPD), chronic heart failure (CHF) and chronic renal failure (CRF) and their family caregivers.
Methods:
Outpatients with advanced COPD (n=73), CHF (n=45) and CRF (n=41) and their family caregivers rated severity of physical and psychological symptoms experienced by the patient using Visual Analogue Scales (VAS). The presence of symptom-related interventions was recorded by patients and family caregivers. Finally, patients and family caregivers rated satisfaction with medical treatment of the patient using VAS. Agreement was determined using intraclass correlation coefficients (ICC) for continuous variables and Cohen's kappa for categorical variables.
Results:
Family caregivers reported a higher number of symptoms than patients (mean [standard deviation; SD]: 8.2 [3.5] versus 7.3 [3.6], respectively [p<0.0005]). For most symptoms, agreement about severity between patients and family caregivers was moderate (ICC: 0.41-0.60). Agreement about satisfaction with medical treatment was fair (ICC [95% confidence interval; CI]: 0.21 [0.05-0.35]). Agreement was poor to moderate for presence of symptom-related interventions (kappa: −0.03-0.54).
Conclusions:
Studies using proxy reporting reflect the views of proxies and do not accurately represent the patients' experience. For clinical care, it's important to pay attention to the perception from the patient as well as the perception from the family caregiver of symptom distress, presence of symptom-related interventions, and satisfaction with treatment.
Introduction
Accurate assessment of symptom distress is necessary to improve symptom management and quality of end-of-life care. Difficulty in obtaining information from terminally ill patients concerning symptom distress and satisfaction with symptom-related interventions poses a challenge in clinical care, as well as in research. Especially in chronic organ failure, unexpected death occurs frequently and prospectively identifying the terminally ill patient is often impossible.9–11 Therefore, retrospective studies using proxy reporting are used to gain insight in symptom distress at the end of life, often relying on bereaved family members to report on symptoms as experienced by the patient. 1,8,12–19 However, the reliability of proxy reporting remains debated. For example, low and inconsistent agreement among cancer patients' ratings of their symptoms and their family members' ratings brings into question the use of family members' views as proxies for the views of patients. 20,21 Agreement was higher for items related to service provision than for symptoms, suggesting that retrospective assessment of end-of-life care services may be valid. 21 Other authors show moderate levels of correlation in symptom distress scores of patients receiving palliative care and their relatives and state that proxy response is a fair substitute in the palliative care setting. 22
Studies evaluating agreement between patients with advanced COPD, CHF, or CRF and their family caregivers concerning severity of symptoms are lacking. Therefore, to date it remains unknown whether and to what extent agreement exists in perception of symptom severity between these patients and their family caregivers. Aims of the present observational study were to examine agreement in severity of symptom distress among patients with advanced COPD, CHF, or CRF and their family caregivers and to study determinants of clinically relevant differences in symptom severity perception between patients and family caregivers. In addition, we aimed to examine agreement in the presence of symptom-related interventions and satisfaction with medical treatment. Based on the previous studies in cancer patients,20,21 we hypothesized a priori that agreement between patients and their family caregivers is inconsistent for symptom severity, but better for satisfaction with treatment and presence of symptom-related interventions.
Methods
Design
This cross-sectional study is part of a longitudinal study concerning self-perceived symptoms and care needs in patients with advanced COPD, CHF, or CRF and the consequences for their closest relatives. 23 The Medical Ethical Committee of the Maastricht University Medical Centre+, Maastricht, the Netherlands, approved this study (MEC 07-3-054). Details of study design and data on symptom distress, health status, advance care planning, and family caregiving were published before.3,23–27 The study was registered at the Dutch Trial Register (NTR 1552).
Study population
Patients with advanced COPD, CHF, or CRF were recruited by their physician specialist at the outpatient clinic of one academic and six general hospitals in the Netherlands in 2008 and 2009. Patients were eligible if they had a diagnosis of severe to very severe COPD (Global initiative for chronic Obstructive Lung Disease (GOLD) stage III or IV, advanced CHF (New York Heart Association (NYHA) class III or IV), or advanced CRF (patients on dialysis). For patients who were contacted for the study but refused participation, data including severity of disease, age, and sex were collected to compare characteristics of participating and nonparticipating patients. Patients were asked to identify the person who spent the most time with them and/or provided most of their care, assistance, and support. 28 All participating patients and their participating family caregivers have given written informed consent.
Instruments
Participants were visited at home or, if CRF patients preferred, during dialysis at the dialysis department. The following patient-related outcomes were assessed: demographics, self-reported comorbidities (Charlson Comorbidity Index 29 ), long-term oxygen therapy (LTOT), need for assistance with personal care, mobility problems, and self-perceived poor health. Blood samples were taken to measure N-terminal pro-b-type natriuretic peptide (NT-proBNP) 30 and creatinine. In CRF patients, blood samples were taken before start of dialysis. Forced expiratory volume in the first second (FEV1) was calculated from the flow–volume curve measured by a handheld pulmonary spirometer. 31 General characteristics of family caregivers, such as gender, age, and relationship with the patient were recorded.
As part of a personal interview, patients and family caregivers were asked separately to rate severity of 18 physical and psychological symptoms experienced by the patient using a Visual Analogue Scale (VAS) for every symptom. 3 VAS is a horizontal straight line, ranging from 0 (no symptom) to 100 mm (maximum severity). 32 Participants had to mark on the line the point that represented the severity of the symptom during the previous 2 weeks. The VAS score has been determined by measuring in millimeters from the left side of the line to the point the patient marked. 32 Symptoms were selected based on previous research. 1 The initial version of the VAS symptom checklist was reviewed by several physician specialists. Based on their expert opinion, several symptoms were added to the VAS symptom checklist. In addition to rating severity of symptoms, participants were asked if they had received treatment (response options: “yes” or “no”) for each reported symptom. Finally, patients and family caregivers were asked to rate their satisfaction with the medical treatment of the patient using VAS. VAS for satisfaction with treatment ranged from 0 (not satisfied) to 100 mm (very satisfied).
Statistics
Categorical variables are described as frequencies, whereas continuous variables were tested for normality and are presented as mean and standard deviation (SD) or median and interquartile range (IQR). Comparison of continuous variables between patients with COPD, CHF, or CRF were done using a one-way analysis of variances (ANOVA) with the Least Significant Difference (LSD) test as post hoc test or Kruskall-Wallis test followed by Mann-Whitney U tests, as appropriate. Categorical variables were compared between patients with COPD, CHF, and CRF using χ2 tests. We calculated the mean number of symptoms with VAS-score >30mm (representing moderate severity 33 ). Mean number of symptoms, symptom VAS scores for severity of individual symptoms, and VAS scores for satisfaction with treatment were compared between patients and their family caregivers using paired sample t tests or Wilcoxon signed-rank tests, depending on the variable distribution. We calculated intraclass correlation coefficients (ICC) to study agreement between patients and family caregivers in mean number of symptoms, symptom severity VAS scores, and VAS scores for satisfaction with medical treatment. Bland and Altman plots 34 were used to show agreement for the three most severe symptoms as reported by patients and caregivers: dyspnea, fatigue, and muscle weakness. To identify determinants of clinically relevant differences (10 mm or more 35 ) between patients and caregivers in VAS scores for dyspnea, fatigue, and muscle weakness, we divided the differences in VAS scores into three categories: VAS score comparable for patients and caregivers, VAS score of the caregiver 10 mm or more lower than VAS score of the patient, and VAS score of the caregiver 10 mm or more higher than VAS score of the patient. Patient and caregiver characteristics were compared between these three categories using ANOVA with LSD as post hoc test for continuous variables and χ2 tests for categorical variables. Percentage agreement in presence of symptom-related interventions was determined using Cohen's kappa. A value for kappa or ICC between 0.81 and 0.99 was defined as almost perfect agreement, 0.61-0.80 as substantial agreement, 0.41-0.60 as moderate agreement, 0.21-0.40 as fair agreement and ≤0.20 as poor agreement. 36 A priori, a two-sided level of significance was set at p≤0.05. 37
Results
Characteristics of patients and family caregivers
In total, 265 patients (105 COPD, 80 CHF, and 80 CRF patients) participated in the longitudinal study (response rate 58.9%). These 265 patients were asked to identify a family caregiver for participation in the study. Family caregivers of 159 patients (60%) (73 COPD, 45 CHF, and 41 CRF patients) agreed to participate. Reasons for no participating family caregiver were: “no family caregiver present” (n=40, 37.8%), “patient was afraid of the burden of the study for the family caregiver” (n=42, 39.6%), “family caregiver refused participation” (n=14, 13.2%), or “participation was not possible because of physical or cognitive impairment of the relative” (n=10, 9.4%). Participants and eligible patients who refused participation or did not have a participating family caregiver were different in some respects. CHF patients who did not participate were older (mean age 78.4 years (8.7), p=0.02) and less often male (45.0%, p=0.001) than CHF participants. COPD participants had more severe disease than nonparticipants (GOLD stage IV, 72.6% versus 51.1%, respectively, p=0.01).
Significant differences were found between COPD, CHF, and CRF patients for outcomes such as age, marital status, comorbidities, support from a professional caregiver and mobility problems (Table 1). Most CRF patients (n=39, 95.1%) underwent hemodialysis and two patients (4.9%) underwent peritoneal dialysis. The majority of family caregivers were female partners of the participating patients. In general, family caregivers of CHF patients were the oldest (Table 1).
Values expressed as mean (SD) or number of patients (%).
p<0.05 versus CHF.
p<0.05 versus CRF.
Nonparametric statistical tests have been used because of skewed data.
N=70.
N=71.
N=44.
Other: brother, sister, parent, daughter-in-law, son-in-law.
COPD, chronic obstructive pulmonary disease; CHF, chronic heart failure; CRF, chronic renal failure; FEV1, forced expiratory volume in the first second; NT-proBNP, N-terminal pro-b-type natriuretic peptide.
Symptom distress: Disagreement between patients and family caregivers
Family caregivers reported a higher number of symptoms than patients (mean [SD] symptoms with VAS score >30 mm: 8.2 [3.5] versus 7.3 [3.6], respectively [p<0.0005]). ICC for the number of symptoms was 0.62 (95% CI: 0.50-0.71) (Fig. 1).

Agreement between the mean number of symptoms with a Visual Analogue Scale (VAS) score >30 mm as reported by patient and family caregiver: scatter plot. Dotted line=line of equality; n=159.
Severity of the symptoms fatigue, coughing, muscle weakness, loss of appetite, low mood, anxiety, panic attacks, edema, and chest pain was rated higher by family caregivers than patients (Table 2). Severity of mouth problems was rated lower by family caregivers than patients. Mean VAS scores were comparable between patients and family caregivers for severity of dyspnea, sleeplessness, pain, pruritus, thirst, muscle cramps, restless legs, and dizziness.
N=159.
N=158.
Wilcoxon signed-rank test was used because of skewed data.
For most symptoms, agreement between patients and caregivers about severity was moderate (ICC 0.41-0.60). Agreement was fair (ICC 0.21-0.40) for severity of muscle weakness, panic attacks, and mouth problems. Agreement was substantial (ICC 0.61-0.80) for dyspnea, loss of appetite, pain, and muscle cramps (Table 2).
Agreement for severity of dyspnea, fatigue, and muscle weakness is shown in Fig. 2. A clinically relevant difference in VAS scores was present in the majority of the patient-caregiver pairs for dyspnea, fatigue, and muscle weakness (64.2%, 67.9%, and 71.1%, respectively). None of the following patient and caregiver characteristics were determinants of a clinically relevant difference: age of the patient, sex of the patient, age of the caregiver, sex of the caregiver, caregiver being the spouse, and caregiver providing help with personal care (Table 3). A lower proportion of CHF than COPD patient-caregiver pairs had a comparable VAS score for fatigue.

Bland and Altman plots showing the difference between Visual Analogue Scale (VAS) score as reported by the patient and family caregiver against the mean VAS score for patient and family caregiver for dyspnea (upper left panel), fatigue (lower panel), and muscle weakness (upper right panel). Area between the dotted lines=no clinically relevant difference in VAS scores; n=159.
Values expressed as mean (SD) or number of patients (%).
Comparable=difference in VAS score between patients and caregivers is between −10 mm and +10 mm.
p<0.05 versus CHF for fatigue; for all other comparisons p>0.05.
COPD, chronic obstructive pulmonary disease; CHF, chronic heart failure; CRF, chronic renal failure; VAS, Visual Analogue Scales.
Satisfaction with treatment: Disagreement between patients and their family caregivers
In general, family caregivers were less satisfied with medical treatment than the patients (median [IQR] VAS score: 73 [55-84] mm versus 80 [72-93] mm, respectively, p<0.0005). Agreement between patients and caregivers was fair (ICC 0.21 [95% CI: 0.05-0.35]) (Fig. 3).

Agreement between Visual Analogue Scale (VAS) scores for satisfaction with medical treatment as reported by patients and their family caregivers: scatter plot. Dotted line=line of equality; n=159.
Agreement about presence of symptom-related interventions was moderate (kappa 0.41-0.60) for dyspnea, muscle weakness, loss of appetite, sleeplessness, low mood, anxiety, mouth problems, pruritus, edema, and muscle cramps (Table 4). For coughing, pain, thirst, dizziness, and chest pain, agreement was fair (kappa 0.21-0.40). Finally, agreement was poor for the symptoms fatigue, panic attacks, and restless legs (kappa ≤0.20).
N=159.
Discussion
Key findings
The present study shows that agreement about symptom distress among clinically stable outpatients with advanced chronic organ failure and their family caregivers is fair to moderate for most symptoms. In addition, agreement between patients and caregivers concerning satisfaction with medical treatment is fair, whereas agreement concerning symptom-related interventions is poor to moderate. Variations between patients' and family caregivers' ratings are not related to general patient or family characteristics.
Disagreement in symptom distress
The present study shows that family caregivers of outpatients with advanced chronic organ failure rate severity of many symptoms higher than patients. This is in accordance with previous studies in advanced cancer patients showing that family members rate symptoms as more severe than patients.20,38 The present findings confirm our hypothesis. Indeed, agreement among patients with advanced chronic organ failure and their family caregivers concerning severity of symptoms was only fair to moderate. Moreover, for the most important symptoms (dyspnea, fatigue, and muscle weakness) a clinically relevant difference existed between the majority of patients' and family caregivers' ratings. Again, this disagreement is in line with previous findings in cancer patients20,21 and shows the complexity of proxy reporting. Information from family caregivers is indirect and the perception of family caregivers may be affected by their own needs and problems. 21 In the current study, the presence of clinically relevant differences in symptom severity was not related to patient or caregiver characteristics, such as, for example, being the spouse of the patient. A previous study also found that agreement between family members and health care professionals was not affected by being the spouse or not. 21
A qualitative study has identified several difficulties raised by bereaved family members of cancer patients concerning evaluation of symptoms of the patient. 39 One of the identified problems was a lack of communication between patients and family caregivers, for example when patients did not communicate about symptoms such as anxiety or depression. Furthermore, family members use event cues, such as admission to hospital, to recall the presence of symptoms, rather than symptom intensity or frequency. In addition, they base their assessment of symptoms on signs commonly associated with the symptom such as the use of analgesics and facial expressions for pain, the expression of fear and agitation for anxiety, and sadness and a reduction in functioning for depression. Finally, even when proxies are unsure, they are willing to give a response. 39
Disagreement in satisfaction with medical treatment
Previous authors suggested that retrospective assessments of family members of cancer patients concerning service provision are valid, whereas symptom assessments are not. 21 The study of Higginson and colleagues showed good correlation between ratings of family members and patients about health care professionals (Spearman correlation 0.62), although family members rated health care professionals higher than patients. 20 Therefore, we hypothesized a priori that agreement between patients with advanced chronic organ failure and their family caregivers would be better for satisfaction with treatment and presence of symptom-related interventions than for severity of symptoms. However, the present study shows the opposite. Indeed, family caregivers were less satisfied with medical treatment than the patients and agreement between patients and caregivers was only fair. Furthermore, agreement concerning the presence of symptom-related interventions was poor to moderate.
Implications for future studies and clinical care
Our study, in the context of these previous studies, shows that the value of proxy reporting in assessment of severity of symptoms, the presence of symptom-related interventions, and satisfaction with medical treatment in advanced chronic organ failure is limited. This highlights the importance of prospective studies including the perception of patients. However, the difficulties in prospectively obtaining information from terminally ill patients can make the use of proxy reporting sometimes unavoidable in end-of-life care. It is important to realize that studies using family caregivers to act as proxies for the views of patients reflect the views of family caregivers. These may be important outcomes, but do not accurately represent the patients' experiences. This should be taken into account in interpreting the results of studies using proxy reporting.
For clinical care, it's important to pay attention to both the perception from the patient as well as the perception from the family caregiver of symptom distress, the presence of symptom-related interventions, and satisfaction with medical treatment. In fact, family caregivers' perceptions of these aspects may have important consequences for family caregivers and the process of family caregiving. Indeed, unresolved symptoms are an important concern for family caregivers.40,41 Furthermore, lower agreement between cancer patients and their family caregivers concerning symptom distress is associated with increased caregiver burden. 38 Finally, caregivers' perceptions of symptom distress and symptom management have been shown to influence adjustment to bereavement in family caregivers of cancer patients. 42 Encouraging patients and their families, for example by a nurse or social worker, to communicate about symptom distress may increase understanding of the patient's experience, may promote the relationship between patient and family caregiver and may facilitate the process of family caregiving.
Limitations
The present study has several limitations that should be considered in interpreting the results. First, some patients refused to identify a family caregiver because they were afraid of the burden of the study for those caregivers. In addition, some of the eligible family caregivers refused to participate. It may be possible that these family caregivers rate symptom distress or medical treatment different than family caregivers who were willing to participate. This confirms the previously reported challenge of recruiting family caregivers of patients with advanced chronic diseases. 43 Second, the majority of participating caregivers were female partners of patients. The present study did not show an association between the presence of a clinically relevant difference in symptom VAS scores and sex of the family caregiver. Nevertheless, a study in patients with advanced cancer has shown a higher level of disagreement in symptom perception between patients and male family caregivers. 38 Consequently, the fact that the experience of male family caregivers is under-represented in this study may have influenced our results. Third, differences were present among patients with COPD, CHF, and CRF. For example, CHF patients were older than CRF or COPD patients. However, age of the patient and age of the caregiver did not show a relationship with the presence of clinically relevant differences in VAS scores. Therefore, we expect that age differences did not significantly influence our results. Fourth, participants were asked to rate symptom severity during the previous 2 weeks. It may be possible that agreement would have been better if participants were asked to rate symptom severity within a shorter time frame. Fifth, low levels for kappa for symptom-related interventions that were rarely reported (such as interventions for panic attacks or restless legs), might be influenced by the fact that these interventions were only reported by a few patients and may not necessarily reflect low rates of overall agreement. 36 Sixth, the present study is a prospective study in patients with advanced chronic organ failure and their family caregivers, and proxy reporting is frequently used in bereavement studies using retrospective assessment of symptoms or terminal care. 1 Therefore, it is difficult to extrapolate these findings to reliability of proxy reporting in retrospective post-bereavement studies. Whether the study is prospective or retrospective can have an impact on the level of agreement between patients and family caregivers. In retrospective assessment of symptoms, family members rate problems as mild or severe and avoid the midpoints of a rating scale, in contrast to what is seen in prospective assessment. Moreover, retrospective assessments may be altered by grief or recall difficulties. 21 Therefore, it might be possible that family caregivers in bereavement studies rate symptoms or terminal care even more differently than patients would have done in prospective assessment. Finally, the current findings need to be interpreted in the light of the number of comparisons that were made in the present study. Nonetheless, multiple findings in the same direction, rather than a single statistically significant result, suggest that these are not due to chance alone.
Conclusions
The present study shows a considerable level of disagreement about symptom distress and symptom-related interventions among clinically stable outpatients with advanced chronic organ failure and their family caregivers. Furthermore, agreement about satisfaction with medical treatment is only fair. This has important consequences for research and clinical care. In interpreting the results from studies using family caregivers to act as proxies for the views of patients, it is necessary to realize that these studies reflect the views of family caregivers, but do not accurately represent the patients' experiences. For clinical care, it's important to pay attention to the perception from the patient as well as the perception from the family caregiver of symptom distress, presence of symptom-related interventions, and satisfaction with medical treatment.
Footnotes
Acknowledgments
This project was supported by: Proteion Thuis, Horn, the Netherlands; CIRO+, centre of expertise for chronic organ failure, Horn, the Netherlands; Grant 3.4.06.082 of the Netherlands Asthma Foundation, Leusden, the Netherlands; and the Weijerhorst Foundation, Maastricht, the Netherlands; Stichting Wetenschapsbevordering Verpleeghuiszorg (SWBV), Utrecht, the Netherlands. No funding source had any role in design and conduction of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and in the decision to submit the manuscript for publication.
The authors are grateful to research nurses Els Verstraeten, R.N, and Mrs. Jamila Dekker-Heuts, R.N., for collection of the data and to Linda Koolen for input of the data. The authors are grateful to the doctors of the following collaborating hospitals and departments for their participation in this study: Maastricht University Medical Centre+ (MUMC+), Maastricht, the Netherlands: Department of Respiratory Medicine, Department of Cardiology, Department of Internal Medicine; Laurentius Hospital, Roermond, the Netherlands: Department of Respiratory Medicine, Department of Internal Medicine; St Jans Gasthuis, Weert, the Netherlands: Department of Cardiology; Màxima Medical Centre, Veldhoven/Eindhoven, the Netherlands: Department of Internal Medicine, Department of Cardiology; Catharina Hospital, Eindhoven, the Netherlands: Department of Respiratory Medicine, Department of Internal Medicine; Atrium Medical Centre, Heerlen, the Netherlands: Department of Cardiology, Department of Internal Medicine; Orbis Medical Centre, Sittard, the Netherlands: Department of Internal Medicine.
Author Disclosure Statement
No competing financial interests exist.
