The CaregiverVoice Survey: A Pilot Study Surveying Bereaved Caregivers To Measure the Caregiver and Patient Experience at End of Life

Abstract

Objective:

To integrate patient and caregiver feedback into end-of-life (EOL) care improvement, we tested the feasibility of a standardized, common instrument to measure care experiences across multiple settings in the last three months of life.

Methods:

We developed and tested a survey, called the CaregiverVoice survey, which combined two validated questionnaires, the FAMCARE-2 and VOICES-SF. A retrospective, observational design was used to survey bereaved caregivers of decedents who had received homecare services in Ontario, Canada.

Results:

In total, 330 surveys were completed (overall response rate of 13%, regional rates ranged from 4% to 83%). There was less than 5% missing data. Most patients received care from multiple settings in the last three months of life, including 60% for which a hospital stay was reported. The overall mean of the 19 FAMCARE-2 items was 1.7 (SD 0.7), with 72% of ratings as 1 very satisfied to 2 satisfied. On VOICES-SF items, 6% of respondents rated “all end-of-life services” as fair or poor, 24% as good, and 70% as excellent or outstanding, with variation depending on care site rated. 13% of caregivers reported that pain management was fair or poor in the last week of life.

Conclusions:

This pilot study provides preliminary evidence that it is feasible to capture the patient and caregiver experience at EOL using a comprehensive survey, though survey distribution method greatly affected response rates. The majority of responses rated care as excellent or very good, although several specific areas for improvement were identified.

Introduction

Many countries have increased attention on patient-reported outcomes and experience measures to capture unique aspects of quality that only patients can provide, which includes end-of-life (EOL) care.¹ The experience of EOL care from the perspectives of patients as well as their caregivers is an important dimension of quality, one that is not well captured in process measures¹ or administrative data analysis.² A recent review of instruments to measure patient and family perceptions of EOL care found 51 different surveys,³ of which only 12 were cited in more than 2 articles. The systematic review noted wide variation in comprehensiveness and in quality domains⁴ covered among the surveys identified. Many EOL care measures are either short satisfaction-type surveys (e.g., Palliative Outcome Scale)⁵ or more extensive interviewer administered phone questionnaires (e.g., Mortality Follow-back Survey).⁶ Although shorter and less expensive to administer, satisfaction scales are prone to ceiling effects, restricted ranges in scores, and acquiescence bias.^7–10 This might be explained by patients and caregivers being appreciative of services received, even if their needs remained unmet. Capturing through questionnaires patients’ experiences—as opposed to merely satisfaction—and the contexts of these perceptions can identify gaps in care, enabling providers and health service planners to make service improvements.¹⁰

Collecting questionnaire data from patients who are actively dying, however, is a formidable challenge, because patients are often too sick and families are too overwhelmed to respond to surveys or interviews, resulting in poor response rates.^11,12 Moreover, collecting data prior to the patient's death means that the care provided immediately before death—which can be most crucial—and bereavement care are not captured. Alternatively, many research studies have used questionnaires for bereaved caregivers,^3,13 which has been found to be an adequate substitute for collecting these data directly from the patient^9,11,14,15 and has the advantage of including the full EOL care trajectory across different settings and providers.¹⁴ A limitation is that bereaved caregiver questionnaires focus mainly on the last setting of care before death.^6,16 EOL surveys also rarely include open-ended items for collecting unique caregiver insights.³ Furthermore, mailed surveys, and especially telephone surveys, are resource-intensive to administer. For instance, the U.S. Consumer Assessment of Healthcare Providers and Systems Hospice Survey developed by the Centers for Medicare & Medicaid Services asks 47 standardized questions of bereaved primary caregivers solely about hospice care with no open-ended questions and is administered by mail or telephone.¹⁷

This pilot study was conducted to test the feasibility of administering a bereaved caregiver survey, called the CaregiverVoice survey. This comprehensive questionnaire examines satisfaction and quality of the EOL care experience for both patients and caregivers across multiple settings, providers, and critical time periods along the EOL trajectory. The CaregiverVoice survey includes open-ended responses for input that might promote quality improvement activities for health service providers. We also tested online administration of the survey to reduce data collection errors and costs and determine scalability. Both the multisetting nature of the questionnaire and the availability of online completion were of particular interest to test the feasibility of the larger goal of having a standardized, common instrument to measure the experience of EOL across all settings of care in the province of Ontario, Canada.

Methods

Study design and population

We developed and tested a survey, called the CaregiverVoice survey, to investigate the experience of EOL care for patients and their caregivers (see Supplementary Appendix 1 for survey items; Supplementary Appendix 1 is available online at www.liebertpub.com/jpm). The survey study design was a retrospective, observational design administered between September 2012 and January 2014. We surveyed bereaved caregivers of decedents who had received formal palliative homecare services in Ontario. The survey was administered by 6 of 14 publicly funded, regional organizations coordinating homecare services in the province. These 6 agencies were chosen because they volunteered to be part of a demonstration project for improving palliative homecare, where assessment of patient experience was part of the evaluation. The regions included represent both urban and rural areas.

CaregiverVoice survey development

We conducted a review of the literature on instruments to measure experience with EOL care. We identified several validated surveys and compared them for ease of use, comprehensiveness of quality domains covered, extent of testing in the desired population, and other qualities. Our final survey incorporated the complete items from two existing validated instruments: FAMCARE-2¹⁸ and VOICES-Short Form,^19,20 as the data collected from those two were complementary.

FAMCARE was selected in part because it is endorsed by Accreditation Canada for assessing palliative care from family caregivers²¹ and is the most widely cited survey for assessing EOL care satisfaction.³ The FAMCARE-2 scale measures patient satisfaction with the general quality of palliative care in a particular setting (e.g., intensive care unit), and has been adapted for use in various settings.^18,22 It is a validated instrument (overall internal consistency α = 0.93) and contains 17 five-point items within four domains: information giving, availability of care, psychological care, and physical care. FAMCARE items are assessed on a five-point scale, from 1 = very satisfied to 5 = very dissatisfied, as well as a don't know option. The original FAMCARE scale has been used extensively in Canada and other countries for examining family satisfaction with care of cancer and noncancer populations, and in homecare settings.²² For our survey, the FAMCARE items were framed in terms satisfaction with the homecare providers in the last three months of life. We added two items: one to assess information continuity and one to assess providers’ respect for cultural diversity (see Appendix 1), both aspects of care that have been identified as important in quality of palliative care frameworks.²³

The VOICES-SF is a validated questionnaire containing 52 items assessing various dimensions of the caregiver's perceptions of the patient's care experiences with providers and services in multiple care settings across in the last three months of life.^19,24 This instrument was used in the National Bereavement Survey (2011–2014) in England.^20,25 The survey features questions on a five-point rating scale, multiple choice, and opened-ended items covering multiple settings and provider types. The VOICES items examine different aspects of care, such as homecare, nursing care, family physician care, accessing urgent care, and symptoms and treatment. Different settings, including home, hospital, long-term care facility, and residential hospice, are also accessed. VOICES also reviews different time periods, such as the last three months of life, the last two days of life, and circumstances surrounding the death. A series of questions are repeated for each relevant setting of care allowing for comparisons (e.g., overall assessment of care in that setting). Minor modifications were made to examine the last week of life beyond the last two days and to item terminology for a Canadian audience. For example, “district nurse” was changed to “homecare nurse.”

Caregivers had a choice to complete the survey on paper or online (palliativecareinnovation.com). The survey contains skip patterns, so that caregivers only respond to items relevant to the types of care the patient received. The survey took approximately 45 minutes to complete.

Data collection

Family caregivers of deceased patients were identified through administrative review of client records. A flexible approach to survey distribution was taken to accommodate the available resources of each site, given that no additional funding or support was provided for data collection. The method of survey distribution varied by region (see Table 1), which allowed for the comparison of these different approaches, and corresponding response rates were obtained. Sites employed a consecutive sampling method within their region, though they started and ended survey distribution at different times within our study period. This sampling method was chosen because the demonstration project was not of sufficient scale to implement a random sampling method. Each organization site used the same study contact letter with their own letterhead and contact information inserted. All sites contacted caregivers via a phone call or mailed study letter six weeks after the patient died, with the exception of one site that mailed the study letter after two weeks (region 5) and one site that wanted to give caregivers more time for grieving and therefore waited up to six months to contact (region 2). The study letters had the same text, which explained the survey, contained the online link to the survey, and provided contact information to request a hardcopy of the survey if needed. One site included a paper survey in all initial letters to caregivers (region 5). Some sites followed a more involved protocol for engaging potential respondents including phone calls and follow-up contacts, often incorporated with bereavement support, to encourage survey completion.

Table 1.

Response Rates and Distribution Method

Region	# distributed	Completed (% online) ^a	Response rate	Distribution method
1	1449	59 (86.4%)	4.1%	Survey letter^b mailed to bereaved caregivers with no further contact. A paper copy of the survey could be requested by the caregiver.
2	398	21 (85.7%)	5.3%	Survey letter mailed to bereaved caregivers after up to 6 months following death with no further contact. A paper copy of the survey could be requested by the caregiver.
3	220	38 (34.2%)	17.3%	Initially, survey letter was mailed to all bereaved caregivers. After 60 letters mailed with no responses, process was changed to include a phone call made by the care coordinator to all caregivers from that point forward, prior to sending letter. If during the call the family preferred a printed copy of survey, the letter and survey was mailed with SASE.
4	169	33 (84.8%)	19.5%	Survey letter mailed to all bereaved caregivers—asked to contact if paper survey required SASE. Paper survey also left in patients’ in-home chart binder, and homecare nurse encouraged caregiver to complete the survey only after the patient died.
5	326	117 (38.5%)	43.7%	Letter and paper survey mailed to all bereaved caregivers 2 weeks after patient death in one mailing. This was followed up with a reminder letter.
6	74	60 (31.7%)	83.3%	Phone calls made to all bereaved caregivers to introduce survey and seek permission to mail survey letter and/or paper survey SASE. Letter and/or paper survey mailed. Mailed reminder letter one week later. Follow-up phone call made to caregiver to remind of survey.

Two respondents who did not indicate region are not included in this table.

All survey letters, besides the regional organizational letterhead, contained the same letter text, which explained the survey, contained the online link to the survey, and provided contact information to request a hardcopy of the survey if needed.

SASE, self-addressed stamped envelope.

An inclusion criterion for survey participation was the ability to read English, as the survey was not available in other languages. Completion of the survey was anonymous, with no tracking of respondents, to protect patient and caregiver privacy, a requirement by ethical officers at some regional sites. The study received approval by the Hamilton Health Sciences – McMaster University research ethics review board, Hamilton, Ontario, Canada. The platform used for the online survey was LimeSurvey (Hamburg, Germany: LimeSurvey Project), which was hosted on a secure server at McMaster University. Completed paper surveys were entered into LimeSurvey by the research team.

Data analysis

Data were exported from the LimeSurvey database to SPSS 20.0 (IBM, Armonk, NY) for statistical computations. The internal consistency of the FAMCARE-related items was calculated using Cronbach's alpha. Pearson correlations (two-tailed) were used between select items to examine caregiver response tendencies between different settings of care. Descriptive statistics were used to summarize caregiver and patient characteristics and perceptions of services used.

Results

In total, 2636 caregivers were approached and 330 surveys were completed for an overall response rate of 13% (regional range of 4% to 83%). Table 1 presents the distribution method and survey response rate for each study region. Phone calls with a follow-up contact resulted in the highest response rates. About half completed the survey online versus on paper. Nearly all (97%) respondents answered all relevant items on the survey, with less than 5% missing data. Almost half (48%) completed an open-ended comment about “what was bad” and 78% about “what was good” about care, which is planned as a separate qualitative analysis.

Deceased patient characteristics are presented in Table 2. Half of patients were male and 62% were 70 years or older. The major illness reported for most patients was cancer (84%). The majority reported European or Caucasian descent (80%) and indicated a Christian-based religion (72%). The caregiver respondents tended to be younger than patients and female (66%). A third of caregivers were the patient's son or daughter and approximately 60% were the patient's spouse.

Table 2.

Demographics of Deceased Patients (n = 330)

Gender	n (%)
Male	155 (47.3)
Female	173 (52.7)
Age
Under 40	1 (0.3)
40–49	15 (4.5)
50–59	33 (10.0)
60–69	77 (23.3)
70–79	87 (26.4)
80–89	85 (25.8)
90+	31 (9.4)
Main diagnosis
Upper gastrointestinal cancer	57 (17.5)
Lung cancer	43 (13.2)
Colorectal cancer	29 (8.9)
Breast cancer	13 (4.0)
Miscellaneous cancer	129 (39.1)
COPD	9 (2.8)
Heart disease	9 (2.8)
Miscellaneous noncancer	30 (9.1)
Unknown diagnosis	6 (1.8)
Ethnic group
European/“Canadian”	263 (79.7)
Chinese	7 (2.1)
South Asian	7 (2.1)
Other^a	14 (4.2)
Missing	39 (11.8)
Religion
Christian	236 (71.5)
No religion	55 (16.7)
Other	21 (6.4)
Missing	18 (5.5)
Settings/providers of care,^b n (%)
Had a palliative care doctor	167 (54.8%)
Had visiting hospice volunteers	61 (20.4%)
Stayed in hospital	188 (59.3%)
Stayed in residential hospice	60 (19.5%)
Stayed in long-term care home	24 (7.8%)

Includes Middle Eastern, Filipino, African Canadian, Caribbean, Latin American, and other ethnicities.

In the last three months of life—multiple settings and providers are possible.

Using FAMCARE to assess homecare satisfaction

Overall satisfaction with homecare services was measured using the FAMCARE scale (see Table 3). The FAMCARE portion of the survey exhibited excellent overall internal consistency, α = 0.97, consistent with that previously reported for the original FAMCARE survey.¹⁸ The overall mean of the 19 FAMCARE items was 1.7 (SD 0.7), with 72% of ratings as 1, very satisfied to 2, satisfied (see Table 3). For the four subscales, mean scores were 1.7 (SD 0.7) for management of physical symptoms and comfortm, 1.8 (SD 0.8) for provision of information, 1.7 (SD 0.7) for family support, and 1.7 (SD 0.7) for patient psychological care. The item rated as most satisfied was “the way in which the homecare providers respected his or her dignity” (mean score 1.4), whereas the least satisfied was “information given about the side effects of treatment” (mean score 1.9).

Table 3.

FAMCARE Mean Scores (n = 323)

FAMCARE-2 subscales	Mean^a	SD
Management of physical symptoms and comfort	1.7	0.7
Provision of information	1.8	0.8
Family support	1.7	0.7
Patient psychological care	1.7	0.7
Questions: Indicate the extent to which you were satisfied with the following in the last three months of the patient's life.
His/her comfort	1.8	0.8
The way in which his/her condition and likely progress were explained by the homecare providers	1.8	0.8
Information given about the side effects of treatment	1.9	0.9
The way in which the homecare providers respected his/her dignity	1.4	0.7
Meetings with the homecare providers to discuss his/her condition and plan of care	1.7	0.9
Speed with which symptoms were treated	1.9	1.0
Homecare providers’ attention to his/her description of symptoms	1.7	0.8
The way in which his/her physical needs for comfort were met	1.7	0.9
Availability of the homecare providers to the family	1.7	0.9
Emotional support provided to family members by the homecare providers	1.8	0.9
The practical assistance provided by the homecare providers (e.g., bathing, respite)	1.8	0.9
The doctor's attention to his/her symptoms	1.8	1.1
The way the family was included in treatment and homecare decisions	1.6	0.9
Information given about how to manage his/her symptoms (e.g., pain, constipation)	1.7	0.9
How effectively the homecare providers managed his/her symptoms	1.8	0.9
The homecare providers’ response to changes in his/her care needs	1.7	0.9
Emotional support provided to him/her by the homecare providers	1.8	0.9
Homecare providers knowing about the problems that had bothered him/her at previous visits	1.8	0.9
Homecare provided in a manner respectful of his/her culture, customs, and faith	1.6	0.7
Grand mean	1.7	0.7

Range from 1 = very satisfied to 5 = very dissatisfied.

Using VOICES to measure experience in last three months of life

Within the VOICES section we found variation both within caregivers’ responses to the items and between respondents, depending on the care trajectory and care setting experienced, which implies that differentiating data were elicited. During the last three months of life, all patients received homecare at some point, slightly over half (55%) reported having a palliative care doctor, and almost 60% reported having a hospital stay (see Table 2). Just over half (53%) needed to contact a health professional for something urgent in the evening or during the weekend at least once or twice, and 24% needed to do so more often. Six percent of respondents rated “all end-of-life services” as fair or poor, 24% as good, and 70% as excellent or outstanding. The providers with the most fair or poor scores were those providing homecare on evenings or weekends (20%), followed by hospital nurses (14%), hospital doctors (13%), and primary care doctors (13%) (see Fig. 1).

FIG. 1.

Percentage of respondents who rated care as “fair” or “poor” across end-of-life care settings (n = 330). Response options were Excellent, Good, Fair, and Poor.

Generally, across all sites of care about 70% of caregivers reported that the patient's pain was completely relieved either all or some of the time or that there was no pain to begin with. Conversely, the settings with the worst pain management were at home, with 31% stating pain was only partially or not at all managed, and during the last hospital admission (25%) (see Fig. 2). Pain relief appears to have been perceived best in hospice compared to other settings.

FIG. 2.

Percentage of respondents who reported that pain was “only partially managed” or “not at all” across end-of-life care settings and trajectories (n = 330). Response options were in respect to the extent to which pain was managed: Does not apply (he/she did not have any pain), Completely all of the time, Completely some of the time, Partially, and Not at all.

Using VOICES to measure experience in last week of life

In the last week, 47% of caregivers reported that pain management was excellent (see Table 4). In terms of relief of other symptoms, spiritual support, and emotional support, 17%, 13%, and 13% of caregivers assessed each of these domains as either fair or poor, respectively. Responses showed 58% died at home, 21% in hospital, 18% in hospice, 2% in a long-term care home, and 1% in a palliative care unit of a hospital. Most patients with a known preference wanted to die either at home (81%) or in a residential hospice (10%). At the time of the death, 71% of caregivers said that they “definitely” got enough help and support at the time of the patient's death versus “to some extent” or “not at all.”

Table 4.

Assessment of Support in the Last Week of Life

	Assessment of support provided
	Excellent		Good		Fair		Poor		Does not apply		Don't know
Domain of support	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)
Relief of pain (n = 311)	145	(46.6)	97	(31.2)	24	(7.7)	16	(5.1)	27	(8.7)	2	(0.6)
Relief of other symptoms (n = 308)	133	(43.2)	106	(34.4)	40	(13.0)	11	(3.6)	10	(3.2)	8	(2.6)
Spiritual support (n = 279)	93	(33.3)	56	(20.1)	23	(8.2)	12	(4.3)	84	(30.1)	11	(3.9)
Emotional support (n = 300)	140	(46.7)	90	(30.0)	23	(7.7)	15	(5.0)	24	(8.0	8	(2.7)
Support to stay where patient wanted to be (n = 311)	204	(65.6)	67	(21.5)	16	(5.1)	9	(2.9)	11	(3.5)	4	(1.3)

Discussion

Our pilot project to test the CaregiverVoice survey provides some preliminary evidence that it was feasible and acceptable to approach bereaved caregivers to capture the patient and caregiver experience at EOL across multiple care settings and domains of support. The limited missing data indicate that respondents were not overburdened to complete the survey, though it is comprehensive and long. Half completed the survey online, which shows promise as a cost-effective mode of distribution. Generally, about 70% of respondents felt that all providers and aspects of care were excellent, 20% felt care was generally good but identified some issues, and 10% reported major problems with the care experienced. Some identified issues included symptom management after hours and during the last week of life. These poorly rated issues serve as examples of how a standardized EOL experience survey could inform frontline providers on where to improve in quality improvement initiatives.

Using validated tools allows us to compare our data to other studies. The FAMCARE scale has been used in diverse palliative care populations: hospital inpatient,²⁶ palliative care unit,²⁷ outpatient clinic,^28–31 hospice,³² and mixed inpatient and homecare settings.^18,33 The latter two have comparable populations and reported overall mean values of 1.9 for advanced cancer outpatients in Toronto, Canada (n = 461)³³ and 1.5 for patients of specialist palliative care services in Australia (n = 461),¹⁸ which are similar to our overall mean value of 1.7. Our cohort is different from these studies, as it is solely a community-based population, and the participating sites were part of a demonstration project for improving palliative care. As such, the study sites are likely better performing than the average homecare region in Ontario, as evidenced by data showing patients in these and similar interventions in the province have a hospital death rate of 21%³⁴ compared to the average of 55% for cancer decedents.³⁵ Similarly, comparing our findings to the National Bereavement Survey in England, which is based on the VOICES instrument, the caregivers in our study rated the EOL care experience as slightly more favorable.³⁶ National Bereavement Survey respondents’ overall assessment of care in the last three months of life was 44% as outstanding or excellent, 34% as good, and 23% as fair or poor, compared to our cohort of 70%, 24%, and 6% for the same categories. In terms of how well pain was relieved while at home, 44% said partially and 7% said not at all in England, whereas our cohort rated pain relief as 28% (partially) and 4% (not at all). In both countries, spiritual support was the least favorably rated among the care domains, which might reflect the biomedical bias in EOL care and challenges with integrating multidisciplinary care.³⁷

Our feasibility study highlighted several important learnings for broader implementation. First, the FAMCARE data demonstrated a ceiling effect, but it has been well used to measure satisfaction, important for accreditation and accountability. Our VOICES data identified specific gaps and strengths in care, having more variation in responses and being less positively skewed, which is more useful for quality improvement. For instance, half of caregivers made specific comments about something bad about the EOL care provided covering several settings. Thus, depending on the focus of the user, the survey could be shortened to either FAMCARE-2 (accountability with respect to satisfaction) or VOICES-SF (quality improvement focus).

Second, the distribution method and response rates varied considerably. Studies involving surveys of bereaved caregivers often report suboptimal response rates.^16,20,38,39 Despite prior evidence that having at least one direct contact with the caregiver and follow-up will lead to higher response rates,^40–42 some sites still chose to mail out the survey with no follow-up. We were also hindered because we could not follow up with nonresponders due to the anonymous nature of the survey. This variation in distribution method, as well as the differences in time of distribution of survey after death (ranging from two weeks to six months), is a reflection of the voluntary and unfunded nature of the pilot. The longer the time between death and survey distribution, the greater the possibility for recall bias, which may dull the intensity of the experience with time. While comparability and generalizability is affected due to these inconsistencies, the pilot highlighted the value of patient-reported outcomes for quality improvement and potential learnings to overcome real-world feasibility challenges.

For future implementation, a consistent protocol for distribution and follow-up, with a standardized timeframe, would be advisable, endorsed from an authoritative source and/or an expectation for quality accreditation. Moreover, since resources are required for direct contact and follow-up, surveyors may want to avoid large population-based samples and instead conduct institutionally led surveys on smaller random samples more regularly, which is more aligned with quality improvement methodology.

Beyond the limitations of varied survey distribution methods, our results are also limited by the structured nature of quantitative data collection (e.g., FAMCARE items), which may not adequately capture the unique context and experience lived by each respondent; the open-ended questions included are an improvement compared to most other surveys but were not extensive.¹⁶ Maintaining anonymity prevented us from targeted follow-up to increase response rates and from linking the caregiver experience to clinical data on the patient. More research is required to determine if anonymity is necessary. Finally, because the survey was only available in English, patients and caregivers who are non-English speakers are likely underrepresented in our data. Further refinement and testing of the CaregiverVoice survey using more representative sampling methods and stakeholder consultation is required to gather more information across diverse causes of death and in varied settings such as hospices, hospitals, and long-term care facilities.

In conclusion, this study piloted an EOL experience survey, a combination of two widely used palliative care instruments, that examined experience and satisfaction in the last three months and last week of life. Our results show that an online and paper survey are feasible means to capture experience and satisfaction ratings at EOL. Even though the patients represented received homecare at EOL, many received other services and died in other settings. Because the CaregiverVoice survey asks about multiple settings and providers, it has potential to be the one common measure to be used across all settings where people die. This research contributes to the aim of standardizing patient and caregiver EOL experience feedback to improve palliative care quality in Ontario, Canada and in large health systems in general.

Footnotes

Acknowledgments

Funding was received for this study from the Ontario Ministry of Health and Long-Term Care.

Author Disclosure Statement

We are not aware of any relationships or support that could be perceived as conflict of interest.

References

, Kiley

, Ast

, et al.: Measuring what matters: Top-ranked quality indicators for hospice and palliative care from the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association. J Pain Symptom Manage, 2015; 49:773–781.

Earle

, Park

, Lai

, et al.: Identifying potential indicators of the quality of end-of-life cancer care from administrative data. J Clin Oncol, 2003; 21:1133–1138.

Lendon

, Ahluwalia

, Walling

, et al.: Measuring experience with end-of-life care: A systematic literature review. J Pain Symptom Manage, 2015; 49:904–915.

National Quality Forum: A National Framework and Preferred Practices for Palliative and Hospice Care Quality. Washington, DC: National Quality Forum, 2006.

Hearn

, Higginson

: Development and validation of a core outcome measure for palliative care: The Palliative Care Outcome Scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care, 1999; 8:219–227.

Teno

, Clarridge

, Casey

, et al.: Family perspectives on end-of-life care at the last place of care. JAMA, 2004; 291:88–93.

Streiner

, Norman

: Health Measurement Scales: A Practical Guide to Their Development and Use, 4th ed. Oxford, UK: Oxford University Press, 2008.

Teno

, Byock

, Field

: Research agenda for developing measures to examine quality of care and quality of life of patients diagnosed with life-limiting illness. J Pain Symptom Manage, 1999; 17:75–82.

Tilden

, Tolle

, Drach

, Hickman

: Measurement of quality of care and quality of life at the end of life. Gerontologist, 2002; 42:71–80.

10.

Young

, Rogers

, Dent

, Addington-Hall

: Experiences of hospital care reported by bereaved relatives of patients after a stroke: A retrospective survey using the VOICES questionnaire. J Adv Nurs, 2009; 65:2161–2174.

11.

Higginson

, Priest

, McCarthy

: Are bereaved family members a valid proxy for a patient's assessment of dying?. Soc Sci Med, 1994; 38:553–557.

12.

Teno

, Casey

, Welch

, Edgman-Levitan

: Patient-focused, family-centered end-of-life medical care: Views of the guidelines and bereaved family members. J Pain Symptom Manage, 2001; 22:738–751.

13.

Higgins

, Prigerson

: Caregiver Evaluation of the Quality of End-Of-Life Care (CEQUEL) Scale: The caregiver's perception of patient care near death. PLoS ONE, 2013; 8:e66066.

14.

Burt

, Shipman

, Richardson

, et al.: The experiences of older adults in the community dying from cancer and non-cancer causes: A national survey of bereaved relatives. Age Ageing, 2010; 39:86–91.

15.

Tang

, McCorkle

: Use of family proxies in quality of life research for cancer patients at the end of life: A literature review. Cancer Invest, 2002; 20:1086–1104.

16.

Burge

, Lawson

, Johnston

, et al.: Bereaved family member perceptions of patient-focused family-centred care during the last 30 days of life using a mortality follow-back survey: Does location matter?. BMC Palliat Care, 2014; 13:25.

17.

Centers for Medicare & Medicaid Services BMD: Consumer Assessment of Healthcare Providers and Systems (CAHPS) Hospice Survey. Centers for Medicare & Medicaid Services, 2016. www.hospicecahpssurvey.org. <spell BMD, access date>

18.

Aoun

, Bird

, Kristjanson

, Currow

: Reliability testing of the FAMCARE-2 scale: Measuring family carer satisfaction with palliative care. Palliat Med, 2010; 24:674–681.

19.

Addington-Hall

, O'Callaghan

: A comparison of the quality of care provided to cancer patients in the UK in the last three months of life in in-patient hospices compared with hospitals, from the perspective of bereaved relatives: Results from a survey using the VOICES questionnaire. Palliat Med, 2009; 23:190–197.

20.

Hunt

, Shlomo

, Addington-Hall

: End-of-life care and achieving preferences for place of death in England: Results of a population-based survey using the VOICES-SF questionnaire. Palliat Med, 2014; 28:412–421.

21.

Accreditation Canada. Hospice, palliative, and end-of-life services. 2010. www.accreditation.ca/hospice-palliative-and-end-life-services. (Last accessed May 25, 2016.)

22.

Kristjanson

: Validity and reliability testing of the FAMCARE Scale: Measuring family satisfaction with advanced cancer care. Soc Sci Med, 1993; 36:693–701.

23.

Ferrell

, Connor

, Cordes

, et al.: The national agenda for quality palliative care: The National Consensus Project and the National Quality Forum. J Pain Symptom Manage, 2007; 33:737–744.

24.

Addington-Hall

, Walker

, Jones

, et al.: A randomised controlled trial of postal versus interviewer administration of a questionnaire measuring satisfaction with, and use of, services received in the year before death. J Epidemiol Community Health, 1998; 52:802–807.

25.

UK Department of Health: First National VOICES Survey of Bereaved People: Key Findings Report. London: UK Department of Health, 2012. www.gov.uk/government/publications/first-national-voices-survey-of-bereaved-people-key-findings. (Last accessed May 25, 2016.)

26.

Ringdal

, Jordhoy

, Kaasa

: Measuring quality of palliative care: Psychometric properties of the FAMCARE Scale. Qual Life Res, 2003; 12:167–176.

27.

Ringdal

, Jordhoy

, Kaasa

: Family satisfaction with end-of-life care for cancer patients in a cluster randomized trial. J Pain Symptom Manage, 2002; 24:53–63.

28.

Carter

, Lewin

, Gianacas

, et al.: Caregiver satisfaction with out-patient oncology services: Utility of the FAMCARE instrument and development of the FAMCARE-6. Support Care Cancer, 2011; 19:565–572.

29.

Follwell

, Burman

, Le

, et al.: Phase II study of an outpatient palliative care intervention in patients with metastatic cancer. J Clin Oncol, 2009; 27:206–213.

30.

, Burman

, Hales

, et al.: The FAMCARE-Patient scale: Measuring satisfaction with care of outpatients with advanced cancer. Eur J Cancer, 2009; 45:3182–3188.

31.

, Burman

, Rodin

, Zimmermann

: Measuring patient satisfaction in oncology palliative care: Psychometric properties of the FAMCARE-patient scale. Qual Life Res, 2009; 18:747–752.

32.

Meyers

, Gray

: The relationships between family primary caregiver characteristics and satisfaction with hospice care, quality of life, and burden. Oncol Nurs Forum, 2001; 28:73–82.

33.

Zimmermann

, Swami

, Krzyzanowska

, et al.: Early palliative care for patients with advanced cancer: A cluster-randomised controlled trial. Lancet, 2014; 383:1721–1730.

34.

Seow

, Brazil

, Sussman

, et al.: Impact of community based, specialist palliative care teams on hospitalisations and emergency department visits late in life and hospital deaths: A pooled analysis. BMJ, 2014; 348:g3496.

35.

Canadian Institute for Health Information: End-of-Life Hospital Care for Cancer Patients. Ottawa: Canadian Institute for Health Information, 2013. http://secure.cihi.ca/free_products/Cancer_Report_EN_web_April2013.pdf. (Last accessed May 25, 2016.)

36.

UK Office for National Statistics: National Bereavement Survey (VOICES). London: Office for National Statistics, 2014. www.gov.uk/government/statistics/national-survey-of-bereaved-people-voices-2014. (Last accessed May 25, 2016.)

37.

Piderman

, Kung

, Jenkins

, et al.: Respecting the spiritual side of advanced cancer care: A systematic review. Curr Oncol Rep, 2015; 17:6.

38.

Aspinal

, Hughes

, Dunckley

, Addington-Hall

: What is important to measure in the last months and weeks of life? A modified nominal group study. Int J Nurs Stud, 2006; 43:393–403.

39.

Brazil

, Bainbridge

, Krueger

, et al.: Family caregiver views on patient-centred care at the end-of-life. Scand J Caring Sci, 2012; 26:513–518.

40.

Aspinal

, Addington-Hall

, Hughes

, Higginson

: Using satisfaction to measure the quality of palliative care: A review of the literature. J Adv Nurs, 2003; 42:324–339.

41.

Lawson

, Van

, Burge

: Challenges and strategies in the administration of a population based mortality follow-back survey design. BMC Palliat Care, 2013; 12:28.

42.

Dillman

: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley, 2000.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB