Quality Indicators for Care of Cancer Patients in Their Last Days of Life: Literature Update and Experts' Evaluation

Abstract

Background:

Quality indicators (QIs) are needed to monitor and to improve palliative care. Care of patients in the last days of life is a discrete phase of palliative care and therefore specific QIs are needed. This study aimed to identify and evaluate current QIs against which to measure future care of patients in the last days of life.

Methods:

To identify QIs for patients in the last days of life an update of the literature and national guidelines was conducted. Subsequently, an international panel of palliative care experts was asked to evaluate the identified QIs: how well they describe care and how applicable they are for care in the last days of life. Also additional QIs were asked.

Results:

In total, 34 QIs for care in the last days were identified in the literature and guidelines. The experts (response rate 58%) agreed with seven QIs as being good descriptors and applicable: concerning a home visit for the family following a patient's death, the presence of a dedicated family room, limited patients receiving chemotherapy, limited need for pain control, gastrointestinal symptoms, and communication from professional to patient and family. The experts also suggested 18 additional topics for QIs for the last days of life.

Conclusion:

Currently no definite set of QIs exist to describe quality of care of patients in their last days of life. New QIs that are focused on care for patients in their last days of life, their relatives, as well as their professional caregivers are needed.

Introduction

Quality, within health care, is a multidimensional concept that requires assessment and evaluation. It can be described as the extent to which services for individuals or populations increase the likelihood of desired outcomes and how consistent they are with current knowledge.¹ Accordingly, quality indicators (QIs) are essential elements for assessing and evaluating the quality of care. This applies to all health care, including the care of dying patients and their relatives—a focus of care that has markedly evolved over the last 40 years. Within the Donabedian model, QIs can monitor the structure (e.g., the context in which health care is provided), the process (e.g., the method by which the health care is provided), or the outcome (e.g., the consequences for the patient of health care).² As such, QIs are usually described with a numerator, denominator, and a performance standard, measuring the frequency with which an event occurred, but also inferring judgment about the quality of care provided.³

The development of QIs is based on the assumption that there is available evidence or a consensus position that can be used to assess the quality of care provided. As many areas of health care interventions operate with a limited evidence base, it is often necessary to combine the existing evidence with consensus techniques; such techniques explore consensus among a group of experts to reach a final aggregate opinion, expert-based knowledge.³

In recent years, palliative care has become a prominent public health issue due to aging populations and concomitant increases in health care expenditure (10% to 12% of all health care costs are spent at the end of life).⁴ The development of shared measures to monitor the quality of care provided, both within and across local, national, and international contexts, has become a major focus of research within palliative care. A recent review⁵ gives a comprehensive overview of published QIs for palliative care, independent of patients' diagnosis and care setting. The authors' findings show that the majority of the scientific literature examining QIs is focused broadly on symptomology within palliative care, with little specific attention on the social and spiritual aspects of palliative care, or on the care delivered in the last days of life. Following these findings, a new set of QIs for palliative care has been developed in the Netherlands, which focused on palliative care in general for all settings and all patients.⁶

There is a general consensus defining the domains contributing to quality of palliative care.^5,7 These eight domains are structure and processes of care, physical aspects, psychological and psychiatric aspects, social aspects, spiritual, religious, and existential aspects, cultural aspects, care of the imminently dying patient, and ethical and legal aspects of care.⁷ Accordingly, there is a need for an identification and evaluation of QIs within these domains with which to measure palliative care, with a particular focus on care of patients in the last days of life. The lack of data on the quality of care delivered to patients in the last days of life is a key barrier in driving up the standards of care.^8,9 Therefore, a set of QIs—sustained by research evidence and/or by experts' opinions—that defines standards for “quality” care may assist clinicians in supporting the complex and dynamic needs of the patient and his or her family at this challenging time. Our study aimed to identify and evaluate QIs that can be used to measure quality of care in the last days of life. The following research questions were addressed:

1. What QIs for care of patients in their last days of life currently exist?

2. Are the existing QIs good descriptors of quality of care of patients in their last days of life?

3. Are these QIs also applicable, that is, can these QIs be measured in the context of care for patients in their last days of life?

4. Are additional QIs for care of patients in their last days of life needed?

Materials and Methods

An update was conducted of the current scientific literature and national guidelines of the nine OPCARE9 countries (Argentina, Italy, Germany, the Netherlands, New Zealand, Slovenia, Sweden, Switzerland, United Kingdom) to identify QIs (containing a numerator, denominator, and a performance standard) for patients in the last days of life (Fig. 1). Based on these results, an electronic questionnaire including the identified QIs was developed to assess (a) how well the identified QIs describe care in the last days of life and (b) how applicable they are for care in the last days of life, according to experts' view. Subsequently, the questionnaire was distributed among an international panel of experts in palliative care and experts in developing QI in palliative care.

FIG. 1.

Search strategy literature update.

Databases	Search Strategy	MeSH Headings
Medline PsychInfo CINAHL	#1 “Palliative Care”[MeSH] OR palliat^* OR “end of life” OR “terminal care” OR “Terminal Care“[MeSH:noexp] OR hospice (all fields) OR “Hospice Care“[MeSH]	Palliative Care Terminal Care Hospice Care
Medline PsychInfo CINAHL	#2 “Dying” OR “die” OR “death” OR “terminally ill” OR “end of life” OR “last days of life” OR “last hours of life” OR ”last week of life” OR “final days of life” OR “final hours of life” OR “final week of life”
MedlinePsychInfoCINAHL	#3 Quality of Health Care [MeSH]	Quality of Health Care
MedlinePsychInfoCINAHL	#4 “Quality measure” OR “quality measures” OR “quality criterium” OR “quality criteria” OR “quality assessment” OR “quality marker” OR “Sign of Quality” OR “Assurance Quality Control” OR “Quality Evaluation” OR “Benchmarking” OR ((Quality[ti] OR performance[ti] OR satisf^[ti]) AND (indicator^[tw] OR criteri^[tw] OR assess^[ti] OR measur^[ti] OR scale[ti] OR validat^[tw]))
	(#1 OR #2) AND (#3 AND #4)
MedlinePsychInfoCINAHL	Limits:Only items with abstractsHumansEnglish, German, Dutch, Italian, Slovenian, Spanish, SwedishAll Adult: 19 + yearsNOT “Cell ” OR “apoptosis”NOT (letter[pt] OR editorial[pt] OR comment[pt]Cancer

Identifying current quality indicators for care in last days of life

To identify current available QIs addressing the care in the last days of life, a four-step search of the scientific literature and national guidelines was undertaken (Table 1).

Table 1.

Four Step Procedure to Identify QIs for Care in the Last Days of Life

Step	Description	Criteria
1	Electronic literature update, using review of Pasman et al.⁵	Combined search strategy based on “QI” and “last days of life” with search strategy of Pasman (Fig. 1)
2	Selection of relevant papers by three researchers (NR, SS, FD)	Selection criteria:(i) Discussion of quality of end-of-life care, palliative care, terminal care(ii) Discussion of QI/measure/criterion/scale(iii) QI/measure/criterion/scale referring to last days of life(iv) Discussion of quality of end-of-life cancer care. All selected relevant papers have been assessed by using a standard data extraction tool.
3	Identification of relevant QIs in papers and national guidelines of palliative care	Data extraction tool for detecting relevant QIs
4	Assessing identified QIs by three researchers (FD,VR, NR)	Criteria:i) QI with a numerator, denominator, and cut-off pointii) Addressing care in the last days of life

QI, quality indicator.

The electronic search yielded 283 relevant abstracts (Fig. 2).

FIG. 2.

Flowchart of literature search on current quality indicators (QIs) in the last days of life.

Subsequently, three researchers analysed the relevance of titles and abstracts of all references. In total, 10 papers^5,10–18 were included for further assessment. Main reasons for exclusion were: not addressing quality of end-of-life, palliative, or terminal care and not containing any measurement of quality. The 10 included papers were assessed full-text for measures referring to the last days of life. Subsequently, nine papers were excluded. Additionally the references of the remaining paper⁵ were assessed and resulted in consideration of three papers: Persuelli et al.,¹⁹ Earle,²⁰ and Grunfeld.²¹ Finally, four papers were included for data extraction of relevant QIs.

The selected papers and national guidelines were assessed for potentially relevant QIs.

Potentially relevant QIs were assessed by the following criteria:

i. to contain a numerator, denominator, and a cut-off point;

ii. to address care in the last days of life.

To assess the second criterion, three independent researchers (FD, VR, NR) assessed if the QI addressed care in the last days of life. QIs had to have some semantic reference to the final week or days of care. Where it was unclear as to whether the QI addressed care in the last days of life, discussion was conducted until consensus was achieved.

Development of the questionnaire

An electronic questionnaire was developed to explore experts' opinion on the capacity of each identified QI to describe care in the last days of life well (i.e., being a good descriptor; it represents the concept being assessed) and to be applicable. Respondents were asked for each QI to rate their level of agreement concerning both characteristics. A 4-point Likert scale was used, ranging from 1 (“I strongly disagree”) to 4 (“I strongly agree”), with a separate “I don't know” option. Respondents could also propose additional QIs in free text. The digital questionnaire was in English (SurveyMonkey^®; SurveyMonkey, Palo Alto, CA) and was pre-tested for comprehensiveness among different non-native English-speaking professionals.

Panel of experts

OPCARE9 is an international multiprofessional collaborative, involving palliative care professionals, for example, physicians, nurses, social workers, psychologists, and researchers. The initial expert panel of the survey consisted of all members of OPCARE9, excluding those actively involved in the development of the questionnaire. A priori it was decided to enlarge this sample by adding extra social workers, psychologists, and spiritual caregivers from participating countries to the panel. The aim of this enlargement was to achieve a good representation of these disciplines, as they were underrepresented in the OPCARE9 collaborative. Moreover, experts in developing QI in palliative care from the nine OPCARE9 countries were added. Finally, 71 experts were invited to participate in the survey by an e-mail including the link to the questionnaire mid 2010. Two reminders were sent during a 3-month follow-up period. Questionnaires returned after the stipulated deadline were not included for analysis.

Analysis

Descriptive statistics were used to analyze demographic variables. All responses, on being a good descriptor and being applicable, were dichotomized (1–2 “not agree,” 3–4 “agree”) and the proportion of experts who agreed/disagreed was calculated. A priori it was agreed that a cut-off level of 75% or more of the respondents was considered as expert consensus. During the process the cut-off level was changed to 80%, to make the results more sensitive to identify good descriptors and applicable quality indicators. To test if palliative care experts and quality-of-care experts rate the QI differently, a Mann Whitney test was used. For the analysis of missing values the percentages were analysed to compare the answering pattern in the first five QIs with the last five. In this analysis the true missing elements, for example, blanks, were separated from the response “I don't know.” For these statistical analyses SPSS 18.0 was used (SPSS, Inc., Chicago, IL). The analysis of content in the free-text questions concerning additional QIs was performed using NVivo 8.0 (QSR International Pty. Ltd., Victoria, Australia)—a qualitative data analysis software tool. The free-text answers were coded by their manifest content according to structure, processes, or outcomes of care. This was initially done independently by two researchers (CL, OL) and then discussed until consensus in coding was reached.

Results

Identified quality indicators for care in the last days of life

In total, 34 QIs (including a numerator, denominator, and a cut-off point) for care in the last days of life were identified; 33 in the scientific literature, and one in the national guidelines of Italy, the Netherlands, Sweden, and Switzerland. Guidelines of the other participating countries were not available at the time of analysis (Table 2).

Table 2.

Number of QIs Identified in Different Sources

Source	QIs for the last days of life	QIs ^*
National guidelines	28	1
Literature search	108	33
Total	136	34

Only QIs with a numerator, denominator, and cut-off point.

QI, quality indicator.

Expert panel

In total, 41 experts responded to the questionnaire (response rate=58%). Different professional backgrounds were represented (Table 3). Of the respondents, 66% were directly involved in patient care, with a mean experience of 11.3 years; and 34% in developing QI in palliative care, with a mean experience of 5.2 years. All participating countries were represented in the panel.

Table 3.

Profile of Expert Panel

	% (n)
Response rate % (n)	58 (41)
Profession
Physicians	34 (14)
Nurses	17 (7)
Social/psychological/spiritual caregivers	21 (9)
Researchers	12 (5)
Other^*	15 (6)
Country
Argentina	7 (3)
Italy	10 (4)
Germany	5 (2)
New Zealand	5 (2)
Slovenia	10 (4)
Sweden	17 (7)
Switzerland	7 (3)
The Netherlands	12 (5)
United Kingdom	27 (11)
Expertise
Palliative care	66 (27)
QI development in palliative care	34 (14)
	Mean (SD)
Years working in palliative care	11 (6)
Years working in QI development in palliative care	5 (5)

Other professions were volunteer coordinator, management, and unknown.

QI, quality indicator.

Quality indicators being a good descriptor and being applicable

For each QI the experts rated the level of being a good descriptor and being applicable for the last days of life (Table 4; available online at www.liebertonline.com/jpm). Eighty percent or more of the respondents agreed with the following seven QIs as being good descriptors and applicable:

(1) More than 90% of all families served by home palliative care services should have received a home visit on the week following patients death to provide support during the mourning process

(2) Presence of dedicated room space for meetings on ICU between clinicians and families

(3) Fewer than 10% of patients who died from cancer received chemotherapy in the last 14 days of life.

Table 4.

All QIs Rated by Experts as Good Descriptors of Quality of Care and Applicable for Care in the Last Days of Life

Current available QIs	Good descriptor (% agree)	Applicable (% agree)
Structure
Presence of dedicated room space for meetings on ICU between clinicians and families^b	90	92
Process
More than 90% of all families served by home palliative care services should have received a home visit on the days following patients death to provide support during the mourning process^b	97	92
Fewer than 4% of patients who died from cancer had more than one ER visit in the last 30 days^a	83	79
Fewer than 10% of patients who died from cancer received chemotherapy in the last 14 days of life^b	81	89
More than 65% of all patients who died from cancer are cared for by a palliative network at home or in a hospice^a	78	89
More than 90% of all families served by home palliative care services should have received a home visit on the day of the patient's death^a	78	89
At least 75% of advanced cancer patients during the final stage should receive care from palliative care services^a	77	82
Fewer than 4% of all patients who died from cancer should have been admitted to the ICU in the last 30 days of life^a	77	83
Fewer than 17% of cancer deaths should occur in an acute setting^a	71	81
Fewer than 4% of patients who died from cancer have had more than one hospitalization in the last 30 days^a	69	81
At least 95% of all patients who receive home palliative care should be enabled to stay at home until death^a	66	87
Fewer than 8% of patients who died from cancer had been admitted for fewer than 3 days in a hospice^a	63	86
More than 45% of all patients who died from cancer had been admitted to a hospice	21	77
Outcome
The score for family insight is 0-1 in at least 75% of all patients during the last week of life^a,c	92	72
The score for pain control is 0-1 in at least 75% of all patients during the last week of life^b,c	89	83
The score for communication from professional to patient and family is 0-1 in at least 75% of all patients during the final week of life^b,c	89	80
The global score for gastrointestinal symptoms does not increase over the initial score during the final week of life in at least 75% of the patients^b,d	88	81
The global score for constipation does not increase over the initial score during the final week of life in at least 75% of the patients^a,d	85	73
The score for communication between professionals is 0-1 in at least 75% of all patients during the last week of life^a,c	87	75
The global score for pain does not increase over the initial score during the final week of life in at least 75% of the patients^b,d	85	82
The score for communication between patient and family is 0-1 in at least 75% of all patients during the last week of life^a,c	83	59
The score for symptom control is 0-1 in at least 75% of all patients during the last week of life^a,c	81	78
The score for patient anxiety control is 0-1 in at least 75% of all patients during the last week of life^c	78	69
The score for patient insight is 0-1 in at least 75% of all patients during the last week of life^c	77	46
The global score for breathing difficulties does not increase over the initial score during the final week of life in at least 75% of the patients^d	77	79
The score for families' anxiety control is 0-1 in at least 75% of all patients during the last week of life^c	71	61
The global score for physical symptoms does not increase over the initial score during the final week of life in at least 75% of the patients^d	71	58
The global score for emotional status does not increase over the initial score during the final week of life in at least 75% of the patients^d	69	53
The global score for global health status does not increase over the initial score during the final week of life in at least 75% of the patients^d	61	53
The global score for social interaction does not increase over the initial score during the final week of life in at least 75% of the patients^d	44	55
The global score for the therapy impact index does not increase over the initial score during the final week of life in at least 75% of the patients^d	40	48
The global score for functional impairment does not increase over the initial score during the final week of life in at least 75% of the patients^d	39	44
The global score for cognitive impairment does not increase over the initial score during the final week of life in at least 75% of the patients^d	37	41
The global score for fatigue does not increase over the initial score during the final week of life in at least 75% of the patients^d	35	41

More than 80% of the respondents agreed that this QI is a good descriptor or applicable.

More than 80% of the respondents agreed that this QI is a good descriptor and applicable.

These items are part of the STAS (Support Team Assessment Schedule), a standardized evaluation instrument for use in palliative care. It comprises five possible answers/each item, ranging from 0 to 4, with lowest score corresponding to no or limited needs. In other words, the symptom does not or only occasionally does, bother the patient.

These items are part of the TIQ (Therapy Impact Questionnaire), which is a multidimensional tool with 36 items, measuring patients' experience of the impact of different items on their quality of life. The initial score is the score when it was first measured.

In at least 75% of all patients during the last week of life no or limited need is present for the following topics:

(4) Pain control

(5) Global score for gastrointestinal symptoms

(6) Global score for pain

(7) Communication from professional to patient and family.

Eighty percent or more of the respondents agreed for six QIs (highlighted with a in Table 4) to be only good descriptors and for eight QIs to be only applicable.

Professional perspectives

Comparative analysis of responses between the experts involved in patient care and those involved in developing QI in palliative care was not significantly different (Mann Whitney tests).

Analysis of missing values

For the first five questions, 1.5% of 41 respondents used the answering category “I don't know” and 6.8% did not answer at all. For the last five questions these categories were respectively 8% and 12.7% of the respondents.

Additional QIs for care in the last days of life

In total, 17 respondents suggested 46 additional QIs that describe the quality of care in the last days of life well (range 1–4 per expert). Eight experts also reported 19 suggestions for QIs that would be applicable. The responses were formulated in general terms and did not describe QIs in detail (i.e., with numerator, denominator, and performance standard). Ten responses were excluded from analysis as they were incomplete or duplicates. All additional QIs were categorized according to characteristic (structure, processes, or outcomes of care). This resulted in 18 structure QIs, 21 process QIs, and 16 outcome QIs (Table 5). In total, the experts suggested 18 additional topics for QIs for care of dying patients (Table 6).

Table 5.

Distribution of Additional QIs ^*

	Total	Structure	Process	Outcome
What other QIs do you consider to be good in describing the quality of care in the last days of life?	40	9	17	14
What other QIs do you consider to be applicable in order to assess the quality of care in the last days of life?	15	9	4	2
Total		18	21	16

Reported by 17 experts.

QI, quality indicator.

Table 6.

Proposed Additional Topics for QIs for Care in the Last Days of Life ^*

	Good descriptor	Applicable
Structure
Access and availability of service and support	2	3
Continuous training or education of staff	3	1
Access to care plans/pathways	1	2
Access to equipment and/or drugs	1	2
Process
Use of care plans/pathways/guidelines	5	1
Extent of drugs and drug prescriptions	3	-
Patients' own influence on decisions	3	-
Offering spiritual support	1	1
Regular team meetings	1	1
Management of infections	1	-
Proportion of patients dying alone	1	-
Quality registration if applicable	1	-
Identification of resources for care delivery	-	1
Outcome
Patient/Family perspectives of care	5	2
Preferred place of death	3	-
Avoidance of pressure ulcers	1	-
Spiritual well-being	1	-
Staff perspectives on the care of the patient	1	-

Reported by 17 experts.

QI, quality indicator.

Discussion

We have identified a range of QIs in the international literature, as well as in national resources. However, international palliative care experts identified a considerable number of other topics for assessing the quality of care of patients in their last days of life and their relatives. Only seven current existing QIs were judged as being a good descriptor of care and applicable in the last days of life. Thus no comprehensive set of QIs for the last days of life is yet present.

As with all research, this study has some limitations. The study used a strict definition of QIs, which resulted in a number of exclusions. Many identified QIs were excluded because they did not meet the a priori set criteria, including a nominator, a denominator, a cut-off point, and semantically referring to end-of-life care, although the strict approach for selecting QIs may also be a strength. Additionally, despite this strict definition, the 34 identified QIs resulted in a rather long questionnaire. Looking at the increase in missing values toward the end of the questionnaire, its length may have influenced the validity of the results. Above that, the questionnaire only assessed the experts' views on the identified QIs.

It is desirable that QIs are evidence-based and psychometrically validated. This study did not aim to validate or to assess psychometric qualities of QIs for end-of-life care, hence these characteristics of the selected QIs cannot be judged. The evaluation of development, including the evidence base²² and linkage between quality measured and improved outcomes²³ should be an objective of a future study. To assess the experts' views on the quality of QI, this study focused on two aspects: being a good descriptor and being applicable in the last days of life. It may be appropriate for “quality” to be based on more than these two aspects. In this international collaborative, it became apparent that aspects of quality (validity, reliability) were interpreted differently, with the concept of “good descriptor” and ”applicability” potentially being subject to misunderstanding.

In this study multiple sources of reference were used: international literature, guidelines, and a large international expert panel. The overall response rate of the experts was reasonable (58%), and efforts were made to guarantee the multiprofessional representation on the expert panel. However, fewer nonmedical experts than expected were included in the panel (n=20). Although it is also not possible to determine the extent to which the panel represents the population of palliative experts, the presence of different professions, combined with the average length of palliative care experience (11 years), suggests that our panel represents a broad and experienced group.

There have been several EU projects working on quality indicators in palliative care, such as PRISMA, EUROPAL, and IMPACT; the latter is still ongoing. It would be sensible to collate the results derived from these projects to improve further development of QIs for end-of-life care. Furthermore, a new set of QIs for palliative care applicable for all settings and all patients has recently been developed.⁶ It can be assumed that care of patients in their last days of life is a discrete phase of palliative care, for both patients and relatives, with its specific characteristics, and therefore the need for specific QIs.²⁴ The definition of this phase needs further refinement, to include a clear understanding of symptoms, problems, and objectives of care against which to measure quality of care.^24,25 The Liverpool Care Pathway for the dying (LCP) gives some guidance in this matter.²⁶ Subsequently, QI development has to be related to the definition of this phase, including the aims of care in the last days of life. Another general difficulty in designing adequate QIs for palliative care is the nature of incurable diseases, as progression leads to a deterioration of the general health status,²⁷ which is even more pronounced in the last days of life. This might in particular be an obstacle in the development of QIs concerning process and outcome. However, to evaluate care of patients in their last days of life, well-designed QIs for this phase are urgently needed. These QIs can then serve for auditing and monitoring care of patients in their last days of life or can be useful for benchmarking. Pasman and fellow workers⁵ presented 11 QIs in the domain of the care of the imminently dying, one for structure of care and 10 for outcomes of care. Further, they found only five QIs for structure of palliative care in general. However, in the present study the additional QIs were evenly distributed for structure, process, and outcome. This yields the discussion regarding which types of QIs are most appropriate for assessing quality of care in the last days of life. Pasman suggested the combination of process and outcome indicators as most suitable for measuring the quality of palliative care,⁵ whereas the respondents in the present study suggested additional topics for structure, process, and outcome indicators to measure care in the last days of life. Overall, the large number of new topics suggested in the present study indicates a need for further development of QIs for care of imminently dying patients.

In conclusion, currently, in Europe and beyond, no definite set of QIs exist to describe quality of care of patients in their last days of life in all possible settings. Further research is needed to develop a set of QIs focused on care in the last days of life including their validation followed by study that assesses and evaluates the psychometric properties of all QI in that set. According to experts, the available existing QIs do not describe care in the last days of life well and are rarely applicable. New QIs that are focused on care for patients in their last days of life, their relatives, as well as their professional caregivers are needed.

Footnotes

Acknowledgments

OPCARE9 aims to improve care in the last days of life by systematically identifying existing knowledge as well as knowledge gaps. The project aims to do this collaboratively across Europe and beyond to integrate knowledge from a range of health care environments and cultures and to avoid duplication of resources and effort. Project outputs include the dissemination of key findings and recommendations for further research and development for care in the last days of life. Further information on OPCARE9, its members, work packages, outputs, and contact details can be found online at

The authors would like to thank the international experts who participated in this study. Also a special thanks to Dr. Stephen Mason for revising the English writing of this paper and to our fellow colleagues of OPCARE9: Jean Clark, Massimo Costantini, Steffen Eychmüller, Carl Johan Fürst, Nicolas Garrique, Margarita Jorge, Urska Lunder, Gustavo De Simone, Ruthmarijke Smeding, Danica Sprajcar, Raymond Voltz, and Lia van Zuylen. Natasja Raijmakers and Maren Galushko contributed equally to this paper.

Author Disclosure Statement

OPCARE9 is funded by the European Commission's Seventh Framework Programme (contract number: HEALTH-F2-2008-202112). No conflicting interests exist.

References

Donaldson

, Field

. Measuring quality of care at the end of life. Arch Intern Med, 1998; 158:121–128.

Donabedian

. The quality of care. How can it be assessed? JAMA, 1988; 260,12:1743–1748.

Campbell

, Braspenning

, Hutchinson

, Marshall

. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care, 2002; 11:358–364.

Higginson

, Koffman

. Public health and palliative care. Clin Geriatr Med, 2005; 21:45–55, viii.

Pasman

, Brandt

, Deliens

, Francke

. Quality indicators for palliative care: A systematic review. J Pain Symptom Manage, 2009; 38:145–156.

Claessen

, Francke

, Belarbi

, Pasman

, van der Putten

, Deliens

. A new set of quality indicators for palliative care: Process and results of the development trajectory. J Pain Symptom Manage, 2011; 42:169–182.

Ferrell

, Connor

, Cordes

, Dahlin

, Fine

, Hutton

, Leenay

, Lentz

, Person

, Meier

, Zuroski

. National Consensus Project for Quality Palliative Care Task Force Members: The national agenda for quality palliative care: The National Consensus Project and the National Quality Forum. J Pain Symptom Manage, 2007; 33:737–744.

Singer

, Bowman

. Quality end-of-life care: A global perspective. BMC Palliat Care, 2002; 1:4.

Seow

, Snyder

, Mularski

, Shugarman

, Kutner

, Lorenz

, Wu

, Dy

. A framework for assessing quality indicators for cancer care at the end of life. J Pain Symptom Manage, 2009; 38:903–912.

10.

Aldasoro

, Alonso

, Ribacoba

, Esnaola

, Olaizola

, Carrera

, Bañuelos

, Rico

. Assessing quality of end-of-life hospital care in a southern European regional health service. Int J Technol Assess Health Care, 2005; 21:464–470.

11.

Llobera

, Esteva

, Benito

, Terrasa

, Rifa

, Pons

. Quality of life for oncology patients during the terminal period. Validation of the HRCA-QL index. Support Care Cancer, 2003; 11:294–303.

12.

Mack

, Nilsson

, Balboni

, Friedlander

, Block

, Trice

, Prigerson

. Peace, Equanimity, and Acceptance in the Cancer Experience (PEACE): Validation of a scale to assess acceptance and struggle with terminal illness. Cancer, 2008; 112:2509–2517.

13.

Mystakidou

, Tsilika

, Kouloulias

, Parpa

, Katsouda

, Kouvaris

, Vlahos

. The "Palliative Care Quality of Life Instrument (PQLI)" in terminal cancer patients. Health Qual Life Outcomes, 2004; 2:8.

14.

Nilsson

, Maciejewski

, Zhang

, Wright

, Trice

, Muriel

, Friedlander

, Fasciano

, Block

, Prigerson

. Mental health, treatment preferences, advance care planning, location, and quality of death in advanced cancer patients with dependent children. Cancer, 2009; 115:399–409.

15.

Park

. Assessment of change of quality of life in terminally ill patients under cancer pain management using the EORTC Core Quality of Life Questionnaire (QLQ-C30) in a Korean sample. Oncology, 2008; 74,Suppl 1:7–12.

16.

Ringdal

, Jordhoy

, Kaasa

. Measuring quality of palliative care: psychometric properties of the FAMCARE Scale. Qual Life Res, 2003; 12:167–176.

17.

Sato

, Miyashita

, Morita

, Sanjo

, Shima

, Uchitomi

. Reliability assessment and findings of a newly developed quality measurement instrument: Quality indicators of end-of-life cancer care from medical chart review at a Japanese regional cancer center. J Palliat Med, 2008; 11:729–737.

18.

Yun

, Kim

, Lee

, Park

, Lee

, Choi

, Lee

, Kim

, Heo

. Patient-reported assessment of quality care at end of life: development and validation of Quality Care Questionnaire-End of Life (QCQ-EOL) Eur J Cancer, 2006; 42:2310–7.

19.

Peruselli

, Marinari

, Brivio

, Castagnini

, Cavana

, Centrone

, Magni

, Merlini

, Scaccabarozzi

, Paci

. Evaluating a home palliative care service: development of indicators for a continuous quality improvement program. J Palliat Care, 1997; 13:34–42.

20.

Earle

. Failing to plan is planning to fail: Improving the quality of care with survivorship care plans. J Clin Oncol, 2006; 24:5112–5116.

21.

Grunfeld

, Lethbridge

, Dewar

, Lawson

, Paszat

, Johnston

, Burge

, McIntyre

, Earl

. Towards using administrative databases to measure population-based indicators of quality of end-of-life care: Testing the methodology. Palliat Med, 2006; 20:769–777.

22.

Berenholtz

, Dorman

, Ngo

, Pronovost

. Qualitative review of intensive care unit quality indicators. J Crit Care, 2002; 17:1–12.

23.

Mularski

, Heine

, Osborne

, Ganzini

, Curtis

. Quality of dying in the ICU: Ratings by family members. Chest, 2005; 128:280–287.

24.

Jocham

, Dassen

, Widdershoven

, Halfens

. Quality-of-life assessment in a palliative care setting in Germany: An outcome evaluation. Int J Palliat Nurs, 2009; 15:338–345.

25.

Ferrell

. Overview of the domains of variables relevant to end-of-life care. J Palliat Med, 2005; 8,Suppl 1:S22–S29.

26.

Ellershaw

. Care of the dying: What a difference an LCP makes! Palliat Med, 2007; 21:365–368.

27.

Ostgathe

, Alt-Epping

, Golla

, Gaertner

, Lindena

, Radbruch

, Voltz

. Hospice and Palliative Care Evaluation (HOPE) Working Group: Non-cancer patients in specialized palliative care in Germany: What are the problems? Palliat Med, 2011; 25:148–152.