The PEACE Project Review of Clinical Instruments for Hospice and Palliative Care

Abstract

Background:

Hospice and palliative care organizations are expanding their use of standardized instruments and other approaches to measure quality. We undertook a systematic review and evaluation of published patient-level instruments for potential application in hospice and palliative care clinical quality measurement.

Methods:

We searched prior reviews and computerized reference databases from 1990 through February 2007 for studies of instruments relevant to physical, psychological, social, cultural, spiritual, or ethical aspects of palliative care, or measuring prognosis, function or continuity of care. Publications were selected for full review if they provided evidence of psychometric properties or practical application of an instrument tested in or appropriate for a hospice or palliative care population. Selected instruments were evaluated and scored for scientific soundness and potential application in clinical quality measurement.

Results:

The search found 1427 publications, with 229 selected for full manuscript review. Manuscripts provided information on 129 instruments which were evaluated using a structured scoring guide for psychometric properties. Thirty-nine instruments scoring near or above the 75th percentile were recommended. Most instruments covered multiple domains or focused on care for physical symptoms, psychological or social aspects of care. Few instruments were available to measure cultural aspects of care, structure and process of care, and continuity of care.

Conclusion:

Numerous patient-level instruments are available to measure physical, psychological and social aspects of palliative care with adequate evidence for scientific soundness and practical clinical use for quality improvement and research. Other aspects of palliative care may benefit from further instrument development research.

Introduction

In response to growing public concern about the quality of life near the end of life, the Institute of Medicine convened two consensus panels to suggest strategies to improve this aspect of healthcare. One major recommendation was to engage in clinical measurement of palliative care outcomes and care processes, for use in quality improvement and research.^1,2 Shortly thereafter, focused attention was given to care practices themselves when in 2005, five major national organizations led the National Consensus Project for Quality Palliative Care (NCP) and endorsed Consensus Practice Guidelines which were updated in 2009.³ The National Quality Forum (NQF) subsequently reviewed and accepted these Guidelines and associated preferred practices.⁴ The NCP practice guidelines and NQF preferred practices address care provided in eight domains: structure and processes of care; physical aspects of care; psychological and psychiatric aspects of care; social aspects of care; spiritual, religious and existential aspects of care; cultural aspects of care; care of the imminently dying patient; and ethical and legal aspects of care.

Providers need a meaningful and consistent way to record and measure the care they provide, as well as the associated patient and family outcomes. Unfortunately, measurement poses unique challenges for the preferred practices and desired patient outcomes of hospice and palliative care. Palliative care domains of quality are inherently patient and family centered, and measures rely on patient, family and clinician reported outcomes rather than on laboratory testing or vital signs.⁵

Furthermore, while many patient-level instruments provide for structured assessments, they have been developed for research and may be burdensome to patients and families or inefficient for busy clinical practice providers. Finally, some aspects of quality, such as the quality of the dying experience, can only be reported by surrogates.⁶

The Centers for Medicare and Medicaid Services (CMS) contracted with The Carolinas Center for Medical Excellence (CCME), the federally designated Quality Improvement Organization for North and South Carolina, to develop a set of tools to enable hospice and palliative care programs to assess their own quality, target actionable shortcomings, and monitor improvement in quality. This project, called the Hospice PEACE Project (“Prepare, Embrace, Attend, Communicate, Empower”), used primary data collection, key informant interviews and literature reviews to recommend approaches to quality measurement in hospice and palliative care. In this manuscript, we report on a systematic review of published patient-level measurement instruments. The overarching objective was to identify instruments most suitable for widespread use in clinical practice and quality improvement interventions.

Methods

We conducted a systematic review of published adult and pediatric patient-level instruments and evaluated their scientific soundness and suitability for clinical use and quality improvement projects in hospice and palliative care.

Literature search strategy

We searched MEDLINE, Database of Reviews of Effects (DARE), Cumulative Index of Nursing and Allied Health Literature (CINAHL), and PsychInfo databases for publications meeting inclusion criteria. Search terms for all databases included “end-of-life,” “terminal illness,” “death,” “dying,” “palliative care,” or “hospice care” combined with the terms “scale,” “instrumentation,” “reliability,” “validity,” and “psychometrics.” We accepted instruments from reference lists of previous systematic reviews by Mularski et al.,⁷ Lorenz et al.,⁸ Higginson and Romer,⁹ and Teno.¹⁰ We searched Medline and DARE from December 2005 through February 2007, since the most recent of these reviews included these databases before that timeframe.⁸ We conducted searches from January 1990 through February 2007 of CINAHL and PsychInfo. Finally, we searched all four databases for pediatric instruments and instruments measuring prognosis in palliative care published between January 1990 and February 2007.

All identified titles were scanned by a project team member to exclude irrelevant articles, and potential exclusions were reviewed by a second team member. The following criteria were used to exclude titles: (1) studies that focused on surgery, chemotherapy, radiotherapy, stents, lasers, and similar technical interventions; (2) studies not having palliative care content; (3) nonsystematic reviews, articles providing only commentary on ethics, legal, or regulatory issues; (4) description of only research methods; (5) editorials, histories, personal narratives; (6) studies about palliative professional education; (7) studies in which the outcomes were laboratory, radiologic tests, or physiologic indicators; and (8) studies where the sample size was less than 30, because a small sample would limit generalizability. Selected titles were then reviewed with abstracts. Abstracts were selected for full manuscript review if all three of the following criteria were met: (1) the study described an instrument relevant to one of the NCP domains; (2) the instrument was designed for or tested in a hospice or palliative care population; and (3) the manuscript presented information about the reliability, validity, variability, or responsiveness of the instrument.

Instrument evaluation

Once potential instruments were identified, they were evaluated for both scientific soundness and characteristics that could make them suitable for clinical and quality improvement uses. Reviewers from the study team used a structured tool to abstract information about each instrument. Instruments were classified as measures of structure, process or outcomes of care, and grouped by NCP quality of care domains (physical, psychological, social, spiritual, ethical, cultural, or structure and process of care) or in the added domains of prognosis, function and continuity of care also relevant to hospice and palliative care practice. Instruments designed to measure global outcomes of care, including satisfaction, quality of life, and quality of death, often addressed several NCP domains and were categorized as “multiple.” Reviewers also recorded information about the measurement instruments' scientific properties—variability, reliability, validity, and responsiveness—and other properties relevant to widespread clinical use in quality improvement, including the target population, generalizability, feasibility, time for administration, look-back time period, type and ease of scoring, and method of administration.

All instruments and abstracted information about them were then reviewed by four investigators (A.S., L.H., F.R., S.Z.). In a consensus process, reviewers discussed and gave a numerical value to evidence supporting both scientific soundness and potential for practical application with hospice and palliative care populations. Each instrument was reviewed using a structured scoring guide adapted from work by Zwakhalen et al.¹¹ Reviewers evaluated instruments on the following criteria: presence of numeric scoring for calculating a quality measure (0–2); evidence of testing in hospice or palliative care (0–2); use of a conceptual framework (0–2); evidence of reliability (0–2); evidence of content validity (0–2); evidence of criterion or construct validity (0–2); responsiveness to intervention (0–2), variability (0–1), clinical feasibility and burden (0–2), and generalizability (0–2), for a total potential score of 0–19 (Table 1). After consensus scoring all instruments, we examined the distribution of scores and selected the 75th percentile (score ≥14) as the cut point to describe higher scoring instruments for all domains. Instruments were recommended if they scored in this upper quartile, or scored 12–13 and filled a critical gap for hospice and palliative care practice.

Table 1.

Criteria To Evaluate Instruments for Hospice and Palliative Care

Aspect	Scoring (Total potential score 0–19)
Scoring system	2 if simple score generated
	1 if complex scoring
	0 if no score
Test population	2 if hospice/palliative care
	1 if seriously ill
	0 if other
Conceptual development	2 if developmental work described
	1 if conceptual/literature review only
	0 if none
Reliability: type and extent	2 if reliability coefficients ≥0.80
	1 if reliability coefficients 0.60–0.79; or if internal consistency only
	0 if limited or no information
Content validity	2 if covers all important aspects well
	1 if covers some aspects well
	0 if does not cover most important aspects well
Validity: type and extent	2 if correlations high, r ≥ 0.60
	1 if correlations modest, r = 0.40–0.59
	0 if correlation r < 0.40 or no information
Responsiveness	2 if evidence strong (intervention)
	1 if some evidence (temporal change related to illness or to usual care)
	0 if no evidence
Variability	1 if described
	0 if not described
Feasibility (frequency, time to administer, method of administration, respondent and health care provider burden)	2 if short/low burden, instructions for use clear, scoring interpretable
	1 if long/some burden, instructions or scoring unclear
	0 if clearly not feasible for hospice/PC setting
Generalizability	2 if applicable to nearly all hospice/PC populations
	1 if applicable to major subpopulation
	0 if poor generalizability for hospice/PC populations

PC, palliative care.

Results

Our search strategy identified 1427 titles, of which 1088 were excluded during title review and 110 were excluded after abstract review. We retained 229 articles for full manuscript review. Reviewed articles included data on 139 individual instruments, 41 of which were not included in prior reviews.^7,10 Ten instruments were excluded after full review because they required a look back period of greater than one week or extended administration time, making them inappropriate for clinical quality improvement. The remaining 129 instruments were scored and classified according to domain. A list of all instruments, as well as detailed individual instrument reviews from the Hospice PEACE Project are available at www.thecarolinascenter.org/default.aspx?pageid=24.

As shown in Figure 1, instruments identified in this review most commonly combined multiple domains (30), or measured outcomes in physical (28), psychological (16), or social (17) domains of care, and these domains accounted for 28 of 39 recommended instruments. We found few instruments measuring aspects of cultural care (1), continuity of care (3), and prognosis (3). Only three instruments explicitly addressed the structure and processes of palliative care—aspects of care that are often targets for quality improvement interventions. Table 2 lists the 39 instruments that scored at or above the 75th percentile (n = 34), or scored just below this level and fill a critical gap in quality of care measurement (n = 5).

Fig. 1.

Number of Instruments Evaluated in each Domian.

Table 2.

Highly Scored Instruments for Domains of Palliative Care

Instruments by domain	Score (0–19)	Primary citations
Multiple domains
European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ)	14	Bjorner et al.¹³ Ford et al.¹⁴ Kaasa et al.^15,16 Stromgren et al.¹⁷
Family Evaluation of Hospice Care (FEHC)	13	Teno et al.¹⁸ Connor et al.^19,20
Functional Assessment of Cancer Treatment (FACT)	15	Cella et al.²¹ Montoya et al.²²
McGill Quality of Life Questionnaire	14	Cohen et al.^23,24 Lo et al.²⁵
Missoula-VITAS Quality of Life Index	16	Byock et al.²⁶ Steele et al.²⁷ Williams et al.²⁸
Palliative Care Outcomes Scale (POS)	13	Hearn and Higginson²⁹ Horton³⁰
Palliative Care QOL Instrument	15	Mystakidou et al.³¹
Quality Care Questionnaire-End of Life (QCQ-EOL)	15	Yun et al.³²
Quality of Death and Dying (QODD)	14	Curtis et al.³³ Patrick et al.³⁴
Quality of End of Life Care and Satisfaction (QUEST)	14	Sulmasy et al.³⁵
Scales for the Evaluation of EOL Care in Dementia (EOL-D)	15	Kiely et al.³⁶
Structure and process
Family Satisfaction with Advanced Cancer Care (FAMCARE)	16	Kristjanson et al.^37,38 Ringdal et al.³⁹
Physical symptoms
Checklist of Nonverbal Pain Indicators (CNPI)	14	Feldt⁴⁰
Discomfort Scale for Dementia of the Alzheimer's Type (DS-DAT)	16	Hurley et al.⁴¹
Douleur Enfant Gustave Roussy (for children) (DEGR)	13	Gauvain-Piquard et al.⁴²
Edmonton Symptom Assessment System (ESAS)	16	Bruera et al.⁴³ Modonesi et al.⁴⁴ Tierney et al.⁴⁵
McGill Pain Questionnaire	14	Melzack^46,47
		Melzack et al.⁴⁸
Pain Assessment in Advanced Dementia (PAIN-AD)	14	Warden et al.⁴⁹
Respiratory Distress Observational Scale (RDOS)	15	Campbell⁵⁰
Structured Interview for Symptoms and Concerns (SISC)	15	Wilson et al.⁵¹
Symptom Monitor	15	Hoekstra et al.⁵²
Psychological symptoms
Agitation Distress Scale	15	Morita et al.⁵³
Communication Capacity Scale	15	Melzak et al.⁴⁶
Cornell Scale for Depression in Dementia (CSDD)	15	Alexopoulos et al.⁵⁴ Kurlowicz et al.⁵⁵
PRIME-MD/PHQ-9	13	Kroenke et al.⁵⁶ Lowe et al.⁵⁷ Robinson and Crawford⁵⁸
Social aspects of care
Bereavement Risk Index (BRI)	14	Beckwith et al.⁵⁹ Kristjanson et al.⁶⁰
		Robinson and Crawford⁶¹
Duke-UNC Social Support Scale	14	Broadhead et al.⁶²
		Herndon et al.⁶³
Family Appraisal of Caregiving Questionnaire (FACQ)	14	Cooper et al.⁶⁴
Hogan Grief Reaction Checklist (HGRC)	15	Hogan et al.⁶⁵
Spiritual aspects of care
Functional Assessment of Chronic Illness Therapy—Spiritual Well-Being (FACIT-Sp)	15	Peterman et al.⁶⁶
Meaning in Life Scale	14	Warner and Williams⁶⁷
Brief Spiritual Beliefs Inventory (SBI-15)	14	Holland et al.⁶⁸
Cultural aspects of care
Cultural Competence Scale	16	Doorenbos et al.⁶⁹
		Schim et al.⁷⁰
Continuity of care
Care Transitions Measure	12	Coleman et al.⁷¹
Ethical/legal aspects of care
Decisional Conflict Scale	17	O'Connor⁷²
		Song and Sereika⁷³
Quality of Communication (QOC)	16	Engelberg et al.⁷⁴
Function
Edmonton Functional Assessment Tool (EFAT)	14	Kaasa et al.¹⁶
Rapid Disability Rating Scale	15	Linn and Linn⁷⁵
Prognosis
Palliative Performance Scale (PPS)	14	Anderson et al.⁷⁶
		Head et al.⁷⁷
		Olajide et al.⁷⁸

Discussion

We performed a systematic review and evaluation of patient-level instruments designed to measure outcomes and quality of care in domains relevant to hospice and palliative care. This review updates prior reviews, and is the first to evaluate instruments for scientific soundness and for practical characteristics that may permit adoption beyond research, in clinical practice and quality improvement. Structured measures facilitate clinical care and quality by allowing clinicians to track results of their efforts, and to compare their practices and outcomes to peers. Instruments developed for research may be long or otherwise impractical for practice, or may not measures things that can be improved in clinical practice. Published palliative care instruments are typically designed to measure outcomes. Fewer instruments measure structure and process of care, which may be more useful in quality improvement interventions designed to modify practices. We found numerous high-quality instruments that include multiple domains of care embedded in larger concepts such as satisfaction with care or quality of dying.

While these outcomes are highly valued, these instruments may not be responsive to efforts to improve pain management practices or enhance spiritual care services. Numerous instruments are available to measure physical, psychological, and social outcomes; hospice and palliative care organizations may find it more difficult to examine the quality of their cultural or spiritual caregiving.

Reviews may be useful to investigators interested in instrument development or outcomes research in hospice and palliative care. Reviewed instruments may also be a resource for clinicians and quality improvement specialists who seek more rigorous approaches to measure and improve the quality of care.

Scientifically sound and clinically feasible patient-level instruments provide clinicians with a more consistent way to record care processes and patient-reported outcomes. Appropriate instruments will be relatively short and simple, take little training to administer and score, be applicable to most patients served, and will measure areas that are actionable by clinicians. Also, use of patient-level instruments may be necessary to permit meaningful quality measurement in palliative care and hospice. Quality assessment relies on quality indicators—the number of patients with a desired care process or outcome divided by the number of patients for whom this process or outcome is relevant.¹² Quality indicators ideally vary with quality of care; however, they will also vary with the quality of medical record documentation available. Recording clinical data with scientifically sound instruments—while not common—has the potential to improve the completeness and consistency of medical records and simultaneously improve data for quality improvement or research. Any mandate to change documentation practices must consider the potential burden of patient-reported instruments during serious illness. Before widespread introduction, it is critical to test this approach and solicit input from patients and families.

Our review has certain limitations. The search strategy emphasized psychometric properties, and may have missed clinical trials of interpersonal, pharmaceutical or technological interventions that used instruments as outcome measures; this omission would result in failure to include some data on the variability and responsiveness of the instruments. Also, the focus on clinical application and quality improvement may result in lower scoring of scientifically sound but more detailed instruments that are still quite valuable for research. To be included in this review, instruments had to be relevant for or applied in a hospice or palliative care population. While the hospice population is clearly defined by a prognosis and an insurance benefit, the population of patients with palliative care needs is broad. Instruments developed for use with critically ill patients, nursing home residents, or other seriously ill patients may have been excluded, yet may have relevance for patients with palliative care needs. Finally, our approach will generate lower scores for instruments that are early in development but have not yet been frequently or rigorously tested. These newer instruments may be of great value to a developing field and should not be prematurely dismissed by researchers or by clinicians. Newer instruments are likely to help fill the gaps for which few measures exist.

Like most of health care, hospice and palliative care providers are just beginning to study the quality of care they deliver. Palliative care focuses on improving patient and family-centered outcomes—aspects of health care more difficult to measure. Many instruments identified in this review have sound conceptual foundations and psychometric properties, but limited evidence that they are responsive to improved practices. As a result, there is a critical gap in their practical application. New approaches to quality of care research and quality improvement should be adopted thoughtfully, with forethought about interventions to promote improvement. Future clinical trials should select reliable and valid instruments for outcome measures and generate greater evidence that these instruments are responsive indicators of improved palliative care. As hospice and palliative care organizations thoughtfully adopt patient-level instruments, they will enhance the capacity for quality measurement and quality improvement.

Footnotes

Acknowledgments

The analyses upon which this publication is based were performed under the following contract: HHSM-500-2006-NC003C (“Utilization and Quality Control Peer Review Organization for the State of North Carolina”). The content of this publication does not necessarily reflect the views or policies of CMS or The Carolinas Center for Medical Excellence. The opinions expressed in this article are those of the authors.

Author Disclosure Statement

No competing financial interests exist.

References

Field

, Cassel

. Approaching Death: Improving Care at the End of Life. Washington, D.C.: National Academy Press, 1997.

Lunney

, Foley

, Smith

, Gelband

. Describing Death in America: What We Need to Know. Washington, D.C.: National Academy Press, 2003.

Clinical Practice Guidelines for Quality Palliative Care, 2004 Pittsburgh, PA: National Consensus Project www.nationalconsensusproject.org. 2009 August 9.

National Quality Forum. National Framework and Preferred Practices for Palliative and Hospice Care. Washington, D.C.: National Quality Forum, 2007.

Teno

, Byock

, Field

. Research agenda for developing measures to examine quality of care and quality of life of patients diagnosed with life-limiting illness. J Pain Symptom Manage, 1999; 17:75–82.

Steinhauser

, Clipp

, Tulsky

. Evolution in measuring the quality of dying. J Palliat Med, 2002; 5:407–414.

Mularski

, Dy

, Shugaman

, Wilkinson

, Lynn

, Shekelle

, Morton

, Sun

, Hughes

, Hilton

, Maglione

, Rhodes

, Rolon

, Lorenz

. A systematic review of measures of end of life care and its outcomes. Health Serv Res, 2007; 42:1848–1870.

Lorenz

, Lynn

, Dy

, Wilkinson

, Mularski

, Shugarman

, Hughes

, Asch

, Rolon

, Rastegar

, Shekelle

. Quality measures for symptoms and advance care planning in cancer: A systematic review. J Clin Oncol, 2006; 24:4933–4938.

Higginson

, Romer

. Measuring quality of care in palliative care services. J Palliat Med, 2000; 3:229–236.

10.

Teno

. Time: Toolkit of Instruments to Measure End-of-Life Care. www.cher.brown.edu/pcoc/toolkit.htm. 2007 March 13.

11.

Zwakhalen

SMG

, Hamers

JPH

, Abu-Saad

, Berger

MPF

. Pain in elderly people with severe dementia: A systematic review of behavioural pain assessment tools. BMC Geriatr, 2006; 6:3

12.

Wenger

, Shekelle

. Assessing care of vulnerable elders: ACOVE project overview. Ann Intern Med, 2001; 135,8 Pt 2:642–646.

13.

Bjorner

, Petersen

, Groenvold

, Aaronson

, Ahiner-Elmqvist

, Arraras

, Bredart

, Fayers

, Jordhoy

, Sprangers

, Watson

, Young

. Use of item response theory to develop a shortened version of the EORTC QLQ-C30 emotional functioning scale. Qual Life Res, 2004; 13:1683–1697.

14.

Ford

, Havstad

, Kart

. Assessing the reliability of the EORTC QLQ-C30 in a sample of older African American and Caucasian adults. Qual Life Res, 2001; 10:533–541.

15.

Kaasa

, Loomis

, Gillis

, Bruera

, Hanson

. Inter-rater reliability of formally trained and self-trained raters using the Edmonton Functional Assessment Tool. Palliat Med, 2000; 14:509–517.

16.

Kaasa

, Wessel

, Darrah

, Bruera

. The EORTC core quality of life questionnaire (QLQ-C30): Validity and reliability when analysed with patients treated with palliative radiotherapy. Eur J Cancer, 1995; 31A:2260–2263.

17.

Stromgren

, Groenvold

, Petersen

, Goldschmidt

, Pedersen

, Spile

, Irming-Pedersen

, Sjogren

. Pain characteristics and treatment outcome for advanced cancer patients during the first week of specialized palliative care. J Pain Symptom Manage, 2004; 27:104–113.

18.

Teno

, Clarridge

, Casey

, Edgman Levitan

, Fowler

. Validation of toolkit after-death bereaved family member interview. J Pain Symptom Manage, 2001; 22:752–758.

19.

Connor

, Tecca

, Lund-Person

, Teno

. Measuring hospice care: The National Hospice and Palliative Care Organization National Hospice Data Set. J Pain Symptom Manage, 2004; 28:16–28.

20.

Connor

, Teno

, Spence

, Smith

. Family evaluation of hospice care: Results from voluntary submission of data via website. J Pain Symptom Manage, 2005; 30:9–17.

21.

Cella

, Tulsky

, Gray

, Sarafian

, Linn

, Bonomi

, Silberman

, Yellen

, Winicour

, Brannon

. The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. J Clin Oncol, 1993; 11:570–579.

22.

Montoya

, Fossella

, Palmer

, Kaur

, Pace

, Yadav

, Simmonds

, Gillis

, Bruera

. Objective evaluation of physical function in patients with advanced lung cancer: A preliminary report. J Palliat Med, 2006; 9:309–316.

23.

Cohen

, Mount

, Bruera

, Provost

, Rowe

, Tong

. Validity of the McGill Quality of Life Questionnaire in the palliative care setting: A multi-centre Canadian study demonstrating the importance of the existential domain. Palliat Med, 1997; 11:3–20.

24.

Cohen

, Mount

, Strobel

, Bui

. The McGill Quality of Life Questionnaire: A measure of quality of life appropriate for people with advanced disease. A preliminary study of validity and acceptability. Palliat Med, 1995; 9:207–219.

25.

RSK

, Woo

, Zhoc

KCH

, Charlotte

, Yeo

, Johnson

, Mak

, Lee

. Quality of life of palliative care patients in the last two weeks of life. J Pain Symptom Manage, 2002; 24:388–397.

26.

Byock

, Merriman

. Measuring quality of life for patients with terminal illness: The Missoula-VITAS quality of life index. Palliat Med, 1998; 12:231–244.

27.

Steele

, Mills

, Hardin

, Hussey

. The quality of life of hospice patients: Patient and provider perceptions. Am J Hosp Palliat Care, 2005; 22:95–110.

28.

Williams

, Selwyn

, Liberti

, Molde

, Njike

, McCorkle

, Zelterman

, Katz

. A randomized controlled trial of meditation and massage effects on quality of life in people with late-stage disease: A pilot study. J Palliat Med, 2005; 8:939–952.

29.

Hearn

, Higginson

. Development and validation of a core outcome measure for palliative care: The Palliative Care Outcome Scale. Qual Health Care, 1999; 8:219–227.

30.

Horton

. Differences in assessment of symptoms and quality of life between patients with advanced cancer and their specialist palliative care nurses in a home care setting. Palliat Med, 2002; 16:488–494.

31.

Mystakidou

, Tsilika

, Kouloulias

, Parpa

, Katsouda

, Kouvaris

, Vlahos

. The “Palliative Care Quality of Life Instrument (PQLI)” in terminal cancer patients. Health Qual Life Outcomes, 2004; 2:8.

32.

Yun

, Kim

, Lee

, Park

, Lee

, Choi

, Heo

. Patient-reported assessment of quality care at end of life: Development and validation of Quality Care Questionnaire-End of Life (QCQ-EOL) Eur J Cancer, 2006; 42:2310–2317.

33.

Curtis

, Patrick

, Engelberg

, Norris

, Asp

, Byock

. A measure of the quality of dying and death. Initial validation using after-death interviews with family members. J Pain Symptom Manage, 2002; 24:17–31.

34.

Patrick

, Engelberg

, Curtis

. Evaluating the quality of dying and death. J Pain Symptom Manage, 2001; 22:717–726.

35.

Sulmasy

, McIlvane

, Pasley

, Rahn

. A scale of measuring patient perceptions of the quality of end-of-life care and satisfaction with treatment: The reliability and validity of QUEST. J Pain Symptom Manage, 2002; 23:458–470.

36.

Kiely

, Volicer

, Teno

, Jones

, Prigerson

, Mitchell

. The validity and reliability of scales for the evaluation of end-of-life care in advanced dementia. Alzheimer Dis Assoc Disord, 2006; 20:176–181.

37.

Kristjanson

. Validity and reliability testing of the FAMCARE Scale: Measuring family satisfaction with advanced cancer care. Soc Sci Med, 1993; 36:693–701.

38.

Kristjanson

, Leis

, Koop

, Carriere

, Mueller

. Family members' care expectations, care perceptions, and satisfaction with advanced cancer care: Results of a multi-site pilot study. J Palliat Care, 1997; 13:5–13.

39.

Ringdal

, Jordhoy

, Kaasa

. Measuring quality of palliative care: Psychometric properties of FAMCARE Scale. Qual Life Res, 2003; 12:167–176.

40.

Feldt

. The checklist of nonverbal pain indicators (CNPI) Pain Manage Nurs, 2000; 1:13–21.

41.

Hurley

, Volicer

, Hanrahan

, Houde

, Volicer

. Assessment of discomfort in advanced Alzheimer patients. Res Nurs Health, 1992; 15:369–377.

42.

Gauvain-Piquard

, Rodary

, Resvan

, Serbouti

. The development of the DEGR(R): A scale to assess pain in young children with cancer. Eur J Pain, 2006; 3:165–176.

43.

Bruera

, Kuehn

, Miller

, Selmser

, Macmillan

. The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. J Palliat Care, 1991; 7:6–9.

44.

Modonesi

, Scarpi

, Maltoni

, Derni

, Fabbri

, Martini

, Sansoni

, Amadori

. Impact of palliative care unit admission on symptom control evaluated by the Edmonton Symptom Assessment System. J Pain Symptom Manage, 2005; 30:367–373.

45.

Tierney

, Horton

, Hannan

, Tierney

. Relationships between symptom relief, quality of life, and satisfaction with hospice care. Palliat Med, 1998; 12:333–344.

46.

Melzack

. The McGill Pain Questionnaire: Major properties and scoring methods. Pain, 1975; 1:277–299.

47.

Melzack

. The short-form McGill Pain Questionnaire. Pain, 1987; 30:191–197.

48.

Melzack

, Ofiesh

, Mount

. The Brompton mixture: Effects on pain in cancer patients. Can Med Assoc J, 1976; 115:125–129.

49.

Warden

, Hurley

, Volicer

. Development and psychometric evaluation of the Pain Assessment in Advanced Dementia (PAINAD) scale. J Am Med Dir Assoc, 2003; 4:9–15.

50.

Campbell

. Psychometric testing of a respiratory distress observational scale. J Palliat Med, 2008; 11:44–50.

51.

Wilson

, Graham

, Viola

, Chater

, deFaye

, Weaver

, Lachance

. Structured interview assessment of symptoms and concerns in palliative care. Can J Psychiatry, 2004; 49:350–358.

52.

Hoekstra

, Bindels

, van Duijn

, Schade

. The symptom monitor. A diary for monitoring physical symptoms for cancer patients in palliative care: Feasibility, reliability and compliance. J Pain Symptom Manage, 2004; 27:24–35.

53.

Morita

, Tsunoda

, Inoue

, Chihara

, Oka

. Communication Capacity Scale and Agitation Distress Scale to measure the severity of delirium in terminally ill cancer patients: A validation study. Palliat Med, 2001; 15:197–206.

54.

Alexopoulos

, Abrams

, Young

, Shamoian

. Cornell Scale for Depression in Dementia. Biol Psychiatry, 1988; 23:271–284.

55.

Kurlowicz

, Evans

, Strumph

, Maislin

. A psychometric evaluation of the Cornell Scale for Depression in Dementia in a frail, nursing home population. Am J Geriatr Psychiatry, 2002; 10:600–608.

56.

Kroenke

, Spitzer

, Williams

. The PHQ-9: Validity of a Brief Depression Severity Measure. J Gen Intern Med, 2001; 16:606–613.

57.

Lowe

, Schenkel

, Carney-Doebbeling

, Gobel

. Responsiveness of the PHQ-9 to psychopharmacological depression treatment. Psychosomatics, 2006; 47:62–67.

58.

Robinson

, Crawford

. Identifying palliative care patients with symptoms of depression: An algorithm. Palliat Med, 2005; 19:278–287.

59.

Beckwith

, Beckwith

, Gray

, Micsko

, Holm

, Plummer

, Flaa

. Identification of spouses at high risk during bereavement: A preliminary assessment of Parkes and Weiss' Risk Index. Hosp J, 1990; 6:35–46.

60.

Kristjanson

, Cousins

, Smith

, Lewin

. Evaluation of the Bereavement Risk Index (BRI): A community hospice care protocol. Int J Palliat Nurs, 2005; 11:610612–618.

61.

Robinson

, Crawford

. Identifying palliative care patients with symptoms of depression: An algorithm. Palliat Med, 2005; 19:278–287.

62.

Broadhead

, Gehibach

, deGruy

, Kaplan

. The Duke-UNC Functional Social Support Questionnaire. Measurement of social support in family medicine patients. Med Care, 1988; 26:709–723.

63.

Herndon

, Fleishman

, Kosty

, Green

. A longitudinal study of quality of life in advanced non-small cell lung cancer: Cancer and Leukemia Group B (CALGB) 8931. Control Clin Trials, 1997; 18:286–300.

64.

Cooper

, Kinsella

, Picton

. Development and initial validation of a family appraisal of caregiving questionnaire for palliative care. Psychooncology, 2006; 15:613–622.

65.

Hogan

, Greenfield

, Schmidt

. Development and validation of the Hogan Grief Reaction Checklist. Death Stud, 2001; 25:1–32.

66.

Peterman

, Fitchett

, Brady

, Hernandez

, Cella

. Measuring spiritual well-being in people with cancer: The functional assessment of chronic illness therapy—Spiritual Well-being Scale (FACIT-Sp) Ann Behav Med, 2002; 24:49–58.

67.

Warner

, Williams

. The Meaning in Life Scale: Determining the reliability and validity of a measure. J Chronic Dis, 1987; 40:503–512.

68.

Holland

, Kash

, Passik

, Gronert

, Sison

, Lederberg

, Russak

, Gaider

, Fox

. A brief spiritual beliefs inventory for use in quality of life research in life-threatening illness. Psychooncology, 1998; 7:460–469.

69.

Doorenbos

, Schim

, Benkert

, Borse

. Psychometric evaluation of the cultural competence assessment instrument among healthcare providers. Nurs Res, 2005; 54:324–331.

70.

Schim

, Doorenbos

, Miller

, Benkert

. Development of a cultural competence assessment instrument. J Nurs Measure, 2003; 11:29–40.

71.

Coleman

, Mahoney

, Parry

. Assessing the quality of preparation for posthospital care from the patient's perspective: The care transitions measure. Med Care, 2005; 43:246–255.

72.

O'Connor

. Validation of a decisional conflict scale. Med Decis Making, 1995; 15:25–30.

73.

Song

, Sereika

. An evaluation of the Decisional Conflict Scale for measuring the quality of end-of-life decision making. Patient Educ Couns, 2006; 61:397–404.

74.

Engelberg

, Downey

, Curtis

. Psychometric characteristics of a quality of communication questionnaire assessing communication about end-of-life care. J Palliat Med, 2006; 9:1086–1098.

75.

Linn

, Linn

. The rapid disability rating scale-2. J Am Geriatr Soc, 1982; 30:378–382.

76.

Anderson

, Downing

, Hill

, Casorso

, Lerch

. Palliative Performance Scale (PPS): A new tool. J Palliat Care, 1996; 12:5–11.

77.

Head

, Ritchie

, Smoot

. Prognostication in hospice care: Can the Palliative Performance Scale Help? J Palliat Med, 2005; 8:492–500.

78.

Olajide

, Hanson

, Usher

, Qaqish

, Schwartz

, Bernard

. Validation of the palliative performance scale in the acute tertiary care hospital setting. J Palliat Med, 2007; 10:111–117.