Assessing Pain in Nonresponsive Hospice Patients: Development and Preliminary Testing of the Multidimensional Objective Pain Assessment Tool (MOPAT)

Abstract

Background:

Accurate assessment in hospice patients who cannot communicate their pain is almost impossible, increasing their risk for unrecognized and inadequately managed pain.

Objective:

The purpose of this article is to describe a series of small-scale projects aimed at developing and refining an instrument to assess acute pain in noncommunicative hospice patients.

Methods:

Project 1 was a clinical project in which focus groups with hospice nurses yielded an adaptation of an existing pain assessment measure that was named the Multidimensional Objective Pain Assessment Tool (MOPAT) and had behavioral and physiological subscales. Projects 2 and 3 tested the MOPAT in 30 cognitively impaired/nonresponsive hospice inpatients and 28 alert and oriented hospice inpatients, with study nurses and hospice nurses rating pain with the MOPAT before and after a pain-relieving intervention and rating its clinical usefulness. Projects 3 and 4 analyzed the reliability, validity, and clinical utility of the MOPAT.

Results:

Overall internal consistency reliability of the MOPAT was demonstrated with Cronbach's α coefficients of 0.79 before and 0.84 after the pain-relieving intervention. The behavioral and physiological subscale scores changed significantly (p < .035) after pain medication, demonstrating sensitivity to changes in pain. Principal components factor analysis revealed two factors matching the subscales and accounting for 66% of the variance. Nearly all the hospice nurses found the MOPAT helpful, easy to use and understand, and conducive to use in daily practice.

Conclusion:

The MOPAT has preliminary evidence of reliability, validity, and clinical utility. Full-scale psychometric testing in hospice and acute care hospital patients is currently underway.

Introduction and Background

Adequate assessment is fundamental to managing pain, and is ideally based on individual self-report because pain is a highly subjective experience.^1,2 Its multiple dimensions—physiological, sensory, cognitive, affective, behavioral, and sociocultural^3,4—can be assessed in most individuals using self-report assessment tools,^1,5–7 even those with partial cognitive impairment. However, in people who cannot communicate their pain at all due to underlying medical conditions and treatments, accurate assessment is almost impossible, putting them at significantly higher risk for unrecognized and inadequately managed pain.^1,5,8,9 Experts have noted the need for tools to assess pain in noncommunicative adult patients for years,^5,10,11 but efforts to date have been limited to only a few populations, for example, critical care, postoperative, elderly, and dementia.^12–15

Because noncommunicative patients in hospice cannot provide self-reports of some of the common dimensions of pain (for example, sensory—how it feels, affective—emotional responses, or cognitive—perceptions of relief ), clinicians must rely on other dimensions of pain such as behavioral (observable pain-related behaviors) and possibly physiological (vital signs or other parameters that may change with pain). At the time this research started, there were no tools available for assessing pain in noncommunicative hospice patients.^4,10

The purpose of this article is to describe the preliminary work undertaken to develop and refine an instrument to assess acute pain in noncommunicative patients in hospice in order to conduct a subsequent full-scale psychometric evaluation of the instrument. Patients targeted for this work were cognitively impaired or nonresponsive by virtue of terminal illness, and could not self-report any aspect of their pain experience. The first step was a clinical project involving hospice nurses, followed by a study of patients with terminal cancer because they comprised the majority of hospice patients at the time, and then the project expanded to a study of hospice patients with additional terminal conditions excepting dementia, as it poses unique problems in behavioral pain assessment.¹⁴ The rationale for focusing on acute pain in hospice is that this type of pain is commonly associated with observable behaviors and possibly physiological parameters, whereas chronic persistent pain may not be as easily detectable. Hospice patients can manifest acute pain when their underlying disease progresses rapidly (as in cancer) or they have breakthrough pain.

Following the initial clinical project, the authors conducted three small-scale interrelated studies that tested an adaptation of an existing pain assessment measure, the Post Anesthesia Care Unit Behavioral Pain Rating Scale (PACU BPRS).^16,17 The PACU BPRS was based on work in postoperative patient populations conducted in the 1960s and 1970s^18–20 and was designed to assess behavioral indicators of pain (e.g., restlessness, vocalizations, etc). Thus, the adapted measure was not new, but was adapted from an old instrument for use in a different setting and population—cognitively impaired and/or nonresponsive (i.e., noncommunicative) patients in the inpatient hospice setting. The collective outcome of the four projects was preliminary evidence of reliability, validity, and clinical utility of the revised instrument, which provided the foundation for a larger multisite psychometric study.

Methods and Results

Project 1 (1994–1995) was a mentored clinical project conducted as a nursing master's degree requirement for which institutional review board (IRB) approval was not required.²¹ The purpose was to explore how hospice nurses in five hospice settings in the southeastern United States assessed pain in patients who were cognitively impaired and/or nonresponsive. Twenty nurses attended an educational program on subjective and objective assessment of pain, completed a knowledge survey about pain assessment, and then participated in focus groups conducted to determine the pain-related behaviors and observations they used to assess pain in nonresponsive and cognitively impaired patients. Their responses were recorded in extensive written notes and analyzed by categories. Nurses reported using several specific pain-related behaviors (restlessness, muscle tension, facial grimacing, and patient sounds or vocalizations) as well as vital signs and other physiological parameters (blood pressure, heart rate, respiratory rate, and diaphoresis) to assess pain. Focus group data confirmed the behavioral indicators on the PACU BPRS (Mateo OM, personal communication with first author, July 1994),¹⁶ an instrument that was known to this article's first author, and guided minor revision of this instrument with permission of the original authors. First, the wording of the behavioral pain indicators (restless, tense muscles, frowning/grimacing, patient sounds) was refined based on hospice nurses' input. Second, the verbal descriptors that corresponded with the scale ratings of 0 (none or normal) to 3 (severe) for each behavioral pain indicator were expanded to better describe hospice patients' behaviors. Third, four physiological pain indicators—blood pressure, heart rate, respirations, and diaphoresis—were added to the measure. After consultation with clinical and measurement experts, scoring of these physiological indicators was dichotomized as 0 (normal or no change from baseline) or 1 (abnormal or change from baseline). The use of a 0–3 scale was deemed impossible because the vital signs of terminally ill people may increase or decrease depending on stage of dying, medical condition, and other factors. One additional item—Sensory Pain Indicator—was added to describe the temporal pattern of pain over time, which is a component of the sensory dimension of pain.³ This item provided three groups of adjectives describing different temporal patterns of pain (brief/momentary/transient, rhythmic/periodic/intermittent, continuous/steady/constant) and was adapted from the Long Form of the McGill Pain Questionnaire.²² The resulting adapted measure was named the Multidimensional Objective Pain Assessment Tool (MOPAT) because it addressed the behavioral, physiological, and sensory dimensions of pain.^3,4 The MOPAT was reviewed by the hospice nurses and minor wording changes were made.

Project 2 (1996–1999) was an instrument-testing study approved by the IRB of the first author's university. The purpose of the study was to test the validity, reliability, and clinical utility of the MOPAT in a target sample of 80 terminally ill cancer patients with pain by comparing 60 cognitively impaired/nonresponsive (CI/NR) patients with 20 alert and oriented (AO) patients (to assess concurrent validity) cared for on the inpatient units of two hospices in the southeastern United States.²³ Eligibility criteria for CI/NR patients were: (1) known to have cancer-related pain, (2) having an exacerbation of previously controlled pain or development of a new pain according to family members and/or hospice nurses, and (3) either nonresponsive or cognitively impaired due to their illness according to hospice nurses. Administration of pain medications was also used as a proxy for presence of pain because there was no reliable or valid method for assessing pain in this population. Eligibility criteria for AO patients were the same as (1) and (2) above, but in addition they were (3) alert and able to provide self-reports of the pain according to hospice nurses. The MOPAT was used by pairs of trained study and hospice nurses to rate pain before and after a pain-relieving intervention. Because the patients in the AO group served as a comparison group, they also completed the self-report McGill Pain Questionnaire Present Pain Intensity (PPI) scale comprised of six response options: none (0), mild (1), discomforting (2), distressing (3), horrible (4), and excruciating (5).²² The trained hospice nurses completed the Clinical Utility Questionnaire (CUQ), which was developed by the investigators and determined to have content validity based on commonly accepted perceptions of clinical utility.^24,25 The CUQ asked nurses to appraise the MOPAT on usefulness, format, amount of time needed to complete, and other relevant parameters. An investigator-generated Patient Data Form was used to record demographic and clinical data on each patient, including the Karnofsky Performance Status (KPS) Scale. In the time period available to conduct the study, 35 patients were accrued at the two hospices—28 AO and 7 CI/NR—and unfortunately the target sample of 60 CI/NR patients was not achieved. The major reason was insufficient personnel to screen and enroll eligible subjects at the two hospices because of logistical, time, and travel requirements. The mean KPS Scale score was 29.3 (standard deviation [SD] = 8.3) for AO patients and 15 (SD = 4.5) for CI/NR patients. Table 1 shows demographic and clinical characteristics of the sample. Neither of the groups used the full range of the MOPAT Behavioral Pain Indicator subscale ratings and many patients had no changes in their Physiological Pain Indicators (see Table 2). The Sensory Pain Indicator was not consistently used by hospice nurses and therefore was not analyzed. CI/NR patients had a higher mean score on the Behavioral Subscale of the MOPAT than the AO patients, demonstrating more restlessness, muscle tension, and vocalizations. They also had a higher mean score on the Physiological Subscale, with more changes in blood pressure and heart rate. The AO patients had a mean McGill PPI Scale score of 2.6 (discomforting/distressing), which was similar to their mean MOPAT Behavioral Subscale score of 2.1 (mild to moderate). Because the target sample for CI/NR patients was not achieved, reliability and validity analyses were not possible. Moreover, there was also little opportunity for hospice nurses to use the CUQ, precluding analysis of clinical utility. To achieve these goals, the investigators decided to conduct a follow-up study in hospice patients.

Table 1.

Demographic and Clinical Characteristics of Patients

Characteristic	Alert and oriented (n = 28)^a	Cognitively impaired/nonresponsive (n = 30)^b
Age	Mean = 60.6 years	Mean = 67.5 years
	SD = 16.5 years	SD = 16.8 years
Karnofsky Scale	Mean = 29.3	Mean = 15.0
	SD = 8.29	SD = 4.5
Gender	Female 61%	Female 55%
	Male 39%	Male 39%
		Missing 6%
Race/Ethnic background	Caucasian 71%	Caucasian 58%
	African American 21%	African American 33%
	Asian 4%	Asian 3%
	Hispanic 4%	Missing 6%
Marital status	Divorced 32%	Single 40%
	Married 25%	Married 27%
	Single 21%	Divorced or separated 12%
	Widowed 21%	Widowed 9%
		Missing 12%
Extent of cancer	Metastatic 39%	Metastatic 86%^c
	Local/Regional 18%	Local 14%
	Unknown 43%
Change in pain	Worsening pain 96%	Worsening pain 71%
	New pain 4%	New pain 29%
Cause of pain	Bone/soft tissue 54%	Bone/soft tissue 43%
	Visceral 14%	Unknown or other 57%
	Nerve 7%
	Unknown or other 25%
Pain Medication	Morphine 39%	Morphine 40%
	Fentanyl 25%	Fentanyl 40%
	Hydromorphone 12%	Other 20%
	Other 24%
Medication route	Oral 47%	Oral 40%
	Transdermal 27%	Transdermal 40%
	Intravenous 20%	Other 20%
	Other 5%

Sample from Project 2.

Combined sample from Projects 2 and 3.

n = 22 with cancer.

SD, standard deviation.

Table 2.

MOPAT Data in Alert and Oriented (A&O) and Cognitively Impaired/Nonresponsive (CI/NR) Patients

Subscale and items	A&O patients (n = 28)^a	CI/NR patients (n = 7)^a	CI/NR patients (n = 23)^b Time 1 (before pain intervention)	CI/NR patients (n = 22)^c Time 2 (after pain intervention)
Behavioral	Pain	Indicators
Restless	None 61%	None 57%	None 13%	None 46%
	Mild 36%	Mild 43%	Mild 22%	Mild 36%
	Moderate 3%	Moderate 0%	Moderate 35%	Moderate 14%
	Severe 0%	Severe 0%	Severe 30%	Severe 4%
Tense muscles	None 36%	None 29%	None 0%	None 23%
	Mild 53%	Mild 57%	Mild 13%	Mild 59%
	Moderate 11%	Moderate 0%	Moderate 44%	Moderate 18%
	Severe 0%	Severe 14%	Severe 44%	Severe 0%
Frowning/Grimacing	None 36%	None 57%	None 0%	None 64%
	Mild 50%	Mild 14%	Mild 13%	Mild 23%
	Moderate 14%	Moderate 14%	Moderate 70%	Moderate 9%
	Severe 0%	Severe 14%	Severe 17%	Severe 4%
Patient sounds	None 90%	None 43%	None 4%	None 27%
	Mild 10%	Mild 43%	Mild 17%	Mild 64%
	Moderate 0%	Moderate 14%	Moderate 61%	Moderate 4.5%
	Severe 0%	Severe 0%	Severe 17%	Severe 4.5%
Mean (SD)	2.07 (0.87)	3.0 (3.3)	8.1 (1.95)	2.1 (2.57)

Physiological	Pain	Indicators
Blood pressure	No change 82%	No change 71%	No change 57%	No change 86%
	Change 18%	Change 29%	Change 43%	Change 14%
Heart rate	No change 75%	No change 71%	No change 22%	No change 73%
	Change 25%	Change 29%	Change 78%	Change 27%
Respirations	No change 86%	No change 86%	No change	No change 64%
	Change 14%	Change 14%	22%, Change 14%	Change 35%
Diaphoresis	No change 96%,	No change 100%,	No change 48%	No change 77%
	Change 4%	Change 0%	Change 52%	Change 23%
Mean (SD)	0.61 (0.87)	0.71 (0.95)	2.5 (1.47)	1.0 (1.02)

Sample from Project 2.

Sample from Project 3.

n = 22 due to missing data.

Project 3 (1999–2002) was initiated to accrue additional CI/NR patients so that psychometric and clinical utility analyses could be performed in this group and to enlist hospice nurses in using the MOPAT so that they could complete the CUQ. The study was approved by the IRB at the first author's university and accrued 23 CI/NR patients cared for in a single inpatient hospice located in the northeastern United States.²⁶ Eligibility criteria were the same as in Project 2, with the exception that medical diagnosis was expanded beyond cancer. Fifteen of the 23 patients had a cancer diagnosis, with diverse remaining diagnoses excepting dementia as noted above. Instruments included the MOPAT and a revised Patient Data Form to accommodate the addition of diagnoses besides cancer. Following training, 20 hospice nurses (representing those available at the site and willing to participate in the study) used the MOPAT before (Time 1) and after (Time 2) a pain-relieving intervention. Because this intervention consisted exclusively of analgesics, the Time 2 MOPAT rating was based on a table indicating onset of relief, thereby enabling an analysis of the MOPAT's sensitivity to changes in pain due mainly to short-acting analgesics. As much as possible, Time 2 ratings were performed by different nurses than Time 1 ratings. The 23 CI/NR patients were added to the 7 patients previously recruited in Project 2, yielding a total of 30 CI/NR patients (see Table 1 for demographic and clinical characteristics). AO patients (n = 28, Project 2) had little variance on the MOPAT subscales as compared with CI/NR patients (see Table 2), and their PPI scores were not correlated with the MOPAT subscales. Analyses then focused on psychometric properties of the MOPAT in CI/NR patients. Internal consistency reliability using Cronbach's α coefficient was 0.85 and 0.78 for the Behavioral and Physiological Subscales, respectively, at Time 1 (when variability was greatest). Evidence of validity was demonstrated in two ways. First, sensitivity to change in pain with a pain-relieving intervention was explored by comparing MOPAT ratings before (Time 1) and after (Time 2) the pain intervention (Fig. 1). Mean scores for the Behavioral and Physiological Subscales were 6.67 and 2.23 at Time 1, and 2.55 and 0.86 at Time 2 (p < .001). Second, an exploratory factor analysis (principal components analysis with Oblimin rotation) supported two components—Behavioral (eigenvalue = 4.11) and Physiological (eigenvalue = 1.20)—with factor loadings ranging from 0.534 to 0.944 and together accounting for 66% of the variance. Because the Sensory Pain Indicator item was not consistently used throughout the study, no analysis was performed. Finally, analysis of CUQs completed by 20 hospice nurses who used the MOPAT revealed that 100% said it was helpful in assessing pain, easy to understand, and quick to complete, and 90% said they would use it in their daily practice and felt that family members could use it with adequate training.²⁷ These data showed preliminary evidence of reliability, validity, and clinical utility.

FIG. 1.

Sensitivity to change in pain with an intervention (Projects 2 and 3). Mean Behavioral (B) scores: T1: 6.67; T2: 2.55; p = 0.000. Mean Physiological (P) scores: T1: 2.23; T2: .86; p = 0.000.

Project 4 (2004–2005) consisted of additional and more detailed analysis of data from Projects 2 and 3 to reexamine reliability and validity and to examine the relationship of MOPAT items with selected demographic and clinical variables in order to revise the instrument for testing in a larger study (DB McGuire, KS Kaiser, K Soeken, unpublished data, 2005). Results revealed that internal consistency reliability across the eight items (four Behavioral, four Physiological) was 0.79 before and 0.84 after pain medication. Before pain medication, all items had corrected item-total correlations > 0.35. Removing the Blood Pressure item, which had a squared multiple correlation (SMC) of only 0.290, would have had a negligible effect, increasing the reliability to only 0.85. After pain medication, two items had corrected item-total correlations < 0.20 and, as before, the Blood Pressure item had a SMC of only 0.154, but removing it would have had only minimal effect, increasing reliability from 0.79 to 0.80. The reliabilities of each subscale (Behavioral, Physiological) were also examined and found to be > 0.75 except for the Physiological after pain medication. Although reliability on this dimension was less than desirable—partially due to lack of variability—other explanations could not be fully explored due to lack of data on medications and other patient factors that might have affected ratings. Both the Behavioral and Physiological subscale scores and each of the individual items demonstrated significant change (p < 0.035) after pain medication (n = 29). Finally, the correlation between research nurse and hospice nurse ratings on items tended to be > 0.40 although sample size was small (n = 7). From 35% to 55% of the variability among scores of the Behavioral and Physiological subscales was due to persons, whereas variability due to items ranged from 0% (Physiological after pain medication) to 12% (r = 0.35). These results confirmed and expanded Project 3's data and provided information on sample size and other patient information (e.g., medication data, vital signs) needed to more fully explain variability in item ratings in a subsequent larger study.

Discussion

Project 1 produced a multidimensional pain assessment instrument (MOPAT) for further testing in cognitively impaired/nonresponsive (CI/NR) hospice patients with sufficient sensitivity and clinical utility of the MOPAT demonstrated in Project 2 to justify additional work in CI/NR hospice patients. Project 3 revealed preliminary evidence of reliability and validity of the MOPAT and demonstrated that hospice staff nurses could be trained to use the MOPAT to rate patients using a standardized protocol. Of note is that the combined sample from Projects 2 and 3 was ethnically diverse (see Table 1), in contrast to most published studies, which have predominantly Caucasian samples. Additional analysis of data from Projects 2 and 3 in Project 4 substantiated the preliminary psychometric results, suggested clinical utility, provided information for a computed sample size in larger prospective psychometric evaluation of the MOPAT, and identified other patient information needed to more fully explain variability in item ratings. Based on these preliminary data, the authors embarked on the design of a prospective, full-scale psychometric study of the MOPAT in hospice as there was still no suitable pain assessment instrument available for patients unable to self-report their pain.⁷

However, the emergent palliative care literature repeatedly identified the need for a measure that could be used to assess pain in noncommunicative patients both within and across settings where palliative care was delivered.^5,9 As noted above, existing instruments had not been tested in completely noncommunicative patients, nor evaluated for use across palliative care settings, regardless of patient population. Because many noncommunicative patients receive palliative care in settings ranging from tertiary acute care hospitals to hospices to long-term care facilities to the home, consistent pain assessment is a challenge, along with documentation and communication of pain across settings and health care providers.^28,29

Continuity of care across these various settings is viewed as a key component of clinical practice guidelines for quality palliative care.^28,29 Adequate pain management is essential for quality care and for quality of life. Because noncommunicative palliative care patients increasingly transition across different settings, it is imperative that a simple, broadly applicable pain assessment tool be available for assessment, documentation, communication, and evaluation of pain-related outcomes.

Thus, as the authors were completing the analysis of Project 4 and planning their larger study in 2005–2006, they shifted their focus to patients receiving palliative care not only in the hospice setting, but also in the tertiary acute care hospital setting, where palliative care is increasingly recognized as essential to overall quality care.^28,29 They procured grant funds to support a large multisite study designed to examine the reliability, validity, and clinical utility of the MOPAT when used by nurses to assess pain in noncommunicative palliative care patients in a large university-affiliated hospital and a community-based inpatient hospice. This research is ongoing, with results to be available in the near future. The production of a reliable, valid, and clinically useful pain assessment tool to monitor the presence and severity of pain in noncommunicative palliative care patients across settings should serve to enhance clinicians' abilities to assess and manage pain in this vulnerable population.

Footnotes

Acknowledgments

Project 2 was funded in part by the Oncology Nursing Foundation Small Grants Program. Project 3 was funded in part by the University of Pennsylvania School of Nursing National Institute of Nursing Research (NINR)-funded P30 Center for Advancing Care in Serious Illness. The authors acknowledge the assistance of the nurses who collected data in the hospice sites for Projects 2 and 3. The current multisite study is funded by the NINR, National Institutes of Health, Bethesda, Maryland (Grant No. 1R01NR009684).

Author Disclosure Statement

No competing financial interests exist.

References

American Pain Society. Principles of Analgesic Use in the Treatment of Acute Pain and Cancer Pain, 6^th. Glenview, IL: American Pain Society, 2008.

International Association for the Study of Pain (IASP) 1994. IASP Pain terminology. http://www.iasp-pain.org/terms-p.html. 2010 July 2.

McGuire

. Comprehensive and multidimensional assessment and measurement of pain. J Pain Sympt Manage, 1992; 7:312–319.

McGuire

. The multiple dimensions of cancer pain: A framework for assessment and management. McGuire

, Yarbro

, Ferrell

. Cancer Pain Management, 2nd. Boston, MA: Jones & Bartlett, 1995; 1–17.

Caraceni

, Cherny

, Fainsinger

, Kaasa

, Poulain

, Radbruch

, De Conno

. Pain measurement tools and methods in clinical research in palliative care: Recommendations of an expert working group of the European Association of Palliative Care. J Pain Symp Manage, 2002; 23:239–255.

Ingham

, Portenoy

. The measurement of pain and other symptoms. Doyle

, Hanks

, Chermey

, Calmas

. Oxford Textbook of Palliative Medicine, 3^rd. New York: Oxford University Press, 2005; 167–184.

McGuire

, Kim

, Lang

. Measuring pain. Frank-Stromborg

, Olsen

. Instruments for Clinical Health-care Research, 3rd. Sudbury, MA: Jones & Bartlett, 2004; 603–644.

Kaiser

. Assessment and management of pain in the critically ill trauma patient. Crit Care Nurs, 1992; 15:14–34.

Paice

, Muir

, Shott

. Palliative care at the end of life: Comparing quality in diverse settings. Am J Hosp Palliat Care, 2004; 21:19–27.

10.

McGuire

. Measuring pain. Frank-Stromborg

, Olsen

. Instruments for Clinical Health-care Research, 2^nd. Boston, MA: Jones & Bartlett, 1997; 528–564.

11.

Puntillo

. Pain in the Critically Ill: Assessment, Management. Gaithersburg, MD: Aspen, 1991.

12.

Gélinas

, Fortier

, Viens

, Fillion

, Puntillo

. Pain assessment and management in critically ill intubated patients: A retrospective study. Am J Crit Care, 2004; 13:126–135.

13.

Gélinas

, Fillion

, Puntillo

, Viens

, Fortier

. Validation of the Critical-Care Pain Observation tool in adult patients. Am J Crit Care, 2006; 15:420–427.

14.

Herr

, Bjoro

, Decker

. Tools for assessment of pain in nonverbal older adultrs with dementia: A state-of-the-science review. J Pain Sympt Manage, 2006; 31:170–192.

15.

Zwakhalen

, Hamers

, Abu-Saad

, Berger

. Pain in elderly people with severe dementia: A systematic review of behavioural pain assessment tools. BMC Geriatr, 2006; 6:3.

16.

Mateo

, Krenzischek

. A pilot study to assess the relationship between behavioral manifestations and self-report of pain in postanesthesia care unit patients. J Post Anesth Care Nurs, 1992; 7:15–21.

17.

Webb

, Kennedy

. Behavioral responses and self-reported pain in post-operative platients. J Post Anesth Care Nurs, 1994; 9:91–95.

18.

Hanken

. The measurement of pain. Newton

, Hunt

, McDowell

, Hanken

. A study of Nurse Action in Relief of Pain. Columbus, OH: The Ohio State University School of Nursing, 1964.

19.

Chambers

, Price

. Influence of nurse upon effects of analgesics administered. Nurs Res, 1967; 16:228.

20.

Bruegel

. Relationship of preoperative anxiety to perception of postoperative pain. Nurs Res, 1971; 20:26.

21.

Kuebler

, McGuire

. Objective assessment of pain in terminally ill, cognitively impaired patients in hospice: A clinical challenge. Abstract presented at: Second International Nursing Research Symposium on Cancer Pain, Seattle, WA, 1995.

22.

Melzack

. The McGill Pain Questionnaire: Major properties and scoring methods. Pain, 1975; 1:277–299.

23.

McGuire

, Yeager

, Dudley

. Behavioral measures of pain in hospice patients. Abstract presented at: International Society of Nurses in Cancer Care 11th International Conference on Cancer Nursing, Oslo, Norway, 2000.

24.

Johnston

. Psychometric issues in the measurement of pain. Finley

, McGrath

. Measurement of Pain in Children and Infants. Seattle, WA: IASP Press, 1998; 5–20.

25.

Waltz

, Strickland

, Lenz

. Measurement in Nursing Research. Philadelphia, PA: FA Davis, 1984.

26.

McGuire

, Reifsnyder

. Testing an instrument to measure pain in non-communicative hospice patients. Abstract presented at: Multinational Supportive Care in Cancer/International Society of Oral Oncology Scientific Meeting, Berlin, Germany, 2003.

27.

McGuire

, Reifsnyder

. Assessing pain in non-communicative hospice patients. Abstract presented at: International Society of Nurses in Cancer Care 13th International Conference on Cancer Nursing, Sydney, Australia, 2004.

28.

National Consensus Project for Quality Palliative Care (NCP). Clinical Practice Guidelines for Quality Palliative Care, 2^nd. Brooklyn, NY: NCP, 2009.

29.

National Quality Forum (NQF). A National Framework and Preferred Practices for Palliative and Hospice Care Quality. Washington, DC: NQF, 2006.