Validation of the Knowledge of Care Options Instrument To Measure Knowledge of Curative,Palliative,and Hospice Care

Abstract

Objective:

Treatment decision-making may be hindered by a lack of knowledge about the care options of curative, palliative, and hospice care. Our purpose was to create and validate an instrument to measure knowledge of these care options, the Knowledge of Care Options (KOCO) instrument.

Methods:

We began by generating a pool of true-false items. Experts (n=32) evaluated items in two rounds of review. For each round, we calculated Content Validity Indexes for each item and for the total scale (S-CVI). Items were revised or dropped as indicated. The clarity and acceptability of KOCO were assessed through cognitive interviews with 10 men and women with cancer. We pilot-tested KOCO with a target population sample of 23 women with metastatic breast cancer as part of larger study testing self-management training materials that included a module on care options.

Results:

Following expert review, the S-CVI was 85.2%, and the KOCO consisted of 11 items. Cognitive interviews showed KOCO to be clear and acceptable. The KR-20 test revealed high internal consistency of 0.89. In the pilot test, the mean pre-test score was 9.3 items correct (SD 1.29). The mean post-test score was 10.21 items correct (SD 0.92). KOCO captured change in knowledge of care options (signed rank test=42.5, p<0.006).

Conclusions:

KOCO is a brief, acceptable instrument capable of assessing knowledge of curative, palliative, and hospice care. Additional testing is needed with larger samples to assess the utility of KOCO for use with various patient populations, family caregivers, and clinicians.

Introduction and Background

Treatment decision-making may be hindered by lack of knowledge about curative, palliative, and hospice care. These care options take different, though overlapping, approaches to health care. While the goal of curative care is to modify disease, the goal of palliative care is to manage disease while providing relief from distressing physical symptoms, and to provide psychosocial and spiritual support. The goal of hospice care is to provide comfort-oriented palliative care to those expected to live for 6 months or less.¹ Research has documented confusion about what each of these care options offers.^2–4 Confusion arises in part because palliative care can be offered concurrently with curative care (e.g., to control pain), and because palliative and hospice care have common components (e.g., spiritual support), as hospice is the final phase of palliative care. Understanding what each of these care options is, and is not, is critical to patients' ability to consider their treatment goals and to plan accordingly.

Curative care is standard, and lack of knowledge about the options of palliative and hospice care contributes to their under-utilization. A recent national survey revealed that 70% of lay public participants (n=800) aged 18 and over were “not at all knowledgeable” about palliative care, but that once informed, they would “very likely” consider using palliative care in the event of serious illness.⁴ For hospice, the median stay is 21.1 days, with more than a third of patients dying or discharged within 7 days of admission.⁵ Delayed or non-use is due in part to misconceptions about the care provided.^6,7 In order to identify gaps in knowledge of curative, palliative, and hospice care, our objective was to create and validate an instrument to measure knowledge of these care options, the Knowledge of Care Options (KOCO) instrument. We developed KOCO for use as a pre- and post-test to assess the efficacy of an educational module on care options being tested as part of a larger study. However, KOCO may be of broader utility because identification of knowledge gaps can help to focus educational efforts for patients, their family members, and clinicians.

Methods and Results

In this initial testing of KOCO, we sought to examine content validity, sensitivity to change, and item frequencies. We followed a standardized process to ensure that KOCO items were representative of all of its content domains.^8,9 This process included identifying the domains, developing operational definitions for each domain, generating items, reviewing of items by experts, assessing via cognitive interviews with a purposive sample, and pilot-testing with a target population. This study was approved by the Yale University Human Investigation Committee.

Identifying content domains

We identified curative, palliative, and hospice care as the domains of interest, conceptualizing them as different care options despite their overlap.

Developing operational definitions

We developed operational definitions for curative, palliative, and hospice care by adapting definitions used by the National Consensus Project for Quality Palliative Care¹ (Table 1).

Table 1.

Operational Definitions for Care Options

Care option	Operational definition
Curative care	The treatment of patients with the intent of curing their disease or condition
Palliative care	Care focused on relief of symptoms of illness, as well as psychological and spiritual issues. The goal is to prevent and/or ease suffering, and to offer patients and their families the best possible quality of life. Palliative care is appropriate at any point in an illness; it is not dependent on prognosis. It can also be provided at the same time as curative and life-prolonging treatment.
Hospice care	Care for people with terminal illnesses (<6 months to live) and their families by meeting the patient's physical, emotional, social, and spiritual needs, as well as the needs of the family. The goals of hospice are to keep the patient as comfortable as possible by relieving or minimizing symptoms; to prepare for a death that follows the wishes and needs of the patient; and to reassure the patient and family members by helping them understand and manage what is happening.

Generating items

Based on various aspects of the operational definitions and on common misconceptions about palliative and hospice care, we generated an initial pool of 22 items covering all content domains. Our goal was for items to pertain to what individuals need to know to distinguish one care option from another. The 22 items were reduced to 13 by eliminating redundancy and optimizing wording. Three items related to curative care, five to palliative care, and five to hospice care.

Reviewing of items by experts

We created a content validity rating form to facilitate experts' review of items. The form included the operational definitions for care options. Using a modification of the method described by Lynn,⁹ experts read each item, rated its relevance to knowledge of care options using a four-point Likert scale for relevance (1=not; 2=somewhat; 3=quite; 4=very), indicated items as true or false, and suggested edits or new items.

We conducted two rounds of expert review. For each round, we calculated Content Validity Indexes (CVI) for each item (I-CVI) and for the total scale (S-CVI). CVIs>80% (based on relevancy ratings of 3 or 4) indicate content validity.¹⁰ I-CVIs were calculated by dividing the number of high-relevancy ratings (3 or 4) per item by the number of experts who responded to the item. Items with a low I-CVI were eliminated or revised per experts' comments. S-CVIs were calculated as the average of the I-CVIs.¹¹ The evolution of KOCO items across both review rounds is presented in Table 2.

Table 2.

Evolution of Knowledge of Care Options (KOCO) Items

Expert review Round One (n=23) 13 items	C	IC	M	I-CVI	Expert review Round Two (n=9) 12 items	C	IC	M	I-CVI	Final KOCO 11 items
Even if my cancer cannot be cured, it can still be treated.	19	3	1	0.82	Even if my cancer can't be cured, it can still be controlled .	6	2	1	0.67	Even if my cancer can't be cured, it can still be treated .
Curative care includes any treatment or medication that seeks to get rid of my cancer.	21	1	1	1.00	Curative care is any treatment or medicine that tries to get rid of my cancer.	8	1	0	0.78	Curative care is any treatment or medicine that tries to get rid of my cancer permanently.
People who have advanced cancer can receive curative care.	7	15	1	0.70	People who have advanced or late stage cancer can get curative care.	7	1	1	0.78	Item dropped
A goal of palliative care is to find a cure for my cancer.	19	2	2	0.91	A goal of supportive care is to find a cure for my cancer.	9	0	0	0.78	A goal of palliative care is to find a cure for my cancer.
Palliative care does not include chemotherapy, radiation, and surgery.	18	3	2	0.86	Supportive care includes chemotherapy, radiation, and surgery.	9	0	0	0.78	Palliative care can include chemotherapy and radiation.
I can receive palliative care at any time during my illness.	20	2	1	1.00	I can get supportive care at any time during my illness.	9	0	0	1.00	I can get palliative care at any time during my illness.
Palliative care helps with pain and other symptoms.	21	1	1	1.00	Supportive care helps manage pain and other symptoms.	9	0	0	0.88	Palliative care helps manage pain and other symptoms.
Psychological and spiritual issues are addressed as part of palliative care.	22	0	1	1.00	Psychological, social, and spiritual issues are a part of supportive care.	8	0	1	1.00	Psychological, social, and spiritual issues are a part of palliative care .
Hospice can help patients find a cure for their disease.	22	0	1	0.86	Hospice can help me find a cure for my disease.	9	0	0	0.78	Hospice can help me find a cure for my disease.
Hospice is for patients who are expected to live 6 months or less.	21	1	1	0.87	Hospice is for patients who will probably live 6 months or less.	9	0	0	1.00	Hospice is for patients who will probably live 6 months or less.
Hospice care hastens death.	22	0	1	0.83	Hospice care makes death happen faster.	8	0	1	0.88	Hospice care makes death happen faster.
The goal of hospice care is to address the physical, emotional, social, and spiritual needs of patients.	22	0	1	1.00	The goal of hospice is to help with physical, psychological , social, and spiritual needs of patients and families .	9	0	0	0.89	One of the goals of hospice care is to help with physical, psychological, social, and spiritual needs of patients and families.
Hospice care treats the basic disease process.	17	1	5	0.45	Item dropped
	S-CVI=0.869					S-CVI=0.852

C, correct response; IC, incorrect response; M, missing response; I-CVI, item Content Validity Index; S-CVI, total scale Content Validity Index.

Bold italic words denote revisions made based on reviewers' comments.

Expert review Round One

In Round One of review, a multidisciplinary panel of 23 experts participated. Experts were chosen based on their research and/or clinical expertise on the three care options and on their varied geographic locations and practice settings.¹² Disciplines included medicine, nursing, palliative care, public health, and social work. In this round, 11of the 13 items had an I-CVI of ≥80%. The S-CVI was 86.9%, providing evidence of content validity. Of the 23 experts, 19 or more agreed on the correct response for 10 of the 13 items. One item, “Hospice care treats the basic disease process,” was dropped due to overall lack of clarity. Items were modified per reviewers' comments. The term “palliative care” was replaced with “supportive care,” as some experts felt “supportive care” was more acceptable. We did not find differences in responses to items by discipline. Following the revisions, KOCO had 12 items.

Expert review Round Two

In Round Two of review, nine national palliative care experts from five states (CA, CT, MA, NY, and VA) participated. Disciplines included medicine, nursing, public health, and psychology. In this round, 6 of the 12 items had an I-CVI of ≥80%, and 4 had an I-CVI of 78%. The S-CVI was 85.2%. Of the nine experts, eight or more agreed on 10 of the 12 items. One of the items with lesser agreement, “People who have advanced cancer can get curative care,” was dropped. This item was deemed problematic because of the varying potential of cure for different cancers at an advanced stage. Two items were modified to improve clarity. Experts commented on the use of the term “palliative care” versus “supportive care.” Feedback included that these terms can be ambiguous and are often used interchangeably. We ultimately returned to “palliative care,” because experts agreed that supportive care generally does not include medical interventions, while palliative care does. The final KOCO had 11 items at an eighth grade reading level (Appendix A). Items were re-ordered so as not to appear by content domain. Scoring is one point per correct response. The total score is based on the number of correct responses, which may be either true or false.

Assessing with a purposive sample

Following expert review, the items were transferred to bubble sheet format. Instructions were included for participants to fill in the bubble they felt corresponded to the correct answer (true/false) for each item. A sample of 10 people with cancer was accrued in October and November 2009 to provide feedback on the clarity and acceptability of the KOCO instrument. Demographic and clinical characteristics for the purposive sample and for the target population sample are presented in Table 3. The purposive sample was recruited from the medical oncology clinic at a cancer center. The sample consisted of six women and four men aged 51–80 years (mean age 66.1 years), who were diverse in their race/ethnicity, education level, marital status, and type and stage of cancer. Participants completed demographic and clinical data forms and then KOCO, with individual cognitive interviews conducted by a trained research staff member immediately following.

Table 3.

Demographic and Clinical Characteristics for Purposive and Target Population Samples

Characteristic	Purposive sample (n=10)		Target population sample (n=23)
Age
Mean (SD)	66.1 years		56.7 (±12.4) years
Range	51–80 years		40–81 years
Age at diagnosis
Mean (SD)			47.9 (±12.2)
Range			30–75 years
Time since diagnosis
Mean (SD)			8.5 (±9.9)
Range			2–40 years
	No.	%	No.	%
Gender
Female	6	60	23	100
Male	4	40
Race
White	7	70	20	87.0
Black	3	30	2	8.7
Hispanic	0	0	1	4.3
Marital status
Married	5	50	17	73.9
Divorced/separated	0	0	3	13.0
Widowed	3	30	1	4.3
Living with partner	0	0	1	4.3
Never married	2	20	1	4.3
Education
Up to 8th grade only	2	20	0	0
High School graduate	2	20	3	13.0
Trade school	0	0	1	4.3
Some college	2	20	4	17.4
College graduate	0	0	8	34.8
Graduate school	4	40	7	30.4
Income
< $50,000	4	40	4	17.4
$50,000–$89,999	1	10	7	30.4
> $90,000	3	30	10	43.5
Unreported	2	20	2	8.7
Religion
Catholic	6	60	12	52.2
Protestant	3	30	5	21.7
Jewish	0	0	3	13.0
Muslim	1	10	1	4.3
Type of cancer
Breast	1	10	23	100
Colon	3	30	0	0
Lung	2	20	0	0
Pancreatic	2	20	0	0
Stomach	1	10	0	0
Sarcoma	1	10	0	0
Stage of cancer
II	1	10	0	0
III	2	20	0	0
IV	5	50	23	100
Unknown	2	20	0	0
Treatment
Chemotherapy	10	100	18	78.3
Radiation	1	10	3	13
Other (e.g., hormone therapy)	0	0	4	17.3

Cognitive interviewing uses verbal probing and think-aloud techniques to assess how individuals interpret questionnaire items and formulate their responses.¹³ During interviews, participants were asked about their understanding of each item, how they might reword or reformat items to improve clarity, and about their overall thoughts about KOCO, to gauge acceptability. Detailed notes were taken. We used content analysis¹⁴ to summarize participants' responses. Interviews and analysis proceeded iteratively.

We conducted univariate analysis for KOCO data. Eight of the 10 participants completed all KOCO items. Of the two participants who had missing responses, one left four items incomplete, and the other left seven incomplete. Items missed by both participants included item 7, “Psychological, social, and spiritual issues are a part of palliative care”; item 8, “Hospice can help me find a cure for my disease”; and item 11, “One of the goals of hospice care is to help with physical, psychological, social, and spiritual needs of patients and families.” The mean raw score (n=8) was 8.7 items correct out of 11 (SD 2.75). Results by item are presented in Table 4.

Table 4.

KOCO Item Analysis for Purposive and Target Population Samples

						Target population sample
	Purposive sample (n=10)					Time 1 (n=23)					Time 2 (n=19)
Item	C	IC	M	%C	Item-total r	C	IC	M	%C	Item-total r	C	IC	M	%C	Item-total r
1. Even if my cancer can't be cured, it can still be treated.	9	0	1	0.90	0.67	18	4	1	0.78	0.19	19	0	0	1.00	1.00
2. Curative care is any treatment or medicine that tries to get rid of my cancer permanently.	10	0	0	1.00	1.00	20	3	0	0.87	−0.08	16	3	0	0.84	−0.10
3. A goal of palliative care is to find a cure for my cancer.	6	3	1	0.60	0.78	13	10	0	0.57	−0.15	16	3	0	0.84	0.07
4. Palliative care can include chemotherapy and radiation.	4	5	1	0.40	0.52	19	4	0	0.83	0.13	17	2	0	0.89	0.11
5. I can get palliative care at any time during my illness.	10	0	0	1.00	1.00	23	0	0	1.00	1.00	19	0	0	1.00	1.00
6. Palliative care helps manage pain and other symptoms.	9	1	0	0.90	0.67	22	1	0	0.96	0.11	17	2	0	0.89	0.04
7. Psychological, social, and spiritual issues are a part of palliative care.	7	1	2	0.70	0.90	21	2	0	0.91	0.17	19	0	0	1.00	1.00
8. Hospice can help me find a cure for my disease.	7	1	2	0.70	0.90	18	5	0	0.78	0.07	18	1	0	0.95	−0.06
9. Hospice is for patients who will probably live 6 months or less.	8	1	1	0.80	0.74	22	1	0	0.96	0.03	17	2	0	0.89	0.04
10. Hospice care makes death happen faster.	10	0	0	1.00	1.00	23	0	0	1.00	1.00	19	0	0	1.00	1.00
11. One of the goals of hospice care is to help with physical, psychological, social, and spiritual needs of patients and families.	7	1	2	0.70	0.47	15	8	0	0.65	0.05	17	2	0	0.89	0.04
Mean raw score (SD)	8.7 (2.75)					9.3 (1.29)					10.21 (0.92)

C, correct response; IC, incorrect response; M, missing response; KOCO, Knowledge of Care Options.

The Kuder-Richardson formula (KR-20) tests internal consistency for instruments with dichotomous response choices. The KR-20 score for KOCO was 0.89, indicating high internal consistency.

Participants felt that all items were clearly worded and no items were modified. Participants understood KOCO's format and were able to complete KOCO within minutes. A participant commented that it was “good, clear, and easy to read.” No one reacted emotionally to any of the items, although one participant commented that KOCO was “slightly off-putting because people are in treatment and have hope, and then they see content about hospice and it can affect them.” Another participant commented “The way it's written stimulates thought.”

A common theme in interviews was a lack of understanding about palliative care. One participant commented “The term, ‘palliative,’ not sure what is involved with it.” Another thought KOCO was “difficult to understand because I don't know terms and types of care enough.” In completing KOCO, some participants were hesitant to respond to items about palliative care, even when encouraged to do so. Nevertheless, the sample had a high mean score; this may be explained by the fact that participants may have known enough to make an educated guess. Alternatively, as with all true-false tests, merely guessing yields a 50% chance of selecting the correct response.

Pilot testing with a target population

We pilot-tested KOCO as part of a larger feasibility study (one group, pre-post test design) that tested a printed self-management guide for women with breast cancer called, Managing Cancer Care: A Personal Guide. The guide consisted of seven modules on the topics of symptom management; care management and goal setting; communication with health care providers; communication with family members; care transitions; uncertainty/empowerment; and care options. The care options module presented content on curative, palliative, and hospice care. The specific aims of the larger study were to improve knowledge of care options, desire and ability to self-manage, communication skills, transitioning, and self-efficacy, and to reduce anxiety, depression, and uncertainty.

To be eligible for the larger study, participants had to be women aged 21 or older, English-speaking, with a diagnosis of metastatic (stage IV) breast cancer, and with a prognosis of at least 3 months. Potential participants were recruited at the infusion clinic at a cancer center between January and May 2010. Potential participants were first approached by their chemotherapy nurse to determine interest in the study. If interested, a research staff member explained the study further and obtained written consent. Participants then completed pre-test data collection, and were given the guide and an orientation to its use. One month later, the research staff member phoned to check in about the participants' use of the guide and to answer any questions. Post-test data collection took place 1 month after the phone call. In addition to KOCO, outcome measures included the Control Preferences Scale,¹⁵ the Medical Communication Competence Scale,¹⁶ the Hospital Anxiety and Depression Scale,¹⁷ the Chronic Disease Self-Efficacy Scale,¹⁸ and the Uncertainty in Illness Scale.¹⁹

We calculated descriptive statistics for demographic, clinical, and KOCO data. Due to the small sample size, we performed the Wilcoxon Signed Rank test to examine differences between pre- and post-test data for all outcome measures, including KOCO. Pearson correlations were used to determine relationships among all measures.

Of the 33 women who were eligible for the study, three were not interested in speaking to a researcher, four refused participation (two did not wish to read health information, and two did not want to take on a new activity), and three were unable to be contacted. Therefore, our sample consisted of 23 women, all of whom completed pre-test data collection. Of these women, two withdrew due to being overwhelmed with their situation, and two died; thus 19 of the 23 participants completed post-test data collection.

The sample was a generally homogenous group of middle to upper middle class, well-educated white women, most of whom were married (73.9%). Participants tended to be young (mean age 56.7 years, range 40–81 years), and over half were Catholic. The average time since diagnosis was 8.5 years (SD 9.9 years, range 2–40 years), and all were receiving treatment.

At pre-test data collection, the mean raw KOCO score was 9.3 items correct (SD 1.29). At post-test data collection, the mean raw score was 10.21 items correct (SD 0.92; Table 4). KOCO captured change in knowledge of care options between the two data collection points (Signed Rank test=42.5, p<0.006). KOCO was not significantly correlated with any of the other outcome measures.

Discussion

Although further development is required, initial testing indicates that KOCO is a content-valid, acceptable means of evaluating patients' knowledge of care options and is sensitive to changes in related knowledge over time. This test can be used in both research and clinical settings to identify gaps in patients' knowledge about care options. While we do not advocate that KOCO replace face-to-face discussions about care options, identification of gaps can help to focus education and patient-clinician discussions that facilitate patients' understanding of their options for care. KOCO might also be used as a diagnostic tool to determine what family caregivers and clinicians understand about these care options. Future research should test this potential use.

Although we tested KOCO with people with cancer, it is likely that KOCO would be efficacious in evaluating knowledge of care options among non-cancer populations as well. The test could easily be tailored to suit any chronic, serious illness, or could be made generic for use with mixed samples. For example, item 10, “Even if my cancer can't be cured, it can still be treated,” could be modified to read “Even if my disease can't be cured, it can still be treated.” Cut-off or “passing” scores that denote “being knowledgeable” about care options should be tailored to suit investigators' purposes.

In creating and determining the correct response to items, we found that the complexity and societal perplexity commonly associated with curative, palliative, and hospice care carried over into our development of KOCO. The nuances among care options necessitated careful wording of items to make them medically accurate, and for most of the expert reviewers to agree upon the correct response. Terminology and conceptual definitions around palliative and supportive care continue to be discussed in the literature.^20–22 Our experience developing KOCO reflected ambiguities in current debate and delivery of care.

Concerns that participants might react negatively to completing test items about palliative and hospice care were largely unfounded. Cognitive interview participants more often described items as thought-provoking rather than disconcerting. Participants who skipped items reported doing so because they did not understand the term “palliative care,” not because they were upset by the items. While one participant described some KOCO items as “slightly off-putting,” it is reasonable to expect that items may prompt individuals to consider their own situations and the type of care they are or could be receiving. While KOCO was not designed to serve as a gauge of individuals' comfort with care options, our interview data reinforce the thought that patients may not be as reticent as is often thought to discuss care options.^23–25

Limitations

KOCO's greatest limitation is likely its true-false format. With a 50-50 chance of selecting the correct response, participants' high KOCO scores may not have reflected their actual knowledge of care options. However, use of a multiple choice format did not seem feasible given the complexity of care options and the challenges we experienced in trying to reach agreement among experts with a simple true-false format. Multiple choice format is also more dependent on test-taking skills and would make KOCO a more complex instrument.

We pilot-tested KOCO with a small, fairly homogenous sample of women with metastatic breast cancer. These women had an average of 8.5 years of experience managing their care, and may have been more knowledgeable about care options than women more recently diagnosed. KOCO should be tested with individuals with different stages of cancer to explore whether people who have moved through the care options are more knowledgeable about them than those who have not.

Another limitation to this study is that we were unable to establish convergent validity, as KOCO did not correlate with the other outcome measures in the feasibility study. However, we selected these outcome measures to evaluate the efficacy of the other intervention modules rather than to further test KOCO's validity. In future tests, we might expect KOCO to correlate with the Uncertainty in Illness Scale, as well as with some self-efficacy scales. A criterion validity study would also be of utility to examine predictive relationships between scores on the KOCO and actual treatment decisions. Additional research with a larger sample is needed to further investigate KOCO's validity.

KOCO's items do not comprehensively represent all aspects of curative, palliative, and hospice care. We were unable to achieve this depth because we wanted to keep KOCO to a reasonable length, and because we needed to drop items on which experts could not agree. Continued development of KOCO may include refinement of item wording to ideally represent care options. Of particular concern would be to refine wording to enable the use of KOCO outside of the United States, where palliative and hospice care are differently structured financially, thereby impacting treatment decision making.

Conclusion

Familiarizing patients, their family members, and clinicians about palliative care continues to be a priority to improve access to palliative care services, as is the development of tools to conduct analyses, assess outcomes, and measure quality of care.²⁶ Although further testing is warranted, KOCO offers a valid and simple means to evaluate basic knowledge of curative, palliative, and hospice care.

Footnotes

Acknowledgments

The authors would like to thank the Round One reviewers, especially Marie Bakitas, Leslie Blatt, Melissa Carlson, Emily Cherlin, Leslie Curry, Lindsay Harris, Matthew Ellman, and Tom Quinn; and the Round Two reviewers, including Elizabeth Bradley, Stephen Connor, Michael DiGiovanna, Patrick Coyne, Betty Ferrell, Diane Meier, Tish Knobf, Ruth McCorkle, and Holly Prigerson. We are also grateful to the patient participants, to the clinic staff at the participating cancer center, and to Deborah Papuga for her assistance with data collection.

This work was supported by a Mentored Research Scholar Grant (MRSG08-292-04-CPPB) from the American Cancer Society.

Author Disclosure Statement

No competing financial interests exist.

References

National Consensus Project for Quality Palliative Care. Clinical Practice Guidelines for Quality Palliative Care, 2nd 2009.

Meier

, Brawley

. Palliative care and the quality of life. J Clin Onc, 2011; 29:2750–2752.

Schulman-Green

, Bradley

, Knobf

, Prigerson

, DiGiovanna

, McCorkle

. Self-management and care transitions in women with advanced breast cancer. J Pain Symptom Manage, 2011; 42:517–525.

Public Opinion Strategies: Public Opinion Research on Palliative Care: A Report Based on Research by Public Opinion Strategies. 2011.

National Hospice and Palliative Care Organization. NHPCO facts and figures: Hospice care in America. Alexandria, VA: National Hospice and Palliative Care Organization, 2010.

Ford

, Nietert

, Zapka

, Zoller

, Silvestri

. Barriers to hospice enrollment among lung cancer patients: A survey of family members and physicians. Pall Supp Care, 2008; 6:357–362.

Feeg

, Elebiary

. Exploratory study on end-of-life issues: Barriers to palliative care and advance directives. Am J Hosp Pall Med, 2005; 22:119–124.

DeVellis

. Scale Development: Theory, Applications, 2nd. Thousand Oaks, CA: Sage Publications, 2003.

Lynn

. Determination and quantification of content validity. Nurs Research, 1986; 35:382–385.

10.

Davis

. Instrument review: Getting the most from your panel of experts. Appl Nurs Res, 1992; 5:194–197.

11.

Polit

, Beck

. The content validity index: Are you sure you know what's being reported: Critique and recommendations. Res Nurs Health, 2006; 29:489–497.

12.

Grant

, Davis

. Selection and use of content experts for instrument development. Res Nurs Health, 1997; 20:269–274.

13.

Knafl

, Deatrick

, Gallo

, Holcombe

, Bakitas

, Dixon

, Grey

. The analysis and interpretation of cognitive interviews for instrument development. Res Nurs Health, 2007; 30:224–234.

14.

Neuendorf

. The Content Analysis Guidebook. Thousand Oaks, CA: Sage Publications, 2002.

15.

Degner

, Sloan

, Venkatesh

. The Control Preferences Scale. Can J Nurs Res, 1997; 29:21–43.

16.

Cegala

, Coleman

, Turner

. The development and partial assessment of the Medical Communication Competence Scale. Health Commun, 1998; 10:261–288.

17.

Zigmond

, Snaith

. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandanavia, 1983; 67:361–370.

18.

Lorig

, Stewart

, Ritter

, González

, Laurent

, Lynch

. Outcome Measures for Health Education and Other Health Care Interventions. Thousand Oaks CA: Sage Publications, 1996.

19.

Mishel

. Reconceptualization of the Uncertainty in Illness Theory. J Nurs Schol, 1990; 22:256–262.

20.

Herrstedt

, Walsh

. Introduction: Supportive care and palliative medicine. Semin Oncol., 2011; 38:335–336.

21.

Kreuter

, Herth

. Supportive and palliative care of advanced nonmalignant lung disease. Respiration, 2011; 82:307–316.

22.

Storey

, Fallon

, Smyth

. The interface between medical oncology and supportive and palliative cancer care. Semin Oncol, 2011; 38:337–342.

23.

Aitini

. Breaking bad news in onco-hematology: new hope, new words? Leuk Lymphoma, 2011; 53:328–329.

24.

Innes

, Payne

. Advanced cancer patients' prognostic information preferences: a review. Palliat Med, 2009; 23:29–39.

25.

Evans

, Tulsky

, Back

, Arnold

. Communication at times of transitions: how to help patients cope with loss and re-define hope. Cancer J, 2006; 12:417–424.

26.

Morrison

, Meier

. The National Palliative Care Research Center and the Center to Advance Palliative Care: a partnership to improve care for persons with serious illness and their families. J Pediatr Hematol Oncol, 2011; 33,Suppl 2:S126–S131.