Abstract
Background:
No reliable instruments exist to measure prognostic awareness and its psychological and behavioral impacts for patients with advanced cancer.
Methods:
We developed the Prognostic Awareness Impact Scale (PAIS) using a qualitative approach. During phase 1, we convened a working group with a transdisciplinary team of clinicians from oncology (n = 2), psychology (n = 2), psychiatry (n = 1), palliative care (n = 3), and survey development (n = 1) to identify key domains of PAIS. Using a consensus-driven process, the team generated an item bank for each domain. During phase 2, we conducted cognitive interviews with 39 patients with advanced cancer to assess the understandability of the PAIS.
Results:
The working group developed a conceptual framework for PAIS, identifying three domains: (1) cognitive understanding of prognosis (capacity to understand intellectually one's prognosis), (2) emotional coping (capacity to process prognostic uncertainty and terminal prognosis), and (3) adaptive response (capacity to use prognostic awareness to inform life decisions). Cognitive interviews revealed that patients had an accurate understanding of most PAIS items. Patients reported difficulty with binary response options for questions pertaining to emotional coping. They expressed difficulty answering numerous questions regarding their cognitive understanding of their prognosis. We revised the PAIS by (1) replacing binary response options with ordinal agreement scales; and (2) reducing the number of items focused on cognitive understanding of prognosis.
Conclusion:
We developed a conceptual framework to capture prognostic awareness and its psychological and behavioral impacts for patients with advanced cancer using the PAIS. Future work should focus on validating the PAIS by testing its psychometric properties.
Background
Patients with advanced cancer often face challenging decisions regarding their medical care, treatment options, and end-of-life care. Given the heterogeneity of the illness trajectory for various cancers, there is substantial variability in prognosis for patients with advanced cancer. Patients' perceptions of their illness and prognosis play an important role in their treatment decision making.1–3 For example, patients who overestimate their chance of survival are often more likely to choose aggressive care at the end of life. 4 Additionally, there is wide variability in patients' willingness to accept certain treatments, such as chemotherapeutics, especially when the possibility of cure is remote.1–3 Studies have also shown that patients' perceptions of their prognosis is associated with their psychological distress.5,6 Collectively, these findings underscore the importance of prognostic awareness and its impact on patients' behaviors and emotional coping with their illness.
Despite the importance of prognostic awareness, reliable and valid instruments to measure prognostic awareness and it behavioral and psychological impact on patients with serious illness are lacking. 7 Clinicians and investigators lack consensus regarding how best to evaluate prognostic awareness. 7 Studies show that the conceptualization of prognostic awareness varies as a unidimensional or multidimensional construct, thereby resulting in discrepancies of assessment.7–9 Moreover, some investigators define prognostic awareness as the awareness of having metastatic or advanced disease, whereas others require the estimation of survival for a specified period of time.4,7,10–15 While some studies focus on terminal illness acknowledgment as a measure of prognostic awareness,12,13,16,17 terminal illness acknowledgment often captures a particularly ill population near the end of life, and may not provide helpful information about prognostic awareness early in the illness course.6,18 Some of the items used in the literature to measure prognostic awareness include those that focus on curability of disease, terminal illness acknowledgment, overall goals of treatment, and desire for prognostic disclosure.4,5,12,16,19 Other items have focused primarily on prognostic and end-of-life communication. 20 As these items focus exclusively on the cognitive domain of prognostic understanding, we lack a comprehensive assessment of how patients understand their prognosis cognitively, process it emotionally, and adapt to it behaviorally, which are critical elements for integrating prognostic knowledge into medical informed decision making.21,22
Discrepancies in prognostic awareness assessments and its association with psychological distress in patients with advanced cancer highlight the need for empirically validated measures of prognostic awareness and its psychological and behavioral impact in this population.5–7,10,13,23–26 In this study, we developed the Prognostic Awareness Impact Scale (PAIS) to comprehensively assess prognostic awareness and its psychological and behavioral impact in patients with advanced cancer. The PAIS encompasses the cognitive, emotional, and behavioral domains of prognostic awareness and their impact on patients' decision making. The PAIS was developed using a 9-person transdisciplinary working group of experts from oncology, psychology, psychiatry, palliative care, and survey development. We then performed cognitive interviews with patients with advanced cancer to ensure that the questions were understandable, answerable, and interpretable as intended by our research team. We revised the PAIS based on the findings from these cognitive interviews.
Methods
Study design
This survey development study proceeded in two phases. During phase 1, we conducted five working group meetings with a transdisciplinary team of expert clinicians and researchers from oncology (n = 2), psychology (n = 2), psychiatry (n = 1), palliative care (n = 3), and survey development (n = 1). Expert review committees are often utilized successfully in survey development and testing typically with 3–10 experts included depending on the needs for the particular instrument.27–30 The transdisciplinary team first reviewed the published literature on prognostic awareness as well as prior items used to measure certain aspects of illness and prognostic understanding, such as acknowledgment of terminal illness, measures of life expectancy, and patient's report of the likelihood of a cure.4,5,12,16,19 We conducted a database search using PubMed, MESH, and Boolean search techniques. Search items included (Prognostic awareness[tiab] OR prognostic understanding[tiab] OR illness understanding OR prognosis) AND (“Cancer”[Mesh] OR “Advanced Cancer”[Mesh] OR “Neoplasms”[Mesh] OR solid tumor OR solid tumour). We also reviewed the references for all articles identified in the search to identify additional relevant citations. We removed duplicates, screened for irrelevant titles and abstracts, and narrowed our search to include studies that involved human subjects. Articles must be published in English and peer reviewed with no restriction on the timeframe for publication. The team then developed a conceptual framework highlighting the key domains to be measured with the PAIS (Fig. 1). The working group then used an iterative consensus-driven process to finalizing the domains of the PAIS. Next, the working group generated an item bank for each domain through an iterative consensus-driven process, utilizing prior literature as well as items from measuring of other related constructs such as peace and acceptance in the cancer experience, 31 coping with uncertainty,32,33 emotional distress, 34 and post-traumatic growth. 35 The initial item bank consisted of 55 items. Two external advisors with expertise in survey development then reviewed all items for additional refinement. The final revised version of the PAIS to be used in phase 2 consisted of 49 items.

Conceptual framework for perceptions of prognosis domains.
In the second phase, we conducted cognitive interviews with 39 patients with advanced cancer to assess whether the PAIS items were understandable, answerable, and interpretable as intended.36–40 We included adult patients (≥18 years) with a diagnosis of advanced solid cancer. We defined patients with advanced solid cancer as those not being treated with curative intent, which we identified based on chemotherapy order entry treatment intent designation or other clinical documentation. We excluded patients with relapsed and refractory hematologic malignancies given the immense prognostic uncertainty in this population and inability to clearly identify a cohort of patients with incurable disease.21,41 Participants were required to speak English or complete the study procedures with minimal assistance from an interpreter. We excluded patients with significant uncontrolled psychiatric disorders or other comorbid disease that the treating clinician believed would prohibit the ability to participate in the study procedures. After completion of the cognitive interviews and their data analysis, the transdisciplinary working group conducted four follow-up meetings to amend the PAIS based on the cognitive interview findings using a consensus-driven approach.
Study procedures
We used a purposeful sampling approach to identify potential study participants and ensure adequate representation of patients with various cancer types. Specifically, we capped enrollment for specific disease groups, including lung, gastrointestinal, and breast cancers, to ensure representation from other cancers. We also purposefully sampled rare tumor disease centers, including melanoma, sarcoma, and head and neck cancers, to achieve adequate representation for other cancer types. A research assistant (R.A.) utilized the oncology clinic schedules to identify potentially eligible patients for study participation. The R.A. then emailed oncology clinicians to ask them whether they have any objections to patient study participation and to confirm the noncurative intent of their treatment. We explicitly communicated with oncology clinicians that we were seeking to enroll participants across varied levels of illness and prognostic awareness to ensure a representative sample. The R.A. then approached patients for study participation during their next scheduled clinic visit to ask about participation, explain study procedures, and obtain written informed consent. This study was approved by the Dana Farber Harvard Cancer Center Institutional Review Board.
After obtaining informed consent, participants completed a demographic questionnaire and the PAIS in a private clinic room. Participants were instructed to complete the PAIS on their own without input from the study team. Following completion of the self-administered questionnaire, the R.A. conducted a cognitive interview to assess patients' understanding of the survey items and the readability of the overall questionnaire. Cognitive interviewing is a specific technique that is often employed during survey development and testing to identify whether the survey items generate the information that the investigators intended and inform revisions of the questionnaire.42–44 We used the “think-aloud” and “verbal probing” strategies to test survey questions and understand participants' cognitive responses during testing (Supplementary Appendix SA1: interview guide).45,46 We utilized a variety of probing techniques, including comprehension, paraphrasing, recall, specific probing, and general probing.45,46 All interviews were audiorecorded and transcribed verbatim. The interviews took ∼45 minutes to complete. Participants did not receive any incentives for study participation. We used a predesigned data collection guide during the cognitive interviews to record the verbalized concerns of the study participants.
Data analysis
We summarized participants' baseline characteristics using frequency and percentage for categorical variables and median ± range for continuous variables. We used Stata (v9.3) for descriptive statistics. At the completion of all interviews, we transcribed the data verbatim into written format, and recorded all participants' perceived difficulties with the PAIS and their proposed solutions. Interviews were analyzed using content analysis framework with NVivo 12 qualitative software. Two independent coders (K.B., A.E.) analyzed the interview content thematically overseen by a qualitative research expert (E.P.). The coders met to develop the thematic framework and coding plan. Each interview was coded twice. Interpretation and analysis of coded transcripts identified items that were (1) easy to understand; (2) difficult; or (3) unsure. To ensure coding reliability, coding discrepancies were resolved through discussion and comparison of raw data. Coding continued until a high level of reliability was established (Kappa ≥0.80). We then generated tables detailing the proposed changes to the PAIS across all domains as well as within each domain.
Results
Phase 1: Conceptual model for the PAIS
Based on the literature search, the transdisciplinary working group identified the need for a cognitive domain to measure patients' cognitive understanding of their prognosis. Based on their clinical and research expertise, the transdisciplinary group discussed the importance of developing a multidimensional instrument that also accounts for the emotional and behavioral aspects of prognostic understanding. Upon further discussion, the transdisciplinary working group identified three domains for PAIS and their subdomains (Fig. 1). The PAIS domains include: (1) cognitive understanding of prognosis, (2) emotional coping with prognosis, and (3) adaptive response. The first domain, cognitive understanding, includes one's ability to acknowledge one's life-limiting illness and to understand the goals of treatment. The second domain, emotional coping, includes a person's capacity to cope with both prognostic uncertainty and the terminal nature of the prognosis. Finally, the third domain, adaptive response, encompasses one's resilient behavior and the capacity to use prognostic awareness to inform life decisions (i.e., prognostic growth) in facing a serious illness. In this conceptual model, patients' cognitive understanding of their prognosis impacts their emotional coping and adaptive response. Additionally, patients' emotional coping with their prognosis also impacts their adaptive response and behavior. The PAIS domains collectively will ultimately impact patient-reported outcomes, such as quality of life and psychological distress, as well as patients' end-of-life decision making.
Phase 2: PAIS cognitive interviews
Patient characteristics
We enrolled 86.6% (39/45) of patients with advanced cancer approached for study participation. Table 1 describes patients' baseline characteristics. Enrolled patients were mostly white (92.3%), with a median age of 64.7 years (range 26.8–84.7). Participants had a diagnosis of gastrointestinal (25.6%, 10/39), lung (25.6%, 10/39), breast (15.4%, 6/39), genitourinary (7.7%, 3/39), or other advanced solid tumor (25.6%, 10/39). Other solid tumors included melanoma, head and neck cancers, glioblastoma, and sarcoma.
Patient Characteristics
Revisions to the PAIS across all domains
Data from cognitive interviews revealed that patients had an accurate understanding of most PAIS items. Table 2 summarizes the common themes from patients' responses to the PAIS that were relevant across all domains as well as the revisions implemented based on these findings. Patients reported difficulty with binary response options for questions pertaining to emotional coping with prognosis. They also expressed emotional difficulty answering numerous questions regarding the cognitive understanding of their prognosis. Some patients also reported the need to clarify the meaning of certain words such as “cure” and “prognosis.” We revised the PAIS based on these findings by (1) replacing binary response options with ordinal agreement scales; (2) reducing the number of items focused on cognitive understanding of prognosis; and (3) defining “cure” and “prognosis” in the introduction to the PAIS.
Revisions to the Prognostic Awareness Impact Scale That Are Relevant Across All Domains
Revisions to the PAIS by individual domain
Table 3 summarizes the data from cognitive interviews and revisions made to each domain. Within the cognitive understanding domain, patients reported difficulty differentiating between their cognitive understanding and their hopes for their prognosis. We revised the question items to focus on patients' knowledge about their prognosis and information relayed to them by their oncologist. Some patients also struggled to answer double-barreled questions, such as “I am still hoping for a cure, even though I understand my cancer is incurable.” These items were removed from the cognitive understanding domain of the PAIS.
Revisions to Individual Domains of the Prognostic Awareness Impact Scale
Within the emotional coping domain, we removed qualifiers or polarizing terminology, such as the word “very” or “anxious,” as patients reported that these terms were too strongly worded or vague. Some patients reported difficulty comprehending items such as “how often do you try to avoid things that remind you of your cancer?” due to challenges in identifying reminders of their cancer diagnosis. We removed these items based on these findings. Finally, all patients reported an accurate understanding of all items within the adaptive response domain. However, one item was removed based on working group consensus as it was felt to be outside the scope of this domain. The finalized PAIS consisted of 34 items after the revisions based on cognitive interviews (Supplementary Appendix SA2).
Discussion
In this study, we developed a conceptual framework to capture the complexity of prognostic awareness and its psychological and behavioral implications for patients with advanced cancer using a consensus-driven transdisciplinary working group and developed the PAIS. The PAIS domains capture the cognitive understanding, emotional coping, and adaptive response necessary to integrate prognostic awareness into patients' decision making for future care. We then revised the PAIS based on cognitive interviews with patients with advanced cancer to ensure its readability and understandability. The cognitive interview findings provide encouraging data on the face and content validity of the finalized 34-item PAIS. Thus, we set the stage to develop a rigorous and validated instrument to measure prognostic awareness and its psychological and behavioral impact on the care of patients with advanced cancer as we plan a future study to further validate the PAIS by testing its psychometric properties, including internal consistency and construct validity.
There is a paucity of literature on empiric approaches to assess prognostic awareness and, therefore, little consensus regarding which approach is most useful for research and clinical purposes.7,23 Additionally, the field lacks agreement on the key domains that would encompass a comprehensive assessment of prognostic awareness and its psychological and behavioral implications.7,23 Given the complexity of this construct, we used rigorous methodology working with a transdisciplinary team of clinicians and researchers from oncology, palliative care, psychology, psychiatry, and survey development to thoughtfully capture the various domains of the PAIS based on expert opinion. This methodology ensured that the PAIS domains were clinically relevant and correspond to our current understanding of the prognostic awareness literature as it relates to patient-reported and end-of-life outcomes.
Even with existing crude measures, investigators have shown that accurate prognostic understanding facilitates better planning at the end of life,7,20,47 in addition to better quality of life for patients and their caregivers, with improved bereavement outcomes.7,12,48 Interestingly, studies of the association between prognostic awareness and psychological distress in patients with advanced cancer have yielded mixed results, in part, due to difficulties in measuring prognostic awareness and how patients emotionally cope with their prognosis accurately.5,6,10,13,24–26 Thus, developing and validating a rigorous PAIS has the potential to move the science forward in supportive oncology and enhance our understanding of the impact of prognostic awareness on patient outcomes.
Having an accurate assessment of patients' prognostic awareness and its psychological and behavioral dimensions may also have substantial clinical implications for how clinicians risk stratify patients for additional support. Figure 2 depicts the potential utility of the PAIS in identifying patients at risk for psychological distress and suboptimal decision making. Identifying patients' level of adaptive coping and cognitive understanding can assist in anticipating patients' distress and possible challenges they may face with medical decision making.23,49 This information can guide clinicians to offer tailored support matching the needs of individual patients based on their prognostic perceptions. This becomes even more relevant with the advances in immunotherapy and targeted therapies in oncology, which have led to an increase in prognostic uncertainty and distress related to uncertainty. 50 Importantly, patients can fall along a continuum within each of the domains of the PAIS, which may have different implications for their clinical care. Incorporating the PAIS into clinical practice will also require careful attention to its potential implications on patient/clinician interactions as well as patients' psychological and existential distress. Thus, a validated instrument of prognostic awareness and its psychological and behavioral impact may have important clinical implications for providing high-quality care and supportive care services for patients with advanced cancer.

Potential utility of the PAIS.
This study has several notable limitations. First, the cognitive interviews were conducted at a single academic center with limited sociodemographic diversity, thereby limiting the generalizability of the findings. It is possible that important constructs, such as spirituality and family involvement, in prognostic decision making play a more prominent role in illness and prognostic understanding for other patient populations, including racial and ethnic minorities. Second, the PAIS framework is theoretical and empirical data are needed to validate its content and the proposed relationship between its domains and patient outcomes. Third, we did not specifically select for patients who have had prognostic discussions with their oncologist. Although this may bias results, it also may be more representative of the cancer population at large. Fourth, the PAIS was not developed for patients with other serious illnesses, or those from cultural backgrounds where prognostic disclosure is not desired. Future work should focus on adapting the PAIS for the needs of other populations, including those with other serious illnesses and various cultural and ethnic backgrounds.
In conclusion, we developed the PAIS, using a novel conceptual framework to capture the complexity of prognostic awareness and its psychological and behavioral impact in patients with advanced cancer, and revised it based on cognitive interviews with this population to ensure its face and content validity. We are currently conducting a study to validate further the PAIS by testing its psychometric properties, including reliability and construct validity. After validation in future studies, the PAIS has the potential to enhance both research inquiries and the future care of patients with advanced cancer. Once validated, future work should examine the PAIS domains longitudinally in patients with cancer, and develop a more comprehensive understanding of how these domains impact patient-reported quality of life, psychological distress, medical decision making, and end-of-life care delivery. Future work should also examine how PAIS can be utilized clinically to risk stratify patients to provide tailored supportive care interventions.
The PAIS will have extensive applications for oncology populations, furthering research and clinical work to promote better care and quality of life for patients with advanced cancer struggling with challenging medical and life decisions.
Footnotes
Acknowledgment
This work was presented in abstract format at the Society of Behavioral Medicine Conference in 2020. The meeting was cancelled due to the COVID-19 pandemic.
Authors' Contributions
All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data. All were involved in drafting the article or revising it critically for important intellectual content. All provided final approval of the article and agree to be accountable for all aspects of the work.
Funding Information
Dr. El-Jawahri is a Scholar in Clinical Research for the Leukemia and Lymphoma Society.
Author Disclosure Statement
No competing financial interests exist.
Supplementary Material
Supplementary Appendix SA1
Supplementary Appendix SA2
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
