Content Validation of Advanced Illness Criteria of a Palliative Care Screening Tool

Introduction

As life expectancy increases, the adult population will live longer with chronic illnesses. The specialty of palliative care can help align medical care to patients' and families' personal goals and values. In addition, integrational palliative care has shown to decrease overall health resource utilization.^1,2

One of the challenges faced by healthcare professionals is early identification of patients and families who may benefit from palliative care services. ³ Timely identification of patients has been evidenced to improve patient-reported outcomes, including symptom burden and quality of life and reduce costs and inpatient utilization of services.^4–8 Screening tools exist; however, their usefulness across settings has not been investigated, nor has a standardized process been identified for recognizing appropriate patients with advanced illness outside of the oncology population. ¹ In addition, many of the existing palliative care screening assessments do not address all criteria recommended by the Center to Advance Palliative Care (CAPC), and have not been evaluated or validated. ²

Despite multiple tools designed to address specific dimensions of disease and health, including functional status ⁹ pain and symptoms, ¹⁰ emotional and spiritual status ¹¹ such as NEST13+, ¹² there is a paucity of evidence illustrating the tools' clinical impact on early initiation of palliative referrals among the medicine and surgery inpatient population. ¹ Furthermore, the lack of clarity regarding the definition of advanced illness has contributed toward the challenges associated with identifying patients. The literature indicates advantages to early palliative care involvement for seriously ill patients and their families by improving patient experience, enhancing the health of populations, and reducing the per capita cost of care (otherwise known as fulfillment of the Institute of Healthcare Improvement's Triple Aim).^13,14 Evidence supports fulfilling the Triple Aim of improving patient family satisfaction, providing higher quality of care at a lower cost, including decreasing hospital length of stay and readmission rates, reducing intensive care unit utilization in the final months of life, and overall health resource utilization. ¹ Although symptom assessment and health outcomes related to palliative care have been reported, there is a dearth of research focusing on provider understanding of advanced illness. ¹⁵ The aim of this study was to determine whether an existing palliative care screening tool (PCST) developed based on the needs of the oncology population is appropriate to evaluate a broader range of patients by measuring content validity. In addition, the study was designed to determine the existence of a potential correlation between the PCST and the Karnofsky Performance Status Scale (KPS), and address provider misunderstanding of advanced illness through an online survey.

Methods

The study was conducted at Western Connecticut Health Network (WCHN), a three hospital network: Danbury Hospital, New Milford Hospital (85 beds), and Norwalk Hospital (328 beds). Danbury Hospital, a 371-bed teaching hospital in Danbury Connecticut, established an interdisciplinary palliative care consult service in 2004, which reaches ∼1000 inpatients per year.

In 2016, the Danbury Hospital Palliative Care Team (PCT) implemented a screening tool into the electronic medical record (EMR) to decrease the time between admission and initiation of a palliative referral. The screening tool is incorporated into the admission assessment for all medicine and surgery patients and identifies patients appropriate for a referral. This tool is modeled after that created by the Palliative Care Service of Hospice of the Bluegrass in Lexington, Kentucky, which has been validated within the oncology population. ² This validated tool is one of the options recommended by the CAPC and identifies eligible patients across multiple diagnoses, as opposed to solely cancer. As other tools are extensive and time consuming, the tool designed by the Palliative Care Service of Hospice of Bluegrass is a one-page quick and easy assessment. The Danbury Hospital PCT adopted the screening tool to address a broader population, specifically all medicine and surgery patients (Supplementary Fig. S1; Supplementary Data available online at www.libertpub.com/jpm).

The screening tool employs a numeric scoring system by identifying the patients' advanced illness, concomitant disease processes, functional status, and other factors that may indicate poor prognosis and increased healthcare utilization. An overall score greater than or equal to “8” will result in an automatic palliative referral. If a patient receives a score greater than or equal to “6,” the PCT is alerted to review the case and determine eligibility. By decreasing the time to referral, the PCT is able to initiate goals of care and enhance communication across the healthcare continuum in a prompt manner.

The Eastern Cooperative Oncology Group (ECOG) Performance Status scale is incorporated into the screening tool to evaluate functional status. Although the KPS and ECOG are functional assessments traditionally used among the oncology population, there is limited evidence regarding their interchangeability among different populations. ¹⁶

An online survey (Supplementary Fig. S2) was created by the investigators and distributed to healthcare practitioners (physicians and nurses) in March 2017 within WCHN to address the understanding of advanced illness. The survey was developed based on informal feedback from nurses and staff about what constitutes as advanced illness in specific categories, representing sections 1 and 2 of the PCST. Practitioners completing the tool lacked clarity on advanced illness, thus our presurvey intention was to modify sections 1 and 2 of the screening tool by adding necessary detail. Feedback on five categories of basic disease processes as major criterion for determination for a palliative care referral was collected. The basic disease processes evaluated included cancer (metastatic/recurrent), advanced chronic obstructive pulmonary disease, stroke with decreased functional status by at least 50% (requiring assistance with activities of daily living [ADLs]), end-stage renal disease on dialysis, advanced cardiac disease, and several other life-limiting illnesses.

A convenience sample of healthcare practitioners were recruited from the following fields or specialties: cardiology, primary care, oncology, pulmonology, nephrology, emergency medicine, and internal medicine. The responses to each question were dichotomous, with the answer option of “yes” or “no.” Participants were asked whether or not specific criterion of a major domain merits a palliative care referral. Content validity was quantified by calculating the content validity ratio (CVR) to identify essential items for a palliative referral. The formula is CVR = (Ne-N/2)/(N/2), in which N is the total number of participants and Ne represents the total number of participants indicating an item as essential. The CVR score for an item can range from −1 to +1; positive values indicate that at least half of the respondents identified the item as essential.^2,17

A retrospective analysis was conducted on a subsample of all patients admitted to Danbury Hospital between June and December 2015 to determine the existence of a potential correlation between the PCST scores with their respective KPS score. We used Spearman's rank correlation and Pearson's correlation to assess the interchangeability between the PCST and the KPS among medicine and surgery patients. Furthermore, we translated the KPS score into the ECOG grade and compared the translated grade with the ECOG score to determine the level of interchangeability. The association was tested using Pearson's chi-square test with Monte Carlo simulation. This study was approved by the Biomedical Research Alliance of New York, Institutional Review Board.

Results

The online survey was distributed to 248 healthcare providers and was completed by 120 (response rate of 48%), including 93 MDs and 27 nurses. The item CVR ranged from −0.58 to 0.88, with an average CVR of 0.29. For subjects with cancer, uncontrolled symptoms, new metastatic/recurrent, and metastatic solid organ yielded positive CVRs (Fig. 1). A total of 80.8% (n = 97) of all respondents indicated that uncontrolled pain and/or symptoms are an essential criterion for a palliative referral, producing the highest CVR (0.62) specific to cancer.

OPEN IN VIEWER

FIG. 1.

Content validity ratios for criterion rated as essential. *Represents dependence. ADL, activities of daily living; LVEF, left ventricular ejection fraction; FTT, failure to thrive.

Advanced lung disease was acknowledged to be appropriate for a palliative care referral if the patient was completely or partially dependent for his or her ADLs or was BiPAP and/or oxygen dependent (0.77; 0.08; 0.38; 0.23). Stroke patients completely dependent for their ADLs and with dysphagia (as a result of the stroke) were identified as essential. Patients with advanced cardiac disease were identified as eligible for referral if they were completely dependent for their ADLs, oxygen dependent or had a left ventricular ejection fraction of <25%. Other conditions, such as advanced liver disease, advanced dementia, human immunodeficiency virus (HIV), malnutrition, Parkinson's disease, and/or major trauma in an elderly patient were all identified as essential items.

The retrospective analysis was conducted among 293 patients. A significant negative correlation was identified between the PCST and the KPS scores, with Pearson's correlation coefficient being −0.660 (p = 4.3*10⁻³⁸), illustrating a linear relationship. The Spearman's rank correlation coefficient is 0.704 (p = 4.314*10⁻⁴⁵), producing a monotone relationship. ECOG and KPS scores within the medicine and surgery patient population are significantly correlated (p = 9.999*10⁻⁰⁵) (Fig. 2).

OPEN IN VIEWER

FIG. 2.

Correlation between Karnofsky and ECOG. Circle sizes are proportional to the frequency of observations. ECOG, Eastern Cooperative Oncology Group.

Discussion

As health systems transition into an increasingly value-based environment, the need to incorporate palliative care early in the disease trajectory has become a quality and financial imperative. ¹⁸ There is a paucity of validated PCSTs that help identify appropriate patients for a timely referral. Before screening tool implementation, palliative referrals were solely dependent on the clinical judgment of healthcare providers. Although the need for further education on palliative care is acknowledged among clinicians, education alone is unlikely to significantly influence practice patterns. ¹⁹

Referral criteria have been developed ²⁰ ; however, validated screening tools specific to the medicine and surgery population do not exist. Advanced disease is a clear indication for a referral, but there is lack of comprehensibility in defining what constitutes “advanced disease” among providers. The diagnoses addressed in the survey are all relevant to palliative care according to recent literature.^21,22 Informal feedback received from nurses and other health providers claimed that the screening tool needed additional detail, as there was a difference of opinion on what constitutes advanced illness. For example, the diagnosis of cardiac disease alone may not be considered an advanced disease; however, cardiac disease concurrent with uncontrolled symptoms at rest and limited functional status would qualify for a referral. This led us to alter the PCST to improve the referral process.

Based on the feedback, CVR, and survey responses, changes were made to the following categories: basic disease conditions, comorbid conditions, functional status, and resource utilization (Supplementary Fig. S1). Cross-sectional feedback collected from healthcare practitioners validated definitions of advanced illness in the subsections 1 and 2 of the PCST. The whole-person assessment is a valuable component of the PCST and is useful in determining prognosis. Furthermore, it helps clinicians initiate the appropriate conversations with patients and families, ensuring alignment with their goals and values.

Our results, in concordance with existing literature, suggest that functional assessment represents an important factor in defining advanced disease and serves as a prognostic indicator. ²¹ The PCST is appropriate to utilize among a broader patient population. A majority of the positive CVRs stemmed from domains coinciding with full or partial ADL dependence. Other criterion identified as essential varied depending on the illness domain. A significant correlation was identified between the PCST scores and KPS scores, suggesting interchangeability across the medicine and surgery population. Our findings propose that using the PCST alone provides an adequate whole patient screen for appropriateness of palliative care.

Our report has several limitations, including a small sample size in the retrospective analysis and only a 48% response rate for the online survey. Despite these limitations, there are various opportunities for further research involving the efficacy of incorporating mandatory functional assessments as prognostic indicators for advanced illness. The PCST is applicable to patients diagnosed with all serious illnesses. We recommend that hospital and community palliative care programs consider embedding PCSTs that include functional status assessments. Incorporation of PCSTs into the EMR system will likely lead to timely palliative care referrals. Effective screening across the continuum of care is an essential step for effective population health management.