Predictors of Six-Month Mortality among Nursing Home Residents: Diagnoses May Be More Predictive than Functional Disability

Abstract

Objective:

Loss of daily living functions can be a marker for end of life and possible hospice eligibility. Unfortunately, data on patient's functional abilities is not available in all settings. In this study we compare predictive accuracy of two indices designed to predict 6-month mortality among nursing home residents. One is based on traditional measures of functional deterioration and the other on patients' diagnoses and demography.

Methods:

We created the Hospice ELigibility Prediction (HELP) Index by examining mortality of 140,699 Veterans Administration (VA) nursing home residents. For these nursing home residents, the available data on history of hospital admissions were divided into training (112,897 cases) and validation (27,832 cases) sets. The training data were used to estimate the parameters of the HELP Index based on (1) diagnoses, (2) age on admission, and (3) number of diagnoses at admission. The validation data were used to assess the accuracy of predictions of the HELP Index. The cross-validated accuracy of the HELP Index was compared with the Barthel Index (BI) of functional ability obtained from 296,052 VA nursing home residents. A receiver operating characteristic curve was used to examine sensitivity and specificity of the predicted odds of mortality.

Results:

The area under the curve (AUC) for the HELP Index was 0.838. This was significantly (α <0.01) higher than the AUC for the BI of 0.692.

Conclusions:

For nursing home residents, comorbid diagnoses predict 6-month mortality more accurately than functional status. The HELP Index can be used to estimate 6-month mortality from hospital data and can guide prognostic discussions prior to and following nursing home admission.

Introduction

Our aim is to help optimize the use of hospice by nursing home patients. Hospice utilization among nursing home patients is growing. Rates of nursing home hospice use more than doubled between 1999 and 2006 (from 14% to 33%),¹ and there is growing evidence that nursing home patients benefit from hospice programs.^2–5 However, because of long hospice stays among a subset of patients in nursing homes,^6–10 the Centers for Medicare and Medicaid Services (CMS) is scrutinizing hospice in nursing homes. CMS has removed functional disability as a qualifying hospice diagnosis.¹¹ Tools are needed that will improve the identification of nursing home residents with a prognosis of 6 months of life or less.

The decision to recommend hospice use is often made after an event leads to acute hospitalization and before return of the resident to the nursing home. Depending on where the decision is made, different data are available. Typically, in nursing homes functional data are available,^12–16 whereas in hospitals diagnostic history is more readily available. Although functional status is a good prognostic indicator for cancer patients, this is not the case for the vast majority of other patients, such as residents with circulatory problems, dementia,^17,18 pneumonia,¹⁹ or cachexia.²⁰ For example, end-stage heart failure is not always accompanied by progressive loss of functional ability.^21,22 Some heart failure patients have no functional deterioration prior to death and enjoy good to excellent quality of life between exacerbations.^23–26 For these patients, reliance on functional ability would lead to inaccurate predictions. We examine the relative role of functional status and diagnoses in predicting mortality for nursing home patients.

Methods

Study approvals

Approval was obtained from the Bay Pines Veterans Administration (VA) Health Care System Institutional Review Board and Research and Development Committee prior to study commencement. Additionally, approval was acquired to conduct the study in the Veterans Administration Informatics and Computing Infrastructure (VINCI), a secure, password protected, virtual workspace with restricted access. VINCI personnel established a unique project workspace to which only approved personnel had access. Data were acquired from the Corporate Data Warehouse, which includes patient medical records, and the Minimum Data Set (MDS). Permission to use these data was acquired from VINCI and the Department of Veterans Affairs Office of Geriatrics and Extended Care Operations. All data were housed and all data analyses conducted in VINCI. All personnel with access to the data were granted access by VINCI personnel, and trained in security and privacy issues.

Sources of data

Data were obtained from electronic health records of veterans across 126 medical centers through VINCI. These data included all diagnoses for hospital admissions within the VA for nursing home patients within the VA system from 2003 to 2012. In addition, we obtained access to the MDS (version 2.0), a separate database kept for 296,052 residents of VA nursing homes. The merge of the two databases yielded 140,699 unique nursing home residents on which both sets of data were available.

Dependent variable

The dependent variable in the study was all-cause mortality within 6 months after discharge from the last hospital admission. Dates of death were verified from the Veterans Affairs Beneficiary Identification and Record Locator System File. The VA uses not only its own records but also reports of death from the Social Security Administration.

Measurement of Barthel Index

We used data obtained from the MDS to construct the Barthel Index (BI) for nursing home patients seen within the VA. The BI is applied frequently in geriatric populations to assess functional status according to an individual's ability to attend to activities of daily living. The index contains 10 items: feeding, bathing, grooming, dressing, bowels, bladder, toilet use, transfers, mobility, and stairs. Each of these items is rated according to level of independence. The sum of itemized scores produces a total score that ranges from 0 to 100 with lower scores indicating dependence. The BI has high inter-rater reliability (r=0.95) and test-retest reliability (r=0.87); as well as high internal consistency (Cronbach's α >0.80) and concurrent validity.^27,28 To derive the BI, a crosswalk was created including items in the BI and corresponding items in the MDS. Assessment scales were then compared, and the MDS scores were converted to an equivalent BI score. For example, the MDS Item “Eating” was assessed as: 0=Independent; 1=Supervision; 2=Limited Assistance; 3=Extensive Assistance; and 4=Total Dependence. Scores on this item were converted to the corresponding BI item “Feeding,” which is assessed as: 0=Unable; 5=Needs help cutting, spreading butter, etc.; and 10=independent. The MDS score for “Eating” was converted to “Feeding” as follows: MDS=3 or 4, BI=0; MDS=1 or 2, BI=5; and MDS=0, BI=10). A computer algorithm was designed to automatically convert MDS to BI scores, applied to the dataset, and total BI scores calculated. Cases missing any of the BI items were excluded. A total of 1,153,131 BI scores for 296,052 unique nursing home residents was calculated. The most recent BI score was used for prediction of mortality.

Measurement of HELP Index

The Hospice ELigibility Prediction (HELP) Index was calculated using the steps proposed by Alemi and Walter²⁹ and later validated in claims-based measures of severity for childhood diseases³⁰ and in predicting severity of heart failure.³¹ The independent variables used to predict probability of mortality in the next 6-moanths included:

1. Hospital diagnoses: Each unique diagnosis, as indicated by the International Classification of Diseases, 9^th Revision (ICD-9), within a hospital discharge was considered a separate indicator variable. This leads to many independent variables, some of which occur too rarely to obtain a stable estimate of the effect. In these situations, the hierarchical structure of the ICD-9 codes was used to identify a broader category. A typical ICD-9 code has 6 digits. For the purposes of specifying diagnoses for this model, the first 5 digits were used to indicate the diagnosis without classifying it into subgroups within the diagnosis. The first 3 digits were used to indicate a large category of diagnoses. There were 8073 unique 6-digit, 1450 unique 5-digit, and 453 unique 3-digit hospital diagnoses codes in the training cases. Hospital diagnoses in the validation set that did not match to 6-, 5-, or 3-digit codes were not scored and de facto were assumed to be irrelevant in predicting mortality.

2. Age on admission: Patients age on admission was calculated as the difference from year of birth to date of admission.

3. Number of diagnoses in the last hospital admission: The number of diagnoses on admission in the training dataset ranged from patients with 1 diagnosis to patients with more than 14 diagnoses on admission.

The HELP Index was measured on 140,699 unique nursing home residents, for whom there were 365,786 admissions and 3,326,486 inpatient diagnoses.

Prediction of mortality in HELP Index

To predict the probability of mortality of a patient in the next 6 months, or to calculate the score for the HELP Index, we used the naïve Bayes approach. In this method, all variables are assumed to be independent of each other. This assumption is clearly false but serves as a first approximation of the estimate of the odds of mortality. In the naïve Bayes approach, the posterior odds of mortality is calculated as: \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}&{Posterior \ Odds \ of \ Mortality}= \\&\quad{Prior \ Odds} \ {*} \ {\mathop \Pi\limits _i} \ {Likelihood \ Ratio_i}\end{align*} \end{document}

Table 1 shows an example of how Bayes model works. The case in Table 1 involves an 81 year old widow with 10 diagnoses on last admission. Some diagnoses (e.g. chronic airway obstruction not elsewhere classified) do not show in the online Table; these are ignored and scored with a likelihood ratio of 1. Once the likelihood ratio associated with each diagnosis has been looked up in the online table, the product of all likelihood ratios is calculated. In this case, we calculate it as 71.86. The product of the likelihood ratio is multiplied by the prior odds of 0.16 to obtain the posterior odds. The posterior odds can be expressed as a probability. This patient has 0.92 probability of dying in the next 6-months. The main reason for this high estimate has to do with lung cancer, anorexia and cachexia. The kidney disease and advanced age also contributed to the high probability of mortality.

Table 1.

Sample Calculations of Probability of Mortality in 6 Months

Variable ^b	Level	Likelihood ratio ^a
Age	81 years old	1.49
Number of diagnoses	10	1.07
Diagnosis 1	162.3: Malignant neoplasm of upper lobe, bronchus or lung	3.18
Diagnosis 2	414.8: Other specified chronic ischemic heart disease	1.41
Diagnosis 3	441.4: Abdominal aneurysm without mention of rupture	1.04
Diagnosis 4	443.9: Peripheral vascular disease, unspecified	0.96
Diagnosis 5	496.: Chronic airway obstruction, not elsewhere classified	Not found
Diagnosis 6	585.4: Chronic kidney disease, stage IV (severe)	1.45
Diagnosis 7	588.81: Secondary hyperparathyroidism of renal origin	1.03
Diagnosis 8	783.0: Anorexia	2.16
Diagnosis 9	787.01: Nausea with vomiting	1.02
Diagnosis 10	799.4: Cachexia	3.06
	Product of all likelihood ratios^a	71.86
	Prior odds of mortality in next 6 months	0.16
	Posterior odds of mortality in next 6 months	11.50
	Probability of mortality in next 6 months^c	0.92

Available online at http://hdl.handle.net/1902.29/11763 or by contacting corresponding author. Items marked as “Not found” are scored as 1, leading to no change in product of all likelihood ratios.

Variables are calculated on last admission.

Calculated as posterior odds divided by one plus the odds.

The likelihood ratios associated with each of the indicators are provided in a comma-separated file available online at http://hdl.handle.net/1902.29/11763. The file can be used to make calculations for any patients with known medical history and for whom coded ICD-9 data and age are available. The file is accompanied with description of columns, as well as a brief instruction on how to make calculations.

Methods of Analysis

In examining the validity of the HELP index, we split the data into two sets: training and validation sets. We used the training data set of 112,897 cases to calculate the severity scores associated with the diagnoses. The validation data set of 27,832 cases was used to test the accuracy of the predictions. The accuracy of predictions was calculated using the Area Under the Curve (AUC).^47–49 We used the pROC function in R to calculate 95% confidence interval for the AUC from 2,000 stratified bootstrap replicates. The parameters of the Barthel Index were not derived from our training data set. Instead, we used the scoring procedure provided in the literature. For this reason, the combined validation and training data sets were used to calculate the accuracy of Barthel Index.

Results

Table 2 demonstrates similarities between the training and validation cohorts. As expected, the majority of subjects in a VA population were white males. Gender, marital status and race did not differ across the training and validation cases except for minor but statistically significant differences in rate of divorce and rate of missing information for race.

Table 2.

Demographics Nursing Home Patients Examined

Characteristics	Percent of training cohort (n=112,867)	Percent of validation cohort (n=27,832)	Z from normal distribution
Gender
Male	96.01%	96.03%	0.93
Female	3.28%	3.11%	1.43
Unknown	0.71%	0.76%	0.81
Marital status
Married	31.47%	31.37%	0.34
Divorced	30.29%	29.65%	2.09^a
Widowed	13.14%	13.15%	0.02
Never married	15.18%	16.15%	1.12
Race
White	60.42%	60.57%	0.47
Black	14.42%	14.20%	0.94
Native American	0.48%	0.53%	1.09
Asian	0.86%	0.77%	1.42
Missing	5.03%	5.73%	2.75^a

Statistically significant at level of 0.05.

Table 3 shows the top 20 most serious diagnoses observed in the data that repeat in at least 100 cases. The majority of diagnoses found to be highly predictive of 6-month mortality included malignancies such as malignant pleural effusion (LR 11.52) and secondary malignant neoplasms (brain and spinal cord (LR 8.03), kidney (LR 7.96), pleura (LR 7.72), some digestive organs (LR 7.60), spleen (LR 7.60), adrenal gland (LR 7.60), retroperitoneum & peritoneum (LR 6.30), lung (LR 5.77), liver (LR 5.67), pancreas (LR 5.62), mediastinum (LR 5.60), and other urinary organs(LR 5.60) in addition to neoplasm-related pain (LR 6.03). The other highly predictive diagnoses were history of cardiac arrest (LR 9.67), cardiogenic (LR 5.93) and septic shock (LR 5.70).

Table 3.

Top 20 Diagnoses Predictive of 6-Month Mortality

Diagnosis, International Classification of Diseases, 9th revision code	# Dead	# Cases	Z Statistic	Likelihood ratio
Malignant Pleural Effusion, 511.81	114	174	19.45	11.53
Cardiac Arrest, 427.5	537	874	40.13	9.67
Hepatorenal Syndrome, 572.4	62	107	13.00	8.36
Secondary malignant neoplasm of brain and spinal cord, 198.3	851	1494	47.49	8.03
Secondary malignant neoplasm of kidney, 198.0	59	104	12.46	7.96
Secondary malignant neoplasm of pleura, 197.2	159	284	20.23	7.72
Secondary malignant neoplasm of other digestive organs and spleen, 197.8	144	259	19.14	7.60
Secondary malignant neoplasm of adrenal gland, 198.7	154	277	19.80	7.60
Malignant neoplasm of brain, unspecified, 191.9	151	293	18.37	6.45
Secondary malignant neoplasm of retroperitoneum and peritoneum, 197.6	136	267	17.25	6.30
Acute myeloid leukemia without mention of remission, 205.00	71	142	12.26	6.07
Defibrination syndrome, 286.6	89	178	13.73	6.07
Neoplasm related pain (acute) (chronic), 338.3	515	1033	32.94	6.03
Cardiogenic shock, 785.52	86	174	13.35	5.93
Secondary malignant neoplasm of lung, 197.0	868	1781	41.90	5.77
Septic shock, 785.52	820	1693	40.49	5.70
Malignant neoplasm of liver, specified as secondary, 197.7	1074	2223	46.24	5.67
Malignant neoplasm of pancreas, part unspecified, 157.9	139	289	16.56	5.62
Secondary malignant neoplasm of mediastinum, 197.1	48	100	9.71	5.60
Secondary malignant neoplasm of other urinary organs, 198.1	107	223	14.50	5.60

Figure 1 provides a Receiver Operator Curve for predicting 6-month mortality. The area under the curve, the C-statistic, was 0.838 for the HELP index and 0.692 for the Barthel index. The HELP index was significantly (alpha levels less than 0.01) more accurate than the Barthel index.

FIG. 1.

Receiver operator curve for predicting 6-month mortality.

Discussion

The HELP index constructed from the history of patients' diagnoses was more predictive of mortality in 6-months than the Barthel functional ability. This suggests that an index based on patients' diagnoses may be more useful than an index that is solely based on functional ability. It is possible, though not tested in the current paper, that if the HELP index was combined with the Barthel Index, it may achieve even higher accuracy levels.

A recent review of prognostic indices for older adults identified 16 validated indices that rely on a mix of functional abilities and/or diagnostic categories; none rely on individual diagnostic codes.^32–48 The cross-validated AUC reported for these indices ranged from 0.68 to 0.82. The HELP index had a cross-validated AUC of 0.84, which was higher than the cross-validated AUC reported for these 16 indices. In 2010, Levy improved the Porock et al. index and obtained an AUC of 0.84. All indices reported in the literature use a mix of functional ability and groups of diagnoses. The HELP index achieved higher or equivalent level of accuracy exclusively with diagnoses.

For example, consider the Advanced Dementia Prognostic Tool.^49–53 These investigators examined nineteen diagnostic categories: (1) diabetes mellitus, (2) arteriosclerotic heart disease, (3) cardiac dysrhythmias, (4) congestive heart failure, (5) hypertension, (6) peripheral vascular disease, (7) stroke, (8) Alzheimer's disease, (9) seizure disorder, (10) Parkinson's disease, (11) chronic obstructive pulmonary disease, (12) anemia, (13) cancer, (14) renal failure, (15) pneumonia or respiratory tract infection, (16) urinary tract infection, (17) other infections, (18) hip fracture prior, and (19) other (nonhip) fractures. The developers of this index found that in predicting mortality, none of these 19 diagnostic categories were as predictive as patients' functional abilities. They therefore abandoned the use of these broad diagnostic categories. In contrast, we relied on specific diagnoses (e.g., not all cancers but specific types of cancers). We found that relying on specific diagnoses leads to an index that is more predictive of mortality than functional disabilities.

There are several theoretical reasons why a model that predicts prognosis from diagnoses may be more accurate than one that is solely based on functional ability. As reviewed earlier, except for major cancers, many patients die without significant functional deterioration. For these patients, an index of functional ability will not provide correct guidance. In addition, many patients have temporary problems with function after illness or a major surgery. These problems often resolve, and thus using these temporary dysfunctions could be misleading. For hospitalized patients, diagnoses are readily available and may be used to conduct an initial screening of patients at elevated risk of dying within the next 6 months.

This study was done using data from nursing home patients within the VA. Veterans who use nursing home services represent a unique population. They differ in significant ways from civilian nursing home residents. For example, veterans are mostly male with multiple comorbidities. The index developed here may not extend to other populations with more women or with fewer comorbid conditions. Research is needed to examine the performance of the index developed here with independent datasets. Although this analysis illustrates that mortality of nursing home residents may be predictable from hospital records, use of this tool requires access to hospital data, which may be difficult in a system that is not integrated. The scoring of the HELP Index is time-consuming. One has to look up the likelihood ratios and find the corresponding ratio among thousands of reported items followed by a series of multiplications. All of this takes time and increases the possibility of error. We encourage automation of the HELP Index in electronic health records such that prognostic information can be generated automatically for clinicians and their patients.

Patients may use different hospitals and a merged dataset may not be available. However, in recent years, the use of electronic health records has made it possible to merge data from various providers and insurers. If these data are available, medical encounters may be more informative than functional ability in predicting patients' prognosis.

Although the HELP Index represents an improvement over current Medicare eligibility guidelines, an index with an AUC of 84% will be in error for 1 in 6 patients. A comprehensive approach may include the use of the HELP Index as an initial screen. For example, one can imagine a two-stage hospice eligibility review. At discharge from the hospital a screen could be performed electronically to determine if patient diagnoses suggest a prognosis of 6 months or less. For patients at risk, additional clinical review of functional ability could be performed to add to the prognostic information and generate appropriate palliative care consultation. Future research needs to explore the performance of combinations of diagnoses and functional ability in predicting mortality among nursing home residents.

Conclusions

Patients' diagnoses may be more accurate than functional disability in predicting 6-month mortality of patients. The HELP index represents a tool to assist clinicians and policy makers in identification of patients who have a prognosis of 6 months or less and who may be appropriate for hospice enrollment based on prognosis.

Footnotes

Acknowledgments

This project was funded by appropriation #3620160 from the VA Office of Geriatrics and Extended Care. We acknowledge the valuable guidance we received from Thomas Edes, MD, Chief of Geriatrics and Extended Care Operations and the National Medical Foster Home coordinators, Dan Goedken and Aida Fonseca. The research team also included James A. Blankenship, Lisa Argyros, Phan Giang, PhD, Willy Liao, MD, Etienne Pracht, PhD, and Bruce Citron, PhD. The views expressed in this paper do not represent the views of the VA.

Author Disclosure Statement

No competing financial interests exist.

References

Levy

, Morris

, Kramer

: Improving end-of-life outcomes in nursing homes by targeting residents at high-risk of mortality for palliative care: Program description and evaluation. J Palliat Med, 2008; 11:217–225.

Wolsfeld

, Hendricks

: VA hospice and palliative care: Preliminary identification of indicators to identify veterans at substantial risk for specialized end-of-life care. RRP-06-323. 2007. Boston: Health Care Financing & Economics (HCFE), VA Boston Healthcare System, Research & Development.

Baer

, Hanson

: Families' perception of the added value of hospice in the nursing home. J Am Geriatr Soc, 2000; 48:879–882.

Munn

, Dobbs

, Meier

, et al.: The end-of-life experience in long-term care: Five themes identified from focus groups with residents, family members, and staff. Gerontologist, 2008; 48:485–494.

Wetle

, Teno

, Shield

, et al.: End of life in nursing homes: Experiences and policy recommendations. Washington, D.C.: AARP Public Policy Institute, 2004.

Luchins

, Hanrahan

, Murphy

, et al.: Criteria for enrolling dementia patients in hospice. J Am Geriatr Soc, 1997; 45:1054–1059.

Hanrahan

, Raymond

, McGowan

, et al.: Criteria for enrolling dementia patients in hospice: A replication. Am J Hosp Palliat Care, 1999; 16:395–400.

Schonwetter

, Han

, Small

, et al.: Predictors of six-month survival among patients with dementia: An evaluation of hospice Medicare guidelines. Am J Hosp Palliat Care, 2003; 20:105–113.

MEDPAC report June 2013 Report to Congress, pp. 135–136. www.medpac.gov/documents/Jun13_EntireReport.pdf (Last accessed March 12, 2014).

10.

Gozalo

, Miller

, Intrator

, et al.: Hospice effect on government expenditures among nursing home residents. Health Serv Res, 2008; 43:134–153.

11.

Federal Register. August 7, 2013. Volume 78, No. 152, p. 48240. Department of Health and Human Services Centers for Medicare and Medicaid Services.

12.

Flacker

, Kiely

: Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc, 2003; 51:213–221.

13.

van der Steen

, Mitchell

, Frijters

, et al.: Prediction of 6-month mortality in nursing home residents with advanced dementia: Validity of a risk score. J Am Med Dir Assoc, 2007; 8:464–468.

14.

Hirdes

, Frijters

, Teare

: The MDS-CHESS Scale: A new measure to predict mortality in institutionalized older people. J Am Geriatr Soc, 2003; 51:96–100.

15.

Porock

, Oliver

, Zweig

, et al.: Predicting death in the nursing home: Development and validation of the 6-month Minimum Data Set Mortality Risk Index. J Gerontol, 2005; 60A:491–498.

16.

Flacker

, Kiely

: Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc, 2003; 51:213–221.

17.

Mitchell

, Kiely

, Hamel

, et al.: Estimating prognosis for nursing home residents with advanced dementia. JAMA, 2004; 291:2734–2740.

18.

Volicer

, Hurley

, Fabiszewski

, et al.: Predicting short-term survival for patients with advanced Alzheimer's disease. J Am Geriatr Soc, 1993; 41:535–540.

19.

van der Steen

, Ooms

, van der Wal

, et al.: Withholding or starting antibiotic treatment in patients with dementia and pneumonia: Prediction of mortality with physicians' judgment of illness severity and with specific prognostic models. Med Decis Making, 2005; 25:210–221.

20.

Briggs

: Who needs a hospice?. Med J Aust, 1992; 156:417–420.

21.

Approaching Death: Improving Care at the End of Life. Field

, Cassel

, (eds). National Academy of Medicine, Washington, D.C., 1997.

22.

Gott

, Barnes

, Parker

, et al.: Dying trajectories in heart failure. Palliat Med, 2007; 21:95–99.

23.

McCarthy

, Phillips

, Zhong

, et al.: Dying with cancer: Patients' function, symptoms, and care preferences as death approaches. J Am Geriatr Soc, 2000; 48(5 Suppl):S110–S121.

24.

Levenson

, McCarthy

, Lynn

, et al.: The last six months of life for patients with congestive heart failure. J Am Geriatr Soc, 2000; 48(5 Suppl):S101–S119.

25.

Kheirbek

, Alemi

, Citron

, et al.: Trajectory of illness for patients with congestive heart failure. J Palliat Med, 2013; 16:478–484.

26.

Skalska

, Wizner

, Więcek

, et al.: Reduced functionality in everyday activities of patients with self-reported heart failure hospitalization - Population-based study results. Int J Cardiol, 2014; 176:423–429.

27.

Cohen

, Marino

: The tools of disability outcomes research functional status measures. Arch Phys Med Rehabil, 2000; 81(Suppl 2):S21–S29.

28.

Hobart

, Lamping

, Freeman

, Langdon , et al.: Evidence-based measurement: Which disability scale for neurologic rehabilitation?. Neurology, 2001; 57:639–644.

29.

Alemi

, Walters

: A mathematical theory for identifying and measuring severity of episodes of care. Qual Manag Health Care, 2006; 15:72–82.

30.

Alemi

, Uriyo

: Accuracy of claims-based measures of severity of childhood illnesses. Health Outcomes Res Med, 2012; 2:e71–e78.

31.

Kheirbek

, Alemi

, Fletcher

: Predicting mortality of heart failure patients: Comorbidities matter. District of Columbia Veterans Affairs Medical Center, 2014. (Available from corresponding author.)

32.

Yourman

, Lee

, Schonberg

, et al.: Prognostic indices for older adults: A systematic review. JAMA, 2012; 307:182–192.

33.

Gagne

, Glynn

, Avorn

, et al.: A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol, 2011; 64:749–759.

34.

Walter

, Brand

, Counsell

, et al.: Development and validation of a prognostic index for 1-year mortality in older adults after hospitalization. JAMA, 2001; 285:2987–2994.

35.

Teno

, Harrell

Jr , Knaus

, et al.: Prediction of survival for older hospitalized patients: the HELP survival model. Hospitalized Elderly Longitudinal Project. J Am Geriatr Soc, 2000; 48(5 Suppl):S16–S24.

36.

Carey

, Covinsky

, Lui

, et al.: Prediction of mortality in community-living frail elderly people with long-term care needs. J Am Geriatr Soc, 2008; 56:68–75.

37.

Carey

, Walter

, Lindquist

, et al.: Development and validation of a functional morbidity index to predict mortality in community-dwelling elders. J Gen Intern Med, 2004; 19:1027–1033.

38.

Mazzaglia

, Roti

, Corsini

, et al.: Screening of older community-dwelling people at risk for death and hospitalization: The Assistenza Socio-Sanitaria in Italia project. J Am Geriatr Soc, 2007; 55:1955–1960.

39.

Lee

, Lindquist

, Segal

, et al.: Development and validation of a prognostic index for 4-year mortality in older adults. JAMA, 2006; 295:801–808.

40.

Schonberg

, Davis

, McCarthy

, et al.: Index to predict 5-year mortality of community-dwelling adults aged 65 and older using data from the National Health Interview Survey. J Gen Intern Med, 2009; 24:1115–1122.

41.

Porock

, Oliver

, Zweig

, et al.: Predicting death in the nursing home: development and validation of the 6-month Minimum Data Set Mortality Risk Index. J Gerontol A Biol Sci Med Sci, 2005; 60:491–498.

42.

Flacker

, Kiely

: A practical approach to identifying mortality-related factors in established long-term care residents. J Am Geriatr Soc, 1998; 46:1012–1015.

43.

Di Bari

, Balzi

, Roberts

, et al.: Prognostic stratification of older persons based on simple administrative data: Development and validation of the “Silver Code,” to be used in emergency department triage. J Gerontol A Biol Sci Med Sci, 2010; 65:159–164.

44.

Fischer

, Gozansky

, Sauaia

, et al.: A practical tool to identify patients who may benefit from a palliative approach: The CARING criteria. J Pain Symptom Manage, 2006; 31:285–292.

45.

Inouye

, Bogardus

Jr , Vitagliano

, et al.: Burden of illness score for elderly persons: risk adjustment incorporating the cumulative impact of diseases, physiologic abnormalities, and functional impairments. Med Care, 2003; 41:70–83.

46.

Pilotto

, Ferrucci

, Franceschi

, et al.: Development and validation of a multidimensional prognostic index for one-year mortality from comprehensive geriatric assessment in hospitalized older patients. Rejuvenation Res, 2008; 11:151–161.

47.

Levine

, Sachs

, Jin

, et al.: A prognostic model for 1-year mortality in older adults after hospital discharge. Am J Med, 2007; 120:455–460.

48.

Drame

, Novella

, Lang

, et al.: Derivation and validation of a mortality-risk index from a cohort of frail elderly patients hospitalised in medical wards via emergencies: The SAFES study. Eur J Epidemiol, 2008; 23:783–791.

49.

Mitchell

, Kiely

, Hamel

, et al.: Estimating prognosis for nursing home residents with advanced dementia. JAMA, 2004; 291:2734–2740.

50.

Volicer

, Hurley

, Fabiszewski

, et al.: Predicting short-term survival for patients with advanced Alzheimer's disease. J Am Geriatr Soc, 1993; 41:535–540.

51.

Marsh

, Prochoda

, Pritchett

, et al.: Predicting hospice appropriateness for patients with dementia of the Alzheimer's type. Appl Nurs Res, 2000; 13:187–196.

52.

Schonwetter

, Han

, Small

, et al.: Predictors of six-month survival among patients with dementia: An evaluation of hospice Medicare guidelines. Am J Hosp Palliat Care, 2003; 20:105–113.

53.

Mitchell

, Miller

, Teno

, et al.: The advanced dementia prognostic tool: A risk score to estimate survival in nursing home residents with advanced dementia. J Pain Symptom Manage, 2010; 40: 639–651.