Identifying Older Patients at High Risk for Emergency Department Visits and Hospitalization

Abstract

Hospitalizations are costly, potentially hazardous for older patients, and sometimes preventable. With Medicare's implementation of hospital penalties for 30-day readmissions on certain index conditions, health care organizations have prioritized addressing those issues that lead to avoidable hospitalizations. Little is known about the utility and feasibility of using standardized tools to identify adults at risk for hospitalizations in primary care. In this study, the goal was to determine, from a sample of 60 adults aged 65 and older, whether the Probability of Repeat Admission (PRA), the Vulnerable Elders Survey (VES-13), or a provider estimate of likelihood of hospitalization could identify patients at high risk for emergency department (ED) visits or hospitalization at 6 and 12 months, while being feasible to administer in a primary care setting. PRA, VES-13, and provider estimate were administered in an outpatient practice. Number of ED visits and hospitalizations at 6 and 12 months were assessed through follow-up phone calls and chart review. PRA and provider estimate were not significant predictors of hospitalizations at 6 months (PRA odds ratio [OR] 1.95; P = 0.39; physician estimate OR 4.33, P = 0.08), but were at 12 months (PRA OR 6.00; P < 0.001; physician estimate OR 2.3; P < 0.05). Additionally, a hospitalization during the prior year was not a significant predictor of hospitalization at 6 months (OR 2.97; P = 0.15) but was at 12 months (OR 3.89, P < 0.05). No tool was a significant predictor of ED visits at either time. PRA and the physician estimate were easy to administer and feasible to implement in a primary care setting.

Introduction

Hospitalizations and subsequent rehospitalizations are costly, potentially hazardous for older patients, and preventable in some cases. With Medicare's implementation of hospital penalties for 30-day readmissions, health care organizations have prioritized the need to better understand and address those issues that lead to avoidable emergency department (ED) visits and hospitalizations, especially for vulnerable older adults.^1,2 Ideally, standardized tools could identify patients at high risk for hospitalization or readmission in order to enable health care teams to focus evidence-based strategies and scarce resources for those at highest risk.

Multiple measures and risk stratification methods have been developed and tested to identify patients who are at high risk for hospitalization or readmission. Use of these tools in health care organizations has been increasing, with the aim of reducing readmissions and costs of care. Overall, most predictive measures of hospitalization have demonstrated poor predictive ability and many models are poorly suited for use in primary care, as they rely on claims data or utilize complex administrative databases.^3

–7 Although some electronic medical record-based measures are being developed and tested, these generally still fail to capture patients' functional status and social factors, which are important predictors of admissions.⁸

Measures administered in primary care settings could allow for measurement of functional status, social factors,⁹ and other knowledge from the primary care provider's ongoing relationship with a patient. Primary care physicians also perceive many ambulatory care-sensitive hospitalizations to be preventable through primary care actions such as monitoring of high-risk patients.² However, little is known about the feasibility and utility of incorporating predictive tools into a primary care practice. The aim of this study was to assess the feasibility and performance of 3 different means of predicting ED visits and hospitalizations in older community-dwelling adults attending a primary care practice: the Probability of Repeat Admission (PRA), the Vulnerable Elders Survey (VES-13), and a physician estimate of likelihood of hospitalization.

Methods

This prospective, 12-month study compared the performance of 3 methods in predicting ED visits and hospitalizations in a sample of older adult patients in an urban primary care practice. To characterize the sample, these variables were used in bivariate associations to examine individual predictors of ED use and hospitalizations. The PRA tool has been widely validated as a self-administered tool and has demonstrated good predictive ability.^3

–7 However, PRA has not yet been used as a predictive tool in a primary care setting during an office visit. PRA is an 8-item tool that uses age, sex, self-rated health, number of hospitalizations in the prior year, number of medical visits in the prior year, presence of diabetes, history of coronary artery disease (CAD), and presence of a formal caregiver to calculate a score ranging from 0.07–0.80. Depending on the scoring method used, patients can be classified as either low-risk/high-risk or low/medium/high risk. The study team used the low-risk/high-risk classification, with a score of 0.50 or more representing high risk.

In addition, studies suggest that predictive measures that include functional measures may be more effective in determining risk of hospitalizations.^10
–12 It was under this premise that VES-13 was chosen as a predictive tool for hospitalizations. The 13 items in VES-13 include age, self-rated health, 6 items on difficulty completing various physical activities, and 5 items assessing the ability to perform instrumental activities of daily living. A score ≥3 classifies the patient as vulnerable. VES-13 has been shown to predict mortality and functional decline in older adults, but not hospitalization.^8,13 Similar to PRA, VES-13 has not been tested as a predictive tool for hospitalizations in a primary care setting.

Additionally, previous studies have indicated that there is a role for provider selection of patients who are likely to be at high risk for hospitalization, as predictive modeling may not always choose the right patients or patients able or willing to participate in interventions.¹⁴ Therefore, the third and final method was a physician estimate to provide insight into whether physicians are accurate at predicting likelihood of hospitalization for their patients. The physician tool had 4 items that asked: (1) how well the physician knew the patient; (2) the likelihood of the patient being hospitalized in the next 30 days, 6 months, or 1 year; (3) whether the physician would be surprised if the patient died within the next year; and (4) the physician's rank of the top 3 factors contributing most to her/his estimate of the patient's risk of hospitalization.

The study used convenience sampling of patients at a geriatrics practice in Philadelphia, PA. A predetermined sample size of 60 was recruited based on the study's available time frame. During an office visit, patients aged 65 and older were asked by a medical assistant if they would be willing to complete a brief survey. If willing, medical assistants administered PRA and VES-13. The patient's provider at that visit, who was blinded to the results of PRA and VES-13, was asked to complete the physician survey. A chart review also was performed that recorded the patient's age, sex, and race, number and presence of specific chronic medical conditions (CAD, congestive heart failure, chronic obstructive pulmonary disease, diabetes, stroke, arthritis, and dementia), number of medications, and prior ED visits and/or hospitalizations in the past year. Patients were telephoned at 6 months and 1 year later and asked whether they had been to the ED and/or hospital in the last 6 months. Based on patient self-report, as well as review of medical records, a final count of ED visits and hospitalizations was determined.

Descriptive statistics were used to characterize the sample by age, sex, race, chronic conditions, medications, and ED visits and hospitalizations in the prior year. To examine individual predictors of ED use and hospitalizations, bivariate associations were examined between these study variables and the outcomes using r values for continuous to continuous relationships, odds ratios (ORs) for continuous to categorical and categorical to categorical relationships, and t tests for categorical predictors with continuous outcomes.

Data were analyzed to determine the comparative efficacy of the 3 tools in predicting hospitalization. Efficacy was assessed by calculating each tool's sensitivity and specificity and its positive and negative predictive values. Logistic regression models also were estimated, and the C-statistic was calculated to examine how well each tool predicted each of the binary outcomes: whether the patients had an ED visit at 6 and 12 months, and an admission at 6 or 12 months. The C-statistic is an indicator of discrimination and accuracy of a prediction model. Values range from 0.5–1: a C-statistic of 0.5 indicates that the model is no better than chance at predicting an outcome, 0.51–0.69 indicates a poor fit, 0.70–0.79 indicates a good fit, and 0.80–1.0 indicates a strong fit.¹⁵ All analyses were completed using SPSS version 23 (IBM Corp., Armonk, NY). The study was approved by the Thomas Jefferson University Institutional Review Board. Surveys for patients were voluntary and results were aggregated without patient identifiers.

Results

Table 1 shows patient demographics. Twelve of the 60 patients visited the ED in 1 year (20%; range zero to 5 visits). For hospitalizations in 1 year, 20 patients were hospitalized (33%; range zero to 5 hospitalizations). Bivariate analysis revealed no association between any of the variables measured and ED visits; additionally, because VES-13 was not significantly associated with any of the study outcomes, no further analyses were conducted using this tool.

Table 1.

Participant Demographics (N = 60)

Variable	N (%)	Mean (SD)
Age		78.16 (7.80)
Sex
Female	44 (73.3)
Male	16 (26.7)
Race
African American	35 (58.3)
White	24 (40.0)
Asian	1 (1.7)
Self-rated Health		1.93 (.94)
Poor	1 (1.7)
Fair	15 (25)
Good	29 (48.3)
Very Good	9 (15)
Excellent	6 (10)
PRA score		.36 (.13)
Low Risk (PRA <0.50)	48 (80)
High Risk (PRA ≥0.50)	12 (20)
VES score		3.73 (2.71)
Not Vulnerable (VES <3)	39 (65)
Vulnerable (VES ≥3)	21 (35)
Comorbidities at baseline (Y/N and mean number)^*	55 (91.8)	1.78 (1.06)
Medications at baseline (Y/N and mean number)	50 (83.3)	8.57 (4.84)
Hosp in previous year at baseline Y/N and mean number)	29 (48.3)	.73 (.94)

Select conditions included congestive heart failure, coronary artery disease, chronic obstructive pulmonary disease, diabetes, arthritis, dementia, and active cancer. Five patients had no chronic conditions (8.3%), 24 had 1 (40%), 12 had 2 (20%), 17 had 3 (28%), and 2 had 4 (3.3%).

PRA, Probability of Repeat Admission; SD, standard deviation; VES, Vulnerable Elders Survey; Y/N, yes/no.

PRA was not a significant predictor of hospitalizations at 6 months in a logistic regression with high-risk (PRA ≥0.500) and low-risk categories (PRA ≤0.499); however, it was a significant predictor at 1 year. For patients in the high-risk category, the odds of hospitalization in the last year were 6.0 times greater at 1 year (P < .001). Similarly, physician estimate was not a significant predictor of hospitalization at 6 months, with an OR of 4.33 (.84–22.67) at 6 months, but was a significant predictor of hospitalization at 1 year (OR = 2.3(.91–8.9); P < .05) at 1 year (Table 2). Additionally, a history of hospitalization during the prior year was not a significant predictor of hospitalization at 6 months (P = 0.15) but was at 1 year (P < 0.05).

Table 2.

Participant Hospitalizations and Odds Ratios of Predictive Tools

Predictor	Odds of hospitalization at 6 months (95% CIs)	Odds of hospitalization at 1 year (95% CIs)
PRA high-risk score	OR 1.95 (.42–9.04); P = .39	OR 6.00 (1.53–23.53); P < .001
Hospitalization in prior year (Y)	OR 2.97 (.69–12.83); P = .15	OR 3.89 (1.23–12.29); P < .05
Physician estimate of hospitalization (likely)	OR 4.33 (.84–22.67); P = .08	OR 2.3 (.91–8.9); P < .05

CI, confidence interval; OR, odds ratio; PRA, Probability of Repeat Admission; Y, yes.

The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and C-statistic for PRA, hospitalization in the prior year, physician estimate, and physician estimate combined with PRA are presented in Table 3. At 6 months, having a hospitalization in the prior year had the highest PPV; at 12 months, physician estimate had the highest PPV of the tools. Of note, at 12 months when the predictive power of physician estimate and PRA are combined, the sensitivity, specificity, and NPV increase relative to that of the physician estimate alone (Table 3).

Table 3.

Sensitivity, Specificity, Positive Predictive Value, Negative Predictive Value, and C-Statistics for Hospitalizations at 6 Months

Test	Reference standard	Sensitivity (%) [95% CI]	Specificity (%) [95% CI]	Positive predictive value	Negative predictive value	C-statistic
PRA	Patient self-reported hospitalizations	30.0 [6.67–65.25]	82.0 [68.56–91.42]	25.0%	85.4%	.560
Hospitalization in Prior Year	Patient self-reported hospitalizations	100 [69.15–100.0]	80.0 [66.3–89.9]	50.0%	100.0%	.630
Physician Estimate	Patient self-reported hospitalizations	80.0 [44.39–97.48]	50.0 [35.53–664.47]	24.2%	92.6%	.660
PRA plus Physician Estimate	Patient self-reported hospitalizations	80.0 [44.39–97.48]	53.85 [39.47–67.77]	25.0%	93.3%	.650
Sensitivity, Specificity, PPV, NPV, and C-statistics for hospitalizations at 12 months
PRA	Patient self-reported hospitalizations	40.0 [19.17, 63.93]	90.0 [76.32, 97.15]	66.7%	75.0%	0.650
Hospitalization in Prior Year	Patient self-reported hospitalizations	70.00 [45.73, 88.03]	62.50 [45.80, 77.26]	63.6%	81.5%	0.663
Physician Estimate	Patient self-reported hospitalizations	43.75 [26.38, 62.33]	78.57 [59.04, 91.65]	70.0%	55.0%	0.669
PRA plus Physician Estimate	Patient self-reported hospitalizations	58.33 [27.75, 84.68]	84.62 [65.11, 95.55]	63.6%	81.5%	0.650

CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; PRA, Probability of Repeat Admission.

Discussion

Given the particular hazards and costs associated with ED visits and hospitalization in the older population, it is imperative to determine a reasonable method by which primary care providers can identify high-risk patients in the primary care setting and target evidence-based interventions. Delirium, medical errors, adverse drug reactions, and falls are common but devastating complications of hospitalization in older adults. As a result, unnecessary hospitalizations should be reduced whenever possible.¹⁶ However, particularly in the primary care setting, there is a lack of reliable, effective tools to differentiate those patients who are at the highest risk of hospitalization,¹⁴ limiting opportunities to direct interventions and resources.

Although many tools have been developed to predict ED visits and hospitalization in older adults, they frequently use administrative data or claims data that would be neither available to a primary care provider, nor feasible to collect in real time in the primary care setting. Despite the need for a simple tool to administer in a primary care setting, the tools currently available have limited predictive ability and often fail to capture functional status and social factors.¹⁷ Previous studies have identified a number of factors that are associated with hospitalizations, including prior hospitalization and health care utilization, specific medical conditions and counts of chronic conditions, severity of illness, social factors, and functional status.^10
–12,18

This study evaluated the performance of PRA in predicting ED visits and hospitalizations; it is widely validated and modestly effective.^3

–7,18 This study also investigated the performance of VES-13, which has been shown to be effective in predicting mortality and functional decline in older adults.⁸ Lastly, this study also evaluated a provider estimate, which some studies have suggested can be as accurate as predictive modeling and may have the added benefit of identifying patients who are more receptive to interventions.¹³ The aim was to determine whether, in a primary care setting, PRA, VES-13, or provider estimate could identify patients at high risk for ED visits or hospitalization in the next 6 months and 1 year, and could be administered easily. This study found that PRA was the most effective tool at predicting hospitalization at 12 months. However, having a hospitalization in the prior year, a data point in PRA, in and of itself was considerably effective at predicting hospitalization at 12 months. Physician estimate of likelihood of hospitalization also was useful in predicting hospitalization at 12 months. Of note, combining PRA with physician estimate improved the predictive power of PRA at 12 months. All predictive tools were easy to administer as part of a routine primary care visit.

None of the tools assessed (PRA, VES-13, or physician estimate) were predictors of an ED visit at 6 months or 1 year. Likewise, self-rated health, age, number of medications, number of specific chronic conditions, and VES-13 were not significantly associated with hospitalization in 6 months or 1 year.

Similar to prior studies, PRA demonstrated high specificity but low sensitivity in predicting hospitalization. It performed well in predicting hospital admission in community-dwelling adults who were categorized as high risk, but it was not reliable for excluding risk of hospitalization for those categorized as low risk. VES-13, on the other hand, identified a large percentage of patients as vulnerable and therefore was less selective in identifying patients as high risk. Primary care providers should be aware of the sensitivity and specificity of both tools, and the potential benefit of adding physician prediction to PRA, particularly for 12-month predictions.

This study has some limitations, including its relatively small size related to being an unfunded pilot study. However, despite the small size, this study was able to detect significant odds of hospitalization at 1 year for PRA, hospitalization in the prior year, and physician estimate of hospitalization. The study team also was able to demonstrate the feasibility of using these tools in a primary care practice. The study's location—a geriatrics primary care practice affiliated with a large urban medical center—also may limit generalizability of the findings. In addition, patient self-reported data were occasionally unreliable, as sometimes patients could not remember exactly how many times they had been to the ED or hospitalized. The accompanying chart review of ED visits and hospital admissions helped mitigate this limitation; however, the charts may not capture ED visits and hospital admissions at outside institutions. Additionally, PRA is a copyrighted tool that requires permission to use.

Future studies are needed to further demonstrate, on a larger scale and for different populations, the comparative effectiveness of different tools for use in the primary care setting to predict hospitalization and rehospitalization for older adults.

Footnotes

Acknowledgment

We would like to thank and share our appreciation of Chad Boult for allowing us to use the Probability of Repeat Admission tool in this study.

Author Disclosure Statement

The authors declare that there are no conflicts of interest. The authors received no financial support for this article.

References

Jencks

, Williams

, Coleman

. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med, 2009; 360:1418–1428.

Freund

, Campbell

, Geissler

, et al. Strategies for reducing potentially avoidable hospitalizations for ambulatory care-sensitive conditions. Ann Fam Med, 2013; 11:363–370.

Podrazik

, Whealan

. Acute hospital care for the elderly patient: its impact on clinical and hospital systems of care. Med Clin North Am, 2008; 92:387–406.

Kansagara

, Englander

, Salanitro

, et al. Risk prediction models for hospital readmission: a systematic review. JAMA, 2011; 306:1688–1698.

Allaudeen

, Schnipper

, Orav

, Wachter

, Vidyarthi

. Inability of providers to predict unplanned readmissions. J Gen Intern Med, 2011; 26:771–776.

Wagner

. Predicting the risk of hospital admission in older persons—validation of a brief self-administered questionnaire in three European countries. J Am Geriatr Soc, 2006; 54:1271–1276.

Dutra

, Moriguchi

, Lampert

, Poli-de-Figueiredo

. Predictive validity of a questionnaire to identify older adults at risk for hospitalization. Rev Saude Publica, 2011; 45:106–112.

Calvillo–King

, Arnold

, Eubank

, et al. Impact of social factors on risk of readmission or mortality in pneumonia and heart failure: systematic review. J Gen Intern Med, 2013; 28:269–282.

Kuo

, Goodwin

. Association of hospitalist care with medical utilization after discharge: evidence of cost shift from a cohort study. Ann Intern Med, 2011; 155:152–159.

10.

Wallace

, Hinchey

, Dimitrov

, Bennett

, Fahey

, Smith

. A systematic review of the probability of repeated admission score in community-dwelling adults. J Am Geriatr Soc, 2013; 61:357–364.

11.

Coleman

, Min

, Chomiak

, Kramer

. Posthospital care transitions: patterns, complications and risk identification. Health Serv Res, 2004; 39:1449–1465.

12.

Miller

, Wessert

. Predicting elderly people's risk for nursing home placement, hospitalization, functional impairment, and mortality: a synthesis. Med Care Res Rev, 2000; 57:259–291.

13.

Min

, Yoon

, Mariano

, Wenger

, et al. The vulnerable elders-13 survey predicts 5-year functional decline and mortality in older ambulatory care patients. J Am Geriatr Soc, 2009; 57:2070–2076.

14.

Freund

, Mahler

, Erler

, et al. Identification of patients likely to benefit from care management programs. Am J Manag Care, 2011; 17:345–352.

15.

Hajian-Tilaki

. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med, 2013; 4:627–635.

16.

Dharmarajan

, Hseih

, Lin

, et al. Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA, 2013; 309:355–363.

17.

Wallace

, McDowell

, Bennett

, Fahey

, Smith

. External validation of the vulnerable elder's survey for predicting mortality and emergency admission in older community-dwelling people: a prospective cohort study. BMC Geriatr, 2017; 17:69.

18.

Wallace

, McDowell

, Bennett

, Fahey

, Smith

. Comparison of count-based multimorbidity measures in predicting emergency admission and functional decline in older community-dwelling adults: a prospective cohort study. BMJ Open, 2016; 6:e013089.