Performance of Identification of Seniors at Risk tool to predict unplanned revisits to the emergency department in 6 months was low in predictive accuracy: A single-center study

Abstract

Background:

Elderly people are at risk for mortality, functional decline, reattendance, and hospitalization after an emergency department visit.

Objective:

The aim of this study was to evaluate the performance of Identification of Seniors at Risk tool to predict unplanned readmissions after an emergency department visit.

Methods:

Records of patients aged ⩾65 years, who completed Identification of Seniors at Risk tool as they were being discharged from the emergency department, were analyzed. Patients were called back at 30th, 60th, 90th, 120th, and 180th days after emergency department discharge to assess their readmission to emergency department. Descriptive statistics and receiver operating characteristic curve analysis were performed.

Results:

This was a single-center study conducted with elderly patients with chronic diseases in a tertiary-level hospital within a period of 10 months with the follow-up calls. During the first month, 1792 patients were admitted to emergency department and 333 patients were aged ⩾65 years. Patients who completed Identification of Seniors at Risk tool as they were discharged from the emergency department were 170 out of 333. In 6 months, 71 patients out of 170, 36 men (50.7%) and 35 women (49.3%), were able to complete the follow-up. For predictive unplanned admissions, specificity and sensitivity at 1, 3, and 6 months were 40%, 40%, 38% and 69%, 78%, 67%, respectively, which demonstrates that sensitivity of Identification of Seniors at Risk tool was higher than its specificity. Although representing poor performance, Identification of Seniors at Risk tool was better at 3 months in predicting health risks for the elderly who have visited emergency department.

Conclusion:

Predictive ability of Identification of Seniors at Risk tool at the usual cutoff ⩾2 points to identify elderly at risk for revisiting emergency department for adverse health outcomes is limited. Multicentered studies, with standardized procedures and well-defined patient profile, are needed to improve the predictive ability of Identification of Seniors at Risk tool to screen elderly who require additional support after hospitalization.

Keywords

Geriatrics emergency department hospitalizations function/functional status

Introduction

Emergency department (ED) visits of the geriatric patients have increased worldwide in the recent years. As new treatments have evolved, people live longer with existing chronic diseases and their need for health services increases. Besides aggravations of chronic diseases, geriatric syndromes such as vision impairment, frailty, urinary and fecal incontinences, falls, and unmanaged polypharmacy attenuate the ED visits. In a survey in the United States, it was found that 60.2% of the patients who had visited ED were aged 75 years and older.¹ Studies emphasized the higher use of emergency services by the older adults as well as the higher urgency of ED visits and higher ED admittance by the elderly.² They stay longer in the ED and have repeat ED visits.² After an ED visit, 25% of elderly people who have attended ED is expected to have a return ED visit in 90 days.³ Another finding is that elderly experience adverse health outcomes after ED discharge more than the younger people.²

Poorer physical functioning and mental health, presence of comorbidities, lack of supplemental insurance, and use of ambulance transport for the initial ED were counted among the short- and long-term predictors of revisit to the ED, hospitalization, or death among older patients who are discharged from the ED.^3–6

As a result of increasing admission rates of elderly patients to the EDs, applicable tools are needed to assess the nature of their ED visits.⁷ To identify older persons at risk for mortality, functional decline, reattendances, and institutionalization, the Identification of Seniors at Risk (ISAR) tool has been developed^8,9 and researchers have used it for prediction of risk for ED visits throughout the world.^10–13

ISAR comprises six questions on functional dependence, recent hospitalization, impaired memory, impaired vision, and polypharmacy reported by patients as dichotomous (e.g. yes/no) responses. Patients with a score of 2 or more are considered to be at risk.⁸ It has been shown that ISAR scores have correlated with baseline functionality and decline in functional status at 6 months after the initial ED visit.⁹ Recently, Galvin et al. performed a meta-analysis and review of prospective or retrospective cohort studies conducted in hospital settings with ISAR tool in older adults following the ED discharge. They have reported that in 10 studies, the sensitivity and the specificity of ISAR 30 were estimated as 0.81 and 0.29, respectively; in 5 studies, the sensitivity and the specificity of ISAR 90 were estimated as 0.84 and 0.38, respectively; and in 5 studies, the sensitivity and the specificity of ISAR 180 were estimated as 0.80 and 0.31, respectively.¹⁴ ISAR tool has been validated in Turkish recently.¹⁵ The adaptation of the ISAR tool has been done before, but its predictive ability for revisiting ED has not been studied in Turkish settings yet. Therefore, this study aimed to evaluate the performance of the ISAR tool to predict unplanned readmissions after an ED visit among patients aged 65 years and older in our country’s settings.

Method

Study design and data collection

This was an observational study to investigate the performance of ISAR tool using the records of patients who were aged 65 years and older and have been admitted to the ED of the urban university hospital during 1 month. As there are no referral chains, patients who are emergent may directly apply to the tertiary-level ED. ISAR tool forms, which have been filled out by patients aged 65 years and older as they were being discharged from the ED, were included. Both these ISAR tool forms and the written consent forms were present for the former study, which was approved by the local ethics committee meeting (2010/09). Informed oral consents for calling them by phone on end points of the study have been obtained from all participants who have filled out ISAR tool for the former study.

The exclusion criteria were determined. ISAR forms were not filled out for patients who were unconscious or have applied ED for trauma effects and those who were hospitalized after diagnosis on the same day of the ED admission or who had psychiatric and/or neurologic diseases; so, records of these patients were not included.

The patients who have filled out the ISAR tool were called back by phone interviews at five different follow-ups at 30th, 60th, 90th, 120th, and 180th days post-discharge to assess their readmission to ED. The information on revisits for ED was obtained either from the patient or from the caregiver when the patients could not give exact information on the hospital visits and medications.

ISAR scale

ISAR scale consists of six questions with a yes/no response format on functional impairment which was present before the ED admission, acute functional decline, hospitalization in the last 6 months before the ED visit, visual impairment, memory impairment, and use of more than three medications every day. Each question is scored 1 if the answer is “yes” and 0 if the answer is “no.” ISAR has a total scale range of 0–6. Patients with a score of 2 or more are considered to be at risk.^8,9 The total point ⩾2 kappa value of the scale was found to be 0.88 (p < 0.001) in the validation study.¹⁵

Statistical analysis

The statistical analyses were performed using SPSS version 19.0. Descriptive statistics are given with median, minimum and maximum values for continuous variables, and frequency and percent for categorical variables. Receiver operating characteristic curve (ROC) analysis is used to evaluate the predictive ability of unplanned readmissions of ISAR tool among individuals at end points of 30, 60, 90, and 180 days. Results are given with sensitivity, specificity, and positive and negative predictive values. For all statistical comparisons, p-value less than 0.05 was assumed as statistically significant.

Results

The study period was 10 months with follow-up calls. Records of patients who have filled out the forms in the first month were included in the study and followed for 6 months, 3 months period were for the analysis.

During the first month, 1792 patients who were older than 18 years have been admitted to ED. There were 333 patients aged 65 years and older. Among them, 163 patients (49%) were hospitalized and those patients were not discharged but were admitted to other specialties after ED admission and thus have become ineligible for the study. The remaining 170 eligible patients (51%) completed the ISAR tool as they were discharged from the ED. All the patients were called back at the end points, but some of the patients have moved or did not want to answer, so 99 patients were excluded from the study in 6 months period. In our study, the number of patients who we were able to follow at five different follow-ups for 180 days was 71 (41.8 %) (Table 1; Figure 1).

Table 1.

Emergency department (ED) adult patient admissions in 4 weeks.^a

The compliance rate was 71 out of 170 patients (41.8%) who have completed the ISAR tool.

Figure 1.

Number of patients at initiation and during follow-up.

Patients had a mean age of 75.78 ± 8.0 years, 49.3% (n = 35) were women. The patients who had systemic chronic diseases were 90.1%. Patients had diabetes mellitus, cardiovascular disease, renal disease, and chronic obstructive lung disease. Participants’ baseline characteristics are presented in Table 2.

Table 2.

Demographic and clinical characteristics (n = 71).

		n	%
Gender	Female	35	49.3
Gender	Male	36	50.7
Systemic disease: DM, CVD, RD, COLD	Yes	64	90.1
Systemic disease: DM, CVD, RD, COLD	No	7	9.9
Who does the patient live with	Spouse	39	54.9
	Children	31	43.7
	Alone	1	1.4
Emergency department visit in previous 90 days	Yes	41	57.7
Emergency department visit in previous 90 days	No	30	42.3
Hospitalization in previous 90 days	Yes	26	36.6
Hospitalization in previous 90 days	No	45	63.4

DM: diabetes mellitus; CVD: cardiovascular disease; RD: renal disease; COLD: chronic obstructive lung disease.

Of the 71 patients, the composite end points occurred at 30th, 60th, 90th, 120th, and 180th days. The sensitivity, specificity, and likelihood ratio (LR) of various ISAR cutoff scores for the composite end point at all time-points are reported in Table 3.

Table 3.

Diagnostic values for the screening instrument 30, 60, 90, 120, and 180 days after discharge from ED.

Readmission day	Cutoff score	Area under ROC	Sensitivity (95% CI)	Specificity (95% CI)	LR+	LR−	p-value
ISAR 30	(⩾2/6)	0.552	68.75 (41.4–88.9)	40.00 (27.0–54.1)	1.15	0.78	0.532
ISAR 60	(⩾2/6)	0.557	70.59 (44.1–89.6)	40.74 (27.6–55.0)	1.19	0.72	0.488
ISAR 90	(⩾2/6)	0.673	77.78 (40.1–96.5)	40.32 (28.1–53.5)	1.30	0.55	0.097
ISAR 120	(⩾2/6)	0.588	76.92 (46.2–94.7)	41.38 (28.6–55.1)	1.31	0.56	0.331
ISAR 180	(⩾2/6)	0.532	66.67 (22.7–94.7)	38.46 (26.7–51.4)	1.08	0.87	0.799

ED: emergency department; ROC: receiver operating characteristic curve; ISAR: Identification of Seniors at Risk; LR+: likelihood ratio of a positive test; LR−: likelihood ratio of negative test.

We found the highest specificity at 4 months as 41.38% and the highest sensitivity at 3 months as 77.78%. Area under the curve (AUC) was measured between 0.532 and 0.673 by ROC analysis at 30th, 60th, 90th, 120th, and 180th days. Neither of them was significant (p > 0.05).

In another analysis, we evaluated ISAR at certain days for ED readmissions where we compared those with score ⩾2 (risky) with those who have scored <2 (normal) at each composite end point. We found no significance among the results (p > 0.05; Table 4). At all end points among patients with a normal score, readmissions were fewer, whereas among patients with a risky score, more readmissions were observed.

Table 4.

Comparison of the risky and normal groups according to ISAR tool at end points.

	Readmission	Normal (<2), n (%)	Risky (⩾2), n (%)	p-value
Day 30	Yes	1 (7.1)	15 (26.3)	0.166
Day 30	No	13 (92.9)	42 (73.7)	0.166
Day 60	Yes	2 (14.3)	15 (26.3)	0.493
Day 60	No	12 (85.7)	42 (73.7)	0.493
Day 90	Yes	0 (0)	9 (15.8)	0.189
Day 90	No	14 (100)	48 (84.2)	0.189
Day 120	Yes	0 (0)	13 (22.8)	0.058
Day 120	No	14 (100)	44 (77.2)	0.058
Day 180	Yes	1 (7.1)	5 (8.8)	1.000
Day 180	No	13 (92.9)	52 (91.2)	1.000

Discussion

Comprehensive Geriatric Assessment (CGA) is useful in clinical decision making, but it takes too long to be used in the ED conditions for elderly patients.^16,17 To screen patients aged 65 years and older in the ED, it has been shown that ISAR may predict ED readmissions.³

Specificity and sensitivity of ISAR

ISAR has been found less specific than sensitive in studies.^11,14,17 At the same end points 30th, 90th, and 180th days when compared to other studies in the literature, sensitivity and specificity of ISAR were similar in our study (Table 3). The results of the retrospective study by Graf et al.¹⁷ were higher than that of ours by 92%, 93%, and 92% at the first, third, and sixth months, respectively. Moons et al.¹¹ have found 79% and 79% at the first and third months which were similar to our results. However, we found the specificity higher than Graf et al.’s¹⁷ study in which they have found 22%, 26%, and 29% at the first, third, and sixth months and were closer to Moons et al.’s¹¹ study in which 37% and 41% were found at the first and third months.

Galvin et al.¹⁴ explained in their study that,

“sensitivity refers to the proportion of older adults who experience an adverse outcome and who were correctly classified as at high risk (ISAR ⩾ 2) whereas specificity refers to those who do not experience an adverse outcome and who were correctly classified as at low risk (ISAR < 2).”

Concurrent with the literature, we found the sensitivity of ISAR tool higher than its specificity which can be interpreted as its true positive rate is higher than the true negative rate. Its ability of identifying those who will have ED revisits is higher than its ability of identifying those who will not have revisits to ED. Around 90% of our patients had chronic systemic diseases such as diabetes mellitus, cardiovascular diseases, renal diseases, and chronic obstructive lung diseases (Table 2). Presence of chronic diseases affects the reoccurrence of adverse outcomes after an ED visit.

We have found low rates for ISAR tool which is consistent with other studies. Our study may be an initial study which may be a reference to local researchers who are planning to use ISAR. Galvin et al.¹⁴ concluded that predictive ability of ISAR tool to be used for screening in the ED was unpresuming, whereas sensitivity was ⩾80% for predicting ED return, emergency hospitalization, and mortality at all time-points. The sensitivity rates in our study were similar, and the specificity was lower when compared to the development cohort of the ISAR tool which had a sensitivity of 72% and specificity of 58% for the cutoff value of 2 or more positive answers and follow-up time of 6 months.⁸ Our patients who were hospitalized prior to the study were 36.6% (n = 26), which may have influenced readmissions.

In a study conducted in our country about the appropriateness of visits to the ED, 19% of all patients were above 65 years of age. Among patients who required ED care, 26.6% patients were above 65 years of age and their visits were appropriate for ED. The rest of the geriatric patients could have used the outpatient facilities; but due to the time of the visit to ED, weekends for instance, some ED visits were found appropriate when reviewed retrospectively they have reported.¹⁸ Also, availability of different health services, such as home-care services, various patterns of ED use in different countries, and patient sample characteristics in the settings where it was performed, changes the predictive validity of the ISAR.^19,20

In a relatively recent study, it was reported that unplanned readmission rate at 30 days was 14.7%, and 23.5% at 90 days. ISAR showed low specificity and positive predictive value (⩽33% and ⩽24%, respectively); however, ISAR tool had low-to-moderate sensitivity (54%–69%) and a high negative predictive value (⩾78%).²¹

In the literature, it was reported that the predictive value of ISAR, which was determined using ROC analysis which takes both specificity and sensitivity into account, was found as “fair” to “poor” in studies carried out in Canada, Europe, and Hong Kong.^{7,10,19,21–24} In a review, it is stated that ISAR’s predictive capacity in terms of AUC and sensitivity was better than its specificity by cutoff points of 2 and 3.²⁵ At 90 days, we found AUC as 0.673, which was similar to the results of former studies by Graf et al.¹⁷ and Moons et al.,¹¹ 0.655 and 0.630, respectively. Although representing poor performance, ISAR was better at 90 days in predicting health risks for the elderly who have visited ED (Table 3).

Country and hospital settings may bring up the differences between the studies. Cultural and social factors may play a role. None of our patients live in the elderly homes. All of our patients except one were living with their spouses or children. This aspect may also be a factor influencing the readmissions to ED, as presence of relatives and caregivers may decrease the readmission rate.

Singler et al.²⁰ have reported that ISAR tool identified elderly patients in the ED with an increased risk of unwanted outcomes and have suggested that results were better when ⩾3 points were used for cutoff. Although the predictive ability was found low, it is suggested that the ISAR tool may be utilized as an initial step to identify populations at high risk and then necessary interventions can be made for functional impairment or depression or high health service utilization after the ED visit.⁹

In a review, ISAR has been found as a useful tool for assessing functional decline in elderly patients presenting to an ED, because it is faster and requires no training.²⁵ However, it has been emphasized that in emergency settings, it may be difficult to complete the ISAR tool, especially in frail elderly, thus much effort must be given to use these tools and thus empowering the staff is needed.²⁶ We think that ISAR with six dichotomous questions is easy to utilize in settings with hospital staff and well-defined patients. Also, ISAR may be modified according to the country profile which is suggested in a recent review.²⁷ Health-care professionals may benefit from integrating the ISAR tool in daily practice at settings where there are limited resources.

It is not advised to use the ISAR tool alone at ED for identifying seniors at risk for adverse outcomes.²⁷ In this study, the average rate of compliance was approximately 41.5% in our patients, which is compatible with the literature. In a similar study, compliance rate ranges from 17% to 54%, with an average of 37%.²⁸ The patients who were in compliance with this study may be using the emergency services more than the other patients.

Strengths and limitations

Among the strengths of this study is that it is a new study on ISAR tool in our country settings. Hospital settings and patient characteristics may influence the outcome of the studies. Whether the patients have attended the same hospital or not where the study was carried out was already checked from the hospital records, but the patients may have attended other hospitals in the same city or in another city.

Although the number of participants who could complete the follow-up seems low when we compare it with other studies, the compliance rate is similar and it should be considered that the period of 6 months follow-up is quite long in an elderly population with chronic diseases.

Conclusion

From this study, it can be concluded that, the predictive ability of ISAR is limited at the usual cutoff ⩾2 points to identify elderly patients at risk of revisiting ED for adverse health outcomes following an initial visit to the ED. The sensitivity of ISAR may increase if used to screen at emergency settings the elderly populations who have been assessed comprehensively by family physicians or geriatrists before by assessment tools and defined as home-care patients or patients with more than one comorbidity. Further work is required to improve the predictive ability of ISAR for screening to identify older people who require additional support after ED discharge.

Footnotes

Acknowledgements

The authors would like to thank Jane McCusker and Josée Verdon for allowing to use the ISAR tool.

Authors’ note

The study was presented as an oral presentation in the 4th Home-Based Health Care Congress in Istanbul, Turkey, 23–25 November 2017.

Author contributions

A.S.D.A. and D.K. wrote the manuscript and made the literature research; A.S.D.A., F.O.K., and Ç.B. designed the study; A.S.D.A., F.O.K., and F.A. collected data; and A.S.D.A., Ç.B., and D.K. analyzed and interpreted the data.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Availability of data and materials

The retrospective data record is present.

Informed consent

Written consent forms were present for the former study. For this retrospective observational study, informed oral consents for calling them by phone on end points of the study have been obtained from all participants who have filled out the ISAR tool for the former study.

Ethical approval and human rights

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee. This study has utilized the results of the validation and reliability study of ISAR tool which has been conducted as the patients were being discharged from the hospital, which was approved by the Medical Faculty Ethics Committee (2010/09). The study was conducted in compliance with the Declaration of Helsinki.

ORCID iD

Didem Kafadar

References

Pitts

Niska

, et al. National Hospital Ambulatory Medical Care Survey: 2006 emergency department summary. Natl Health Stat Report 2008; 7: 1–38.

Aminzadeh

Dalziel

WB.

Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med 2002; 39(3): 238–247.

McCusker

Healey

Bellavance

, et al. Predictors of repeat emergency department visits by elders. Acad Emerg Med 1997; 4(6): 581–588.

Friedmann

Jin

Karrison

, et al. Early revisit, hospitalization, or death among older persons discharged from the ED. Am J Emerg Med 2001; 19(2): 125–129.

Chin

Jin

Karrison

, et al. Older patients’ health-related quality of life around an episode of emergency illness. Ann Emerg Med 1999; 34(5): 595–603.

Rubenstein

. The emergency department: a useful site for CGA? J Am Geriatr Soc 1996; 44: 601–602.

Di Bari

Salvi

Roberts

, et al. Prognostic stratification of elderly patients in the emergency department: a comparison between the “Identification of Seniors at Risk” and the “Silver Code.” J Gerontol A Biol Sci Med Sci 2012; 67(5): 544–550.

McCusker

Bellavance

Cardin

, et al. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 1999; 47(10): 1229–1237.

Dendukuri

McCusker

Belzile

The identification of seniors at risk screening tool: further evidence of concurrent and predictive validity. J Am Geriatr Soc 2004; 52(2): 290–296.

10.

McCusker

Bellavance

Cardin

, et al. Prediction of hospital utilization among elderly patients during the 6 months after an emergency department visit. Ann Emerg Med 2000; 36: 438–445.

11.

Moons

De Ridder

Geyskens

, et al. Screening for risk of readmission of patients aged 65 years and above after discharge from the emergency department: predictive value of four instruments. Eur J Emerg Med 2007; 14(6): 315–323.

12.

Salvi

Morichi

Grilli

, et al. Predictive validity of the Identification of Seniors At Risk (ISAR) screening tool in elderly patients presenting to two Italian emergency departments. Aging Clin Exp Res 2009; 21(1): 69–75.

13.

Hoogerduijn

Schuurmans

Korevaar

, et al. Identification of older hospitalised patients at risk for functional decline, a study to compare the predictive values of three screening instruments J Clin Nurs 2010; 19(9–10): 1219–1225.

14.

Galvin

Gilleit

Wallace

, et al. Adverse outcomes in older adults attending emergency departments: a systematic review and meta-analysis of the Identification of Seniors At Risk [ISAR] screening tool. Age Ageing 2017; 46(2): 179–186.

15.

Akça

ASD

Toprak

Hocagil

, et al. Validation and reliability of the Turkish version of Identifying Seniors At Risk (ISAR) tool. Poster presentation, TAHEV Congress of 8th Family Medicine Fall School, Congress Book, 24–28 September 2014. Antalya, Turkey (in Turkish).

16.

Guttman

Afilalo

Guttman

, et al. An emergency department-based nurse discharge coordinator for elder patients: does it make a difference? Acad Emerg Med 2004; 11(12): 1318–1327.

17.

Graf

Giannelli

Herrmann

, et al. Identification of older patients at risk of unplanned readmission after discharge from the emergency department—comparison of two screening tools. Swiss Med Wkly 2012; 141: w13327.

18.

Oktay

Cete

Eray

, et al. Appropriateness of emergency department visits in a Turkish university hospital. Croat Med J 2003; 44(5): 585–591.

19.

Edmans

Bradshaw

Gladman

, et al. The Identification of Seniors at Risk [ISAR] score to predict clinical outcomes and health service costs in older people discharged from UK acute medical units. Age Ageing 2013; 42(6): 747–753.

20.

Singler

Heppner

Skutetzky

, et al. Predictive validity of the identification of seniors at risk screening tool in a German emergency department setting. Gerontology 2013; 60: 413–419.

21.

Braes

Moons

Lipkens

, et al. Screening for risk of unplanned readmission in older patients admitted to hospital: predictive accuracy of three instruments. Aging Clin Exp Res 2010; 22(4): 345–351.

22.

Salvi

Morichi

Grilli

, et al. Screening for frailty in elderly emergency department patients by using the Identification of Seniors At Risk [ISAR]. J Nutr Health Aging 2012; 16(4): 313–318.

23.

Buurman

van den Berg

Korevaar

, et al. Risk for poor outcomes in older patients discharged from an emergency department: feasibility of four screening instruments. Eur J Emerg Med 2011; 18(4): 215–220.

24.

Yim

Rainer

Graham

, et al. Emergency department intervention for high-risk elders: identification strategy and randomised controlled trial to reduce hospitalisation and institutionalization. Hong Kong Med J 2011; 17(3 Suppl. 3): 4–7.

25.

Beaton

Grimmer

Tools that assess functional decline: systematic literature review update. Clin Interv Aging 2013; 8: 485–494.

26.

Asomaning

Loftus

Identification of seniors at risk (ISAR) screening tool in the emergency department: implementation using the plan-do-study act model and validation results. J Emerg Nurs 2014; 40(4): 357–364.

27.

Yao

Fang

Lou

, et al. A systematic review of the identification of seniors at risk (ISAR) tool for the prediction of adverse outcome in elderly patients seen in the emergency department. Int J Clin Exp Med 2015; 8(4): 4778–4786.

28.

Warburton

Parke

Church

, et al. Identification of seniors at risk: process evaluation of a screening and referral program for patients aged > or =75 in a community hospital emergency department. Int J Health Care Qual Assur Inc Leadersh Health Serv 2004; 17: 339–348.