Usability of the Surprise Question by Nurses and Patients’ Families for Risk Stratification in Emergency Patients: A Prospective Cohort Study

Abstract

The Surprise Question is an effective tool to identify patients in need of palliative care. But it is unknown whether the Surprise Question can effectively predict adverse outcomes in Emergency patients. Therefor this study is to determine the utility of the modified Surprise Question for risk stratification in emergency patients. And assessed if the modified Surprise Question could be used by different healthcare personnel. Nurses and patients’ families were asked to respond as “Yes” or “No” to the modified Surprise Question for each patient. The outcome was resuscitation unit admission. Logistic regression was used to determine covariant significantly associated with resuscitation unit admission. The area under the curve for the second Surprise Question response by nurses was 0.620, which improved to 0.704 when the responses of nurses and patients’ families were in agreement. The clinical impression of nurses is a valuable tool to predict altered conditions for medium-acuity patients, and the diagnostic accuracy improves when responses of patients’ families and nurses agreed. The clinical impression of nurses is a valuable tool to predict altered conditions for medium-acuity patients, and the diagnostic accuracy improves when responses of patients’ families and nurses agreed.

Keywords

nurses surprise question emergency treatment risk assessment patients

What do we already know about this topic?

The Surprise Question: “Would I be surprised if this patient died in the next 12 months?” is an effective tool to identify patients in need of palliative care and was found to predict adverse outcomes in the emergency department.

How does your research contribute to the field?

This paper assessed if the modified Surprise Question could be used by nurses.

What are your research’s implications towards theory, practice, or policy?

This paper evidences that the Surprise Question is a valuable tool to predict altered conditions for medium-acuity patients in emergency department.

Background

With the rapid development of emergency medicine, visits to the emergency department have increased dramatically worldwide.¹ Important diagnostic and treatment decisions must be made within a short period after emergency department admission. However, it is more important to assign a matched care setting for patients based on the disease severity.² Therefore, a triage system is crucial to optimize emergency department resources, which mainly classifies patients who do and do not need immediate intervention based on urgency.³

The Emergency Severity Index (ESI) is a widely adopted triage system with proven practicability and accuracy.^4,5 The ESI includes five levels; Levels 1 and 2 are based on patient acuity and Levels 3 through 5 on resource allocation.⁶ To create treatment space for high-acuity patients in the crowded environment of the emergency department, the “fast track” model is now generally accepted. With these lower-acuity patients handled first, the medium-acuity (ie, ESI 3) cases now wait for the longest for care and are often a lower priority for regular emergency department resources,^7,8 despite the variation of required resources, complexity, monitoring intensity, and nursing grades. Further, the initial ESI system may omit potentially critically ill patients, and the condition of emergency patients is prone to rapid changes. Importantly, in previous, large emergency department cohort studies, most patients were classified as medium-acuity, ranging from 48% to 65%.^9-11 Currently, there is no useful screening tool to identify medium-acuity patients who are likely to deteriorate to a life-threatening status since the fast-paced emergency environment requires something easy to operate and interpret in a short period.

The Surprise Question: “Would I be surprised if this patient died in the next 12 months?” is an effective tool to identify patients in need of palliative care^12,13 and was found to predict adverse outcomes in the emergency department,^14-16 although the accuracy widely varied.¹⁷ Most studies used the Surprise Question for patients with cancer and organ failure and long-term mortality as the primary endpoint.^18-20 However, it is unknown whether the Surprise Question can effectively predict adverse outcomes in patients with Emergency Severity Index Level 3 in the emergency department. Nurses and patients’ families are might find it easier to open up to about patients’ preferences and could be good assessors of the Surprise Question. Therefore, the aim of this study is to assess the predictive value of the SQ when used by nurses working on the emergency department and patients’ families for risk stratification in medium-acuity patients.

Methods

Study Design

The PSQS study was conducted in the emergency department observation unit of a general hospital in Sichuan, China from December 2020 to May 2021. All emergency patients were first triaged by the Emergency Service Index system. Patients with Emergency Service Index Levels 1 or 2 were admitted to the resuscitation or intensive care units for treatment. Patients with ESI 3 to 5 were evaluated by doctors in the general diagnosis unit. If patients need further intervention, monitoring, or hospitalization, they were stayed in the emergency department and accompanied by family members.

Study Population

In total, 1411 patients with ESI 3 who visited the emergency department from December 1, 2020, to February 28, 2021 were consecutively recruited for this study. Patients were excluded if they were less than 18 years old, pregnant, left the emergency department within 24 hours, had communication disorders, or refused to participate. Finally, 1186 patients were included. Figure 1 presents the study’s participant flow chart. Based on our pilot experiment, the resuscitation unit admission incidence rate was about 10% among the nurses who answered “No” to the modified Surprise Question, and about 2% among nurses who answered “Yes.”

Figure 1.

Study flow chart.

The Modified Surprise Question

The original Surprise Question is: “Would I be surprised if this patient died in the next 12 months?.” Patients in our study were relatively stable and had very low mortality (less than 1%). Thus, nurses who cared for the patients were asked to answer the modified Surprise Question: “Would you be surprised if the patient’s condition deteriorated to life-threatening?” within two hours after admission to the observation unit. The answer choices were “Yes,” “No,” or “Unclear.” We also asked the modified Surprise Question to the family members. The families of patients who were unwilling to answer this question were allowed to remain silent. Although the family members lack relevant professional knowledge, they are the main caregivers outside the hospital and one of the most important sources of information on the patient’s condition. The modified Surprise Question was asked a second time approximately 24 hours later. We could only dynamically evaluate the modified Surprise Question twice since most patients stayed in the observation unit for less than 72 hours. We classified the answer “No” as a positive response, meaning that the nurse or family member predicts that the patient will experience a serious deterioration in the emergency department.

Competency Inventory for Registered Nurses

There were 65 nurses in the emergency department, of which 58 participated in this study. Age, seniority, professional title, and core competence scores of nurses who answered the modified Surprise Question were obtained. The nurse core competence scale used the Competency Inventory for Registered Nurse, designed by Liu et al.²¹ The overall reliability of Cronbach’s α was 0.89 and included seven dimensions: critical thinking and scientific research ability, clinical nursing, leadership ability, interpersonal, ethical and legal practice, professional development, and educational consultation. In total, 55 items were scored by Likert 5 score. The total scale score was 220, and the higher the score, the stronger the core competence of the nurse. A total score of 164 to 220 (average item score >3) indicated strong core competence, 110 to 164 (average item score 2-3) indicated moderate core competence, and <110 (average item score <2) indicated weak core competence. The nurses were divided into two groups based on the cut-off value of 164 as only one nurse had a score <110.

Primary Endpoint

The patients with Emergency Service Index Level 3 primarily wait for admission after the necessary treatment in the emergency department and are considered at relatively low mortality risk. However, patients with serious illnesses may not be identified at triage, and a patient’s condition is often complex and rapidly changing in the emergency department. Therefore, if the patient showed unstable vital signs and life-threatening symptoms in the observation unit (ie, Emergency Service Index Level 1 or 2), they were transferred to the resuscitation unit after confirmation by emergency physicians. Thus, the primary endpoint was resuscitation unit admission.

Data Collection

The socio-demographic characteristics collected include the age, sex, and marital status of the patients and their accompanying family members. Vital signs (including temperature, pulse, respiration, blood pressure, and blood oxygen saturation), chief complaints, diagnosis, Charlson Comorbidity Index, and the length of stay were recorded by two independent, trained researchers. The Charlson Comorbidity Index is a validated and reliable tool for measuring the number and severity of comorbid diseases, with a total score between 0 and 31 that positively correlates with mortality.²² The Emergency Service Index levels were recorded based on triage nurses’ evaluations. The National Early Warning Score was calculated by heart rate, blood pressure, respiration rate, oxygen saturation, body temperature, and consciousness.²³ The Modified Early Warning Score replaces oxygen saturation with urine volume based on the National Early Warning Score.²⁴ Finally, the quick Sepsis-related Organ Failure Assessment consisted of blood pressure, respiration rate, and consciousness.²⁵ Differences between the two data checks were judged by another researcher.

Statistical Analyses

Parametric variables were reported as means ± standard deviations and compared using variance analysis. Nonparametric variables were reported as medians and interquartile ranges (25th-75th percentiles) and compared using the Mann–Whitney U test. Categorical variables were reported as frequencies and percentages and compared using the chi-square test.

Because only approximately 1% of answers from nurses or family members were “Unclear,” our analyses did not consider this response. A logistic regression model was used to examine the association of the modified Surprise Question responses with resuscitation unit admission. To determine whether these relationships were independent risk factors, the model was adjusted for age, sex, vital signs on admission, emergency department length of stay, the Charlson Comorbidity Index, common chief complaints, and complicating malignant tumors. A univariate logistic regression were used to calculated odds-ratio. A multivariable logistic regression model was used to assess the interactions between the modified Surprise Question responses and the Competency Inventory for Registered Nurse score and resuscitation unit admission. The areas under the receiver operator characteristic curve were calculated to estimate the judgment efficiency of the modified Surprise Question for resuscitation unit admission, sensitivity, specificity, positive negative predictive values, and accuracy.

A two-tailed P < .05 was statistically significant for all tests. All statistical analyses were performed using SPSS version 26.0 (IBM Corp, Armonk, NY, USA) and R software 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria).

Ethical Approval

This study complied with the Declaration of Helsinki, and the local Human Ethical Committee approved the study protocol. All participants provided written informed consent.

Results

Baseline Characteristics

In total, 1186 participants (average age 51.1 ± 18.9 years) were included (Figure 1). Among them, 682 (57.5%) were men. Sixty-seven patients (5.6%) were admitted to the resuscitation unit, nine of which (0.8%) died during hospitalization. Participants transferred to the resuscitation unit were older, had higher Charlson Comorbidity Index scores, Modified Early Warning Scores, and emergency department length of stays than those who were not transferred. Table 1 and Supplemental Table 1 compare the baseline characteristics.

Table 1.

Baseline Characteristics of MA Patients.

Characteristics	Non-RUA (n = 1119)	RUA (n = 67)	P value
Age (year)	53.8 ± 18.9	59.1 ± 17.1	.025
Male sex	641 (57.3)	41 (61.2)	.529
Vital signs
Temperature (°C)	36.7 ± 0.7	36.6 ± 0.6	.495
SBP (mmHg)	128.6 ± 21.2	129 ± 18.7	.871
DBP (mmHg)	78.7 ± 14.8	78.1 ± 14.6	.745
Heart rate (/min)	94.8 ± 18.4	94.5 ± 19	.915
Respiratory rate (/min)	20.2 ± 1.5	20.2 ± 1.8	.637
Oxygen saturation (%)	97.5 ± 1.8	97.5 ± 2.0	.806
Common chief complaints			.424
Musculoskeletal-related	50 (4.5)	3 (4.5)
Respiratory-related	201 (18.0)	14 (20.9)
Urological-related	85 (7.6)	3 (4.5)
Endocrinic-related	72 (6.4)	5 (7.5)
Neurological-related	177 (15.8)	4 (6.0)
Gastrointestinal-related	291 (26.0)	22 (32.8)
Cardiovascular-related	85 (7.6)	7 (10.4)
Hematological-related	158 (14.1)	9 (13.4)
Malignant tumor	130 (11.6)	8 (11.9)	.936
Charlson Comorbidity Index	5 (2-7)	6 (2-7)	.049
Risk score
NEWS	1 (0-3)	2 (1-3)	.229
MEWS	2 (1-4)	3 (1-4)	.037
qSOFA			.075
0	902 (80.6)	48 (71.6)
1	200 (17.9)	16 (23.9)
2	17 (1.5)	3 (4.5)
Length of stay
ED	2 (1-4)	1 (1-3)	<.001
ED plus hospitalization	10 (4-16)	7 (2-16)	.082

Note. Values are expressed as N (%), mean ± SD, and median (25th-75th).

SBP = systolic blood pressure; DBP = diastolic blood pressure; ED = emergency department; NEWS = National Early Warning Score; MEWS = Modified Early Warning Score; qSOFA = quick Sepsis-related Organ Failure Assessment; RUA = resuscitation unit admission; MA = medium-acuity.

Nurses’ Characteristics

There were 58 participating registered nurses (median age 28 [range, 26-31] years); 10 (17.2%) were males. The nurses had a median work experience of 7 (range, 2-9) years and a median Competency Inventory for Registered Nurse score of 157 (range, 136-169). Of all nurses, 20 (37.8%) had a Competency Inventory for Registered Nurse score of ≥164.

The Modified Surprise Question and Resuscitation Unit Admission

For the first modified Surprise Question (Figure 2), there was a similar incidence of resuscitation unit admission regardless of the answer from nurses (“No” 6.9% vs “Yes” 4.6%, P = .084) or patients’ families (“No” 7.2% vs “Yes” 5.3%, P = .309). Notably, nurses with a Competency Inventory for Registered Nurse score of ≥164 who answered “No” had a significantly higher percentage of patients admitted to the resuscitation unit than the “Yes” group (10.1% vs 3.9%, P < .001).

Figure 2.

Associations between the first Surprise Question and resuscitation unit admission incidences.

For the second modified Surprise Question (Figure 3), nurses (“No” 8.3% vs “Yes” 3.2%, P < .001) and patients’ families (“No” 9.4% vs “Yes” 4.5%, P = .008) who answered “No” had a significantly higher percentage of patients admitted to the resuscitation unit than the “Yes” group. These results were consistent regardless of the nurses’ Competency Inventory for Registered Nurse score.

Figure 3.

Associations between the second Surprise Question and resuscitation room admission incidences.

Logistic Regression Analyses

The odds ratio for resuscitation unit admission in the multivariable adjusted logistic regression models (Table 2) comparing the “Yes” and “No” responses to the first modified Surprise Question was 1.452 (95% confidence interval: 0.871-2.419, P = .152) for all nurses, 0.851 (95% confidence interval: 0.252-1.686, P = .582) for nurses with Competency Inventory for Registered Nurse scores of <164, 2.802 (95% confidence interval: 1.384-5.672, P = .004) for nurses with Competency Inventory for Registered Nurse scores of ≥164, and 1.179 (95% confidence interval: 0.441-3.148, P = .743) for patients’ families.

Table 2.

Adjusted and Unadjusted Odds Ratios for the Association of Surprise Question and Resuscitation Unit Admission by Logistic Regression Analysis.

Variables	Univariable analysis		Multivariable analysis
Variables	OR (95% CI)	P value	OR (95% CI)	P value
Response to first SQ (no vs yes)
All nurses	1.549 (0.940-2.553)	.086	1.452 (0.871-2.419)	.152
Nurses with CIRN <164	0.621 (0.315-1.226)	.170	0.851 (0.252-1.686)	.582
Nurses with CIRN ≥164	2.778 (1.404-5.497)	.003	2.802 (1.384-5.672)	.004
Patients’ families	1.393 (0.734-2.647)	.311	1.179 (0.441-3.148)	.743
Response to second SQ (no vs yes)
All nurses	2.741 (1.603-4.687)	<.001	2.801 (1.620-4.844)	<.001
Nurses with CIRN <164	2.899 (1.206-6.968)	.017	2.759 (1.116-6.822)	.028
Nurses with CIRN ≥164	4.478 (1.969-10.184)	<.001	4.461 (1.904-10.45)	.001
Patients’ families	2.208 (1.215-4.014)	.009	2.325 (1.250-4.323)	.008

Note. Multivariable logistic analysis was adjusted by age, sex, systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, oxygen saturation, common chief complaints, malignant tumor, emergency department length of stay, Emergent Severity Index, and Charlson Comorbidity Index. OR = odds ratio; SQ = Surprise Question; CIRN = Competency Inventory for Registered Nurses.

P < .05 for interaction of response to SQ, CIRN, and resuscitation room admission.

After adjusting for confounding factors, the assessment of the second modified Surprise Question (Table 2) “No” response found that all nurses (odds ratio: 2.801, 95% confidence interval: 1.620-4.844, P < .001), nurses with Competency Inventory for Registered Nurse scores of <164 (odds ratio: 2.759, 95% confidence interval: 1.116-6.822, P = .028), nurses with Competency Inventory for Registered Nurse scores of ≥164 (odds ratio: 4.461, 95% confidence interval: 1.904-10.45, P = .001), and patients’ families (odds ratio: 2.325, 95% confidence interval: 1.250-4.323, P = .008) were independent predictors of resuscitation unit admission.

Discrimination Analyses

At admission, the area under the receiver operating characteristic curve (Table 3) was 0.543 (95% confidence interval: 0.473-0.612, P = .240) for the National Early Warning Score, 0.574 (95% confidence interval: 0.501-0.647, P = .043) for the Modified Early Warning Score, and 0.547 (95% confidence interval: 0.473-0.622, P = .194) for the quick Sepsis-related Organ Failure Assessment for resuscitation unit admission. The first modified Surprise Question response from nurses and patients’ families did not have predictive value for resuscitation unit admission (P > .05). However, the area under the curve for the second modified Surprise Question from all nurses was 0.620 (95% confidence interval: 0.553-0.686, P = .001), which was greater than the National Early Warning Score, the Modified Early Warning Score, and the quick Sepsis-related Organ Failure Assessment (P < .05). The modified Surprise Question answered by all nurses had 70% sensitivity, 54% specificity, 8% positive predictive value, and 97% negative predictive value. Notably, when the modified Surprise Question responses by patients’ families and nurses agreed, the area under the curve improved to 0.704 (95% confidence interval: 0.629-0.779, P < .001), and the diagnostic accuracy improved to 84%.

Table 3.

Discrimination, Sensitivity, Specificity, Accuracy, PPV, and NPV of Surprise Question and Emergent Severity Index for Resuscitation Room Admission.

Predictors	ROAUC (95% CI)	P for ROAUC	Sensitivity	Specificity	Accuracy	PPV	NPV
“No” to first SQ
All nurses	0.554 (0.484-0.625)	0.135	0.58	0.53	0.53	0.07	0.95
Nurses with CIRN <164	0.441 (0.344-0.538)	0.234	0.44	0.44	0.44	0.05	0.92
Nurses with CIRN ≥164	0.624 (0.536-0.711)	0.009	0.68	0.57	0.62	0.09	0.98
Patients’ families	0.531 (0.446-0.617)	0.461	0.29	0.78	0.75	0.07	0.95
“No” to second SQ
All nurses	0.620 (0.553-0.686)	0.001	0.7	0.54	0.55	0.08	0.97
Nurses with CIRN <164	0.622 (0.520-0.724)	0.032	0.74	0.5	0.51	0.07	0.98
Nurses with CIRN ≥164	0.675 (0.582-0.768)	<0.001	0.74	0.61	0.58	0.1	0.96
Patients’ families	0.582 (0.496-0.669)	0.052	0.39	0.78	0.76	0.09	0.96
Nurses plus patients’ families^a	0.704 (0.629-0.779)	<0.001	0.53	0.86	0.84	0.18	0.97
Risk score at admission
NEWS^a	0.543 (0.473-0.612)	0.240	0.52	0.53	0.53	0.06	0.95
MEWS^a	0.574 (0.501-0.647)	0.043	0.42	0.69	0.68	0.08	0.95
qSOFA score^a	0.547 (0.473-0.622)	0.194	0.28	0.81	0.78	0.08	0.95

ROAUC = area under the receiver operator characteristic curve; SQ = Surprise Question; CIRN = Competency Inventory for Registered Nurses; NEWS = National Early Warning Score; MEWS = Modified Early Warning Score; qSOFA = quick Sepsis-related Organ Failure Assessment.

Compared with second SQ by all nurses, P < .05 for ROAUC comparison. The ROAUCs of SQ were calculated by bivariate model, that of NEWS and MEWS were calculated by continuous variable, and that of qSOFA was calculated by three-variable model. Sensitivity and specificity of NEWS, MEWS, and qSOFA for resuscitation room admission were calculated according to the cut off value of the largest Youden index.

Discussion

In this study, the associations between the answers to the modified Surprise Question from nurses and patients’ families changed with resuscitation unit admission events. The answers to the first modified Surprise Question asked within two hours after observation unit admission indicated that only nurses with a strong core competence (ie, Competency Inventory for Registered Nurse scores of ≥164) could predict rapid deterioration by their clinical intuition. Importantly, as the time in the observation unit increases (eg, after 24 hours), the modified Surprise Question answer “No” from nurses and patients’ families had independent predictive value regarding resuscitation unit admission.

To our knowledge, there is no specific assessment tool for the prognosis of medium-acuity patients in the emergency department. We compared the discrimination, sensitivity, specificity, and accuracy of the modified Surprise Question with that of the National Early Warning Score, the Modified Early Warning Score, and the quick Sepsis-related Organ Failure Assessment, which are used in different settings to detect at-risk patients for deterioration, intensive care unit admission, and mortality in the emergency department.^26-29 Our results suggested that only the initial Modified Early Warning Score has poor predictive power for the deterioration of patients with Emergency Service Index Level 3. These results demonstrate that it is difficult to identify the prognosis of medium-acuity patients with stable vital signs using a screening tool comprised of only vital signs. Further, the modified Surprise Question responses from nurses the next day could better discriminate resuscitation room admission than the Modified Early Warning Score, suggesting that the clinical intuition of nurses considers more factors than vital signs, making it a more valuable tool to predict deterioration. These suspected at-risk patients also need more attention and monitoring so that treatment plans can be changed in time.

Previous research has asked the Surprise Question regarding patients and validated a positive predictive value.^30,31 In our pilot experiment, the degree of cooperation from patients was very poor, likely generating fear. Therefore, we conducted this study by questioning the patients’ family members and also found a negative to positive change in the predictive value, similar to the nurses. This result may be partly owing to physician-patient communication, but further study is required to explain this phenomenon. Similar to previous studies, our results also found lower discriminatory values for assessments by nonprofessionals than professionals. More importantly, the diagnostic accuracy improved when professionals and nonprofessionals agreed.^30,31 The specificity was also higher for answers provided by the patients’ families than by nurses in our study, supporting the importance of shared clinical decision-making.³²

The biggest difference between this study and previous studies is the primary endpoint. In this study, the primary endpoint is worsening disease rather than mortality owing to the differences in the clinical setting and the disease severity.^{13,14,18,33,34} A meta-analysis analyzed the association between the Surprise Question and 6 to 18-month mortality and found that the pooled prognostic characteristics of sensitivity, specificity, and the area under the curve were 0.67, 0.80, and 0.81, respectively.¹² However, our results show that sensitivity was greater than specificity when nurses judge the likelihood of worsening disease, perhaps because medical providers are more cautious when judging death versus disease progression. This also explains why more than 50% of nurses answered “No” in our study. The discrimination in our study (area under the curve: 0.62) is generally weaker than in other studies.^12,20,35,36 However, a previous study also found lower discriminatory values for endpoints other than mortality.³⁰ Another study about the Surprise Question conducted at the emergency department measured acute morbidity as the endpoint and also resulted in a lower area under the curve for nurses (0.68) and patients (0.54), similar to our results.³¹

Few studies have explored interactions between the Surprise Question responses, Competency Inventory for Registered Nurse scores, and outcomes. Our study demonstrated that nurses with a strong core competence (Competency Inventory for Registered Nurse of ≥164) have a stronger predictive ability for prognosis than nurses without. Nurses with higher scores also had significantly more work experience in our study (data not shown). Consistently, previous studies found that nurses with more experience were more accurate in their Surprise Question prediction.^13,37 These results emphasize the importance of empowering junior nurses to acquire higher core competencies through education and training programs to provide better patient care.³⁸

There are limitations to this study. First, we used a modified Surprise Question and used resuscitation unit admission as the primary endpoint, both of which are not validated. However, some other studies have also used a modified Surprise Question in other clinical settings.¹⁵ Second, the participating nurses had only a few years of professional experience, and we reported that clinical intuition has more predictive value in more experienced professionals. Thus, the finding in this study may differ in other emergency departments. Third, a previous study found that the patients’ prognosis accuracy was higher when both nurses and physicians agreed.^15,30 However, physicians did not participate in this study due to the rapid rotation of emergency department unit physicians in our hospital. Thus, the importance of teamwork cannot be investigated. Fourth, the next-day answers by nurses and patients’ families had higher predictive power, but that may be because the answers changed based on history, physical and laboratory examinations, and diagnostic tests. As such, we will explore this concept in a future study.

Conclusions

In the very early stage of emergency department treatment, the judgment of nurses with relatively few years of clinical experience regarding patient prognosis was not accurate. However, emergency department nurses should be aware that their clinical intuition is a valuable tool for predicting altered conditions of patients with stable vital signs. Thus, individualized nursing plans can be made in advance. The diagnostic accuracy of the modified Surprise Question improved when the responses of the patients’ families and the nurses were aligned, highlighting the importance of shared decision-making. Future studies are needed to verify the effectiveness of the modified Surprise Question used in this study and to confirm if implementing the modified Surprise Question in the emergency department changes medical decisions and improves patient prognosis.

Supplemental Material

sj-docx-1-inq-10.1177_00469580231163089 – Supplemental material for Usability of the Surprise Question by Nurses and Patients’ Families for Risk Stratification in Emergency Patients: A Prospective Cohort Study

Supplemental material, sj-docx-1-inq-10.1177_00469580231163089 for Usability of the Surprise Question by Nurses and Patients’ Families for Risk Stratification in Emergency Patients: A Prospective Cohort Study by Yi Liu, Dongze Li, Yu Jia, Jing Yu, Fanghui Li, Yongli Gao, Yan Ma, Zhi Wan, Zhi Zeng and Wei Zhang in INQUIRY: The Journal of Health Care Organization, Provision, and Financing

Supplemental Material

sj-docx-2-inq-10.1177_00469580231163089 – Supplemental material for Usability of the Surprise Question by Nurses and Patients’ Families for Risk Stratification in Emergency Patients: A Prospective Cohort Study

Supplemental material, sj-docx-2-inq-10.1177_00469580231163089 for Usability of the Surprise Question by Nurses and Patients’ Families for Risk Stratification in Emergency Patients: A Prospective Cohort Study by Yi Liu, Dongze Li, Yu Jia, Jing Yu, Fanghui Li, Yongli Gao, Yan Ma, Zhi Wan, Zhi Zeng and Wei Zhang in INQUIRY: The Journal of Health Care Organization, Provision, and Financing

Footnotes

Acknowledgements

We would like to thank all the participants of this project and investigators for collecting the data.

Author Contributions

YL, ZZ, and WZ conceived of the study design. DL, YJ, JY, FL, YL, YG, and ZW collected the epidemiological and clinical data. DL, YJ, YL, and YM summarized data, and performed the statistical analysis. DL, YJ, YM, and YC interpreted the data, and drafted the manuscript. All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Data Availability Statement

The corresponding author will provide the data that support the results of this research upon reasonable request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported financially by grants from National Key Research and Development Program of China (No. 2020AAA0105000, 2020AAA0105005), Sichuan Science and Technology Program (No. 2022YFS0279, 2021YFQ0062, 2022JDRC0148, 2023YFS0240), Sichuan Provincial Health Commission (ZH2022-101), Sichuan University West China Nursing Discipline Development Special Fund Project (HXHL20017, HXHL20046, HXHL21016)

ORCID iD

Yi Liu

Supplemental Material

Supplemental material for this article is available online.

References

Pines

Hilton

Weber

, et al. International perspectives on emergency department crowding. Acad Emerg Med. 2011;18:1358-1370. doi:10.1111/j.1553-2712.2011.01235.x

Tam

Chung

Lou

CK.

A review of triage accuracy and future direction. BMC Emerg Med. 2018;18:58. doi:10.1186/s12873-018-0215-0

FitzGerald

Jelinek

Scott

Gerdtz

. Emergency department triage revisited. Emerg Med J. 2010;27:86-92. doi:10.1136/emj.2009.077081

Elias

Damle

Casale

, et al. A web-based tool for patient triage in emergency department settings: validation using the emergency severity index. JMIR Med Inform. 2015;3:e23. doi:10.2196/medinform.3508

McHugh

Tanabe

McClelland

, et al. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med. 2012;19:106-109. doi:10.1111/j.1553-2712.2011.01240.x

Shelton

The Emergency Severity Index 5-level triage system. Dimens Crit Care Nurs. 2009;28:9-12. doi:10.1097/01.dcc.0000325106.28851.89

McCarthy

Zeger

Ding

, et al. Crowding delays treatment and lengthens emergency department length of stay, even among high-acuity patients. Ann Emerg Med. 2009;54:492-503.e494. doi:10.1016/j.annemergmed.2009.03.006

Bullard

Villa-Roel

Bond

Vester

Holroyd

Rowe

BH.

Tracking emergency department overcrowding in a tertiary care academic institution. Healthc Q 2009;12:99-106. doi:10.12927/hcq.2013.20884

van der Wulp

Schrijvers

van Stel

HF.

Predicting admission and mortality with the Emergency Severity Index and the Manchester Triage System: a retrospective observational study. Emerg Med J. 2009;26:506-509. doi:10.1136/emj.2008.063768

10.

Levin

Toerper

Hamrock

, et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med. 2018;71:565-574.e562. doi:10.1016/j.annemergmed.2017.08.005.

11.

Raita

Goto

Faridi

Brown

DFM

Camargo

Hasegawa

Emergency department triage prediction of clinical outcomes using machine learning models. Crit Care. 2019;23:64. doi:10.1186/s13054-019-2351-7

12.

Downar

Goldman

Pinto

Englesakis

Adhikari

NK.

The “surprise question” for predicting death in seriously ill patients: a systematic review and meta-analysis. CMAJ. 2017;189:E484-e493. doi:10.1503/cmaj.160775

13.

Flierman

van Rijn

Willems

, et al. Usability of the surprise question by nurses to identify 12-month mortality in hospitalized older patients: a prospective cohort study. Int J Nurs Stud. 2020;109:103609. doi:10.1016/j.ijnurstu.2020.103609

14.

Zeng

Eugene

Supino

Would you be surprised if this patient died in the next 12 months? Using the surprise question to increase palliative care consults from the emergency department. J Palliat Care. 2020;35:221-225. doi:10.1177/0825859719866698

15.

Haydar

Strout

Bond

Han

PK.

Prognostic value of a modified surprise question designed for use in the emergency department setting. Clin Exp Emerg Med. 2019;6:70-76. doi:10.15441/ceem.17.293

16.

Aaronson

George

Ouchi

, et al. The surprise question can be used to identify heart failure patients in the emergency department who would benefit from palliative care. J Pain Symptom Manage. 2019;57:944-951. doi:10.1016/j.jpainsymman.2019.02.007

17.

Jennings

Marks

Lum

HD.

The surprise question as a prognostic tool #360. J Palliat Med. 2018;21:1529-1530. doi:10.1089/jpm.2018.0348

18.

Moss

Lunney

Culp

, et al. Prognostic significance of the “surprise” question in cancer patients. J Palliat Med. 2010;13:837-840. doi:10.1089/jpm.2010.0018

19.

Moss

Ganjoo

Sharma

, et al. Utility of the “surprise” question to identify dialysis patients with high mortality. Clin J Am Soc Nephrol. 2008;3:1379-1384. doi:10.2215/cjn.00940208

20.

White

Kupeli

Vickerstaff

Stone

How accurate is the ‘surprise question’ at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med. 2017;15:139. doi:10.1186/s12916-017-0907-4

21.

Liu

Kunaiktikul

Senaratana

Tonmukayakul

Eriksen

Development of competency inventory for registered nurses in the People’s Republic of China: scale development. Int J Nurs Stud. 2007;44:805-813. doi:10.1016/j.ijnurstu.2006.01.010

22.

Charlson

Pompei

Ales

KL.

A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-383. doi:10.1016/0021-9681(87)90171-8

23.

Smith

Redfern

Pimentel

, et al. The National Early Warning Score 2 (NEWS2). Clin Med. 2019;19:260. doi:10.7861/clinmedicine.19-3-260

24.

Subbe

Kruger

Rutherford

Gemmel

Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94:521-526. doi:10.1093/qjmed/94.10.521

25.

Singer

Deutschman

Seymour

, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315:801-810. doi:10.1001/jama.2016.0287

26.

Nannan Panday

Minderhoud

Alam

Nanayakkara

PWB.

Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): a narrative review. Eur J Intern Med. 2017;45:20-31. doi:10.1016/j.ejim.2017.09.027

27.

Mitsunaga

Hasegawa

Uzura

, et al. Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department. PeerJ. 2019;7:e6947. doi:10.7717/peerj.6947

28.

RSL

Leung

Brabrand

, et al. qSOFA is a poor predictor of short-term mortality in all patients: a systematic review of 410,000 patients. J Clin Med. 2019;8:61. doi:10.3390/jcm8010061

29.

Liu

Luo

Zhang

, et al. Quick Sequential Organ Failure Assessment as a prognostic factor for infected patients outside the intensive care unit: a systematic review and meta-analysis. Intern Emerg Med. 2019;14:603-615. doi:10.1007/s11739-019-02036-0

30.

Zelis

Mauritz

Kuijpers

LIJ

, et al. Short-term mortality in older medical emergency patients can be predicted using clinical intuition: a prospective study. PLoS ONE. 2019;14:e0208741. doi:10.1371/journal.pone.0208741

31.

Rohacek

Nickel

Dietrich

, et al. Clinical intuition ratings are associated with morbidity and hospitalisation. Int J Clin Pract. 2015;69:710-717. doi:10.1111/ijcp.12606

32.

Gustin

Jr.

Shared decision-making. Anesthesiol Clin. 2019;37:573-580. doi:10.1016/j.anclin.2019.05.001

33.

Maes

Van Den Noortgate

De Brauwer

, et al. Prognostic value of the Surprise Question for one-year mortality in older patients: a prospective multicenter study in acute geriatric and cardiology units. Acta Clin Belg. 2022;77:286-294. doi:10.1080/17843286.2020.1829869

34.

Rauh

Sullivan

Camacho

, et al. Validation of the surprise question in gynecologic oncology: a one-question screen to promote palliative care integration and advance care planning. Gynecol Oncol. 2020;157:754-758. doi:10.1016/j.ygyno.2020.03.007

35.

Moor

Tak van Jaarsveld

Owusuaa

, et al. The value of the surprise question to predict one-year mortality in idiopathic pulmonary fibrosis: a prospective Cohort Study. Respiration. 2021;100:780-785. doi:10.1159/000516291

36.

Malhotra

Tao

Wang

, et al. Performance of the surprise question compared to prediction models in hemodialysis patients: a Prospective Study. Am J Nephrol. 2017;46:390-396. doi:10.1159/000481920

37.

Da Silva Gane

Braun

Stott

Wellsted

Farrington

. How robust is the ‘surprise question’ in predicting short-term mortality risk in haemodialysis patients? Nephron Clin Pract. 2013;123:185-193. doi:10.1159/000353735

38.

Zhou

Wang

Hao

Factors associated with core competencies of emergency-room nurses in tertiary hospitals in China. Jpn J Nurs Sci. 2020;17:e12337. doi:10.1111/jjns.12337

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.03 MB