Performance of CURB-65,PSI,and APACHE-II for predicting COVID-19 pneumonia severity and mortality

Abstract

No prognostic tools for the prediction of COVID-19 pneumonia severity and mortality are available. We explored whether CURB-65, PSI, and APACHE-II could predict COVID-19 pneumonia severity and mortality. We included 167 patients with confirmed COVID-19 pneumonia in this retrospective study. The severity and 30-day mortality of COVID-19 pneumonia were predicted using PSI, CURB-65, and APACHE-II scales. Kappa test was performed to compare the consistency of the three scales. There was a significant difference in the distribution of the scores of the three scales (P < 0.001). Patients with PSI class ⩽III, CURB-65 ⩽1, and APACHE-II-I all survived. The ROC analysis showed the areas under the curve of the PSI, CURB-65, and APACHE-II scales were 0.83 (95% CI, 0.74–0.93), 0.80 (95% CI, 0.69–0.90), and 0.83 (95% CI, 0.75–0.92), respectively. Our findings suggest that PSI and CURB-65 might be useful to predict the severity and mortality of COVID-19 pneumonia.

Keywords

APACHE-II COVID-19 pneumonia CURB-65 mortality Pneumonia Severity Index

Coronavirus disease 2019 (COVID-19) is caused by a novel enveloped RNA beta coronavirus.¹ COVID-19 symptoms worsen rapidly and high mortality has been reported globally.² The World Health Organization (WHO) has declared a public health emergency of international concern over the global outbreak of COVID-19.³ As of 30 May 2021, there were 169,597,415 confirmed patients of COVID-19 and 3,530,582 deaths worldwide.⁴

It is important to evaluate the severity of COVID-19, since the outcomes of non-severe patients were favorable compared to severe patients. A reliable tool of assessing the severity of pneumonia can help clinicians better estimate the prognosis of their patients and therefore a more suitable treatment approach can be decided. Pneumonia Severity Index (PSI) and CURB-65 (confusion, urea, respiratory rate, blood pressure, age ⩾65) are commonly used to evaluate the severity of community and hospital-acquired pneumonia.^5,6 Moreover, a previous study showed the CURB-65 scale could predict the mortality of community-acquired pneumonia (CAP).⁷ The Acute Physiology and Chronic Health Evaluation (APACHE) II system is widely applied in the intensive care unit (ICU).⁸ Several studies have evaluated common intensive care severity scores to assess COVID-19 pneumonia severity and mortality. Artero et al. showed that performance of PSI and CURB-65 were better than quick Sequential Organ Failure Assessment (qSOFA: respiratory rate≥22/min, altered mentation, or systolic blood pressure ≤100 mmHg) at predicting mortality in patients with COVID-19 pneumonia. qSOFA score is simple, and qSOFA score≥2 predicted the severity of COVID-19, with a sensitivity of 26.7%.⁹ The study by Myrstad et al.¹⁰ showed that National Early Warning Score 2 (NEWS2) score was superior to qSOFA, Systemic Inflammatory Response Syndrome (SIRS) criteria and CRB-65 in predicting COVID-19 pneumonia severe and mortality. The finding from Ji et al.¹¹ suggested that the CALL (comorbidity, age, lymphocyte, and Lactate Dehydrogenase; LDH) score could predict the progression risk in patients with COVID-19 pneumonia. However, Ihle-Hansen et al.¹² found CRB-65 and qSOFA scoring systems were not suitable for evaluating the severity and mortality of COVID-19. Satici et al.¹³ showed that PSI performed better than CURB-65 in assessing the mortality of COVID-19, and adding CRP levels to PSI did not improve the prediction of 30-day mortality. Thus the performance of these scoring systems in evaluating the severity and mortality of COVID-19 pneumonia is controversial. In this cross-sectional study, we aimed to test the hypothesis that PSI, CURB-65, and APACHE-II scales are associated with the severity and 30-day mortality in patients with COVID-19 pneumonia.

Methods

Study population

Patients with confirmed COVID-19 pneumonia who were admitted to Tumor Center of Union Hospital, Tongji Medical College, Huazhong University of Science and Technology (Wuhan, China) from February 15, 2020 to March 17, 2020 were included in this retrospective study. A confirmed case of COVID-19 was defined as a positive result on real-time reverse transcription-polymerase chain reaction (RT-PCR) assay of respiratory specimens.¹⁴ Our study was conducted in line with the Declaration of Helsinki. The Fujian Medical University Union Hospital Ethics Committee approved the study protocol (2020XGFKY002) and written informed consent was obtained from a legally authorized representative for each patient. All hospitalized patients were treated according to the COVID-19 Diagnosis and Treatment Protocol issued by National Health Commission of China.¹⁵ Patients who were younger than 18 years, had a DNR (Do Not Resuscitate) order or were pregnant were excluded. The patient flow chart is shown in Figure 1.

Figure 1.

Patient flow chart.

Data collection

Clinical records were reviewed and retrospectively collected by J.C and S.R. The epidemiological, demographic, clinical, and laboratory data on admission and during treatment were extracted using a standardized data form. Epidemiological and demographic data included age, sex, profession, and comorbidities (malignant tumors, congestive heart failure, cerebrovascular disease, kidney disease, and liver disease). Clinical profile of the COVID-19 patients included mental status, heart rate, respiratory rate, blood pressure, body temperature, and radiographic signs of pleural effusion. Laboratory findings included arterial pH, blood urea nitrogen, glucose, hematocrit, and partial pressure of arterial O₂ within 24 h after COVID-19 diagnosis.

Two physicians (G.L and R.C) reviewed the chest CT images independently. In cases of disagreement, a consensus was reached after discussion with L.C, a senior respiratory physician and H.L, a senior radiologist. We contacted attending doctors and other healthcare providers when data were missing or questionable. Two clinicians (J.C and S.R) assessed the PSI,¹⁶ CURB-65, and APACHE-II scales independently. A consensus was reached by discussion when a disagreement occurred.

PSI is a scale containing 19 different variables, 91 points (risk classes IV–V) is the cut-off value for PSI,¹⁷ which means a poor prognosis. We defined a cutoff value ≥91 for severe COVID-19 pneumonia. CURB-65 includes five variables (confusion, BUN >7 mmol/l, respiratory rate ≥30 bpm, systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg, and age ≥65 years). Three or more positive variables indicate a poor prognosis.¹⁸ We defined a cutoff value ≥3 for severe COVID-19 pneumonia. APACHE II score is widely applied in ICU. A score ≥15 indicates critically ill patients. In the study of Huang et al.,¹⁹ APACHE II scores at cutoff values ≥10 achieved a higher accuracy in predicting mortality. We divided the APACHE II score into three levels (<10, 10–14, and ⩾15).

Outcomes

The patients were divided into non-survivor group and survivor group. In addition, they were divided into non-severe group and severe group. The primary outcome of the study was severe COVID-19 defined as any of the following: respiratory rate >30 bpm; severe respiratory distress; or SPO₂ ⩽93% on room air, which was based on the guideline of Clinical Management of Severe Acute Respiratory Infection When COVID-19 is suspected published by WHO.²⁰ The secondary outcome was 30-day mortality. Patients who were discharged after recovery or transferred to another hospital or treated for more than 30 days were considered as survival.

Statistical analysis

We summarized data with mean value with standard deviations or median value with interquartile range, and categorical data as counts with percentages. We used the T-test or Mann-Whitney test to compare the differences in continuous variables, and the Chi-square test or Fisher’s exact test to compare the differences in categorical variables where appropriate. To compare the consistency of the three scales, we divided the scales into three levels of mild, medium, and severe (Supplemental Table S2). Kappa test was performed to compare the consistency of the three scales. The receiver operating characteristics (ROC) and the area under the curve (AUC) were described to assess the association between the severity of COVID-19 and three scales.

Results

A total of 167 patients with confirmed COVID-19 pneumonia were included in this retrospective study. The difference in the components of three scales between non-survivors and survivors, and non-severe patients and severe patients are shown in Table 1. The overall 30-day mortality rate was 3.59% (6/167). The non-survivors were more likely to have previous cerebrovascular disease, lower systolic pressure, lower arterial PH, lower partial pressure of arterial O₂, and blood urea nitrogen ⩾7 mmol/l. Compared to patients with non-severe COVID-19, patients with severe COVID-19 were more likely to have previous cerebrovascular disease, higher heart rate, higher breathing rate, lower systolic pressure, lower arterial pH, lower partial pressure of arterial O₂, and blood urea nitrogen ⩾7 mmol/l.

Table 1.

The difference in the components of three scales between non-survivors and survivors, non-severe patients and severe patients.

Characteristic	All Patients (N = 167)	Outcomes		P-value	Disease severity		P-value
Characteristic	All Patients (N = 167)	Survival (N = 161)	Death (N = 6)	P-value	Non-severe (N=138)	Severe (N = 29)	P-value
Gender—no./total no. (%)
Male	84 (50.3)	78 (48.4)	6 (100.0)	0.028	64 (46.4)	20 (69.0)	0.04
Female	83 (49.7)	83 (51.6)	0 (0.0)	0.028	74 (53.6)	9 (31.0)	0.04
Age—no./total no. (%)
⩾65	106 (63.5)	101 (62.7)	5 (83.3)	0.417	82 (59.4)	24 (82.8)	0.02
<65	61 (36.5)	60 (37.3)	1 (16.7)	0.417	56 (40.6)	5 (17.2)	0.02
Tumor disease—no./total no. (%)
Yes	12 (7.2)	12 (7.5)	0 (0.0)	>0.999	9 (6.5)	3 (10.3)	0.439
No	155 (92.8)	149 (92.5)	6 (100.0)	>0.999	129 (93.5)	26 (89.7)	0.439
Digestive diseases—no./total no. (%)
Yes	6 (3.6)	5 (3.1)	1 (16.7)	0.2	3 (2.2)	3 (10.3)	0.066
No	161 (96.4)	156 (96.9)	5 (83.3)	0.2	135 (97.8)	26 (89.7)	0.066
Congestive heart failure—no./total no. (%)
Yes	7 (4.2)	6 (3.7)	1 (16.7)	0.23	4 (2.9)	3 (10.3)	0.101
No	160 (95.8)	155 (96.3)	5 (83.3)	0.23	134 (97.1)	26 (89.7)	0.101
Cerebrovascular disease—no./total no. (%)
Yes	60 (35.9)	55 (34.2)	5 (83.3)	0.023	42 (30.4)	18 (62.1)	0.002
No	107 (64.1)	106 (65.8)	1 (16.7)	0.023	96 (69.6)	11 (37.9)	0.002
Kidney disease—no./total no. (%)
Yes	11 (6.6)	10 (6.2)	1 (16.7)	0.34	5 (3.6)	6 (20.7)	0.004
No	156 (93.4)	151 (93.8)	5 (83.3)	0.34	133 (96.4)	23 (79.3)	0.004
Abnormal mental state—no./total no. (%)
Yes	13 (7.8)	8 (5.0)	5 (83.3)	<0.001	5 (3.6)	8 (27.6)	<0.001
No	154 (92.2)	153 (95.0)	1 (16.7)	<0.001	133 (96.4)	21 (72.4)	<0.001
Heart rate ⩾125 bpm/min—no./total no. (%)
Yes	5 (3.0)	4 (2.5)	1 (16.7)	0.169	1 (0.7)	4 (13.8)	0.003
No	162 (97.0)	157 (97.5)	5 (83.3)	0.169	137 (99.3)	25 (86.2)	0.003
Breathing ⩾30/min—no./total no. (%)
Yes	10 (6.0)	8 (5.0)	2 (33.3)	0.043	2 (1.4)	8 (27.6)	<0.001
No	157 (94.0)	153 (95.0)	4 (66.7)	0.043	136 (98.6)	21 (72.4)	<0.001
Systolic pressure <90 mmHg—no./total no. (%)
Yes	2 (1.2)	0 (0.0)	2 (33.3)	0.001	0 (0.0)	2 (6.9)	0.029
No	165 (98.8)	161 (100.0)	4 (66.7)	0.001	138 (100.0)	27 (93.1)	0.029
Body temperature <35°C or ⩾40°C—no./total no. (%)
Yes	0	0	0	—	0	0	—
No	167 (100.0)	161 (100.0)	6 (100.0)	—	138 (100.0)	29 (100.0)	—
Arterial pH <7.35—no./total no. (%)
Yes	2 (1.2)	0 (0.0)	2 (33.3)	0.001	0 (0.0)	2 (6.9)	0.029
No	165 (98.8)	161 (100.0)	4 (66.7)	0.001	138 (100.0)	27 (93.1)	0.029
Pleural effusion—no./total no. (%)
Yes	15 (9.0)	15 (9.3)	0 (0.0)	>0.999	12(8.7)	3 (10.3)	0.727
No	152 (91.0)	146 (90.7)	6 (100.0)	>0.999	126(91.3)	26 (89.7)	0.727
Blood sodium <130 mmol/l—no./total no. (%)
Yes	2 (1.2)	1 (0.6)	1 (16.7)	0.071	1 (0.7)	1 (3.4)	0.318
No	165 (98.8)	160 (99.4)	5 (83.3)	0.071	137 (99.3)	28 (96.6)	0.318
Blood glucose ⩾14 mmol/l—no./total no. (%)
Yes	12 (7.2)	12 (7.5)	0 (0.0)	>0.999	8 (5.8)	4 (13.8)	0.226
No	155 (92.8)	149 (92.5)	6 (100.0)	>0.999	130 (94.2)	25 (86.2)	0.226
Hematocrit <30%—no./total no. (%)
Yes	15 (9.0)	15 (9.3)	0 (0.0)	>0.999	11 (8.0)	4 (13.8)	0.299
No	152 (91.0)	146 (90.7)	6 (100.0)	>0.999	127 (92.0)	25 (86.2)	0.299
Partial pressure of arterial O2 <60 mmHg—no./total no. (%)
Yes	4 (2.4)	2 (1.2)	2 (33.3)	0.006	1 (0.7)	3 (10.3)	0.017
No	163 (97.6)	159 (98.8)	4 (66.7)	0.006	137 (99.3)	26 (89.7)	0.017
Blood urea nitrogen ⩾9 mmol/l—no./total no. (%)
Yes	8 (4.8)	5 (3.1)	3 (50.0)	0.001	3 (2.2)	5 (17.2)	0.004
No	159 (95.2)	156 (96.9)	3 (50.0)	0.001	135 (97.8)	24 (82.8)	0.004
Blood urea nitrogen ⩾7 mmol/l—no./total no. (%)
Yes	16 (9.6)	11 (6.8)	5 (83.3)	<0.001	6 (4.3)	10 (34.5)	<0.001
No	151 (90.4)	150 (93.2)	1 (16.7)	<0.001	132 (95.7)	19 (65.5)	<0.001
PSI score—Median [IQR]	69.00 [52.50, 87.00]	66.00 [52.00, 84.00]	154.50 [138.50, 168.25]	<0.001	62.50 [50.25, 80.00]	115.00 [81.00, 134.00]	<0.001
CURB-65 score—Median [IQR]	1.00 [0.00, 1.00]	0.00 [0.0, 1.00]	3.00 [3.00, 3.75]	<0.001	0.00 [0.00, 1.00]	2.00 [1.00, 2.00]	<0.001
APACHE-II score—Median [IQR]	5.00 [3.00, 6.00]	5.00 [3.00, 6.00]	19.00 [16.75, 19.75]	<0.001	4.00 [3.00, 5.00]	9.00[5.00, 12.00]	<0.001

Clinical characteristics and laboratory findings of patients with COVID-19 pneumonia are shown in Table 2. Primary onset symptoms were similar between survivors and non-survivors, and between patients with severe disease and non-severe disease. Compared to survivors, non-survivors were more likely to have higher blood urea nitrogen (mmol/l) (9.16 [7.35, 11.25] vs 4.30 [3.45, 5.40], P < 0.001) and lower lymphocyte count (×10⁹) (0.30 [0.18, 0.68] vs 1.35 [0.93, 1.68], P = 0.003). Similarly, patients with severe COVID-19 were more likely to have lower blood lymphocyte count (×10⁹) (0.85 [0.64, 1.27] vs 1.37 [1.06, 1.72], P < 0.001), higher serum interleukin-6 levels (pg/ml) (15.22 [11.19, 29.62] vs 6.41 [4.51, 13.76], P < 0.001), and higher serum interleukin-10 levels (pg/ml) (4.61 [3.90, 5.58] vs 4.20 [3.41, 4.87], P = 0.013) compared to the non-severe patients (Table 2).

Table 2.

Clinical characteristics and laboratory findings of patients with COVID-19 pneumonia.

Characteristic	All Patients (N = 167)	Outcomes		p-Value	Disease severity		p-Value
Characteristic	All Patients (N = 167)	Survival (N = 161)	Death (N = 6)	p-Value	Non-severe (N = 138)	Severe (N = 29)	p-Value
Respiratory diseases—no./total no. (%)
Yes	14 (8.4)	13 (8.1)	1 (16.7)	0.414	10 (7.2)	4 (13.8)	0.269
No	153 (91.6)	148 (91.9)	5 (83.3)	0.414	128 (92.8)	25 (86.2)	0.269
Hypertension—no./total no. (%)
Yes	50 (29.9)	48 (29.8)	2 (33.3)	>0.999	37 (26.8)	13 (44.8)	0.073
No	117 (70.1)	113 (70.2)	4 (66.7)	>0.999	101 (73.2)	16 (55.2)	0.073
Diabetes—no./total no. (%)
Yes	32 (19.2)	32 (19.9)	0 (0.0)	0.597	25 (18.1)	7 (24.1)	0.444
No	135 (80.8)	129 (80.1)	6 (100.0)	0.597	113 (81.9)	22 (75.9)	0.444
Fever—no./total no. (%)
Yes	116 (69.5)	114 (70.8)	2 (33.3)	0.071	96 (69.6)	20 (69.0)	>0.999
No	51 (30.5)	47 (29.2)	4 (66.7)	0.071	42 (30.4)	9 (31.0)	>0.999
Cough—no./total no. (%)
Yes	109 (65.3)	106 (65.8)	3 (50.0)	0.419	90 (65.2)	19 (65.5)	>0.999
No	58 (34.7)	55 (34.2)	3 (50.0)	0.419	48 (34.8)	10 (34.5)	>0.999
Expectoration—no./total no. (%)
Yes	23 (13.8)	22 (13.7)	1 (16.7)	0.595	21 (15.2)	2 (6.9)	0.374
No	144 (86.2)	139 (86.3)	5 (83.3)	0.595	117 (84.8)	27 (93.1)	0.374
Dyspnea—no./total no. (%)
Yes	41 (24.6)	38 (23.6)	3 (50.0)	0.159	29 (21.0)	12 (41.4)	0.031
No	126 (75.4)	123 (76.4)	3 (50.0)	0.159	109 (79.0)	17 (58.6)	0.031
Aspartate aminotransferase (U/l)—Median [IQR]	28.00 [21.00, 37.00]	28.00 [21.00, 37.00]	52.00 [44.00, 57.00]	0.022	27.00 [21.00, 36.00]	35.50 [24.75, 53.25]	0.011
Alanine aminotransferase (U/l)—Median [IQR]	37.40 [33.70, 40.90]	37.60 [34.08, 40.92]	32.90 [29.00, 33.30]	0.012	38.30 [34.90, 41.10]	32.55 [29.62, 36.25]	<0.001
Albumin (g/l)—Median [IQR]	25.00 [19.00, 48.50]	25.00 [19.00, 47.00]	59.50 [27.25, 73.00]	0.219	25.00 [18.00, 44.75]	35.00 [21.00, 61.00]	0.096
Neutrophil count (×10⁹)—Median [IQR]	3.33 [2.55, 4.77]	3.31 [2.55, 4.73]	9.64 [3.01, 11.37]	0.098	3.25 [2.58, 4.35]	3.50 [2.32, 7.18]	0.342
Lymphocyte count (×10⁹)—Median [IQR]	1.35 [0.92, 1.67]	1.35 [0.93, 1.68]	0.30 [0.18, 0.68]	0.003	1.37 [1.06, 1.72]	0.85 [0.64, 1.27]	<0.001
Leukocyte count (×10⁹)—Median [IQR]	5.25 [4.34, 7.19]	5.24 [4.34, 7.05]	12.18 [9.97, 13.58]	0.016	5.27 [4.38, 7.04]	5.09 [3.91, 9.70]	0.801
Monocytes count (×10⁹)—median [IQR]	0.48 [0.36, 0.62]	0.48 [0.36, 0.60]	0.59 [0.30, 0.70]	0.821	0.48 [0.36, 0.60]	0.44 [0.35, 0.65]	0.967
Hemoglobin (g/l)—median [IQR]	123.53 (15.98)	123.60 (16.15)	121.67 (11.55)	0.772	125.12 (14.91)	116.00 (18.87)	0.005
Blood platelet (×10⁹)—mean (SD)	215.00 [166.00, 272.00]	219.00 [173.00, 274.00]	129.50 [99.00, 142.75]	0.001	223.00 [179.00, 274.75]	164.00 [128.00, 217.00]	0.003
Creatinine (mmol/l)—median [IQR]	74.00 [64.00, 87.00]	74.00 [64.00, 87.00]	117.50 [62.50, 261.75]	0.249	73.00 [64.00, 86.00]	84.00 [67.50, 111.00]	0.047
Blood urea nitrogen (mmol/l)—median [IQR]	4.40 [3.50, 5.50]	4.30 [3.45, 5.40]	9.16 [7.35, 11.25]	<0.001	4.30 [3.40, 5.40]	5.15 [4.02, 8.17]	0.017
Glomerular filtration rate—median [IQR]	80.40 [69.20, 94.90]	80.80 [70.20, 94.75]	57.70 [23.97, 112.88]	0.361	80.80 [71.00, 94.90]	74.05 [46.58, 91.65]	0.12
IL-6 (pg/ml)—median [IQR]	7.38 [4.76, 17.23]	7.14 [4.72, 17.05]	17.04 [14.04, 246.06]	0.08	6.41 [4.51, 13.76]	15.22 [11.19, 29.62]	<0.001
IL-4 (pg/ml)—median [IQR]	2.75 [2.10, 3.48]	2.83 [2.11, 3.51]	2.05 [2.01, 2.21]	0.159	2.84 [2.11, 3.58]	2.41 [2.06, 3.08]	0.349
IL-10 (pg/ml)—median [IQR]	4.29 [3.44, 5.01]	4.26 [3.43, 5.00]	5.14 [4.89, 10.38]	0.066	4.20 [3.41, 4.87]	4.61 [3.90, 5.58]	0.013
IL-2 (pg/ml)—median [IQR]	3.14 [2.67, 3.98]	3.18 [2.67, 4.04]	2.81 [2.67, 2.81]	0.259	3.27 [2.70, 3.98]	2.84 [2.50, 4.15]	0.444
TNF-α (pg/ml)—median [IQR]	2.88 [2.25, 5.21]	2.89 [2.25, 5.27]	2.27 [2.15, 2.70]	0.265	2.88 [2.29, 5.25]	2.91 [2.12, 4.55]	0.682
IFN-γ (pg/ml)—median [IQR]	2.79 [2.21, 3.56]	2.83 [2.21, 3.60]	2.21 [2.12, 2.46]	0.259	2.79 [2.23, 3.54]	2.64 [2.05, 3.79]	0.938
Cytokines (pg/ml)—Median [IQR]	24.00 [18.00, 31.00]	24.50 [18.00, 31.00]	18.00 [13.50, 25.50]	0.404	25.00 [18.50, 31.50]	22.00 [16.50, 29.00]	0.237

We found a significant difference in the distribution of the three scales in patients with different outcomes and disease severity (P < 0.001, Table 3). No patients with PSI class ⩽III, CURB-65 ⩽1, and APACHE-II-I died, while 1 (16.7%) and 3 (83.3%) patients with PSI-IV and PSI-V died, respectively. The mortality rate in patients with a CURB-65 score of 2 and ⩾3 was 16.7% (1/6) and 83.3% (5/6), respectively. For patients with APACHE-II class II and class III the mortality rates were 16.7% (1/6) and 83.3% (5/6), respectively.

Table 3.

Distribution of three scales in patients with different outcomes and disease severity.

Scales	Outcomes (n/%)		P-value*	Disease severity (n/%)		P-value*
Scales	Survival (N = 161)	Death (N = 6)	P-value*	Non-severe (N = 138)	Severe (N = 29)	P-value*
PSI class			<0.001			<0.001
I/II‡	88 (54.7)	0 (0.0)		83 (60.1)	5 (17.2)
III	43 (26.7)	0 (0.0)		40 (29.0)	3 (10.3)
IV	26 (16.1)	1 (16.7)		14 (10.1)	13 (44.8)
V	4 (2.5)	5 (83.3)		1 (0.7)	8 (27.6)
CURB-65 score			<0.001			<0.001
0	83 (51.6)	0 (0.0)		77 (55.8)	6 (20.7)
1	62 (38.5)	0 (0.0)		56 (40.6)	6 (20.7)
2	14 (8.7)	1 (16.7)		5 (3.6)	10 (34.5)
⩾3	2 (1.2)	5 (83.3)		0 (0.0)	7 (24.1)
APACHE-II class			<0.001			<0.001
I	153 (95.0)	0 (0.0)		135 (97.8)	18 (62.1)
II	7 (4.3)	1 (16.7)		3 (2.2)	5 (17.2)
III	1 (0.6)	5 (83.3)		0 (0.0)	6 (20.7)

Fisher’s exact test.

‡

the number of cases with PSI-I was 0.

PSI and CURB-65 had substantial consistency in the evaluation of patients’ disease severity (Kappa = 0.69, P < 0.001) (Table 4). However, PSI and APACHE-II had moderate consistency (Kappa = 0.54, P < 0.001). During the matching process, the results revealed that 5 (83.3%) patients who died during hospitalization had higher scores on all three scale systems.

Table 4.

Comparison of three scales on the severity of the patients with COVID-19 pneumonia.

Scales	PSI			Kappa
Scales	Mild	Moderate	Severe	Kappa
CURB-65				0.69
Mild	129	15	1
Medium	2	12	1
Severe	0	0	7
APACHE-II				0.54
Mild	130	21	2
Medium	1	6	1
Severe	0	0	6

The performance of PSI, CURB-65, and APACHE-II scales to assess the severity of COVID-19 pneumonia are shown in Table 5. PSI had higher sensitivity but lower specificity than CURB-65. APACHE-II had the lowest sensitivity. The highest Youden index was found at the cut off of IV in PSI (0.62), ⩾2 in CURB-65 (0.55), and ⩾II in APACHE-II (0.36). The receiver operating characteristic (ROC) curves of the three scales are shown in Figure 2. The areas under the curve in the ROC analysis of PSI, CURB-65, and APACHE-II were 0.83 (95% CI, 0.74–0.93), 0.80 (95% CI, 0.69–0.90), and 0.83 (95% CI, 0.75–0.92), respectively.

Table 5.

The ability of three scales to evaluate the severity of COVID-19 pneumonia.

Scales	Sensitivity (%)	Specificity (%)	+LR	−LR	YI
PSI
⩾III	82.8 (63.5, 93.5)	60.1 (51.4, 68.3)	2.08	0.29	0.43
⩾IV	72.4 (52.5, 86.6)	89.1 (82.4, 93.6)	6.66	0.31	0.62
V	27.6 (13.4, 47.5)	99.3 (95.4, 100.0)	38.07	0.73	0.27
CURB-65
⩾1	79.3 (59.7, 91.3)	55.8 (47.1, 64.2)	1.79	0.37	0.35
⩾2	58.6 (39.1, 75.9)	96.4 (91.3, 98.7)	16.18	0.43	0.55
⩾3	24.1 (11.0, 43.9)	100.0 (96.6, 100.0)		0.76	0.24
APACHE-II
⩾II	37.9 (21.3, 57.6)	97.8 (93.3, 99.4)	17.45	0.63	0.36
III	20.7 (8.7, 40.2)	100.0 (96.6, 100.0)		0.79	0.21

LR: positive likelihood ratio; −LR: negative likelihood ratio; YI: Youden index = (sensitivity + specificity) −1.

Figure 2.

Receiver operator characteristic curve (ROC) and area under the curves (AUC) for the severity of COVID-19 pneumonia patient for three scales.

Discussion

In the present study, we found a significant difference in the distribution of the scores of the PSI, CURB-65, and APACHE-II scales in patients with different outcomes and disease severity. Moreover, PSI and CURB-65 both performed well in the evaluation of patients’ disease severity (Kappa = 0.69, P < 0.001). Our results are in consistency with those reported by Artero et al.⁹ and Satici et al.¹³ Thus our findings suggest that PSI and CURB-65 might be useful to predict the severity and mortality of COVID-19 pneumonia.

A previous study showed that patients with COVID-19 pneumonia who had compromised respiratory status on admission were at a higher risk of developing severe disease and worse outcomes.²¹ Identifying the seriousness of COVID-19 is essential for clinicians to make decisions whether or not to take more intensive supportive treatment for the patients.²² PSI, CURB-65, and APACHE-II scales have been well certified and widely applied for assessing pneumonia and general critical illness.^7,19,23 In our study, we found that no patients with PSI class ⩽III (131/167, 78.4%), CURB-65 ⩽1 (145/167, 86.8%), and APACHE-II-I (153/167, 91.6%) died, the mortality rate in patients with PSI class IV, CURB-65 = 2, and APACHE-II class II is 16.7%, and the mortality rate in patients with PSI class V, CURB-65≥3, and APACHE-II class III is 83.3%. There was a significant trend of increasing mortality risk with a higher PSI, CURB-65, and APACHE-II risk class. Thus, our findings suggest that these three scales are useful tools to predict the mortality in patients of COVID-19 pneumonia, and are consistent with previous findings. For example, the PSI and CURB-65 scales could identify low-risk patients with COVID-19 pneumonia.²⁴ Moreover, the APACHE-II score of the survivors was significantly lower than that of non-survivors.¹⁹ Therefore, the three scales could be reliably applied to identify low-risk and high-risk patients.

The AUCs of PSI, CURB-65, and APACHE-II were 0.83 (95% CI, 0.74–0.93), 0.80 (95% CI, 0.69–0.90), and 0.83 (95% CI, 0.75–0.92), respectively, indicating that the three scales are good predictive tools of the severity of COVID-19 pneumonia. The sensitivity of PSI was higher than CURB-65 and APACHE-II, indicating it could more correctively sort out patients who would develop into severe COVID-19 pneumonia, but the specificity was lower than CURB-65, which means that more non-high-risk patients would be falsely predicted to be severe.

The sensitivity of APACHE-II class ⩾II was only 37.9 (21.3, 57.6). However, the sensitivity of severity assessment for patients with COVID-19 pneumonia is extremely important, so APACHE-II should be used with caution. The result is different from findings reported by Cheng et al.²⁵ who found that the APACHE-II score was a strong predictor of COVID-19 pneumonia severity and mortality, They included a total of 53 cases. We think APACHE-II score is unsuitable for COVID-19 pneumonia for the following reasons. First, the score is complex, contains many variables and is not easy to manipulate. Second, it is no specific for respiratory system, respiratory failure is the hallmark for COVID-19 pneumonia patients, even absence of accompanying circulatory failure.²⁶ Third, the proportion of scores in elderly patients is high. Finally, COVID-19 patients are accompanied with various comorbidities, however, there is a lack of scoring for comorbidities in APACHE-II score.²⁷

Five of the six non-survivors included in this study had the highest levels in all three scales (PSI-V, CURB-65 score ⩾3, and APACHE-II-III). However, no patients with PSI class ⩽III (131/167, 78.4%), CURB-65 ⩽1 (145/167, 86.8%), and APACHE-II-I (153/167, 91.6%) died. Therefore, all three scales performed well in evaluating the mortality of mild and severe patients. A previous study showed a combination of PSI and CURB-65 scales could reliably access CAP.²⁸ Moreover, PSI and CURB-65 scales were almost equally sensitive and specific for predicting mortality among hospitalized patients with CAP with PSI class ⩾IV and CURB-65 score ⩾2.²⁹ Similarly our results showed that PSI is in good consistency with CURB-65.

Due to a very small sample of death in our study, the sensitivity and specificity of the three scales can’t be further compared to predict the mortality of patients with COVID-19 pneumonia. Notably, one non-survivor who was 85 years old, with a previous cerebrovascular disease and abnormal mental state during hospitalization, was rated as medium (PSI-IV, CURB-65 of 2, and APACHE-II-II) on all three scales. This example suggests we should pay more attention and vigilance to old COVID-19 patients with underlying chronic diseases.

There were some limitations in the present study. This is a single-center, retrospective study with small sample size. Our findings need to be validated in further multicenter larger studies. Due to retrospective nature of the present study, we did not have a predefined statistics protocol to calculate the sample size. The parameters of these scales were only a binary variable. They did not fully reflect the severity of COVID-19 pneumonia. Also, we should consider other factors for COVID-19, such as D-dimer, body mass index, IL-6 levels, which are not included in PSI, CURB-65, APACHE-II. In the study by Zhou et al.,³⁰ D-dimer levels >1 μg/ml were shown to be associated with higher mortality (OR 18.42 (95% CI 2.64–128.55)). In a retrospective study by Nadkarni et al.,³¹ which included 4389 hospitalized patients with COVID-19, anticoagulantion resulted in lower mortality and intubation. Finally, the effects of treatments on mortality were not analyzed.

Conclusion

In conclusion, our study provided a preliminary assessment tool of the severity of COVID-19 pneumonia patients using PSI, CURB-65, APACHE-II scales. Our findings suggest that PSI and CURB-65 are useful to assess the severity and mortality of COVID-19 pneumonia; these scales might be used by clinicians to better manage their patients with COVID-19 pneumonia. More external validation studies are needed to verify our findings.

Supplemental Material

sj-docx-1-eji-10.1177_20587392211027083 – Supplemental material for Performance of CURB-65, PSI, and APACHE-II for predicting COVID-19 pneumonia severity and mortality

Supplemental material, sj-docx-1-eji-10.1177_20587392211027083 for Performance of CURB-65, PSI, and APACHE-II for predicting COVID-19 pneumonia severity and mortality by Junnian Chen, Bang Liu, Houwei Du, Hailong Lin, Cunrong Chen, Shanshan Rao, Ranjie Yu, Jingjing Wang, Zhiqiang Xue, Yixian Zhang and Yanghuang Xie in European Journal of Inflammation

Footnotes

Acknowledgements

We thank all the patients who consented to donate their data for analysis and the medical staff members who are on the front line of caring for patients. We thank Dr. Lianming Liao at the Center of Laboratory Medicine, Union Hospital and Dr. Jin Wei from School of Health Sciences, College of Health and Medicine, University of Tasmania, Australia, for their help in language improvement.

Author contributions

J.C, B.L, and H.D wrote original draft and collected the references. L.H, C.C, S.R, R.Y, J.W, Z.X, Y.Z, and Y.X provided clinical support. H.D and C.C helped to revise the manuscript. Dr. H Du was listed as a co-first author for his substantial contribution to manuscript critical revision, data analysis, interpretation, and drafting responses to reviewers. All co-authors agreed and approved Dr Du’s contribution and authorship. Dr. H Du approved the version to be published.

Availability of data and material

The data in the present study is not publicly accessible. Qualified researchers who are interested in obtaining the data can contact the corresponding author (Email: chenjunnian2020@outlook.com).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Natural Science Foundation of Fujian, China (grant numbers.2018J01309).

Ethical approval

Our study was conducted in line with the Declaration of Helsinki. The Fujian Medical University Union Hospital Ethics Committee approved the study protocol (2020XGFKY002), and written informed consent was obtained from the patients for their anonymized information to be published in this article.

ORCID iD

Junnian Chen

Code availability Statistical

Analysis of data was performed on R software, version 3.6.3 (R Foundation for Statistical Computing, AT&T Bell Laboratories).

Supplemental material

Supplemental material for this article is available online.

References

Zhao

, et al. (2020) Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. Lancet 395: 565-74.

Onder

Rezza

Brusaferro

(2020) Case-fatality rate and characteristics of patients dying in relation to COVID-19 in Italy. JAMA 323(18): 1775–1776.

World Health Organisation (2020) Rolling updates on coronavirus disease (COVID-19). https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen (accessed 31 January 2020).

World Health Organisation. (2021).WHO coronavirus disease (COVID-19) dashboard. https://covid19.who.int/ (accessed 1 June 2021).

Ilg

Moskowitz

Konanki

, et al. (2019) Performance of the CURB-65 score in predicting critical care interventions in patients admitted with community-acquired pneumonia. Ann Emerg Med 74: 60–68.

Murillo-Zamora

Medina-Gonzalez

Zamora-Perez

, et al. (2018) Performance of the PSI and CURB-65 scoring systems in predicting 30-day mortality in healthcare-associated pneumonia. Med Clin (Barc) 150: 99–103.

Lim

Eerden

Laing

, et al. (2003) Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax 58: 377–382.

Knaus

Draper

Wagner

, et al. (1985) APACHE II: a severity of disease classification system. Crit Care Med 13: 818–829.

Artero

Madrazo

Fernández-Garcés

, et al. (2021) Severity scores in COVID-19 pneumonia: a multicenter, retrospective, cohort study. J Gen Intern Med 36(5): 1338–1345.

10.

Myrstad

Ihle-Hansen

Tveita

, et al. (2020) National early warning score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality form COVID-19. A prospective cohort study. Scan J Traum Res Emerg Med 28: 66.

11.

Zhang

, et al. (2020) Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score. Clin Infect Dis 71(6): 1393–1399.

12.

Ihle-Hansen

Berge

Tveita

, et al. (2020) COVID-19: symptoms, course of illness and use of clinical scoring systems for the first 42 patients admitted to a Norwegian local hospital. Tidsskr Nor Laegeforen 140(7): tidsskr.20.0301.

13.

Satici

Demirkol

Sargin Altunok

, et al. (2020) Performance of pneumonia severity index and CURB-65 in predicting 30-day mortality in patients with COVID-19. Int J Infect Dis 98: 84–89.

14.

Mao

Jin

Wang

, et al. (2020) Neurologic manifestations of hospitalized patients with coronavirus disease 2019 in Wuhan, China. JAMA Neurol 77(6): 683–690.

15.

National Health Commission of the People’s Republic of China Diagnosis and Treatment Protocol for COVID-19 (trial version 5, revised version, 4 February 2020). http://www.nhc.gov.cn/yzygj/s7653p/202002/d4b895337e19445f8d728fcaf1e3e13a.shtml (accessed 5 February 2020).

16.

Marras

Gutierrez

Chan

(2000) Applying a prediction rule to identify low-risk patients with community-acquired pneumonia. Chest 118: 1339–1343.

17.

Fine

Hough

Medsger

, et al. (1997) the hospital admission decision for patients with community acquired pneumonia. Arch Intern Med 157: 36–44.

18.

Ranzani

Prina

Menéndez

, et al. (2017) New sepsis definition (Sepsis-3) and community-acquired pneumonia mortality: a validation and clinical decision-making study. Am J Respir Crit Care Med 196(10): 1287–1297.

19.

Huang

Xuan

, et al. (2017) The value of APACHE II in predicting mortality after paraquat poisoning in Chinese and Korean population. Medicine 96(30): e6838.

20.

World Health Organisation (2020) Clinical management of severe acute respiratory infection when COVID-19 is suspected. https://www.who.int/publications-detail/clinical-management-of-severe-acute-respiratory-infection-when-novel-coronavirus-(ncov)-infection-is-suspected. (accessed 5 May 2020).

21.

Guan

, et al. (2020) China MTEG. Clinical characteristics of coronavirus disease 2019 in China. N Eng J Med 382: 1708–1720.

22.

Kolditz

Ewig

Klapdor

, et al. (2015) Community-acquired pneumonia as medical emergency: predictors of early deterioration. Thorax 70: 551–558.

23.

Fine

Auble

Yealy

, et al. (1997) A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 336: 243–250.

24.

Niederman

MS.

(2009) Making sense of scoring systems in community acquired pneumonia. Respirology 2009; 14: 327–335.

25.

Cheng

Yang

, et al. (2021) Pneumonia scoring systems for severe COVID-19: which one is better. Virol J 18(1): 33.

26.

Richardson

Hirsch

Narasimhan

, et al. (2020) Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA 323(20): 2052–2059.

27.

Guan

Liang

Zhao

, et al; China Medical Treatment Expert Group for COVID-19 (2020) Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis. Eur Respir J 55(5): 2000547.

28.

Niederman

Feldman

Richards

. (2006) Combining information from prognostic scoring tools for CAP: an American view on how to get the best of all worlds. Eur Respir J 27: 9–11.

29.

Varshochi

Kianmehr

Naghavi-Behzad

, et al. (2013) Correspondence between hospital admission and the pneumonia severity index (PSI), CURB-65 criteria and comparison of their predictive value in mortality and hospital stay. Infez Med 21: 103–110.

30.

Zhou

, et al. (2020) Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 395(10229): 1054–1062.

31.

Nadkarni

Lala

Bagiella

, et al. (2019) Anticoagulation, bleeding, mortality, and pathology in hospitalized patients with COVID-19. J Am Coll Cardiol 76(16): 1815–1826.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB