Practical Risk Scoring System for Predicting Severity of COVID-19 Disease

Key Points

Introduction

SARS-CoV-2, the viral agent that causes the COVID-19 disease, has—from its humble beginnings in Wuhan China—become an international pandemic. ¹ This novel virus initially presented and circulated in Wuhan, Hubei, China. ¹ Since the start of the pandemic in Wuhan, the clinical characteristics and risk factors for COVID-19 severity (such as mortality) had been studied for the affected Chinese civilian population. ¹ However, as aptly pointed out by Zhang et al, there are significant differences in the population, health service capacity, and population ethnic backgrounds that limit the generalizability of studies performed on the Chinese population to other populations. ² Although there have been multiple studies for risk scoring, the literature has been particularly sparse for the American veteran population.^3-10 Risk scoring population algorithms have been well-established based on the unique characteristics of the Chinese population or the European population, both of which is a different population from the American Veteran population for which the literature is sparse.^2,8–10 Studies performed on the population within the United States of America have generally consisted of observations of particular risk factors and its impact on severity.^11-13 Further meta-analyses and reviews have delineated risk factors predicting worse prognosis without a scoring system to stratify patient prognoses; these potential risk factors have been acknowledged as identified by the Centers for Disease Control and Prevention (CDC).^14-17 These risk factors have included laboratory parameters, such as Rahman et al.’s experience in Bangladesh with hematological abnormalities predicting COVID-19 severity. ¹⁷ Nonetheless, the development of a risk scoring system based on the multiple risk factors is sparse for the American Veteran population. As part of a unique study to explore the significantly affected American Veteran population, we developed the Jhala Risk Scoring System (JRSS) that helps in predicting prognosis in a regional veteran population; the JRSS is particularly useful and non-invasive as it can be utilized without invasive procedures, diagnostic imaging, or phlebotomy.

Methods

A retrospective review of all SARS-CoV-2 reverse transcriptase-polymerase chain reaction (RT-PCR) tests collected and performed at the regional Veterans Administration Medical Center (VAMC) serving the Philadelphia and surrounding areas from March 17th, 2020 to May 20th, 2020. Data was collected and analyzed in the same year. This review took place with the appropriate IRB approval at the institution. The results of these tests were reviewed and separated based on whether the test was positive (or detected) for SARS-CoV-2 or negative. As this study was for prognosis after infection with SARS-CoV-2, only positive test results were considered, and tests were further sorted by patient. As patients may have more than one positive test result, redundant test results were removed to ensure each patient was only reviewed once. A medical chart review was performed to document the patient’s relevant clinical and laboratory characteristics during the disease course. Clinical characteristics documented included age, gender, ethnicity, pulmonary related past medical history, cardiovascular related past medical history, diabetes history, smoking history, and clinical course during time of infection. Assessed also were whether certain laboratory parameters during the disease course exceeded set limits that had previously been documented to be associated with worse outcomes, specifically D-Dimer (>1 mg/L), C-reactive protein (>10 mg/dL), lactose dehydrogenase (>245 U/L), troponin (>2 times the upper limit of normal), ferritin (>500 ng/mL), creatine phosphokinase (>2x the upper limit of normal), and the absolute lymphocyte count (<800/microliter). Only laboratory parameters that had been obtained clinically during the patient’s disease course were considered.

The data was compiled and assessed per the JRSS (see Figure 1). The JRSS examines the patient’s age (1 point > 55 years of age, 2 points if >65 years of age), ethnicity (1 point if African-American), pulmonary past medical history (2 points if any past medical history exists), cardiovascular past medical history (1 point if just history of hypertension, 2 points if there is any other history), diabetes past medical history (1 point if history exists), and smoking history (2 points if any history of smoking tobacco exists). A score that is 7 or greater was deemed to be of higher risk. As laboratory data collected upon presentation may present additional information suggesting elevated risk, a laboratory addendum of additional points based on the laboratory parameters would be added for those with a score less than 7. Points to be added would range from 1 point (1 laboratory parameter exceeded set limit), 2 points (2 or 3 laboratory parameters exceeded set limits), or 3 points (4 or more laboratory parameters exceeded set limits).

OPEN IN VIEWER

Figure 1.

A flow chart of the Jhala Risk Scoring System (JRSS) starting first with clinical assessment of history. Some who score ⩾7 on history alone would be deemed elevated risk. For the rest, there is further points to be added based on laboratory parameters that may subsequently put the patient in the elevated risk category. Combined scores ⩾7 is elevated risk category, and those <7 would not be classified as elevated risk patients.

The performance of the JRSS was assessed by standard statistical analysis to determine the odds ratio as well as the 95% confidence interval of this odds ratio. ¹⁸ If the lower limit of the 95% confidence interval for the odds ratio is not below 1, the odds ratio would be considered statistically significant as the P value would be <.05; this was also confirmed by direct calculation of the P value for each finding. ¹⁹ In order to assess and verify the prognostic capability of the JRSS in predicting negative outcomes, the following negative outcomes were considered for the statistical calculation: hospitalization or requirement for medical monitoring, death, intubation during disease course, or admission to the intensive care unit (ICU).

Results

There was a total of 187 unique patients with positive SARS-CoV-2 test results were reviewed. Out of these 187 patients, 45 had a JRSS score (without considering laboratory data) of 7 or greater (Table 1). Twenty of these high score patients were admitted to the Medical Intensive Care Unit (MICU). Seven were admitted to the medical floor. Five continued to reside in a long-term care facility (community living center or CLC). Thirteen did not require hospital admission or required medical monitoring at all. Ultimately for those 45 patients with JRSS scores (without considering laboratory data) of 7 or greater, 9 patients expired (1 for reasons unrelated to COVID-19). Nine had been intubated at some point during their hospitalization (only 6 of the intubated patients had expired). One patient was admitted to a hospital outside of Veteran Affairs without corresponding hospitalization data available other than that the patient did not expire, and was thus excluded from the numbers for the statistical calculations for whether intubation was required or patient required ICU care due to these facts being unknown. One patient was already admitted to the surgical intensive care unit for unrelated reasons and developed COVID-19 midway through his treatment course; as this patient had not been admitted to the intensive care unit for reasons related to COVID-19, this patient was excluded from the statistics of those admitted to the ICU. Therefore, for negative outcomes of hospitalization or required medical monitoring and death, 187 patients were included; 186 were included for statistics on whether intubation was required; and 185 patients were included for statistics on whether the patient was admitted to the ICU.

OPEN IN VIEWER

Table 1.

A-B. Tabulated summary results comparing patients requiring hospitalization and risk scores. Those with risk scores ⩾7 had higher statistically significant odds of being hospitalized (odds ratio of 7.15 with 95% confidence interval of 3.63-14.08). One patient was admitted outside of the VA system, and thus was excluded from the intubation or admitted to ICU categories as this information was unknown. An additional 1 patient became positive after he was already in the surgical ICU for another medical reason and was thus excluded from the admitted to ICU calculations. Part A—JRSS with laboratory data consideration, B—JRSS without considering any available laboratory data.

Jhala Risk Scoring System (JRSS) statistics with laboratory data consideration
A		Hospitalized or required medical monitoring		Death		Intubation required		Admitted to ICU
		Yes	No	Yes	No	Yes	No	Yes	No
Risk score ⩾ 7	Yes	58	16	16	58	15	59	36	38
	No	38	75	7	106	4	108	19	92
Odds in high risk group			58/16 or 3.63	16/58 or 0.28		15/59 or 0.25		36/38 or 0.95
Odds in lower risk group			38/75 or 0.51	7/106 or 0.07		4/108 or 0.04		19/92 or 0.21
Odds Ratio			7.15	4.18		6.86		4.59
Upper limit of odds ratio (95% CI)			14.08	10.74		21.63		8.98
Lower limit of odds ratio (95% CI)			3.63	1.63		2.18		2.34
P-value			<.0001	.003		.001		<.0001
Sensitivity			60%	70%		79%		65%
Specificity			82%	65%		65%		71%
Jhala Risk Scoring System (JRSS) statistics if available laboratory data is ignored
B		Hospitalized or required medical monitoring		Death		Intubation required		Admitted to ICU
		Yes	No	Yes	No	Yes	No	Yes	No
Risk score ⩾ 7	Yes	32	13	9	36	9	36	20	25
	No	64	78	14	128	10	131	35	105
Odds in high risk group			32/13 or 2.46	9/36 or 0.25		9/36 or 0.25		20/25 or 0.8
Odds in lower risk group			64/78 or 0.82	14/128 or 0.11		10/131 or 0.076		35/105 or 0.33
Odds Ratio			3	2.29		3.275		2.4
Upper limit of odds ratio (95% CI)			6.19	5.71		8.67		4.84
Lower limit of odds ratio (95% CI)			1.45	0.92		1.24		1.19
P-value			.003	.38		.02		.01
Sensitivity			33%	39%		47%		36%
Specificity			86%	78%		78%		81%

Abbreviations: CI, confidence interval; JRSS, Jhala Risk Scoring System.

In contrast, 142 patients had a JRSS score of less than 7 (or 0-6) based on clinical history alone. As laboratory values at the time of the disease course would add additional information that may highlight at risk patients, a laboratory addendum was computed for each of these patients. Once the laboratory addendum was computed, an additional 29 patients had scores of 7 or greater (Table 1). Of these 29, 26 patients were either admitted to the intensive care unit (16 patients), medical floor (8 patients), or were being followed long term in the monitored CLC (2 patients). Seven expired (6 in MICU and 1 in CLC), of which 5 had been intubated prior to death. Of those who did not die, just one was intubated. Notably, the lower limit of the 95% confidence interval increased once the measured laboratory parameters were accounted for.

For those who had a lower score (6 or less), there were 113 patients (Table 1). The overwhelming majority of the patients in this category never required any hospitalization or medical monitoring (75 patients) for COVID-19. The rest, 38 patients, were monitored as part of their long-term care in the CLC (10 patients), were on the medical floor for monitoring (7 patients), and only 21 were in the intensive care unit. Of these patients, of which 3 were intubated (2 intubated patients expired), 7 patients expired during their disease process.

The JRSS identified a subset of patients who may be at higher risk of requiring hospitalization. Considering both the initial scoring and the laboratory addendum, 74 potentially higher risk patients (with a score > 7) had been identified, of which 58 required hospitalization or continued medical monitoring in the CLC, for an admission/monitoring rate of 78%, in contrast to those who did not score highly (only 33% admission monitoring rate). The odds ratio for an elevated score, calculated from the above data, is 7.15 with a 95% confidence interval ranging from 3.63 to 14.08; this indicates a statistically significant association between a score of ⩾7 and requirement for hospitalization or continued medical monitoring (see Table 1).^18-19 The JRSS with laboratory addendum was also successful in predicting death, requirement for intubation, and requirement to be admitted to an ICU (odds ratios of 4.18, 6.86, and 4.59, all statistically significant as in Table 1).

The JRSS without considering any available laboratory data demonstrated a continued ability to predict hospitalization, intubation, and ICU admission in a statistically significant manner, but the lower limit of the 95% confidence interval approached more closely to 1 for all markers of worse prognosis, indicating both that inclusion of the available laboratory data strengthened the algorithm and that the JRSS could be used without laboratory data (see Table 1). The inclusion of laboratory data strengthened the predictive value of the JRSS in all domains (predicting hospitalization or requirement for medical monitoring, intubation, death, and intensive care unit usage), though it is also noted that the performance of laboratory tests assessed by the JRSS was statistically linked to contact with the medical system due to COVID-19 (see Table 1).

Discussion

As previously have been described in the literature, such as in the well-established studies of the Chinese and European population, the cumulation of risk factors can be utilized to model prognosis and identify which patients may have worse or better prognosis after infection with SARS-CoV-2.^1-7 The JRSS system is based in part on the pathogenicity of the SARS-CoV-2 virus where the virus targets certain organ systems (past medical history for pulmonary or cardiovascular systems), and on the previously observed and known risk factors for severe disease.^1-13,20-22 Similarly, these risk scoring models, while they may help identify patients at higher risk of worse prognosis, do not completely replace clinical judgment of each individual case as in both instances, a minority of patients who score poorly do well and other patients who score well do poorly.^1-7 Nonetheless, the presence of a risk scoring algorithm appropriately attuned to a representative population group can be a tool to predict negative outcomes and help in the shepherding of healthcare resources in these pandemic times. Importantly, scoring systems, which have been utilized for many different areas of medicine, provide a formal standardized assessment that can supplement physician judgment; decisions based on probable prognosis does not always need to be based on an individual clinician’s gut feeling alone. ²³ The JRSS is particularly useful in that it does not rely on invasive procedural or diagnostic imaging for risk stratification, and can be performed with clinical information alone without any blood draws for laboratory data if needed. While some patients are not identified as higher risk if laboratory data is ignored, the algorithm still identifies in a statistically significant manner a subset of higher risk patients, which can still usefully guide clinical judgment. The JRSS normally relies on routinely collected laboratory tests and is thus a potentially useful tool in common practice. However, despite this usefulness, this type of risk scoring has noted only sparsely with the unique characteristics of the Veteran population of the United States. It is most important to note that populations differ in substantial ways including but not limited to ethnic composition, availability of healthcare, and other innumerable characteristics that may prevent generalizability from one country’s population to another; therefore, a risk scoring system based on a population in China or Europe may not necessarily apply to the American population on a different continent. ² In this attempt to apply a risk scoring system on a population within the United States, specifically the Veteran population, that is known to be different from the Chinese and European civilian populations, a successful segregation of higher risk patients was delineated that may help efforts to predict patient outcomes and triage medical resources. Furthermore, this study involved the more vulnerable Veteran population that has already been well documented to differ from the non-Veteran civilian American population.^24-26

The success of the algorithm also helps confirm prior studies demonstrating a higher risk of more severe outcomes with potential risk factors such as age, ethnic background, pre-existing conditions (such as cardiovascular disease, pulmonary disease, or diabetes), smoking habits, and known laboratory parameters that portend a worse prognosis and could be gathered at the time of the initial presentation or visit.^1-17 The association of worse outcomes with the variables accounted for by the JRSS makes sense from a pathogenicity and literature standpoint.^1-17 The variables accounted for have either been reported as risk factors in multiple prior studies or involve an organ system that would be directly affected by the SARS-CoV-2 infection.^1-17 One of the risk factors, ethnicity, may be a marker of worse prognosis due to its very strong and well documented non-biological association with social and health inequalities or discrimination within society at large. ¹³ On review of the data, there was a statistically significant association between the number of laboratory tests ordered within the JRSS and need for hospitalization or continued medical monitoring. These laboratory values, as markers of lymphopenia or inflammation have been strongly associated with worse outcomes with pathogenic reasons for this.^1-17,20-22 This published history therefore would tend to argue that there exists a true biologic reason for the association with worse prognosis with these laboratory parameters. While this study is limited in being a single regional VAMC institutional study involving only Veteran patients, this study is an important contribution to ensuring that risk scoring systems have been systematically studied in additional populations whereby it may be useful. Further directions suggested by this study may include looking at the broader US population, examining if the JRSS might be even more robust if the laboratory testing assessed was routinely performed on a higher proportion of patients with COVID-19, or even examining which laboratory parameters contribute most to an accurate algorithm.

Conclusion

In this very study of a comprehensive risk scoring system for a patient population with COVID-19 within the United States, undertaken in a regional VAMC, found that the JRSS, which assessed points based on age, ethnicity, pre-existing conditions, smoking habit, and available laboratory parameters had predictive value in determining which patients may be at elevated risk of hospitalization. The JRSS is simple enough that it does not require for its utilization any invasive procedures, diagnostic imaging, or even blood draws for laboratory studies; laboratory data on set parameters is simply incorporated into the algorithm if it is available. The determination early on by a standardized risk scoring system (based on patients in the United States) of which patients may require more extensive medical services may help clinicians implement better strategies for more effective use of limited medical resources and anticipate needs earlier in the patients’ course.