Practical Risk Scoring System for Predicting Severity of COVID-19 Disease

Abstract

Objectives:

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of COVID-19 disease, has become an international pandemic with numerous casualties. It had been noted that the severity of the COVID-19 disease course depends on several clinical, laboratory, and radiological factors. This has led to risk scoring systems in various populations such as in China, but similar risk scoring systems based on the American veteran population are sparse, particularly with the vulnerable Veteran population. As a simple risk scoring system would be very useful, we propose a simple Jhala Risk Scoring System (JRSS) to assess the severity of disease risk.

Methods:

Results:

The JRSS, based on age, ethnicity, presence of any lung disease, presence of cardiovascular disease, smoking history, and diabetes history with laboratory parameters correlated and predicted (with statistical significance) which patients would be hospitalized.

Conclusion:

The JRSS may play a role in informing which COVID-19 positive patients in the emergency room/urgent care for risk stratification.

Keywords

Clinical Prognosis SARS-CoV-2 evidence-based medicine risk scoring COVID-19 epidemiology quality assurance clinical pathology

Key Points

(1) SARS-CoV-2 has become a world-wide pandemic that has affected population groups throughout the world.

(2) Risk scoring systems, based on known associations of clinical characteristics and disease severity, may help in predicting prognosis.

(3) The JRSS was developed based on the American Veteran population in the major metropolitan city and surrounding areas of Philadelphia.

Introduction

SARS-CoV-2, the viral agent that causes the COVID-19 disease, has—from its humble beginnings in Wuhan China—become an international pandemic.¹ This novel virus initially presented and circulated in Wuhan, Hubei, China.¹ Since the start of the pandemic in Wuhan, the clinical characteristics and risk factors for COVID-19 severity (such as mortality) had been studied for the affected Chinese civilian population.¹ However, as aptly pointed out by Zhang et al, there are significant differences in the population, health service capacity, and population ethnic backgrounds that limit the generalizability of studies performed on the Chinese population to other populations.² Although there have been multiple studies for risk scoring, the literature has been particularly sparse for the American veteran population.^3-10 Risk scoring population algorithms have been well-established based on the unique characteristics of the Chinese population or the European population, both of which is a different population from the American Veteran population for which the literature is sparse.^2,8–10 Studies performed on the population within the United States of America have generally consisted of observations of particular risk factors and its impact on severity.^11-13 Further meta-analyses and reviews have delineated risk factors predicting worse prognosis without a scoring system to stratify patient prognoses; these potential risk factors have been acknowledged as identified by the Centers for Disease Control and Prevention (CDC).^14-17 These risk factors have included laboratory parameters, such as Rahman et al.’s experience in Bangladesh with hematological abnormalities predicting COVID-19 severity.¹⁷ Nonetheless, the development of a risk scoring system based on the multiple risk factors is sparse for the American Veteran population. As part of a unique study to explore the significantly affected American Veteran population, we developed the Jhala Risk Scoring System (JRSS) that helps in predicting prognosis in a regional veteran population; the JRSS is particularly useful and non-invasive as it can be utilized without invasive procedures, diagnostic imaging, or phlebotomy.

Methods

A retrospective review of all SARS-CoV-2 reverse transcriptase-polymerase chain reaction (RT-PCR) tests collected and performed at the regional Veterans Administration Medical Center (VAMC) serving the Philadelphia and surrounding areas from March 17th, 2020 to May 20th, 2020. Data was collected and analyzed in the same year. This review took place with the appropriate IRB approval at the institution. The results of these tests were reviewed and separated based on whether the test was positive (or detected) for SARS-CoV-2 or negative. As this study was for prognosis after infection with SARS-CoV-2, only positive test results were considered, and tests were further sorted by patient. As patients may have more than one positive test result, redundant test results were removed to ensure each patient was only reviewed once. A medical chart review was performed to document the patient’s relevant clinical and laboratory characteristics during the disease course. Clinical characteristics documented included age, gender, ethnicity, pulmonary related past medical history, cardiovascular related past medical history, diabetes history, smoking history, and clinical course during time of infection. Assessed also were whether certain laboratory parameters during the disease course exceeded set limits that had previously been documented to be associated with worse outcomes, specifically D-Dimer (>1 mg/L), C-reactive protein (>10 mg/dL), lactose dehydrogenase (>245 U/L), troponin (>2 times the upper limit of normal), ferritin (>500 ng/mL), creatine phosphokinase (>2x the upper limit of normal), and the absolute lymphocyte count (<800/microliter). Only laboratory parameters that had been obtained clinically during the patient’s disease course were considered.

The data was compiled and assessed per the JRSS (see Figure 1). The JRSS examines the patient’s age (1 point > 55 years of age, 2 points if >65 years of age), ethnicity (1 point if African-American), pulmonary past medical history (2 points if any past medical history exists), cardiovascular past medical history (1 point if just history of hypertension, 2 points if there is any other history), diabetes past medical history (1 point if history exists), and smoking history (2 points if any history of smoking tobacco exists). A score that is 7 or greater was deemed to be of higher risk. As laboratory data collected upon presentation may present additional information suggesting elevated risk, a laboratory addendum of additional points based on the laboratory parameters would be added for those with a score less than 7. Points to be added would range from 1 point (1 laboratory parameter exceeded set limit), 2 points (2 or 3 laboratory parameters exceeded set limits), or 3 points (4 or more laboratory parameters exceeded set limits).

Figure 1.

A flow chart of the Jhala Risk Scoring System (JRSS) starting first with clinical assessment of history. Some who score ⩾7 on history alone would be deemed elevated risk. For the rest, there is further points to be added based on laboratory parameters that may subsequently put the patient in the elevated risk category. Combined scores ⩾7 is elevated risk category, and those <7 would not be classified as elevated risk patients.

The performance of the JRSS was assessed by standard statistical analysis to determine the odds ratio as well as the 95% confidence interval of this odds ratio.¹⁸ If the lower limit of the 95% confidence interval for the odds ratio is not below 1, the odds ratio would be considered statistically significant as the P value would be <.05; this was also confirmed by direct calculation of the P value for each finding.¹⁹ In order to assess and verify the prognostic capability of the JRSS in predicting negative outcomes, the following negative outcomes were considered for the statistical calculation: hospitalization or requirement for medical monitoring, death, intubation during disease course, or admission to the intensive care unit (ICU).

Results

There was a total of 187 unique patients with positive SARS-CoV-2 test results were reviewed. Out of these 187 patients, 45 had a JRSS score (without considering laboratory data) of 7 or greater (Table 1). Twenty of these high score patients were admitted to the Medical Intensive Care Unit (MICU). Seven were admitted to the medical floor. Five continued to reside in a long-term care facility (community living center or CLC). Thirteen did not require hospital admission or required medical monitoring at all. Ultimately for those 45 patients with JRSS scores (without considering laboratory data) of 7 or greater, 9 patients expired (1 for reasons unrelated to COVID-19). Nine had been intubated at some point during their hospitalization (only 6 of the intubated patients had expired). One patient was admitted to a hospital outside of Veteran Affairs without corresponding hospitalization data available other than that the patient did not expire, and was thus excluded from the numbers for the statistical calculations for whether intubation was required or patient required ICU care due to these facts being unknown. One patient was already admitted to the surgical intensive care unit for unrelated reasons and developed COVID-19 midway through his treatment course; as this patient had not been admitted to the intensive care unit for reasons related to COVID-19, this patient was excluded from the statistics of those admitted to the ICU. Therefore, for negative outcomes of hospitalization or required medical monitoring and death, 187 patients were included; 186 were included for statistics on whether intubation was required; and 185 patients were included for statistics on whether the patient was admitted to the ICU.

Table 1.

A-B. Tabulated summary results comparing patients requiring hospitalization and risk scores. Those with risk scores ⩾7 had higher statistically significant odds of being hospitalized (odds ratio of 7.15 with 95% confidence interval of 3.63-14.08). One patient was admitted outside of the VA system, and thus was excluded from the intubation or admitted to ICU categories as this information was unknown. An additional 1 patient became positive after he was already in the surgical ICU for another medical reason and was thus excluded from the admitted to ICU calculations. Part A—JRSS with laboratory data consideration, B—JRSS without considering any available laboratory data.

Jhala Risk Scoring System (JRSS) statistics with laboratory data consideration
A		Hospitalized or required medical monitoring		Death		Intubation required		Admitted to ICU
		Yes	No	Yes	No	Yes	No	Yes	No
Risk score ⩾ 7	Yes	58	16	16	58	15	59	36	38
Risk score ⩾ 7	No	38	75	7	106	4	108	19	92
Odds in high risk group			58/16 or 3.63	16/58 or 0.28		15/59 or 0.25		36/38 or 0.95
Odds in lower risk group			38/75 or 0.51	7/106 or 0.07		4/108 or 0.04		19/92 or 0.21
Odds Ratio			7.15	4.18		6.86		4.59
Upper limit of odds ratio (95% CI)			14.08	10.74		21.63		8.98
Lower limit of odds ratio (95% CI)			3.63	1.63		2.18		2.34
P-value			<.0001	.003		.001		<.0001
Sensitivity			60%	70%		79%		65%
Specificity			82%	65%		65%		71%
Jhala Risk Scoring System (JRSS) statistics if available laboratory data is ignored
B		Hospitalized or required medical monitoring		Death		Intubation required		Admitted to ICU
		Yes	No	Yes	No	Yes	No	Yes	No
Risk score ⩾ 7	Yes	32	13	9	36	9	36	20	25
Risk score ⩾ 7	No	64	78	14	128	10	131	35	105
Odds in high risk group			32/13 or 2.46	9/36 or 0.25		9/36 or 0.25		20/25 or 0.8
Odds in lower risk group			64/78 or 0.82	14/128 or 0.11		10/131 or 0.076		35/105 or 0.33
Odds Ratio			3	2.29		3.275		2.4
Upper limit of odds ratio (95% CI)			6.19	5.71		8.67		4.84
Lower limit of odds ratio (95% CI)			1.45	0.92		1.24		1.19
P-value			.003	.38		.02		.01
Sensitivity			33%	39%		47%		36%
Specificity			86%	78%		78%		81%

Abbreviations: CI, confidence interval; JRSS, Jhala Risk Scoring System.

In contrast, 142 patients had a JRSS score of less than 7 (or 0-6) based on clinical history alone. As laboratory values at the time of the disease course would add additional information that may highlight at risk patients, a laboratory addendum was computed for each of these patients. Once the laboratory addendum was computed, an additional 29 patients had scores of 7 or greater (Table 1). Of these 29, 26 patients were either admitted to the intensive care unit (16 patients), medical floor (8 patients), or were being followed long term in the monitored CLC (2 patients). Seven expired (6 in MICU and 1 in CLC), of which 5 had been intubated prior to death. Of those who did not die, just one was intubated. Notably, the lower limit of the 95% confidence interval increased once the measured laboratory parameters were accounted for.

For those who had a lower score (6 or less), there were 113 patients (Table 1). The overwhelming majority of the patients in this category never required any hospitalization or medical monitoring (75 patients) for COVID-19. The rest, 38 patients, were monitored as part of their long-term care in the CLC (10 patients), were on the medical floor for monitoring (7 patients), and only 21 were in the intensive care unit. Of these patients, of which 3 were intubated (2 intubated patients expired), 7 patients expired during their disease process.

The JRSS identified a subset of patients who may be at higher risk of requiring hospitalization. Considering both the initial scoring and the laboratory addendum, 74 potentially higher risk patients (with a score > 7) had been identified, of which 58 required hospitalization or continued medical monitoring in the CLC, for an admission/monitoring rate of 78%, in contrast to those who did not score highly (only 33% admission monitoring rate). The odds ratio for an elevated score, calculated from the above data, is 7.15 with a 95% confidence interval ranging from 3.63 to 14.08; this indicates a statistically significant association between a score of ⩾7 and requirement for hospitalization or continued medical monitoring (see Table 1).^18-19 The JRSS with laboratory addendum was also successful in predicting death, requirement for intubation, and requirement to be admitted to an ICU (odds ratios of 4.18, 6.86, and 4.59, all statistically significant as in Table 1).

The JRSS without considering any available laboratory data demonstrated a continued ability to predict hospitalization, intubation, and ICU admission in a statistically significant manner, but the lower limit of the 95% confidence interval approached more closely to 1 for all markers of worse prognosis, indicating both that inclusion of the available laboratory data strengthened the algorithm and that the JRSS could be used without laboratory data (see Table 1). The inclusion of laboratory data strengthened the predictive value of the JRSS in all domains (predicting hospitalization or requirement for medical monitoring, intubation, death, and intensive care unit usage), though it is also noted that the performance of laboratory tests assessed by the JRSS was statistically linked to contact with the medical system due to COVID-19 (see Table 1).

Discussion

As previously have been described in the literature, such as in the well-established studies of the Chinese and European population, the cumulation of risk factors can be utilized to model prognosis and identify which patients may have worse or better prognosis after infection with SARS-CoV-2.^1-7 The JRSS system is based in part on the pathogenicity of the SARS-CoV-2 virus where the virus targets certain organ systems (past medical history for pulmonary or cardiovascular systems), and on the previously observed and known risk factors for severe disease.^1-13,20-22 Similarly, these risk scoring models, while they may help identify patients at higher risk of worse prognosis, do not completely replace clinical judgment of each individual case as in both instances, a minority of patients who score poorly do well and other patients who score well do poorly.^1-7 Nonetheless, the presence of a risk scoring algorithm appropriately attuned to a representative population group can be a tool to predict negative outcomes and help in the shepherding of healthcare resources in these pandemic times. Importantly, scoring systems, which have been utilized for many different areas of medicine, provide a formal standardized assessment that can supplement physician judgment; decisions based on probable prognosis does not always need to be based on an individual clinician’s gut feeling alone.²³ The JRSS is particularly useful in that it does not rely on invasive procedural or diagnostic imaging for risk stratification, and can be performed with clinical information alone without any blood draws for laboratory data if needed. While some patients are not identified as higher risk if laboratory data is ignored, the algorithm still identifies in a statistically significant manner a subset of higher risk patients, which can still usefully guide clinical judgment. The JRSS normally relies on routinely collected laboratory tests and is thus a potentially useful tool in common practice. However, despite this usefulness, this type of risk scoring has noted only sparsely with the unique characteristics of the Veteran population of the United States. It is most important to note that populations differ in substantial ways including but not limited to ethnic composition, availability of healthcare, and other innumerable characteristics that may prevent generalizability from one country’s population to another; therefore, a risk scoring system based on a population in China or Europe may not necessarily apply to the American population on a different continent.² In this attempt to apply a risk scoring system on a population within the United States, specifically the Veteran population, that is known to be different from the Chinese and European civilian populations, a successful segregation of higher risk patients was delineated that may help efforts to predict patient outcomes and triage medical resources. Furthermore, this study involved the more vulnerable Veteran population that has already been well documented to differ from the non-Veteran civilian American population.^24-26

The success of the algorithm also helps confirm prior studies demonstrating a higher risk of more severe outcomes with potential risk factors such as age, ethnic background, pre-existing conditions (such as cardiovascular disease, pulmonary disease, or diabetes), smoking habits, and known laboratory parameters that portend a worse prognosis and could be gathered at the time of the initial presentation or visit.^1-17 The association of worse outcomes with the variables accounted for by the JRSS makes sense from a pathogenicity and literature standpoint.^1-17 The variables accounted for have either been reported as risk factors in multiple prior studies or involve an organ system that would be directly affected by the SARS-CoV-2 infection.^1-17 One of the risk factors, ethnicity, may be a marker of worse prognosis due to its very strong and well documented non-biological association with social and health inequalities or discrimination within society at large.¹³ On review of the data, there was a statistically significant association between the number of laboratory tests ordered within the JRSS and need for hospitalization or continued medical monitoring. These laboratory values, as markers of lymphopenia or inflammation have been strongly associated with worse outcomes with pathogenic reasons for this.^1-17,20-22 This published history therefore would tend to argue that there exists a true biologic reason for the association with worse prognosis with these laboratory parameters. While this study is limited in being a single regional VAMC institutional study involving only Veteran patients, this study is an important contribution to ensuring that risk scoring systems have been systematically studied in additional populations whereby it may be useful. Further directions suggested by this study may include looking at the broader US population, examining if the JRSS might be even more robust if the laboratory testing assessed was routinely performed on a higher proportion of patients with COVID-19, or even examining which laboratory parameters contribute most to an accurate algorithm.

Conclusion

In this very study of a comprehensive risk scoring system for a patient population with COVID-19 within the United States, undertaken in a regional VAMC, found that the JRSS, which assessed points based on age, ethnicity, pre-existing conditions, smoking habit, and available laboratory parameters had predictive value in determining which patients may be at elevated risk of hospitalization. The JRSS is simple enough that it does not require for its utilization any invasive procedures, diagnostic imaging, or even blood draws for laboratory studies; laboratory data on set parameters is simply incorporated into the algorithm if it is available. The determination early on by a standardized risk scoring system (based on patients in the United States) of which patients may require more extensive medical services may help clinicians implement better strategies for more effective use of limited medical resources and anticipate needs earlier in the patients’ course.

Footnotes

Declaration of Conflicting Interests:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding:

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jeffrey Petersen

References

Lei

, et al. Clinical characteristics, associated factors, and predicting COVID-19 mortality risk: a retrospective study in Wuhan, China. Am J Prev Med. 2020;59:168-175.

Zhang

Fok

Zhao

, et al. Generalizability of COVID-19 mortality risk score model. Am J Prev Med. 2020;59:e249-e250.

Huang

Wang

Chen

Yang

Yao

Liang

. A novel risk score to predict diagnosis with Coronavirus disease 2019 (COVID-19) in suspected patients: a retrospective, multicenter, and observational study. J Med Virol. 2020;92:2709-2717.

Shang

Liu

Wei

, et al. Scoring systems for predicting mortality for severe patients with COVID-19. EClinicalMedicine. 2020;24:100426.

Liang

, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. 2020;180:1081-1089.

Knight

Pius

, et al. Risk stratification of patients admitted to hospital with COVID-19 using the ISARIC WHO clinical characterisation protocol: development and validation of the 4C mortality score. BMJ. 2020;370:m3339.

Galloway

Norton

Barker

, et al. A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: an observational cohort study. Infect J. 2020;81:282-288.

Osborne

Veigulis

Arreola

Röösli

Curtin

. Automated EHR score to predict COVID-19 outcomes at US Department of Veterans Affairs. PLoS One. 2020;15:e0236554.

Ioannou

Locke

Green

, et al. Risk factors for hospitalization, mechanical ventilation, or death among 10 131 US veterans with SARS-coV-2 infection. JAMA Network Open. 2020;3:e2022310.

10.

Petersen

Jhala

. Practical risk scoring system for predicting severity of COVID-19 disease. Am J Clin Pathol. 2021;156(Supplement 1):S133.

11.

Kim

Bostwick

. Social vulnerability and racial inequality in COVID-19 deaths in Chicago. Health Educ Behav. 2020;47:509-513.

12.

Magleby

Westblade

Trzebucki

, et al. Impact of Severe Acute Respiratory Syndrome Coronavirus 2 viral load on risk of intubation and mortality among hospitalized patients with Coronavirus disease 2019. Clin Infect Dis. 2021;73(11):e4197-e4205.

13.

Holmes

Enwere

Williams

, et al. Black-White risk differentials in COVID-19 (SARS-COV2) transmission, mortality and case fatality in the United States: translational epidemiologic perspective and challenges. Int J Environ Res Public Health. 2020;17:4322.

14.

Cappuccio

Siani

. Covid-19 and cardiovascular risk: susceptibility to infection to SARS-CoV-2, severity and prognosis of Covid-19 and blockade of the Renin-Angiotensin-Aldosterone system. An evidence-based viewpoint. Nutr Metab Cardiovasc Dis. 2020;30:1227-1235.

15.

Turan

Hakim

Dashraath

Jeslyn

WJL

Wright

Abdul-Kadir

. Clinical characteristics, prognostic factors, and maternal and neonatal outcomes of SARS-CoV-2 infection among hospitalized pregnant women: a systematic review. Int J Gynecol Obstet. 2020;151:7-16.

16.

Centers for Disease Control and Prevention (CDC). Coronavirus Disease 2019 (COVID-19): Assessing Risk Factors for Severe COVID-19 Illness. Accessed December 22, 2021. https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-discovery/assessing-risk-factors.html. Last updated November 30, 2020.

17.

Rahman

Shanjana

Tushar

, et al. Hematological abnormalities and comorbidities are associated with COVID-19 severity among hospitalized patients: experience from Bangladesh. PLoS One. 2021;16(7):e0255379.

18.

Tenny

Hoffman

. Odds Ratio. StatPearls Publishing; 2020.

19.

Altman

Bland

. How to obtain the P value from a confidence interval. BMJ. 2011;343:d2304-d2304.

20.

Pourbagheri-Sigaroodi

Bashash

Fateh

Abolghasemi

. Laboratory findings in COVID-19 diagnosis and prognosis. Clin Chim Acta. 2020;510:475-482.

21.

Sandoval

Januzzi

Jaffe

. Cardiac Troponin for assessment of myocardial injury in COVID-19: JACC review topic of the week. J Am Coll Cardiol. 2020;76:1244-1258.

22.

Vargas-Vargas

Cortés-Rojo

. Ferritin levels and COVID-19. Rev Panam Salud Publica. 2020;44:e72.

23.

Ferguson

. The rationale for developing scoring systems for clinical practice. Thorac Surg Clin. 2007;17:343-351.

24.

Agha

Lofgren

VanRuiswyk

Layde

. Are patients at Veterans Affairs medical centers sicker? A comparative analysis of health status and medical resource use. Arch Intern Med. 2000;160:3252-3257.

25.

Eibner

Krull

Brown

, et al. Current and projected characteristics and unique health care needs of the patient population served by the Department of Veterans Affairs. Rand Health Quarterly. 2016;5:13.

26.

Morgan

Teal

Reddy

Ford

Ashton

. Measurement in veterans Affairs health services research: veterans as a special population. Health Serv Res. 2005;40:1573-1583.