Exploratory COVID-19 death risk score based on basic laboratory tests and physiological clinical measurements

Gui-ying Dong, Fei-fei Jin, Qi Huang, Chun-bo Wu, Ji-hong Zhu, Tian-bing Wang Emergency Department, Peking University People’s Hospital, Beijing 00044, China

2 Trauma Center, Peking University People’s Hospital, Key Laboratory of Trauma and Neural Regeneration (Peking University), Ministry of Education, Beijing100044, China

3 Department of Endocrinology and Metabolism, Peking University People’s Hospital, Beijing100044, China

KEYWORDS: COVID-19; 30-day mortality; Prediction model

INTRODUCTION

During the global COVID-19 pandemic, medical staff from various countries summarized some experiences, and as one of the most severely affected countries, China has successfully responded to mass casualty events in Wuhan.[1]Without prior evidence, it is important to identify deteriorated patients in order to provide critical care as soon as possible.[2]

Although the observation of biomarkers for the prediction of disease severity and prognosis in patients with COVID-19 has made promising achievements,clinical studies based on a single biomarker may not be able to assess the clinical changes of patients and provide treatment in clinical practice.[3-6]It is more comprehensive and objective to adopt various indicators for the prediction of disease severity and prognosis in patients with COVID-19.[7-11]However, due to the unfamiliarity with COVID-19 and worry about the incompleteness of manifestations and complications from the clinic, researchers used several laboratory tests to diagnose and assess the severity. Many severity scoring systems and prediction models are too complex or impractical to be widely applied.[11]With the emergency and severity of the pandemic, it is more urgent for the overall medical system, especially the emergency department, to practice necessary and appropriate testing,rather than aimless screening.[12]

Therefore, this study aimed to use clinically accessible parameters and routine laboratory tests to extrapolate a model for predicting the 30-day mortality rate of patients with COVID-19.

METHODS

Study population and study design

We conducted a retrospective study of 295 hospitalized patients at Tongji Hospital of Tongji Medical College, Huazhong University of Science and Technology, which was one of the exclusively designated institutions for COVID-19 patients in Wuhan, China,from February 8, 2020, to March 8, 2020. The outcome of the present study was the 30-day mortality rate.The study was performed under the framework of theDeclaration of Helsinkiand approved by the local ethics committee (2021-PHB-343). The application for written informed consent was waived due to the study design. In this study, we referenced the guideline recommendations of Strengthening the Reporting of Observational Studies in Epidemiology (STROBE).[13]

All COVID-19 patients older than 18 years were eligible for the study. The exclusion criteria were as follows: patients with an arrest or moribund caused by other diseases at arrival, those who refused endotracheal intubation and mechanical ventilation, those with other acute infections or inflammation, and those with incomplete clinical data (age or vasoactive drug dosage).A flow chart of this study is shown in Figure 1.

Figure 1. The flow chart of this study.

Data collection and definitions

Data used in this study were anonymized and deidentified before analysis from electronic health-care record (EHR) systems. Demographic and clinical data were collected upon admission. Data on gender, age,comorbidities, sequential organ failure assessment(SOFA) score,[14]“confusion, uremia, respiratory rate,blood pressure, and over age 65 years” (CURB-65)score,[15]Glasgow coma score (GCS),[16]and vital signs were obtained. Laboratory tests, including white blood cell (WBC), platelet (Plt), hemoglobin (Hb), highsensitivity C-reactive protein (hsCRP), interleukin(IL)-6, fibrinogen (FIB), high sensitivity troponin(hsTnI), N-terminal pro-B-type brain natriuretic peptide (NT-proBNP), serum creatinine (Scr), alanine aminotransferase (ALT), ferritin, and D-dimer levels and the ratio of peripheral oxygen saturation (SpO2) to the fraction of inspired oxygen (FiO2)[17]were screened every other day from admission until 7 d. The highest and lowest values were selected for the analysis.

Statistical analysis

SPSS 25.0 (IBM Inc., USA) and R version 3.6.1(http://www.r-project.org) were used for data analysis.The Kolmogorov-Smirnov test was used for normality test. Continuous variables were presented as the median(interquartile range [IQR]), and were compared using the Mann-WhitneyUtest. Categorical variables were presented asn(%) and were compared using the Chisquared test or Fisher’s exact test as appropriate.Statistical significance was set atP＜0.05.

Sample size calculation

Sample size was calculated using the Power Analysis and Sample Size (PASS) software (2021). Based on cohort data, the mortality rate was 21%.[18]The area under curve (AUC) was calculated to quantify the discriminatory performance of the prediction score.An AUC of 0.5 indicates a very poor discriminatory performance, whereas an AUC of 1.0 corresponds to an excellent accuracy. To satisfy this difference with 80%power at 5% significance (two-tailed), the performance of the mortality prediction model was assumed to be between 0.75 and 0.91, requiring at least 25 deaths, and the number of patients in the derivation group was 119.

Model development

In light of the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) recommendations,[19]a preliminary list of 11 candidate variables was selected based on a review of the now available evidence.[6,20-21]Univariate logistic regression analyses were conducted to recognize the unadjusted association between potential predictors and 30-day mortality. Multiple logistic regression was used with a stepwise backward approach of the positive variables using the Bayesian information criterion to develop the prediction score.P＜0.1 was used to indicate whether there was an interaction.[22]

Model validation

The model’s discrimination for 30-day mortality was examined using the AUROC. The calibration was appraised using the Hosmer-Lemeshow goodness-offit test. The bootstrap sampling approach was then used to create new data to perform internal validation and sampling of the whole dataset using 100 repetitions with replacement. Moreover, we compared the AUC with the existing SOFA and CURB-65 scores, as they have been verified in a larger database.[23,24]

To evaluate the clinical utility of the CDRS, a decision curve analysis (DCA) was conducted to compare the net benefit of the CDRS, SOFA, and CURB-65 in predicting 30-day mortality under different threshold probabilities.

RESULTS

Patient characteristics

The demographics and general characteristics of the 278 patients are listed in Table 1. The median age of the patients was 64 (IQR, 52-71 years), and 146 patients(52.52%) were men. A total of 215 patients (77.34%)survived and 63 (22.66%) died. Compared with the nonsurvival group, the percentage of younger patients[25]was significantly higher (63.26% vs. 31.75%,P＜0.001) in the survival group. There was no significant difference between the two groups in the middle-aged patients(30.70% vs. 41.27%,P=0.185). Fewer patients in the survival group needed mechanical ventilation (5.11% vs.95.24%,P＜0.001) and vasoactive drug therapy (1.86%vs. 66.67%,P＜0.001) than those in the non-survival group. The non-survival group was more likely to have underlying cardiovascular disease (30.16% vs. 15.81%,P=0.011), chronic pulmonary disease (17.46% vs. 6.51%,P=0.008), and chronic liver disease (7.94% vs. 1.86%,P=0.017) than the survival group.

Multivariable logistic regression model derivation of the CDRS

Table 2 summarizes the results of the binomial logistic regression analyses. The 30-day mortality was positively associated with age, respiratory rate (RR), and WBC, D-dimer, hsCRP, IL-6, ferritin, hsTnI, Scr, and ALT levels (P＜0.001).

Multivariate analysis showed that RR (OR=1.185,95%CI: 1.032-1.361,P=0.016), hsCRP (OR=1.018,95%CI: 1.003-1.033,P=0.017), D-dimer (OR=1.101,95%CI: 0.986-1.229,P=0.089) were significantly associated with 30-day mortality. The score was extrapolated as follows: CDRS=-10.245+(0.022×hsCRP)+(0.172×D-dimer)+(0.203×RR) (supplementary Table 1).

Validation of the CDRS

Table 3 shows that the CDRS has predictive validity(AUC=0.984, 95%CI: 0.969-0.998,P＜0.001), which was similar to those of the SOFA (AUC=0.975, 95%CI:0.945-1.000,P＜0.001) and CURB-65 (AUC=0.971,95%CI: 0.949-0.994,P＜0.001). The Hosmer-Lemeshow goodness of fit test stated that the CDRS had good calibration (P=0.85).

Internal validation of the CDRS model indicated statistical optimism in the score (AUC=0.980, 95%CI:0.965-0.995).

The bias-corrected curve, generated through a bootstrap method, showed a slight deviation from the reference line, although the predicted 30-day mortality remained in good agreement with the actual 30-day mortality (Figure 2).

Net benefit of using CDRS

DCA (Figure 3) showed that CDRS had a positive net benefit at a predicted threshold probability between 1% and 95% when treating patients with COVID-19 as if all would have died or survived (i.e., treat-all or treat-none strategies). When the predicted threshold probability was 1% to 95% for the CDRS, SOFA, and CURB-65 scores, the net benefits were positive for all scores. Regarding its clinical use, CDRS provided a similar net benefit to CURB-65 when the predicted threshold probability ranged between 1% and 95%(Figure 3 and supplementary Table 2), which was less than that using the SOFA score.

Figure 2. Calibration curves constructed using the bootstrap (n=278).CDRS: COVID-19 death risk score.

Figure 3. The decision curve analysis (DCA) curves of medical intervention in patients with the CDRS, SOFA, and CURB-65.CDRS: COVID-19 death risk score; SOFA: sequential organ failure assessment; CURB-65: confusion, uremia, respiratory rate, blood pressure and age ≥ 65 years.

Table 1. Baseline demographics and clinical characteristics

Table 2. Univariate and multivariate regression analysis of the risk factors for 30-day mortality

Table 3. The performance comparison of CDRS with SOFA and CURB-65

DISCUSSION

Since the first case of COVID-19 occurred, the fifth round of the epidemic has been reported in which Omicron was the predicted main-stream variant, and its spreading has been significant.[26]South Africa researchers suggested that Omicron might spread faster,but the clinical manifestations are similar to those of the past variants. Existing public health prevention measures,along with the diagnostic and therapeutic experience that has remained effective against COVID-19, should be effective against the Omicron variant.[27]

Although the effective treatment for COVID-19 remains unknown, we have a better understanding of selecting laboratory tests that are appropriate for the diagnosis and treatment of COVID-19 patients.[12]However, optimal prediction models for COVID-19 are required worldwidely, especially at the point of care and during a sudden shortage of medical resources. There are few research on simple-to-collect data-based prediction models in COVID-19 patients.[28-30]

This study assembled a cohort of 278 patients with COVID-19 and developed and validated a new clinical risk model to predict 30-day mortality in view of three variables that are easily available in clinical practice.The statistical results derived by adhering to TRIPOD standards[19]showed that the hsCRP and D-dimer levels and RR were independently associated with the 30-day mortality of hospitalized patients with COVID-19, and the new CDRS model provides the best discrimination to predict the 30-day mortality (AUC=0.984). Meanwhile,the CDRS model achieved a net benefit of the thresholds for risk stratification. Thus, the model has potential for clinical utility and generalizability. The model was well calibrated and would be convenient in a variety of healthcare situations, especially in regions with inadequate human and material resources.

Recent studies have demonstrated that the CURB-65 (specific-pneumonia-severity score) and SOFA(severity of the patient’s whole condition score) could predict mortality in a large population of patients with COVID-19 reliably.[23,24]However, the SOFA score is based on 12 parameters of the patient’s six organs,while CURB-65 comprises five separate elements. Both scores include a subjective evaluation of the state of consciousness. No predictors other than those commonly found in electronic records were required in CDRS,unlike SOFA and CURB-65. Moreover, CDRS is simpler to use to generate an information system automatically.

There are some limitations to this research. This study was carried out in a single center with a small size of patients. The performance was based on a comparison with SOFA and CURB-65, and bootstrapping was utilized for internal validation. Therefore, CDRS requires rigorous external validation in multicenter studies on a large sample population in a real-world setting.

CONCLUSIONS

Among patients with COVID-19, CDRS was simple to be used in clinical practice and powerful to deliver satisfactory performance at the point of care. CDRS could be introduced to risk assessment as an early warning tool with bedside variables.

Funding:This study was supported by the base supporting 2021 mobile digital hospital system engineering research center of the Ministry of Education (2194000024).

Ethical approval:The study was performed under the framework of theDeclaration of Helsinkiand approved by the local ethics committee (2021-PHB-343). The application for written informed consent was waived due to the study design.

Conflicts of interest:The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Contributors:conceptualization, methodology, and writing:GYD; formal analysis: GYD, FFJ, and QH; resources: GYD;review and editing: CBW and JHZ; funding: TBW. All authors have read and agreed to the published version of the manuscript.

All the supplementary files in this paper are available at http://wjem.com.cn.