Highlights

  • Although certain risk factors have been associated with morbidity and mortality in patients with Coronavirus disease 2019 (COVID-19), emergency department (ED) derived risk prediction models specific to COVID-19 are lacking.

  • The CRI is an accurate and validated risk prediction tool scoring system that uses clinical data upon presentation to the ED to predict the need for mechanical ventilation or inpatient death.

  • Variables independently associated with the primary outcome were age ≥ 65 years, chronic obstructive pulmonary disease, chronic kidney disease, cerebrovascular disease, initial D-dimer > 1.1 µg/mL, platelet count < 150 K/µL, and severity of SpO2:FiO2 ratio.

  • The CRI can guide early triage, inform treatment strategies, and empower physician decision-making during the COVID-19 pandemic.

Introduction

Coronavirus disease 2019 (COVID-19) is caused by the novel severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), which was first identified in Wuhan, China, in December 2019, and has since caused a global pandemic [1].

The clinical manifestations of COVID-19 vary from asymptomatic to severe disease [2]. In a multicenter case series from New York, 14% of discharged patients required ICU admission [3], with high inpatient mortality [3]. Data so far has demonstrated that underlying comorbid conditions, inflammatory markers, and hypoxic respiratory failure are associated with a higher incidence of adverse outcomes, including mechanical ventilation and mortality [3,4,5,6].

During a pandemic, a mass influx of patients can quickly exhaust available resources overwhelming an entire healthcare system. Under such circumstances, a simple risk model that can accurately predict patient outcomes would be valuable in determining early prognosis and aid in triaging patients presenting to the emergency department (ED). Multiple risk models have been described, but to our knowledge, no large study has been externally validated.

Herein, we describe the characteristics and findings of patients hospitalized for COVID-19 at a tertiary care healthcare system. Our study aimed to develop the COVID-19 risk index (CRI): a pragmatic tool to assess the risk for requiring mechanical ventilation or death upon presentation to the ED. We also aimed to validate the risk model within a larger 38-center external hospital registry.

Materials and methods

Derivation cohort patient population

We analyzed the records of patients consecutively admitted to Henry Ford Health System in Southeast Michigan between March and May 2020. Patients selected were ≥ 18 years of age and were hospitalized with a confirmed SARS-CoV-2 diagnosis. Patients were excluded if they were transferred to or out of our center or developed cardiac arrest prior to presentation. Records were reviewed retrospectively.

SARS-CoV-2 was diagnosed by onsite molecular diagnostic testing for the identification of SARS-CoV-2 RNA using reverse transcription polymerase chain reaction (RT-PCR). Our method has been validated against the Centers for Disease Control and Prevention (CDC) reference method to meet or exceed the level of detection required under Food and Drug Administration (FDA) Emergency Use Authorization (EUA) guidelines.

Definition of outcomes

The primary outcome was a composite of the need for mechanical ventilation or death during hospitalization. The secondary outcome was inpatient mortality.

Definitions of variables and predictor selection

Variables measured at hospital admission (Table 1) were collected at hospital admission. These variables included demographics, symptoms on admissions, vital signs, comorbidities, laboratory, and radiologic data obtained on admission. Data collection was manually extracted from electronic health records. Symptoms were deemed positive if endorsed within 24 h of presentation. Laboratory data were manually extracted. Baseline results refer to initial blood samples collected in the ED or the first values within 24 h of admission. Comorbid conditions were identified based on admission and discharge diagnoses. This study was approved by the Institutional Review Board (IRB #13774), with a waiver of informed consent. Serum ferritin, D-dimer, creatinine phosphokinase, high-sensitivity troponin-I, and lactate dehydrogenase level cutoffs were used based on the median. Lymphopenia was defined as an absolute lymphocyte count < 1000 counts/uL [7], and anemia as a hemoglobin level < 11 g/dL. Troponin levels were categorized according to the upper 99th percentile as per prior work [8]. Immunosuppression was defined as a history of autoimmune disease, immunodeficiency disorders, transplantation, immunosuppressant use (any disease-modifying agent or prednisone ≥ 10 mg or equivalent for more than 14 days), human immunodeficiency virus, or active malignancy. Cardiovascular disease was defined as any coronary artery disease, heart failure, or cardiac arrhythmia. Peripheral capillary oxygen saturation (SpO2) values were obtained from pulse oximetry vital logs. For patients not on mechanical ventilation, the fraction of inspired oxygen (FiO2) was estimated by multiplying liter flow per minute by 0.03 and adding that to 0.21, in accordance with original mSOFA investigations [9]. SpO2:FiO2 ratios were categorized similarly to the Berlin criteria for acute respiratory distress syndrome [10]. Data regarding mean arterial pressure, supplemental oxygen delivery, and Glasgow Coma Scale were ascertained from ED vital logs.

Table 1 Clinical characteristics of patients on presentation from the derivation cohort
Full size table

A univariate analysis was performed, and the variables that were significantly associated with our primary outcome were included in logistic regression. The variables that remained statistically associated (P ≤ 0.05) with the primary outcome after multivariable regression were used to create the CRI index.

Statistical analysis

Statistical Package for Social Sciences (SPSS, version 25, IBM, Armonk, New York) and Statistical Analysis Software (SAS) was used for data cleaning and analysis. The data were analyzed by independent statisticians: O.C. and E.P. for the derivation cohort and C.A. and M.Y. for the validation cohort. Descriptive statistical analyses were obtained for all included study variables and are summarized in Supplementary Table 1. Categorical variables are expressed as frequency or percentage, whereas continuous variables are presented as mean and standard deviation or median and interquartile range based on the normality of the data. First, univariate analysis was performed by using χ2 test or Fisher exact test for categorical variables and the T-test or the Mann–Whitney U test for continuous variables. Multivariate logistic regression analysis using the backward Wald’s method was then performed to identify the significant variables to be included in the CRI to predict the primary outcome. Different groups among the CRI scores were compared using χ2 test for categorical variables and analysis of variance or Kruskal Wallis tests for continuous variables based on the normality of the data. Then, the time-to-event analysis was performed using Kaplan–Meier curves, where significance was calculated using the log-rank test. A receiver-operating characteristics (ROC) curve was added to address the discriminatory power of the CRI, and the area under the ROC curve was reported. Statistical analyses were considered significant if P < 0.05.

Validation cohort

The newly developed index was validated with the MI-COVID19 database. The MI-COVID-19 database is a statewide multiple health system collaborative quality initiative (CQI) that was sponsored by Blue Cross Blue Shield of Michigan and Blue Care Network. The aim of this initiative was to improve the quality of hospitalized patients with COVID-19. The database includes 38 hospitals across the state of Michigan. The median bed size is 391 (IQR: 250–537), 81% are non-profit, and 93% self-identify as teaching hospitals. Patients admitted at the Henry Ford Hospital system (used as a derivation cohort) were excluded. The same inclusion and exclusion criteria as the derivation cohort were applied. Validation and the creation of ROC curves were performed through logistic regression modeling. Calibration was performed in a similar fashion. Each patient was assessed for the number of significant factors (as defined in the original cohort) that were affecting them at the time of admission and assigned a score. The total score compared to the outcome was first assessed for each patient as an unadjusted rate. Logistic regression was then used to estimate the predicted probabilities of having the outcome compared to the total score. These predicted probabilities were used to calculated adjusted rates of the outcome within each score category. Finally, the unadjusted rates were compared to the adjusted rates (Fig. 1).

Fig. 1
figure 1

A The discriminatory power for the COVID-19 risk index (CRI) within the derivation (A) and validation (B) cohorts by receiver-operating characteristic (ROC) area under the curve (AUC)

Full size image

Results

We reviewed the charts of a total of 1024 patients admitted between March 8 and May 1, 2020. Of those, three were transferred in or out of our institution, and one presented with cardiac arrest and were therefore excluded. A total of 1020 patients were included in the final analysis for the derivation cohort. Table 1 shows the baseline clinical characteristics of the overall derivation cohort and is categorized into those with or without the primary outcome of death or need for mechanical ventilation. Characteristics included demographics, symptoms on presentation, comorbid conditions, medications, and laboratory/imaging data.

Using the significant univariate predictors, a multivariate logistic regression analysis was performed to identify the independent predictors of our primary outcome. Variables included in the multivariate regression model included: age ≥ 65, gender, body mass index, hypertension, smoking history, diabetes mellitus, cardiovascular disease, chronic obstructive pulmonary disease, history of immunosuppression, anemia, chronic kidney disease, ferritin ≥ 600 (ng/mL), troponin > 18 (ng/dL), D-dimer ≥ 1.1 (µg/mL), creatinine phosphokinase ≥ 10 (IU/L), lactate dehydrogenase ≥ 340 (IU/L), and SpO2:FiO2 categories (Table 1). Of these variables, the significant multivariate predictors were age ≥ 65 years, chronic obstructive pulmonary disease, chronic kidney disease, cerebrovascular disease, initial D-dimer > 1.1 µg/mL, platelet count < 150 K/µL, and severity of SpO2:FiO2 ratio (Derivation cohort: Table 2, Validation cohort: Table 4). These variables were used to derive the CRI.

Table 2 Multivariable predictors of death or intubation in the COVID-19 risk index (CRI) derivation cohort
Full size table

Patients’ CRI scores were calculated according to the sum of CRI points they had on presentation to the ED (Table 2). The incidence of the primary outcome increased steadily with higher CRI scores (Table 3) in both the derivation and validation cohorts (area under ROC: 0.84 [derivation] and vs. 0.74 [validation]).

Table 3 Primary outcome within different score groups within CRI categories
Full size table

To validate our index, we analyzed the predictive capability of the CRI in an external cohort of 1799 sampled hospitalized patients. Supplementary Table 1 includes the baseline characteristics of the validation cohort. The validation cohort had a similar trend and distribution of the primary endpoint according to the CRI model (Table 3). The calibration plot (Fig. 2) reveals a close adherence between the observed and expected primary outcomes within the validation cohort.

Fig. 2
figure 2

Calibration plot

Full size image

Discussion

We describe a new risk predictive model for patients admitted with COVID-19 using easily acquired data elements in the ED. The CRI predicts the need for mechanical ventilation or inpatient death.

Although certain clinical characteristics are associated with increased mortality in COVID-19 [11], a validated disease-specific model to assess risk at the time of presentation to the ED is lacking. The CRI encompasses seven variables that were independently associated with the need for mechanical ventilation or death in patients hospitalized with COVID-19. Patients score points for age ≥ 65 years, chronic obstructive pulmonary disease, chronic kidney disease, cerebrovascular disease, initial D-dimer > 1.1 µg/mL, platelet count < 150 K/µL, and severity of SpO2:FiO2 ratio (Table 2). All the above variables have been shown to be associated with worse outcomes in patients with COVID-19 [12,13,14]. However, not all variables were shown to be independently associated with the primary outcome.

There have been other clinical risk predictive models used to assist with decision-making [7, 15,16,17], but the CRI offers several advantages. The variables are objective and easily obtained at the time of presentation, and the scoring system is simple. A low CRI was associated with a lower risk for mechanical ventilation or death in both the derivation and validation cohorts, and the rates increased with higher scores. The score can assist in the triage of patients with lower scores to units with lower provider-to-patient ratios given the low likelihood for mechanical ventilation or death. Conversely, a higher score confers an increased risk of mechanical ventilation or death, mandating closer monitoring during admission (Table 3). In comparison to other score models, the CRI was validated against a larger and more heterogeneous population. It also utilized data from a more predictable time point (admission) compared to the variable time points utilized by other models.

Temporal trends indicate that the demographics and the approach to the care of COVID-19 patients have changed as the disease has evolved and clinical and research experience accumulates. The initial aggressive approach to initiate mechanical ventilation has tempered. While changing practice variations can challenge the applicability of risk models, the CRI uses objective and plausible metrics that have a strong association with the underlying disease state and we believe will maintain relevance (Table 4).

Table 4 Univariate odds ratio of score components within the validation cohort
Full size table

During a pandemic, and especially in areas of high case density, clinical tools and prediction models can aid in appropriate patient triage and resource allocation. The CRI is a useful tool that weighs the chance of survival with risk for ICU care, mechanical ventilation, or death, allowing for objective and efficient ED triage and management of patients admitted with COVID-19. We, therefore, suggest obtaining the CRI on admission for such patients.

Limitations

Our study is retrospective and therefore is subject to limitations. Our cohort was limited to hospitalized patients and cannot be generalized to all patients diagnosed with COVID-19 as the model captures an earlier state of patient demographics, disease management, outcomes, and vaccination. Future prospective and multicenter studies should re-assess the CRI to further determine generalizability as the disease patterns change and new variants are discovered.

Conclusions

In conclusion, the CRI is an important tool that can guide early triage, inform treatment strategies, and empower physician decision-making during the COVID-19 pandemic.