Abstract
The significant impact of COVID-19 worldwide has made it necessary to develop tools to identify patients at high risk of severe disease and death. This work aims to validate the RIM Score-COVID in the SEMI-COVID-19 Registry. The RIM Score-COVID is a simple nomogram with high predictive capacity for in-hospital death due to COVID-19 designed using clinical and analytical parameters of patients diagnosed in the first wave of the pandemic. The nomogram uses five variables measured on arrival to the emergency department (ED): age, sex, oxygen saturation, C-reactive protein level, and neutrophil-to-platelet ratio. Validation was performed in the Spanish SEMI-COVID-19 Registry, which included consecutive patients hospitalized with confirmed COVID-19 in Spain. The cohort was divided into three time periods: T1 from February 1 to June 10, 2020 (first wave), T2 from June 11 to December 31, 2020 (second wave, pre-vaccination period), and T3 from January 1 to December 5, 2021 (vaccination period). The model’s accuracy in predicting in-hospital COVID-19 mortality was assessed using the area under the receiver operating characteristics curve (AUROC). Clinical and laboratory data from 22,566 patients were analyzed: 15,976 (70.7%) from T1, 4,233 (18.7%) from T2, and 2,357 from T3 (10.4%). AUROC of the RIM Score-COVID in the entire SEMI-COVID-19 Registry was 0.823 (95%CI 0.819–0.827) and was 0.834 (95%CI 0.830–0.839) in T1, 0.792 (95%CI 0.781–0.803) in T2, and 0.799 (95%CI 0.785–0.813) in T3. The RIM Score-COVID is a simple, easy-to-use method for predicting in-hospital COVID-19 mortality that uses parameters measured in most EDs. This tool showed good predictive ability in successive disease waves.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Background
The COVID-19 pandemic, a disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has had a huge impact on healthcare systems worldwide and resulted in more than 523 million known infections and well over 6.2 million deaths globally as of May 24, 2022 [1]. The disease’s spectrum ranges from a minor illness that can be treated on an outpatient basis to severe acute respiratory failure that may require admission to the intensive care unit (ICU) or death [2,3,4].
The early detection of patients with COVID-19 who may have worse outcomes is a priority [5,6,7]. A highly accurate tool for predicting the clinical course of this disease could be very useful for risk stratification, clinical decision-making, and ultimately for reducing mortality. A number of prognostic models for COVID-19 have been proposed since the beginning of the pandemic. Several studies have established a link between the most severe cases of COVID-19 and blood cell count-derived ratios, such as the neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), or neutrophil-to-platelet ratio (NPR) [8,9,10,11,12].
Our group created a prognostic nomogram (Risk of In-hospital Mortality Score in COVID-19 (RIM Score-COVID)) that is highly accurate for predicting in-hospital mortality [13]. The RIM Score-COVID includes five variables commonly measured upon a patient’s arrival at the emergency department (ED): age, sex, baseline oxygen saturation (SpO2), C-reactive protein (CRP) level, and NPR. Applying the nomogram to the validation cohort, the area under the receiver operating characteristics curve (AUROC) was 0.861 (95% CI 0.823–0.900).
The Spanish Society of Internal Medicine (SEMI, for its initials in Spanish) sponsored a nationwide COVID-19 patient registry in which 150 Spanish hospitals participated. Over 24,000 patients over 18 years of age were included. Several studies derived from the SEMI-COVID-19 Registry have reported clinical characteristics with prognostic value [4, 14,15,16] as well as prognostic scores [17, 18]. However, since the start of the pandemic, multiple factors have changed the course of disease because the clinical management [20], available resources, and treatments indicated have evolved since the beginning of the pandemic [19, 21]. Therefore, the objective of this study was to validate the prognostic utility of the RIM Score-COVID in the SEMI-COVID-19 Registry in three time periods: patients admitted in the first wave, patients admitted after first wave but before the start of vaccination, and patients admitted after the start of vaccination [22].
Methods
The RIM Score-COVID was validated using the SEMI-COVID-19 Registry, which retrospectively compiles data on the index admission of patients ≥ 18 years of age with COVID-19 confirmed microbiologically through a reverse transcription polymerase chain reaction (RT-PCR) or antigen test and with hospital discharge or in-hospital death from January 2020 to December 2021. More in-depth information about the justification, objectives, methodology, and preliminary results of the SEMI-COVID-19 Registry have recently been published [4, 14, 15].
Study population and registry information
Patients from the SEMI-COVID-19 Registry were treated at their attending physician’s discretion, according to local protocols and clinical judgement. Patients included in open-label clinical trials could be included in the registry, provided that all information about treatment was available.
An online electronic data capture system (DCS) was developed, which included a database manager along with procedures for the verification of data and contrasting of information against the original medical record to ensure the best possible quality of data collection. Patient identifiable data were dissociated and pseudonymized.
Data collection
Data were collected retrospectively and included approximately 300 variables grouped under various headings: (1) inclusion criteria, (2) epidemiological data, (3) RT-PCR and serology data, (4) personal medical and medication history, (5) symptoms and physical examination findings at admission, (6) laboratory (blood gasses, metabolic panel, complete blood count, coagulation) and diagnostic imaging tests, (7) additional data at 7 days after admission or at admission to the intensive care unit (ICU), (8) pharmacological treatment during the hospitalization (antiviral drugs, immunomodulators, antibiotics) and ventilatory support, (9) complications during the hospitalization, and (10) progress after discharge and/or 30 days from diagnosis. Variables were obtained from the patient’s electronic medical record [4].
To validate the results obtained with the initial RIM Score-COVID model, the cohort of patients was divided into three groups based on admission date.
The first time period (T1) was February 1 to June 10, 2020 (first wave). The cut-off point of June 10, 2020 was selected for two reasons: first, because it was the date that the last patient was included in the original validation model for the RIM Score-COVID and second, because it is the approximate date of transition between the first and second waves of the COVID-19 pandemic in Spain.
The second time period (T2) was from June 11 to December 31, 2020 (second wave and pre-vaccination period). The second cut-off point of December 31, 2020 was selected for two reasons: first, because vaccination in Spain began soon before on December 26, 2020 and second, because it is the approximate date of transition between the third and fourth waves in Spain.
The third time period (T3) was from January 1 to December 5, 2021. The third cut-off point of December 5, 2021 was selected because it was the end of data inclusion in the SEMI-COVID-19 Registry for this work. T3 is the vaccination period.
RIM Score-COVID
The RIM Score-COVID is a logistic regression model that includes five variables commonly measured upon the patient’s arrival at the ED: age, sex, baseline SpO2, CRP level, and NPR value. The NPR is the ratio of neutrophil count (× 109 cells/L) to platelet count (× 1011 cells/L). The initial nomogram was developed in a cohort of 2,543 patients with COVID-19 requiring hospital admission, of which 1,955 (76.8%) patients were included in the final analysis. The patients were randomly divided into training and validation cohorts; applying the nomogram to the latter cohort, the area under the receiver operating characteristics curve (AUROC) was 0.861 (95% CI 0.823–0.900). The nomogram developed is shown in Fig. 1 [13]. It should be noted that the RIM Score-COVID was developed in a cohort of COVID-19 patients admitted during the first wave of the pandemic, when there was a multitude of hospitalized patients, a shortage of RT-PCR diagnostic tests for SARS-CoV-2, and frequent changes in the diagnostic and treatment protocol proposed by the Spanish Ministry of Health. For several weeks in the first wave, COVID-19 was diagnosed based solely on clinical and radiological criteria.
Statistical analysis
Data were expressed as medians (interquartile range) for continuous variables or as numbers and percentages for categorical variables. The normality of the distribution was assessed using the Kolmogorov–Smirnov test. The Kruskal–Wallis test was used for comparison of continuous independent variables (Kolmogorov–Smirnov test < 0.05 in all). Pearson's Chi-square test was performed to compare categorical variables between groups. ROC curves and their corresponding 95% AUC confidence intervals (CI) were calculated [23]. The Statistical Package for Social Sciences SPSS version 23.0 (Armonk, NY, USA) was used to perform all statistical analyses. Significance was established as p < 0.05. The Python package "scikit-learn" (1.0.1) was used to generate ROC curves and their corresponding AUC. Confidence intervals were estimated through bootstrapping 1000 tenfold cross-validation resampling, also calculated with scikit-learn 1.0.1.
Ethical aspects
The study was conducted in accordance with ethical principles and was conducted pursuant to good clinical practice standards and the principles of the latest versions of the Declaration of Helsinki (2013) and the Oviedo Convention (1997). This work was approved by the Institutional Research Ethics Committee of Málaga on March 27, 2020 (Ethics Committee code: SEMI-COVID-19 27–03-20), as per the guidelines of the Spanish Agency of Medicines and Medical Products. All patients gave informed consent. All data collected, processed, and analyzed in this work were anonymized and used only for the purposes of this project. TRIPOD statement guidelines were adhered to in the execution and reporting of the study [24].
Results
Data from 24,514 patients included in SEMI-COVID-19 Registry were analyzed, of which 22,566 (92.1%) were included. All patients were over 18 years old and none died in the ED. Patients admitted prior to February 1, 2020 (n = 15); those with platelet counts higher than × 106 cells/mm3 (n = 6); those with total leukocyte counts lower than the sum of total lymphocytes and total neutrophils (n = 481); and patients missing CRP (n = 767), SpO2 (n = 510), total neutrophils (n = 107), total platelets (n = 46), or sex (n = 16) were excluded, as shown in the flowchart in Fig. 2.
The cohort of patients was divided into time periods according to the aforementioned cut-off points. A total of 15,976 (70.7%) were included in T1, 4,233 (18.7%) in T2, and 2,357 in T3 (10.4%). In total, 4,238 (18.7%) patients died: 3,113 (73.4%) in T1, 702 (16.5%) in T2, and 423 (9.9%) in T3. As to the mortality rate, the proportion of patients who died was highest in T1 (19.4%), followed by T3 (17.9%), and was the lowest in T2 (16.5%) (Table 1).
Table 2 shows the demographic data, comorbidities, evaluation of patients in the ED, as well as the outcomes of all patients included in the study. The groups of patients diagnosed in the different time periods were compared and significant differences were found in terms of age, sex, diabetes mellitus, hypertension, and cancer. Regarding the first evaluation in the ED, all parameters studied were significantly different between the time periods, as were the percentage of patients admitted to the ICU and the percentage of deaths (Table 2).
To predict the probability of death based on variables measured upon hospital admission, ROC curves and their corresponding AUC for the NPR model of the RIM Score were calculated for the total number of patients and for each time period. The AUC of the RIM Score-COVID in the Spanish SEMI-COVID-19 Registry was 0.823 (95% CI 0.819–0.827). The AUC for each time period was 0.834 (95% CI 0.830–0.839) for T1, 0.792 (95% CI 0.781–0.803) for T2, and 0.799 (95% CI 0.785–0.813) for T3 (Fig. 3 A–D).
Discussion
According to our results, the RIM score-COVID (NPR model) is an effective and simple tool for predicting the risk of in-hospital mortality in patients with COVID-19. This nomogram incorporates the NPR, a novel ratio proposed by our group that is able to be calculated using values found in a complete blood count [13]. To our knowledge, this is the first study in which a nomogram designed in the first wave has been validated using data from later periods of the COVID-19 pandemic.
The RIM Score-COVID nomogram uses five accessible, routine, and simple-to-measure parameters commonly obtained in the patient’s first assessment in the ED. An AUC value of 0.861 was obtained in the validation cohort during the nomogram’s development [13]. Values close to 0.8 were reached in the three different time periods, during which the disease was managed differently due to greater knowledge of COVID-19 as the pandemic progressed, the emergence of different strains, and the start of a mass vaccination campaign. This reflects the RIM Score-COVID’s prognostic capacity for in-hospital mortality due to COVID-19 and, therefore, its clinical usefulness.
Several previous studies have published clinical scores that attempt to predict the risk of worse outcomes in patients with COVID-19 [16, 17, 24,25,26,27,28]. However, some of these studies have methodological limitations, including small patient samples, an unrepresentative selection of the control group, or short or incomplete follow-up periods [28].
On the other hand, many of the nomograms and scores published to date are complex, as they include many parameters (some up to 23) and show a lower predictive ability than ours [25,26,27,28,29,30]. Other scores have demonstrated a predictive ability similar to ours, but were developed with a smaller number of patients, do not include training and validation cohorts [31, 32], use parameters that are not accessible in many hospital EDs [33,34,35], or use imaging methods that cannot always be reported objectively in real time [35].
Two scores have been developed from the SEMI-COVID-19 Registry. The first was the PRIORITY model, which included age, dependency, cardiovascular disease, chronic kidney disease, dyspnea, tachypnea, confusion, systolic blood pressure, SpO2 ≤ 93%, and a requirement for oxygen. The AUC value of this model was 0.794 (95% CI: 0.775–0.813) in the validation cohort [17]. The other model developed and evaluated was a machine learning-based in-hospital COvid-19 Disease Outcome Predictor (CODOP), developed in a multicontinental retrospective study that included patients in the SEMI-COVID-19 Registry. CODOP uses 12 clinical parameters that are commonly measured at hospital admission and associated with the pathophysiology of COVID-19 (platelets, eosinophils, neutrophils, monocytes, CRP, creatinine, lactate dehydrogenase, sodium, potassium, glucose, and D-dimer) as well as age. CODOP has a high discriminative ability up to 9 days before clinical resolution (AUROC: 0.90–0.96, 95% CI: 0.879–0.970) [18].
A mortality prediction score has recently been published that is based on a cohort of 6,161 patients with COVID-19 on arrival at the ED of 127 Spanish hospitals. This score is similar to the RIM Score-COVID in that it uses the parameters of age, sex, SpO2, and NLR. It also includes two other parameters: dyspnea, which, although dichotomous (yes/no), is a subjective variable, and estimated glomerular filtration rate, which may not be so easy to obtain. However, its predictive capacity is lower than that of the RIM Score-COVID, with an AUC in the validation cohort of 0.845 (0.819–0.870) [36].
The first wave of COVID-19 posed an unprecedented challenge to healthcare systems around the world [37,38,39]. During those first months, limited resources, the scarcity of diagnostic tests, patients' own fears of going to hospitals, and the lack of knowledge of the disease led to a scenario that—despite the current overload of healthcare systems has fortunately not been repeated with such intensity for the time being [40, 41].
These aspects are reflected when comparing the cohorts of the three time periods: patients admitted in the first wave were older, had a more severe inflammatory state on arrival at the ED, and there were significantly higher percentages of patients admitted and deceased than in the successive waves [40, 42]. However, age, presence of comorbidities, and inflammatory status on arrival at the ED did not differ in deceased patients from the three time periods, with the exception of CRP, which, although elevated in both, was significantly different.
Nevertheless, despite the temporal differences that have led to different hospital care scenarios, the emergence of new SARS-CoV-2 strains, the varying availability of resources, the use of treatments with a higher degree of clinical evidence, and the start of the vaccination campaign, the RIM Score-COVID nomogram has continued to demonstrate prognostic capacity in the three time cohorts, with an AUC in successive waves of 0.79 and 0.80.
Some factors, such as age, hypoxemia, abnormal blood count ratios, and increased acute phase reactants, have been identified as risk factors for mortality and worse prognosis [18, 19, 43]. Early identification of patients at risk for moderate or severe forms of COVID-19 could lead to more forceful clinical behavior in the ED and to reducing admission thresholds using blood count-derived ratios in patients with COVID-19, especially the novel NPR.
Our findings may reflect an underlying inflammatory state that would become evident on the nomogram when weighing and combining complete blood count (CBC) parameters with the other factors identified as relevant. CBC-derived ratios, including the NPR, have been shown to be independent markers of mortality and worse prognosis in COVID-19 patients [11,12,13].
Our study has some limitations. The research has been performed at different time points during the pandemic with all the implications described previously. However, this could also be considered a strength, as it has allowed for the validation of a simple nomogram in patients with confirmed SARS-CoV-2 infection. The study has been performed on patients in Spain, but given the demographic variations among patients from different regions of Spain, it is reasonable to expect the model to perform similarly in other populations. On the other hand, although patients with COVID-19 who were discharged home from the ED were not included in the study, all patients who required hospital admission were included; therefore, our model should be applied to patients who require hospital admission for COVID-19. Lastly, the data were collected from electronic medical records by attending physicians, so important information may have been missing.
During the different study periods, the treatment protocols for patients with COVID-19 changed based on the scientific evidence available at that time. This has led to changes in the therapeutic approach to patients in the different periods/waves of the disease. We believe this aspect lends greater value to our findings since, without assessing the treatments administered, the information obtained from the patient on arrival at the ED is independent of the therapeutic protocols indicated when predicting the risk of in-hospital death due to COVID-19. This logic can also be extended to the effects of different SARS-CoV-2 strains, since although the RIM Score-COVID was developed when the Alpha variant was predominant, its discriminatory capacity has also been proven with later strains, such as the Delta variant. Further studies are needed to demonstrate its discriminatory capacity in new variants such as Omicron. Finally, the SEMI-COVID-19 Registry was not designed to collect information on vaccination in T3, so these data could not be analyzed. However, Spain is one of the countries where vaccination against SARS-CoV-2 has been very successful, with a high percentage of vaccination coverage in population. Vaccination has been shown to reduce the severity of SARS-CoV-2 infection. However, the progression of COVID-19 infection in patients vaccinated against SARS-CoV-2 is well described and related to the time that has elapsed since full vaccination and host characteristics (high comorbidity and immunosuppression). This suggests that decreased immunity and impaired immune responses after vaccination could help explain the high mortality rate [44].
Conclusion
The RIM Score-COVID is an easy and practical quantitative prediction tool. The nomogram uses objective parameters that are easy to obtain and reproducible in most healthcare facilities that treat COVID-19 patients worldwide. It can be calculated at no added cost and without the need for additional laboratory tests. This tool has been validated in a large cohort of patients hospitalized for COVID-19 in different periods of the pandemic in a large number of Spanish hospitals.
These assessments provide further evidence on this highly accurate tool’s predictive ability of the risk of in-hospital COVID-19 mortality, which showed good prognostic ability in successive waves of the pandemic. Further studies are needed to determine the use of this nomogram to support clinical judgment in other populations and healthcare settings around the world.
Data availability
J-M C-R. and J-M R-R have full access to the data and are the guarantors for the data.
References
World Health Organization [Internet]. WHO Coronavirus Disease (COVID-19) Dashboard [Cited 2021 May]. Available from: https://covid19.who.int/
Wu YC, Chen CS, Chan YJ (2020) The outbreak of COVID-19: An overview. J Chin Med Assoc 83:217–220
Richardson S, Hirsch JS, Narasimhan M, Crawford JM, McGinn T, Davidson KW et al (2020) Presenting Characteristics, Comorbidities, and Outcomes among 5700 Patients Hospitalized with COVID-19 in the New York City Area. JAMA 323:2052–2059
Casas-Rojo JM, Antón-Santos JM, Millán-Núñez-Cortés J, Umbreras-Bermejo C, Ramos-Rincón JM, Roy-Vallejo E et al (2020) Clinical characteristics of patients hospitalized with COVID-19 in Spain: Results from the SEMI-COVID-19 Registry. Rev Clin Esp 220:480–494
Hashmi HAS, Asif HM (2020) Early Detection and Assessment of Covid-19. Front Med (Lausanne) 7:311. https://doi.org/10.3389/fmed.2020.00311
Gong J, Ou J, Qiu X, Jie Y, Chen Y, Yuan L et al (2020) A Tool for Early Prediction of Severe Coronavirus Disease 2019 (COVID-19): A Multicenter Study Using the Risk Nomogram in Wuhan and Guangdong. China Clin Infect Dis 71:833–840
Garibaldi BT, Fiksel J, Muschelli J, Robinson ML, Rouhizadeh M, Perin J et al (2021) Patient Trajectories Among Persons Hospitalized for COVID-19: A Cohort Study. Ann Intern Med 174:33–41
Liu J, Liu Y, Xiang P, Pu L, Xiong H, Li C et al (2020) Neutrophil-to-lymphocyte ratio predicts critical illness patients with 2019 coronavirus disease in the early stage. J Transl Med 18:206
Mehta P, McAuley DF, Brown M, Sanchez E, Tattersall RS, Manson JJ, HLH Across Speciality Collaboration, UK (2020) COVID-19: consider cytokine storm syndromes and immunosuppression. Lancet 395:1033–1034
Yang AP, Liu JP, Tao WQ, Li HM (2020) The diagnostic and predictive role of NLR, d-NLR and PLR in COVID-19 patients. Int Immunopharmacol 84:106504
Jimeno S, Ventura PS, Castellano JM, García-Adasme SI, Miranda M, Touza P et al (2021) Prognostic implications of neutrophil-lymphocyte ratio in COVID-19. Eur J Clin Invest 51:e13404
López-Escobar A, Madurga R, Castellano JM, Ruiz de Aguiar S, Velázquez S, Bucar M et al (2021) Hemogram as marker of in-hospital mortality in COVID-19. J Investig Med 69:962–969
López-Escobar A, Madurga R, Castellano JM, Velázquez S, Suárez Del Villar R, Menéndez J et al (2021) Risk score for predicting in-hospital mortality in COVID-19 (RIM Score). Diagnostics (Basel) 11:596
Ramos-Rincon JM, Buonaiuto V, Ricci M, Martín-Carmona J, Paredes-Ruíz D, Calderón-Moreno M, SEMI-COVID-19 Network et al (2021) Clinical characteristics and risk factors for mortality in very old patients hospitalized with COVID-19 in Spain. J Gerontol A Biol Sci Med Sci 76:e28–e37
Rubio-Rivas M, Mora-Luján JM, Montero A, Aguilar García JA, Méndez Bailón M, Fernández Cruz A, SEMI-COVID-19 Network et al (2022) The use of corticosteroids or tocilizumab in COVID-19 based on inflammatory markers. J Gen Intern Med 37:168–175
Díaz-Simón R, Lalueza A, Lora-Tamayo J, Rubio-Rivas M, Mendo CL, Martínez ML, SEMI-COVID-19 Network et al (2021) Clinical characteristics and risk factors of respiratory failure in a cohort of young patients requiring hospital admission with SARS-CoV2 infection in spain: results of the multicenter SEMI-COVID-19 registry. J Gen Intern Med 36:3080–3087
Martínez-Lacalzada M, Viteri-Noël A, Manzano L, Fabregate M, Rubio-Rivas M, Luis García S, SEMI-COVID-19 Network et al (2021) Predicting critical illness on initial diagnosis of COVID-19 based on easily obtained clinical variables: development and validation of the PRIORITY model. Clin Microbiol Infect 27(12):1838–1844
Klén R, Purohit D, Gómez-Huelgas R, Casas-Rojo JM, Antón-Santos JM, Nunez-Cortes JM et al (2022) Development and evaluation of a machine learning-based in-hospital COvid-19 disease outcome predictor (CODOP): a multicontinental retrospective study. Elife 11:e75985
RECOVERY Collaborative Group, Horby P, Lim WS, Emberson JR, Mafham M, Bell JL, Linsell L et al (2021) Dexamethasone in hospitalized patients with Covid-19. N Engl J Med 384:693–704
WHO Solidarity Trial Consortium, Pan H, Peto R, Henao-Restrepo AM, Preziosi MP, Sathiyamoorthy V, Abdool Karim Q et al (2021) Repurposed Antiviral Drugs for Covid-19 - Interim WHO Solidarity Trial Results. N Engl J Med 384:497–511
Gottlieb RL, Vaca CE, Paredes R, Mera J, Webb BJ, Perez G et al (2022) Early Remdesivir to Prevent Progression to Severe Covid-19 in Outpatients. N Engl J Med. N Engl J Med 386:305–315
Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC et al (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 12:77
Collins GS, Reitsma JB, Altman DG, Moons KG (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:7594
Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E et al (2020) Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ 369:1328
Jehi L, Ji X, Milinovich A, Erzurum S, Merlino A, Gordon S et al (2020) Development and validation of a model for individualized prediction of hospitalization risk in 4,536 patients with COVID-19. PLoS ONE 15:e0237419
Zhou Y, He Y, Yang H, Yu H, Wang T, Chen Z et al (2020) Development and validation a nomogram for predicting the risk of severe COVID-19: A multi-center study in Sichuan. China PLoS One 15:e0233328
Jehi L, Ji X, Milinovich A, Erzurum S, Rubin BP, Gordon S et al (2020) Individualizing Risk Prediction for Positive Coronavirus Disease 2019 Testing: Results From 11,672 Patients. Chest 158:1364–1375
Huang J, Cheng A, Lin S, Zhu Y, Chen G (2020) Individualized prediction nomograms for disease progression in mild COVID-19. J Med Virol 92:2074–2080
Liang W, Liang H, Ou L, Chen B, Chen A, Li C, China Medical Treatment Expert Group for COVID-19 et al (2020) Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med 180:1081–1089
Chen R, Liang W, Jiang M, Guan W, Zhan C, Wang T et al (2020) Risk factors of fatal outcome in hospitalized subjects with coronavirus disease 2019 from a nationwide analysis in China. Chest 158:97–105
Ji D, Zhang D, Xu J, Chen Z, Yang T, Zhao P et al (2020) Prediction for Progression Risk in Patients With COVID-19 Pneumonia: The CALL Score. Clin Infect Dis 71:1393–1399
Cai L, Zhou X, Wang M, Mei H, Ai L, Mu S et al (2021) Predictive Nomogram for Severe COVID-19 and Identification of Mortality-Related Immune Features. J Allergy Clin Immunol Pract 9:177–84.e3
Dong YM, Sun J, Li YX, Chen Q, Liu QQ, Sun Z et al (2021) Development and Validation of a Nomogram for Assessing Survival in Patients With COVID-19 Pneumonia. Clin Infect Dis 72:652–660
Chen H, Chen R, Yang H, Wang J, Hou Y, Hu W et al (2021) Development and validation of a nomogram using on admission routine laboratory parameters to predict in-hospital survival of patients with COVID-19. J Med Virol 93:2332–2339
Yu Y, Wang X, Li M, Gu L, Xie Z, Gu W et al (2020) Nomogram to identify severe coronavirus disease 2019 (COVID-19) based on initial clinical and CT characteristics: a multi-centre study. BMC Med Imaging 20:111
Berenguer J, Borobia AM, Ryan P, Rodríguez-Baño J, Bellón JM, Jarrín I et al (2021) Development and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score. Thorax 76:920–929
Adeyemo OO, Tu S, Keene D (2021) How to lead health care workers during unprecedented crises: A qualitative study of the COVID-19 pandemic in Connecticut, USA. PLoS ONE 16:e0257423
Sun S, Xie Z, Yu K, Jiang B, Zheng S, Pan X (2021) COVID-19 and healthcare system in China: challenges and progression for a sustainable future. Global Health 17(1):14. https://doi.org/10.1186/s12992-021-00665-9
Mollinedo-Gajate I, Villar-Álvarez F, Zambrano-Chacón M, Los Á, Núñez-García L, de la Dueña-Muñoz L, López-Chang C et al (2021) First and Second Waves of Coronavirus Disease 2019 in Madrid, Spain: Clinical Characteristics and Hematological Risk Factors Associated With Critical/Fatal Illness. Crit Care Explor. 3:e0346
Asch DA, Sheils NE, Islam MN, Chen Y, Werner RM, Buresh J et al (2021) Variation in US Hospital Mortality Rates for Patients Admitted With COVID-19 During the First 6 Months of the Pandemic. JAMA Intern Med 181:471–478. https://doi.org/10.1001/JAMAINTERNMED.2020.8193
Mahmoud M, Carmisciano L, Tagliafico L, Muzyka M, Rosa G, Signori A, GECOVID Study Group et al (2021) Patterns of Comorbidity and In-Hospital Mortality in Older Patients With COVID-19 Infection. Front Med (Lausanne). 8:726837
AbuRuz S, Al-Azayzih A, ZainAlAbdin S, Beiram R, Al HM (2022) Clinical characteristics and risk factors for mortality among COVID-19 hospitalized patients in UAE: Does ethnic origin have an impact. PLoS ONE 17(3):e0264547
Grasselli G, Zangrillo A, Zanella A, Antonelli M, Cabrini L, Castelli A et al (2020) Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to icus of the lombardy region. Italy. JAMA. 323(16):1574–1581
Moreno-Perez O, Ribes I, Boix V, Martinez-García MÁ, Otero-Rodriguez S, Reus S, On behalf the COVID-19 ALC research group et al (2022) Hospitalized patients with breakthrough COVID-19: Clinical features and poor outcome predictors. Int J Infect Dis 118:89–94
Acknowledgements
We gratefully acknowledge all the investigators and staff from the SEMI-COVID-19 Registry who participated in the collection of the patient data (see Appendix 1).
Funding
There are no sources of funding for this manuscript.
Author information
Authors and Affiliations
Consortia
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
The project was approved by the Institutional Research Ethics Committee of Málaga on 27 March 2020 (Ethics Committee code: SEMI-COVID-19 27–03-20), as per the guidelines of the Spanish Agency of Medicines and Medical Products. All patients gave informed consent.
Informed consent
Only patients who had previously given consent for their medical records to be used for medical research were included in this registry. Data confidentiality and patient anonymity were always maintained, in accordance with Spanish regulations on observational studies.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Awards obtained: the RIM Score-COVID project was winner of the EpidemiXs–COVID Warriors Challenge of the V National Health Hackathon. Madrid, November 2020.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ramos-Rincón, JM., Ventura, P.S., Casas-Rojo, JM. et al. Validation of the RIM Score-COVID in the Spanish SEMI-COVID-19 Registry. Intern Emerg Med 18, 907–915 (2023). https://doi.org/10.1007/s11739-023-03200-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11739-023-03200-3