Introduction

Despite the advances in the early recognition and treatment, sepsis still represents one of the leading causes of morbidity and mortality worldwide [1], being the tenth leading cause of death in the general population, and the second leading cause of death in non-coronary Intensive Care Units (ICU) [2]. The majority of patients diagnosed with sepsis in hospitals are admitted through the Emergency Department (ED) [3]. Sepsis is a rapidly evolving condition, which can respond to the initial treatment, or progress to septic shock. Treatment [4] is most effective when initiated early with a significant proportion of patients spending several hours in the ED after being diagnosed with sepsis [5].

In this era of economic restraints, with few ICU beds and the increasing availability of High-Dependency Units (HDU), it is useful to identify patients at high risk of adverse outcomes early to properly resuscitate them.

Several scoring systems have been developed to predict mortality in critically ill patients. We selected two prognostic scores, both conceived as outcome prediction tools: Acute Physiology and Chronic Health Evaluation (APACHE) II [6] and Mortality in Emergency Department Sepsis (MEDS) [7]. APACHE II is a validated prognostic score designed for use in the ICU; MEDS is the only score specifically designed to stratify septic patients presenting to the ED. Conversely, the Sequential Organ Failure Assessment (SOFA) score [8] represents a simple and objective score that allows for calculation of both the number and the severity of organ dysfunction. Predisposition, Infection, Response and Organ Damage (PIRO) score [9] was designed as a staging system for septic patients. Finally, qSOFA was recently developed with the goal to identify adult patients with suspected infection, who are likely to have poor outcomes [10].

The largest proportion of validation studies includes all patients admitted to the ICU, but not specifically septic patients; in a minority of cases, the scores were tested in ED septic patients. We, therefore, decided to perform this analysis to test the aforementioned scores in a population of septic not-intubated patients, managed in an HDU.

ED-HDU is a clinical setting that is gaining widespread diffusion in Italy. Italian standards for Hospital Organization (Decreto No 70, 2015, April 2nd,) provides for creation of HDU beds in every Hospital with an ED. The ED-HDU is a sub-intensive care unit, with availability of advanced monitoring, managed by emergency physicians; all patients are admitted from the ED, according to bed availability. Within 48 h from ED admission, the ED-HDU physicians must decide the optimal patients’ disposition, choosing between the ordinary ward and the intensive or sub-intensive care facilities. Because our ED-HDU does not have invasive mechanical ventilators, intubated patients or those potentially requiring intubation within 24 h are directly admitted to the ICU. In a previous analysis of a population of septic patients, we demonstrate that ED-HDU allows a significant reduction in ICU admissions, leading to a relevant cost reduction [11].

The aims of this study are: (1) to evaluate the prognostic performance of the most commonly used prognostic scores in septic patients managed in an ED-HDU; (2) to evaluate the incremental prognostic value of SOFA score calculated at ED admission and after 24 h, over age, the previous medical conditions, evaluated through Charlson Comorbidity Index, vital signs, and lactate level.

Materials and methods

Study design and settings

This study is a retrospective review of all consecutive patients admitted from June 2008 to April 2016 in the ED-High-Dependency Unit (HDU), at Careggi University-Hospital (Florence, Italy), an urban, 1600-bed tertiary care centre with 56,000 ED visits per year. We included all subjects with a diagnosis of sepsis, severe sepsis, or septic shock (according to the 2001 SCCM/ESICM/ACCP/ATS/SIS criteria [12]). All patients were 18 years of age or older and not-intubated.

Study population

Patients were identified according to ED-HDU admission diagnosis from electronic medical records. For each patient, basic demographic data and previous medical conditions were collected from medical records using a standardized collection template; the variables needed to construct the scores were collected at ED admission (T0) and after the first 24 h (T1) of ED-HDU admission. For a single missing value at T1, a replacement was calculated using the mean value of the T0 value and a value collected at 48 h and reported in the ED-HDU medical record; otherwise, it was considered as a missing value in the analysis.

Because the Glasgow Coma Scale (GCS) and the respiratory rate were not always recorded exactly in the ED charts, these items were scored by deduction from other items in the chart. For GCS values that were not noted exactly, the description ‘alert’ was considered ‘15’, and a change in mental status was scored as ‘14’. Similarly, for respiratory rates, we converted ‘normal respiratory rate’ into 18/min, as this value is considered normal in all the scores, and ‘tachypneic’ to 30/min, as this is considered abnormally high in all scores as well.

The lactate clearance was defined by the equation [(T0 lactate − T1 lactate)/T0 lactate] × 100%.

Calculation of the scores

We retrospectively calculated MEWS [13], qSOFA [10], Charlson Comorbidity Index [14], SOFA [8], APACHE II [6], MEDS [7], and PIRO. Charlson Comorbidity index assigns a score between 1 and 6 points to a range of diseases (one point for myocardial infarction, congestive heart failure, peripheral arterial disease, cerebrovascular disease, dementia, chronic pulmonary disease, connective tissue disease, ulcer disease, mild liver disease, and diabetes without organ damage; two points for diabetes with organ damage, hemiplegia, severe renal disease, and non-metastatic cancer; three points for severe liver disease; six points for metastatic cancer and human immunodeficiency virus infection), and the sum of these points serves as a measure of the burden of comorbidity.

We chose to use the PIRO model as proposed by Howell et al. [15], because it was developed in ED patients presenting with a wide range of disease severity: this model was, therefore, more suitable for our patients compared with other PIRO versions derived from ICU patients. In both MEDS and PIRO scores calculation, we did not include immature neutrophils (or “bands”) percentage, since it is not routinely measured in our country (Italy). MEDS score and Charlson Comorbidity Index were only evaluated at T0, because they both include several parameters which do not change in the first 24 h; APACHE II score was calculated based on the worst values of the first 24 h after ED admission, and, by design, it is calculated once after the first 24 h of hospital admission. qSOFA was calculated once, based on the first vital signs collected at ED entry (T0); by retrospective design, we considered altered mentation as to any GCS value lower than 15. SOFA and MEWS scores were calculated both at T0 and T1, because they include parameters that can significantly vary after a few hours, and a serial evaluation reflects a real change in a patient’s clinical condition and the effect of therapeutic interventions. The delta SOFA (ΔSOFA) was calculated as the difference between the T1 and T0 SOFA score. Patients who died during the first 24 h (n = 30) were not included in T1 score calculation.

A phone follow-up was performed to assess day-28 mortality rate for patients discharged before this term. The primary outcome is day-28 mortality; the secondary outcome is the need for ICU admission.

The study is consistent with the principles of the Declaration of Helsinki of clinical research involving human subjects. Informed consent was obtained from all individual participants included in the study when they were called for follow-up.

Statistical analysis

Continuous variables were reported as mean ± standard deviation, and comparison between two groups was performed with the Student t test for unpaired data. Categorical data were analyzed using contingency tables and performing χ 2 test. The scoring systems were reported as medians and interquartile range (IQR). All score comparisons between different groups have been performed by Mann–Whitney’s test for non-parametric data. We assessed discrimination and calibration of the scoring systems in predicting mortality: discrimination was tested by building receiver operating characteristics (ROC) curves, and calculating the area under the curves (AUCs) with 95% confidence interval (CI). To evaluate differences in trend among different scores, we employed the analysis of variance (ANOVA) for repeated measures. To identify incremental prognostic value of SOFA score compared to the previous medical conditions, vital signs, and biomarkers, a survival analysis using Cox’s logistic regression was performed. p < 0.05 were considered as significant. All statistical analyses were carried out using SPSS version 21 (SPSS Statistics Inc., Armonk, New York, USA).

Results

Between June 2008 and April 2016, 765 patients were admitted to the ED-HDU with a diagnosis of sepsis/severe sepsis/septic shock. Twenty-three patients were excluded as 28-day follow-up was incomplete. The study includes 742 patients (Table 1) with a 28-day mortality rate of 31%.

Table 1 Demographic and clinical characteristics of the study population
Full size table

Performance of prognostic scores

All examined scores are significantly higher in non-survivors compared with survivors (Table 2). The different scores show a modest-to-moderate discrimination (AUC: T0 SOFA 0.695, 95% CI 0.653–0.737; T1 SOFA 0.741, 95% CI 0.700–0.781; T0 MEWS 0.662, 95% CI 0.618–0.705; T1 MEWS 0.729, 95% CI 0.685–0.774; PIRO: 0.646, 95% CI 0.603–0.690; APACHE II 0.756, 95% CI 0.718–0.794; qSOFA 0.625, 95% CI 0.579–0.671; Charlson Comorbidity Index 0.596, 95% CI 0.552–0.640; MEDS 0.674, 95% CI 0.633–0.715, all p < 0.001); calibration, evaluated by Hosmer–Lemenshow test, is not significant for all scores (data not shown).

Table 2 Scoring systems, laboratory tests, and vital signs at Emergency Department admission and after 24 h
Full size table

Mortality rate significantly increases across higher SOFA score quartiles (Fig. 1a) and with increasing number of organ dysfunction (Fig. 1b). Non-survivors have a significantly higher ΔSOFA compared to survivors (0.37 ± 2.15 vs. −0.28 ± 1.88, p < 0.0001). ANOVA for repeated measures demonstrates an increasing trend of SOFA scores over the first 24 h in non-survivors, and a decreasing trend in survivors (p between subjects <0.0001); MEWS score improvement is significantly blunted in non-survivors compared with survivors (p = 0.002).

Fig. 1
figure 1

28-day mortality rate for SOFA score quartiles (a) and according to the number of organ dysfunction (b)

Full size image

Lactate concentrations are significantly higher in non-survivors compared with survivors both at T0 and T1; a lactate clearance ≥10% over 24 h is significantly more common among survivors (74 vs. 65%, p = 0.026) compared with non-survivors. Considering the secondary outcome need of ICU admission, all scores were evaluated only retrospectively. MEWS at T1 (admitted in ICU: median 4, IQR 2.25–6; not admitted to ICU: median 3, IQR 2–5, p = 0.011), APACHE II (admitted in ICU: median 19, IQR 15–24; not admitted to ICU: median 16, IQR 13–21, p = 0.001), qSOFA (admitted in ICU: median 2, IQR 1–3; not admitted to ICU: median 2, IQR 1–3, p = 0.023), and SOFA at both T0 (admitted in ICU: median 6, IQR 4–9; not admitted to ICU: median 5, IQR 3–7, p < 0.001) and T1 (admitted in ICU: median 6, IQR 4–9; not admitted to ICU: median 4, IQR 3–7, p = 0.001) are significantly higher in patients who require admission to an ICU compared with patients admitted to an ordinary ward.

We compared the subcomponents of the PIRO score between patients with sepsis from a respiratory tract infection and patients with an intra-abdominal source. Only the T0 Organ component is significantly lower in respiratory sepsis compared with other infection origin sites (4.5 ± 2.8 vs. 5.3 ± 2.9, p = 0.002). All other domains do not demonstrate any significant difference between septic syndromes from a different origin.

Multivariate analysis

To assess the incremental prognostic value of SOFA score over the previous medical conditions and other indices of sepsis severity, we performed a Cox multivariate analysis of survival, including T0 and T1 parameters (Table 3). This analysis was performed only for the SOFA score, because the other scores (APACHE, MEDS, and PIRO) already include several of these variables. Analysis for these scores would, therefore, be incorrect due to a significant risk of a double counting. We aimed at inserting variables in the same order that they become available during the real-life evaluation of a septic patient in the ED. In the first two steps, a more advanced age is significantly associated with an increased mortality risk; comorbidities, expressed through Charlson Comorbidity Index, maintain an independent prognostic value even when age-adjusted. In models 3 and 4, we added parameters commonly used to assess sepsis severity: vital signs, grouped in MEWS score, and dichotomized lactate concentration (≤ or >2 mmol/L). MEWS score demonstrate an incremental prognostic value both at T0 and T1, while lactate concentration proves to be an independent prognostic ability only at T1. Finally, we entered SOFA score, which holds a significant incremental prognostic value both at T0 and T1. After introduction of sepsis severity variables, comorbidities lose their independent prognostic value.

Table 3 Incremental prognostic value of SOFA score over age, previous medical conditions evaluated through Charlson Comorbidity Index, vital signs, and lactate level: stepwise multivariate analysis at Emergency Department admission (T0) and after 24 h (T1)
Full size table

In a Cox multivariate survival analysis, mortality rate significantly increases per increasing qSOFA (RR 1.282, 95% CI 1.126–1.458, p < 0.0001), adjusted by age (RR 1.034, 95% CI 1.019–1.049, p < 0.0001), Charlson score (RR 1.063, 95% CI 1.019–1.116, p = 0.014), and a lactate concentration >2 (RR 1.754, 95% CI 1.287–2.390, p < 0.0001).

We finally dichotomized our population according to age (≤65 and >65 years) and SOFA score (≤5 and >5, according to the median value in this population), to suggest a cutoff, which might identify an increased mortality risk. As shown in Fig. 2, older patients with a SOFA >5 show a disproportionally high mortality rate.

Fig. 2
figure 2

Mortality rate according to age group (patients aged ≤ or >65 years) and T1 SOFA score (≤ or >5)

Full size image

Discussion

In a large population of non-intubated septic patients, we demonstrate that SOFA score, which includes parameters available in the ED, is significantly higher in non-survivors and patients who need ICU admission; discrimination ability is fair at ED admission, and it is moderate after 24 h. An incremental prognostic value is demonstrated over the previous medical conditions, vital signs, and lactate concentration at ED admission and after 24 h. The trend over this period demonstrates an increase in non-survivors and a decrease in survivors. Sepsis is a rapidly evolving disease and the initial framework does not appear to be exhaustive: the earliest SOFA score represents baseline severity of illness, while the change in SOFA score reflects events during the course of hospitalization, such as clinical deterioration or inadequate response to therapeutic interventions.

In several previous studies, both on ICU [16–21] and ED patients [22, 23], the initial, mean, and highest SOFA score correlates with mortality; in a study by Jones et al. [24], which confirms the prognostic value of SOFA score variation, the trend was evaluated in the first 72 h, a time frame too long for the management of patients in the ED setting. Compared with existing studies, our population is significantly larger, and it only includes septic patients; the timing of our serial evaluation, at ED admission, and after 24 h appears to be adequately tailored for an Emergency Medicine setting, including both ED and ED-HDU. Even considering the retrospective design of the study, our results support the possible employment of SOFA for prognostic stratification of patients admitted in an ED-HDU. Finally, in a multivariate analysis, SOFA score gives additional prognostic information over the previous medical conditions and indices of sepsis severity.

In our study population, age [25] maintains an independent prognostic value in all the multivariate analysis steps, confirming that it heavily weights septic patients’ outcomes; the suggested cutoff of 65 years and SOFA score of 5 must be interpreted with caution, as they are not prospectively validated [26]. Non-survivors show significantly higher lactate concentration and a reduced clearance compared with survivors. Nevertheless, lactate concentration at the moment of sepsis diagnosis, adjusted by SOFA score, does not demonstrate an independent prognostic value, while it does at T1. At the moment of the first diagnosis, high lactate concentration has a multifactorial pathogenesis, including increased production and decreased clearance, and it is not necessarily related to hypoperfusion. Persistence of high lactate concentration after 24 h is independently associated with an untoward prognosis. Despite the relevant heterogeneity of the populations and the different timing of the measurements, many studies demonstrate that a change in lactate concentration over the first hours of treatment represents a valuable tool in the evaluation of critically ill patients, and persistence of high concentrations might indicate a poor response to treatment [27–29].

Overall, the evaluation performed at ED entrance, which included T0 SOFA, qSOFA, T0 MEWS, PIRO score, and MEDS, shows a fair prognostic discrimination ability; the new proposed qSOFA demonstrates an independent prognostic value even after correction for age, Charlson score, and presence of an increased lactate concentration.

We also included APACHE II score, because it remains the most employed score in the medical community. APACHE II proves to be a good discrimination between different outcomes, but other studies [30] demonstrate the difficulty of applying APACHE II in the ED for its late availability and the large amount of required variables. MEDS and PIRO confirm a fair prognostic discrimination ability: in the previous studies, MEDS performs poorly in patients with severe stages of septic disease [30, 31]. There is a mild discriminatory ability with PIRO, but no significant score difference is observed with respect to a different infection site [32].

A Task Force recently evaluated which clinical criteria best identifies infected patients most likely to have sepsis [33]. After the interrogation of large data sets of hospitalized patients with presumed infection, they demonstrate a moderate predictive validity for SOFA score, comparable with our results. Considering the ease of use and the comparable performance with other more complex scores, the Task Force recommends using a change in baseline of the total SOFA score of two points or more to represent organ dysfunction, which is a key criterion for the diagnosis of sepsis. Using Sepsis-3, sepsis requires life-threatening organ dysfunction; adopting these criteria would select for sicker patients, because we included septic patients with an infection but not end-organ dysfunction. We cannot exclude that different selection criteria had an impact upon our results. All available studies, including those employed to formulate Sepsis-3, are retrospective, and this framework limits the translation of a retrospectively validated score into a clinical decision rule [34]: in future years, we plan to employ SOFA to confirm the diagnosis of sepsis. This might become an opportunity for a prospective evaluation of the score’s prognostic value.

There are some limitations in this report. This study is retrospective. It was also conducted in a single center, and ED-HDUs are not widespread, so our results may not be generalizable, especially outside of European countries. Moreover, as all the patients were admitted to the ED-HDU, they were not-intubated: we cannot exclude that different selection criteria would have an impact upon the results.

Conclusion

In a large population of non-intubated septic patients, we demonstrate that SOFA score is significantly higher in patients with an adverse outcome in terms of 28-day mortality and ICU admission need, compared with patients with a good outcome. An incremental prognostic value over the previous medical conditions, vital signs, and lactate concentration is demonstrated at both ED admission and at 24 h. An age >65 years with a SOFA score >5 identifies septic patients with a very high mortality.