Introduction

Identifying patients with suspected infection who are at risk of developing sepsis is crucial in the emergency department (ED) because sepsis and related adverse outcomes can be prevented with early recognition and prompt treatment [1]. Several risk scoring systems have been developed for patients at risk of developing sepsis. However, these systems have been found to be more effective for critically ill patients, especially those being treated in intensive care units (ICU) [2]. In the ED setting, an easy-to-use, fast, and accurate scoring system is needed. Although systemic inflammatory response syndrome (SIRS) criteria have traditionally been used to diagnose sepsis, quick sequential organ failure assessment score (qSOFA) was adopted after the Sepsis‑3 recommendations [3]. However, underdiagnosis and delay in sepsis recognition are still a major concern in the ED [3, 4].

Although a number of studies have evaluated the predictive performance of SIRS and qSOFA for identifying sepsis and sepsis-related adverse outcomes, most were either not conducted with an infection-suspected ED population, were retrospective, or had a low number of study patients [4]. Also, most of the previous studies focused on mortality as an adverse outcome, and adverse outcomes such as hospitalization and mortality as a composite outcome after ED visits were not studied. The aim of this study was to determine the performance of SIRS, qSOFA, and positive combinations of both scoring systems for predicting adverse outcomes as a composite outcome in patients with suspected infection in the ED.

Methods

Design, setting, and population

This single-center, prospective cohort study was conducted at the ED of an academic tertiary care facility with an annual census of 60,000 patients. The patients admitted to the ED between October 2017 and April 2018 were screened for eligibility for this study. All patients who were > 18 years old, suspected of any type of infection by the treating emergency physician, and who agreed to participate in the study were included during working hours on weekdays. Of those unable to consent at the time of enrollment, informed consent was obtained from the caregivers. Pregnant patients, patients who were prisoners, and those who were not ordered a complete blood count by treating physicians were excluded from the study. Institutional review board approval was obtained for the study (KU GOKAEK 2017/12.6, 2017/246).

Study protocol

The participants’ demographic information, vital signs, comorbidities, and laboratory test results, including white blood cells (WBC), hemoglobin, platelets, bilirubin, and lactate levels, were recorded. Each patient’s qSOFA and SIRS scores were calculated (Table 1). A qSOFA score of ≥ 2 or SIRS criteria of ≥ 2 was defined as risk-positive for adverse outcome. Subsequently, two positive combinations of qSOFA and SIRS were defined. For combination‑1, either a qSOFA or a SIRS criteria of ≥ 2; for combination‑2, both qSOFA score and SIRS criteria of ≥ 2 were defined as risk-positive for adverse outcome. The patients who needed close monitoring, parenteral antibiotics, and intravenous fluids were admitted to non-ICU hospital wards, and those who required additional mechanical ventilation or inotropes were admitted to the ICU. The patients were followed up by phone and from electronic medical health records on day 30 after the initial ED visit to determine whether a composite outcome occurred. Composite outcome was defined as all-cause mortality, ICU, or non-ICU hospital admission within 30 days of the ED visit.

Table 1 Definition of qSOFA score and SIRS criteria
Full size table

Outcome measures

The primary outcome of this study was the performance of qSOFA, SIRS, qSOFA + SIRS combination‑1 and combination‑2 in predicting 30-day composite outcome in an ED population. The secondary outcome was all-cause mortality and sepsis-related ICU or non-ICU hospital ward admission within 30 days.

Data analysis

The statistical analysis was performed with SPSS version 20 (SPSS for Windows, IBM Corp., Armonk, NY, USA). Normality of distribution was tested with the Kolmogorov–Smirnov test. Non-normally distributed continuous variables were expressed median and interquartile range (IQR), and categorical variables were expressed with percentages. The correlation between the ordinal variables were evaluated in the crosstabs and presented with Spearman correlation coefficient. The risk ratio (RR) of developing a composite outcome for qSOFA, SIRS and qSOFA + SIRS combinations were calculated. The Vassarstat.net statistical calculator tool was used to calculate the sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR−), and positive and negative predictive values (PPV and NPV) [5] to determine the predictive performance of the scoring systems. The significance was set at a p value of 0.05.

Results

During the study period, 9709 patients were screened. Of the patients, 577 were suspected of a clinical infection. After excluding 218 patients, a total of 350 patients were analyzed for the study (Fig. 1). Of the analyzed patients, 178 (50.9%) were male, and the median age was 63 (IQR 27). The patients’ demographic and clinical features are shown in Table 2. qSOFA was ≥ 2 in 82 patients and SIRS was ≥ 2 in 267 patients. The distribution of the qSOFA and SIRS scores was presented in Fig. 2. There was no significant correlation between qSOFA and SIRS for predicting composite outcome (r = 0.15; p = 0.005) (Table 3).

Fig. 1
figure 1

Patient flowchart

Full size image
Fig. 2
figure 2

Number of the patients with composite outcome according to the distribution of a SIRS criteria and b qSOFA score

Full size image
Table 2 Demographic and clinical features of the patients
Full size table
Table 3 Correlation between qSOFA and SIRS
Full size table

The composite outcome occurred in 211 (60.3%) patients within 30 days: mortality in 84 (24%), ICU admission in 78 (22.3%), and non-ICU hospital ward admission in 154 (44%). The performance of qSOFA in predicting composite outcome had a sensitivity of 0.34, a specificity of 0.93, an LR+ of 4.74, and an LR− of 0.71. For SIRS, the sensitivity was 0.81, the specificity was 0.31, the LR+ was 1.17, and the LR− was 0.61; for qSOFA + SIRS combination‑1, the sensitivity was 0.84, the specificity was 0.28, the LR+ was 1.17, and the LR− was 0.57; for combination‑2, the sensitivity was 0.31, the specificity was 0.96, the LR+ was 7.25 and the LR− was0.72. The RR of developing a composite outcome for patients was 1.69 (95% CI 1.47–1.95) with using qSOFA+, with SIRS+ it was 1.33 (95% CI 1.05–1.69), with combination‑1 it was 1.37 (95% CI 1.06–1.78) and with combination‑2 it was 1.76 (95% CI 1.54–2.01). The performance of the scores for predicting composite outcome, mortality, ICU, and non-ICU hospital ward admission are shown in Table 4.

Table 4 Predictive performance of qSOFA, SIRS and qSOFA + SIRS combinations for mortality, non-ICU and ICU admission
Full size table

Discussion

In this prospective study, the specificity of qSOFA and qSOFA + SIRS combination‑2 were found to be the highest for predicting 30-day composite outcomes in patients with suspected infection in the ED. Furthermore, the sensitivity of SIRS and qSOFA + SIRS combination‑1 were highly sensitive in this study. These results suggest that the use of qSOFA and qSOFA + SIRS combination‑2 in decisions for patient disposition, and SIRS and qSOFA + SIRS combination‑1 in the screening of risk for adverse outcomes provide better utility for emergency physicians. The use of the qSOFA + SIRS combination‑2 increased the specificity by 3–11% compared to qSOFA alone. Similarly, the use of qSOFA + SIRS combination‑1 also increased the sensitivity by 3% compared to SIRS alone. Unlike the previous studies, this study also reported the predictions for composite outcome and non-ICU hospitalization. Compared to the literature, qSOFA and SIRS showed a similar trend in sensitivity and specificity for predicting mortality and ICU admissions [2,3,4]; however, qSOFA showed the highest specificity for predicting composite outcome, and SIRS showed the highest sensitivity for predicting ICU admission.

The Sepsis‑3 task force suggested the use of qSOFA as a screening tool for high-risk patients in a non-ICU population where a laboratory evaluation is not practical [6]. This recommendation has commonly been adopted by emergency physicians to triage patients with suspected infection [7]. However, this study suggests that qSOFA would not be a reliable exclusion method for predicting adverse outcomes in the ED because of its lower sensitivity. Conversely, it had excellent specificity and predictive performance for composite 30-day outcome following the ED visit. Several meta-analyses and systematic reviews have evaluated the predictive performance of qSOFA and SIRS for mortality in a non-ICU population [8,9,10]. Although similar sensitivity and specificity values were reported in these meta-analyses, the main limitation of these meta-analyses is that most of the data were pooled from retrospective studies with significant heterogeneity between studies. In the meta-analysis by Franchini et al., the pooled sensitivity and specificity were reported as 51% and 78% for qSOFA and 86% and 27% for SIRS in predicting mortality [8]. In another meta-analysis by Jiang et al. [9], qSOFA and SIRS to predict mortality in ED patients with infection were compared. The pooled sensitivity and specificity were reported as 42% and 88% for qSOFA and 81% and 41% for SIRS. Another meta-analysis by Song et al. [10], which evaluated the predictive ability of qSOFA and SIRS for in-hospital mortality, acute organ dysfunction, and ICU admission, reported a sensitivity of 51% and specificity of 83% for qSOFA and a sensitivity of 86% and specificity of 29% for SIRS in predicting in-hospital mortality. Moreover, for predicting ICU admission, qSOFA had a sensitivity of 53% and specificity of 75%, and SIRS had a sensitivity of 91% and specificity of 14% [10]. In this study, the sensitivity and specificity of qSOFA and SIRS were in line with the literature for predicting mortality. However, compared to the literature, in this study, qSOFA showed a better performance for predicting ICU admission, with a sensitivity of 60% and specificity of 91%. This difference might be caused by the varied design, population, and settings between the studies.

A recent ED-based study that evaluated the performance of qSOFA and SIRS in the diagnosis of infection reported that SIRS was more accurate than qSOFA in predicting established infection in the ED with the area under the curve (AUC) being 0.647 vs. 0.582, respectively [11]. However, none of these scores would be useful due to their low AUCs. Also, it was suggested that the combined application of SIRS and qSOFA could improve the diagnostic accuracy for predicting infection and adverse outcome [11]. In another retrospective study, the predictive performance of qSOFA + SIRS combination (corresponded to the qSOFA + SIRS combination‑1 for this study) was reported with a sensitivity of 78.2% and specificity of 48.4% for predicting mortality in patients with surgical sepsis in the ED. Use of qSOFA + SIRS combination‑1 provided an increase in sensitivity and specificity of 6% and 7%, respectively, compared to SIRS alone [12]. In this study, qSOFA + SIRS combination‑1 led to a 6% increase in sensitivity and 2% decreased specificity, which resulted in 93% sensitivity and 25% specificity for predicting mortality compared to the use of SIRS alone. This is a higher sensitivity value than reported in the literature. Moreover, the sensitivity of qSOFA + SIRS combination‑1 was 84% for predicting composite outcome in general and 91% for predicting ICU admission. The performance of this combination was slightly better than those previously reported. The prospective nature of this study increases the reliability of the study results. To our knowledge, no prior study has evaluated the qSOFA + SIRS combination‑2 in the literature. A score of ≥ 2 in both qSOFA and SIRS provided an excellent specificity by increasing the specificity by 3–11% compared to qSOFA alone which resulted a specificity of 96% for predicting 30-day composite outcome in patients with suspected infection in the ED. Therefore, use of qSOFA + SIRS combination‑2 can provide benefit for clinicians in the risk assessment of adverse outcome or decision of patient disposition in the ED.

Limitations

This study has a number of limitations. First, because this was an observational study, the tests and treatments ordered were at the treating physicians’ discretion. Different treatments and transitions of care might have affected the study results. Second, this study was conducted in a tertiary care facility where most of the study population was critically ill. However, 25% of the eligible population admitted to the ED with the signs and symptoms suggesting simple infections such as cystitis, cellulitis, or upper respiratory tract infection. Because of the treating physicians did not order WBC test for this population, SIRS criteria could not be calculated, and this population was not analyzed for this study. Including this population might affect the diagnostic performance of qSOFA in this study. Third, there is no universal criterion for admission of infectious patients from the ED. Although a standardized local criterion was used for non-ICU hospitalization and ICU admission, admission criteria might vary between healthcare facilities. Fourth, because the blood, urine, or tissue cultures were not ordered routinely in all patients, definitive diagnosis of infection was not confirmed by the results of culture in this study. Therefore, the performance of qSOFA, SIRS, and the qSOFA + SIRS combinations in the patients with confirmed infection could not be evaluated in this study.

Conclusion

The results of this prospective study suggest that qSOFA and qSOFA + SIRS combination‑2 could be used in decisions for patient disposition because of its high specificity and LR + . However, qSOFA would not be a judicious option for screening high-risk patients in the ED. SIRS and qSOFA + SIRS combination‑1 would be better tools for excluding adverse outcome risks due to their high sensitivity. Although it has been suggested that the SIRS criteria might be useful for screening for sepsis in terms of triage, it would not be feasible because laboratory tests are required to calculate the scoring system.