Background

Sepsis, a life-threatening organ dysfunction caused by dysregulated host response to infection, affects millions of people around the world and kills as many as one in four each year [1,2,3]. The Emergency Department (ED) is often the initial setting for the diagnosis and treatment of acute sepsis, and has up to nearly 850,000 emergency department visits annually in the United States [4]. Prompt identification and appropriate treatment of sepsis in ED are crucial to improve the patient outcome [5]; therefore, early and specific marker of sepsis would be useful in ED, especially when clinical signs and symptoms are still insufficient for diagnosis [6].

Systemic inflammatory response syndrome (SIRS) was used as a screen criteria to patients with infection for the diagnosis of sepsis since 1992 [7]; however, more and more evidence pointed out its high false-positive rates in the diagnosis of sepsis [8]. Recently, the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) Task Force proposed the new quick Sequential (Sepsis-related) Organ Failure Assessment (qSOFA) to replace SIRS as a new screening tool for the identification of patients with sepsis [9]. The qSOFA score ranges from 0 to 3 points (one point for each of the following: respiratory rate > 21 breaths/min; systolic arterial blood pressure ≤ 100 mmHg; or altered mental status), and an increase of at least two points indicates a high mortality of sepsis patients. Several studies were recently published to compare the predictive performance of qSOFA with SIRS score for mortality in these patients [10]. However, conflicting conclusions were generated, especially in the patients in ED, and a meta-analysis was needed to verify the predictive performance of qSOFA score.

Several meta-analyses on the prognostic or diagnostic accuracy of qSOFA or SIRS were indeed published [11,12,13,14]; however, meta-analysis specially focusing on the ED patients was not yet published. The aim of this study was to obtain summary estimates of prognostic and diagnostic performance of qSOFA in outside ICU especially ED patients with infection. We also analyzed different subgroups of ED patients to make more precise conclusions.

Methods

Search strategy

We performed a comprehensive database searching in Medline (via PubMed), the Cochrane Library, ISI Web of Knowledge, and Science Direct for studies that evaluated the qSOFA score and/or SIRS as a tool for predicting the prognosis of sepsis in ED patients from the inception of each database through Dec 2017. The following terms were used: Quick Sepsis-related Organ Failure Assessment (qSOFA), systematic inflammatory response syndrome (SIRS), sepsis, septic, infection, prognosis. We also searched the reference list of each primary study identified and articles dealing with literature review. We also conducted searches of abstracts from major conferences. The titles and abstracts of studies which were potentially relevant were scanned and the full articles were reviewed when the studies seemed to meet the criteria or when information was insufficient to exclude them.

Study selection criteria

Studies were included if they met the following criteria.

  1. (1)

    Adult patients outside of ICU with suspected or confirmed infection or sepsis.

  2. (2)

    RCT, propensity-matched cohort study (prospective or retrospective), or historically controlled study.

  3. (3)

    qSOFA as a predictive tool for predicting mortality or diagnosis of sepsis.

  4. (4)

    Sufficient data to calculate absolute numbers of true-positive, false-positive, false-negative, and true-negative results.

Data extraction and quality assessment

Following the initial screening, full articles were independently reviewed by two reviewers (Y-CL, Y-YL) with application of the same inclusion criteria. Disagreements were resolved by consensus. A predefined form was used to extract data from each study. We only included publications written in English. Two reviewers (Y-CL, Y-YL) independently assessed risk of bias of the included studies using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 tool [15]. Disagreements were resolved by consensus. The qSOFA (positive if ≥ 2) and SIRS (positive if ≥ 2) score were defined following the proposed guideline [16, 17].

Statistical analysis

We presented the data as mean values for continuous variables and as frequencies for categorical variables. Results of individual studies were presented graphically by plotting sensitivity and specificity estimates on one-dimensional forest plots. Meta-analysis was performed by fitting bivariate models to our data. The estimates of pooled sensitivity, specificity, positive and negative likelihood ratios, with their 95% confidence intervals (CI) were calculated starting from parameter estimates obtained from bivariate models [18]. The effect of some predefined sources of heterogeneity (prospective vs retrospective study design; single-center vs multi-center; in-hospital mortality vs 28- or 30-day mortality; scores measured at ED arrival vs worst value; suspected vs confirmed infection; sepsis vs septic shock; overall mortality ≥ 10% vs overall mortality < 10%) was assessed by fitting bivariate meta-regression models with the inclusion of covariate terms. Bivariate meta-regression random effect model was also used in verifying the prognostic accuracy of qSOFA and SIRS criteria in diagnosis of sepsis. We performed all analysis using Review Manager (RevMan), and Stata15. A P value < 0.05 was considered statistically significant.

Results

Search results and description of studies

After screening titles and abstracts and completing full-text reviews, 24 studies met the inclusion criteria and were selected (Fig. 1). Table 1 described the characteristics of the 24 studies selected [9, 19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41]. Of the included studies, five studies (21%) were prospective studies, and the remaining were retrospective studies or hoc analysis. Seventeen studies (71%) included patients only in ED [19, 20, 25,26,27,28,29, 31, 33,34,35,36,37,38,39,40,41].

Fig. 1
figure 1

Study selection. Some studies were excluded for more than one reason

Full size image
Table 1 Studies characteristics included in the meta-analysis
Full size table

In the 17 studies from ED patients, 13 studies (76%) used in-hospital mortality as outcome measure, and four studies (24%) used 28- or 30-day mortality. Ten studies (59%) included patients with diagnosis of suspected or confirmed infection, and seven studies (41%) followed the diagnosis of sepsis or septic shock. Eleven studies (65%) measured the qSOFA at ED arrival, and six studies (35%) measured at the worst value during ED stay. The overall mortality rate in nine studies (53%) were over 10%, while eight studies (47%) were less than 10%. The mean age of patients in eight studies (47%) were over 65, and seven studies (41%) were less than 65, and data were not shown in the other two studies.

Quality assessment

Quality assessments using QUADAS-2 criteria are summarized in Fig. 2. Overall, the quality of the studies was deemed satisfactory.

Fig. 2
figure 2

Risk of bias graph for the included studies

Full size image

Prognostic accuracy for mortality using positive qSOFA scores

Figures 3 and 4 showed the forest plots and the 95% confidence regions of the sensitivity and specificity of qSOFA criteria reported in the 24 included studies. The pooled sensitivity of qSOFA across all included studies was 0.58 (95% CI 0.47–0.67), and the specificity was 0.69 (95% CI 0.48–0.84). The pooled estimates of positive and negative likelihood ratios were 1.8 (95% CI 1.1–3.0) and 0.62 (95% CI 0.51–0.70), respectively (Table 2).

Fig. 3
figure 3

Forest plots of sensitivity and specificity for qSOFA in predicting mortality in included studies

Full size image
Fig. 4
figure 4

Bivariate summary points of (specificity, sensitivity) and their 95% confidence regions for qSOFA in predicting mortality in included studies

Full size image
Table 2 Prognostic accuracy for mortality using positive qSOFA scores
Full size table

In seventeen studies specially focused on the ED patients, the pooled sensitivity of qSOFA across all included studies was 0.54 (95% CI 0.43–0.65), and the specificity was 0.77 (95% CI 0.66–0.86). The pooled estimates of positive and negative likelihood ratios were 2.4 (95% CI 1.8–3.2) and 0.6(95% CI 0.51–0.70), respectively (Table 2). The forest plots and the 95% confidence regions of the sensitivity and specificity are shown in Figs. 5 and 6.

Fig. 5
figure 5

Forest plots of sensitivity and specificity for qSOFA in predicting mortality in ED studies

Full size image
Fig. 6
figure 6

Bivariate summary points of (specificity, sensitivity) and their 95% confidence regions for qSOFA in predicting mortality in ED studies

Full size image

Subgroup analysis for positive qSOFA scores in predicting mortality

Table 3 show the results of univariate meta-regression analysis (prospective, retrospective, single-center, multi-center, in-hospital mortality, 28- or 30-day mortality, scores measured at ED arrival or worst value, suspected or confirmed infection, sepsis or septic shock, overall mortality ≥ 10%, overall mortality < 10%) in identifying potential sources of heterogeneity in studies specially focused on ED patients. Results showed that outcome definition and overall mortality were important sources of heterogeneity. Studies that used in-hospital mortality showed a higher sensitivity and lower specificity than studies that used 28- or 30-day mortality. Studies with overall mortality < 10% showed higher specificity than studies with overall mortality ≥ 10%.

Table 3 Subgroup analysis to examine the prognostic accuracy of qSOFA criteria in ED
Full size table

Diagnostic accuracy for sepsis using positive qSOFA scores and SIRS criteria in ED patients

To determine the role of qSOFA and SIRS in the diagnostic accuracy for sepsis, we used the Sepsis3.0 criteria [16] (Sepsis is defined as life-threatening organ dysfunction caused by a dysregulated host response to infection; organ dysfunction can be identified as an acute change in total SOFA score ≥ 2 points consequent to the infection) and focused the studies on ED patients. Five studies included the diagnosis accuracy of sepsis using qSOFA while three studies using SIRS; forest plots for the sensitivity and specificity analysis are shown in Fig. 7. The pooled sensitivity and specificity of positive qSOFA score for diagnosis of sepsis were 0.54 (95% CI 0.50–0.58) and 0.67 (95% CI 0.65–0.68), respectively. The pooled sensitivity and specificity of positive SIRS criteria were 0.72 (95% CI 0.67–0.77) and 0.71 (95% CI 0.69–0.73), respectively. There is no difference in the accuracy of diagnosis of sepsis between positive qSOFA scores and SIRS criteria (P > 0.05) (Table 4).

Fig. 7
figure 7

Forest plots of sensitivity and specificity for qSOFA and SIRS criteria in diagnosis of sepsis in included studies

Full size image
Table 4 Covariate analysis to verify the prognostic accuracy of qSOFA and SIRS criteria in ED
Full size table

Discussion

Our meta-analysis evaluated the prognostic capability for predicting mortality and diagnosing sepsis of qSOFA in adult patients outside of ICU with suspected infection. We found that qSOFA was poorly sensitive but highly specific for prediction of mortality. For the studies specially focused on the ED patients, studies that used in-hospital mortality had a higher sensitivity and lower specificity than studies that used 28- or 30-day mortality. Studies with overall mortality < 10% had a higher specificity than studies with overall mortality ≥ 10%. We also found that there is no difference in the accuracy of diagnosis of sepsis between positive qSOFA and SIRS criteria.

Sepsis is a common emergency, and its prognosis is highly affected by the early diagnosis and treatment in the ED [6]. The Sepsis3.0 task force designed qSOFA criteria to replace SIRS to identify patients with suspected infection who would require early diagnosis and treatment [9]. Interestingly, our meta-analysis has found that qSOFA has a poor sensitivity for predicting mortality and diagnosing sepsis in outside of ICU adult patients especially ED patients. As early recognition of sepsis and promptly providing treatment is crucial to improve outcomes, our result will cause great concern about the use of qSOFA in the ED patients.

Our meta-analysis focus on the patients outside of ICU, especially subgroup analysis in ED patients, which is quite different with meta-analysis that published before [11,12,13,14]. Our study confirmed that qSOFA was poorly sensitive (0.54) and moderately specific (0.77) for prediction mortality in ED patients, which was consistent with two studies published before. In Fernando’s study [11], the sensitivity and specificity of qSOFA and SIRS in ED patients subgroup are (0.467, 0.813) and (0.836, 0.306), and in Song’s study [13], the sensitivity and specificity of qSOFA in ED patients subgroup are (0.47, 0.85).

Our meta-analysis has several limitations. First, large amount of heterogeneity still exists among the included studies, although we have investigated some source of heterogeneity through subgroup analysis. Second, the definition of suspected infection varied among studies. Third, some studies applied qSOFA or SIRS score only to specific patients, such as older patients [26], community-acquired pneumonia patients [37], ED patients admitted to intensive care unit [19, 39], which is a potential source of bias. Finally, we found only five studies using positive qSOFA and three studies using positive SIRS criteria for assessing the accuracy of diagnosis of sepsis. The conclusion from this limited data is still farfetched.

Conclusions

In conclusion, our meta-analysis found that qSOFA showed a poor performance in predicting mortality of infected patients outside of ICU, especially in ED. Both qSOFA and SIRS were moderately accurate in diagnosing sepsis. Therefore, combining qSOFA and SIRS may be helpful in predicting mortality. Future studies should focus on combining qSOFA, SIRS, and other Point of Care Testing to accurately assess the diagnosis and prognosis of ED infected patients.