Introduction

Infection is a common disease which is on the rise and has a high mortality in developed countries [1,2,3]. Both the identification of sepsis and the risk stratification of patients with an acute infection remain a challenge for physicians.

Kaukonen et al. studied the sensitivity of the use of two or more systemic inflammatory response syndrome (SIRS) criteria for the diagnosis of severe sepsis [4]. They found that these criteria did not identify one out of eight patients with severe sepsis, suggesting limited sensitivity. Williams et al. compared the performance of several scores in different groups of patients based on mortality and concluded that the discrimination decreased in those risk groups with higher mortality for each score [5].

Recently, new definitions for sepsis have been published [6]. The quick Sepsis-related Organ Failure Assessment (qSOFA) score is a new screening tool including three clinical variables (Glasgow Coma Scale score ≤ 13, systolic blood pressure < 100 mmHg and respiratory rate ≥ 22 per min), which was developed to identify patients suspected of having sepsis. Although the predictive validity of the qSOFA score has been shown to be good, it was derived from large retrospective databases and, therefore, requires prospective validation. Other prospective studies have evaluated the prognostic accuracy of different scores [7, 8], but none were focused on determining the accuracy of the qSOFA score in older patients with acute infection.

Many parameters of these mortality risk scores are related to host inflammatory response to infection. The physiological changes related to ageing, comorbidity, polypharmacy and geriatric syndromes in older patients may inhibit adequate response to infection, thereby reducing the prognostic prediction capacity of these scores in these patients [9,10,11]. Moreover, there has been a rise in emergency department (ED) visits by older patients due to infection [3], and it remains unknown whether the new sepsis diagnostic criteria are useful to identify older patients with high short-term mortality or whether other factors should be taken into account in risk stratification. In this sense, new strategies are needed for risk stratification during the first assessment in the ED in order not to delay the initiation of appropriate therapeutic measures due to an identification error in this age group. A recent prospective cohort study including patients ≥75 years of age concluded that the classical sepsis criteria were not useful to stratify the 30-day risk of mortality in older patients. These authors developed a new predictive model named the GYM score (Glasgow coma score < 15, tachYpnea > 20 bpm and severe co-Morbidity evaluated as a Charlson Comorbidity Index ≥ 3) that showed better accuracy in predicting 30-day mortality than sepsis criteria in older patients [12]. Nevertheless, this score has not yet been externally validated.

Considering all of the above, the aims of the present study were to determine the accuracy of the SIRS criteria and the qSOFA and GYM scores to predict all-cause 30-day mortality in older patients attended for acute infection in EDs.

Methods

Study design

We performed a prospective observational cohort study including all the patients aged 75 years or older attended for an acute infection in 69 Spanish EDs over three seasonal periods of 2 days each (1st and 22nd October 2015, 12th and 19th January 2016, and 13th and 27th April 2016). The Ethical Committee of the reference centre approved the study. All the patients or legal guardians provided informed consent to participate in the study.

Patient selection

We included older patients ≥75 years of age who did not have severe functional dependence (Barthel index >40), were clinically diagnosed with an acute infection in the participating EDs and who accepted to participate in the study.

Study setting

The EDs participating in the study are included in the network of centres of the Infectious Disease Group of the Spanish Emergency Medicine Society (INFURG-SEMES) (see addendum).

Definition and collection of variables

We collected demographic data (age and gender), comorbidities (Charlson index), basal functional status (Barthel index), clinical and analytical data in the ED (altered metal status according to the Glasgow Coma Scale, heart and respiratory rates, temperature, blood pressure and oxygen saturation, and leucocyte count) and the site of infection (urinary, respiratory, intra-abdominal, skin and soft tissue or other infections).

Classical sepsis was defined as the presence of infection and SIRS. SIRS was considered if two or more of the following parameters were presented: body temperature > 38 °C or < 36 °C, heart rate > 90 beats per minute, hyperventilation documented by a respiratory rate > 20 breaths per minute or PaCO2 < 32 mmHg and a leucocyte count > 12,000 cells/μL or < 4000/μL. Hypotension was defined as the presence of systolic blood pressure < 90 mmHg or mean blood pressure < 70 mmHg. An altered mental status was defined as a score of <15 on the Glasgow Coma Scale. Severe comorbidity was determined as a Charlson index of 3 or more. Severe functional status was considered with a Barthel index ≤ 40 points. The primary outcome was all-cause 30-day mortality after the ED visit.

The variables were registered in an electronic case report form. The different criteria and parameters were previously defined by the group based on the current clinical guidelines and were subsequently reported to the investigators by the principal investigator of each centre. Follow-up was carried out by the principal investigator of each centre by consulting the electronic clinical charts and/or by telephone to determine death within the first 30 days after being attended in the ED.

Statistical analysis

The quantitative variables are expressed as mean and standard deviation (SD) and the qualitative variables are expressed as absolute and relative frequencies. Qualitative variables were analysed using the Chi-square or Fisher exact tests if more than 25% of the expected frequencies were less than 5, and the Student’s t-test was used for quantitative variables.

The areas under the curve (AUCs) were calculated for SIRS criteria and the qSOFA and GYM scores. The sensitivity, specificity, positive and negative likelihood ratio and the 95% confidence intervals (CIs) were calculated for ≥ 2 SIRS criteria and a qSOFA score ≥ 2 and GYM score ≥ 1. The AUC and the characteristics of the scores were compared by a non-parametric test. We considered an α error less than 0.05. The statistical analyses were performed using the statistical package SPSS 18.0® (SPSS Inc., Chicago, IL, USA) and STATA 12.0 (StataCorp LP, College Station, TX, USA).

Results

We included 1071 (60.3%) out of 1776 infected patients assessed in the ED. Seventy patients did not accept to participate in the study, 191 patients had missing values and 444 patients had severe functional dependence (Fig. 1).

Fig. 1
figure 1

Flow diagram of the study

Full size image

The mean age of the patients was 83.6 (SD 5.6) years and 544(50.8%) were males. Table 1 shows the characteristics of the patients included in the study. At the time of the first assessment in the ED, 555 (51.8%) had ≥ 2 SIRS criteria, 83 (7.7%) had a qSOFA score ≥ 2 and 610 (56.9%) had a GYM score ≥ 1.

Table 1 Characteristics of the patients included in the study
Full size table

Seventy-two patients (6.5%) died within 30 days after the ED visit. The 30-day mortality was 5.3% for patients with a qSOFA score < 2 vs. 24.1% in those with a qSOFA score ≥ 2 (absolute difference, 18.8%; 95% CI 10.8–29.1). The 30-day mortality was 4.8% for patients with < 2 SIRS criteria vs. 8.5% in those with ≥ 2 SIRS criteria (absolute difference, 3.6%; 95% CI 0.62–6.7). Lastly, the 30-day mortality was 3.0% in patients with a GYM score < 1 vs. 9.5% in those with a GYM score ≥ 1 (absolute difference, 6.5%; 95% CI 3.6–9.3).

Table 2 shows the results of the univariate analysis of each score and criteria in relation to the 30-day mortality. Table 3 shows the prognostic performance of the SIRS criteria and the two scores according to the respective cut-off points. In order to predict the 30-day-mortality, the sensitivity of SIRS ≥ 2, qSOFA score ≥ 2 and GYM score ≥ 1 was 65.3% (95% CI 53.1–75.9), 27.8% (95% CI 18.2–39.8) and 80.6% (95% CI 69.2–88.6), respectively. The best specificity was showed by qSOFA score ≥ 2, with 93.7% (95% CI 91.9–95.1). The negative likelihood ratio was 0.71 for SIRS ≥ 2, 0.77 for a qSOFA score ≥ 2 and 0.43 for a GYM score ≥ 1.

Table 2 The 30-day mortality of the SIRS, qSOFA score and GYM score model categories [n (%)]
Full size table
Table 3 The diagnostic performances of scores to predict the 30-day-mortality
Full size table

Table 3 also shows the diagnostic yield of a cut-off of ≥ 1 for the qSOFA score. A GYM score ≥ 1 and a qSOFA score ≥ 2 showed the highest sensitivity (p < 0.001) and specificity (p < 0.001), respectively (Table 4).

Table 4 Comparison of the sensitivity and specificity results between the different scores for clinically significant cut-off points
Full size table

The AUC for 30-day mortality of the GYM score was 0.73 (95% CI 0.66–0.79; p < 0.001), being 0.69 for the qSOFA score (95% CI 0.61–0.76; p < 0.001) and 0.65 for SIRS criteria (95% CI 0.59–0.72; p < 0.001). The statistical differences comparing receiver operating characteristic (ROC) curves were: GYM score vs. SIRS, p = 0.066; qSOFA vs. SIRS, p = 0.469; and GYM score vs. qSOFA, p = 0.070 (Fig. 2).

Fig. 2
figure 2

ROC curves of SIRS, qSOFA and GYM risk models. Comparison of the ROC curve and AUC of GYM score, qSOFA score and SIRS to predict 30-day mortality: GYM score vs. SIRS, p = 0.066; qSOFA score vs. SIRS, p = 0.469; GYM score vs. qSOFA score, p = 0.070

Full size image

Finally, based on the number of GYM score criteria, the 30-day-mortality was 2.7%, 6.3%, 15.2% and 34.6% for 0, 1, 2 or 3 criteria, respectively (Fig. 3).

Fig. 3
figure 3

Frequency of 30-day mortality of the GYM score

Full size image

Discussion

The present study provides additional evidence regarding the limitations of the SIRS criteria in identifying high-risk older patients with acute infection. In the present study, we found that the GYM score showed a good prognostic capacity, and a cut-off point ≥ 1 had the highest sensitivity compared to the SIRS criteria and the qSOFA score. Therefore, a GYM score ≥ 1 may be a useful screening tool to identify 30-day mortality in older patients attended with acute infection in EDs. In regard to the qSOFA score, we observed that a cut-off ≥ 2 had the highest specificity, albeit with very poor sensitivity, in older patients. Although the qSOFA score showed good diagnostic accuracy in our study, this was lower than what has been previous described in an all-age-group population [7, 13].

The identification of high-risk patients at arrival to the ED is a priority in acute care. Older patients are a highly complex population in which diagnosis and adequate risk stratification are more difficult, usually requiring more time and resources [14], which may substantially contribute to worse outcomes and ED overcrowding [15, 16]. Nevertheless, the scores currently available have not been specifically validated in older patients.

An important issue to consider in clinical practice is that decisions are based on a concrete, previously established, cut-off point, which turns these quantitative scores into qualitative variables for decision-making. In this sense, it is probably more important to consider the diagnostic performance of the established cut-off point than its predictive capacity measured by the ROC curve when assessing the effectiveness of the score [17].

Despite having a higher specificity, the use of the qSOFA score has been questioned because of its low sensitivity compared to the classical sepsis criteria [14, 18]. To assess the usefulness of a diagnostic test, we must keep in mind its objective [19]. When the consequence of a diagnostic error is very high in prognostic terms, it is very important to use a highly sensitive test to exclude the possibility of disease. A good example of this situation is sepsis, a very severe, albeit treatable, disease [20]. The most adequate diagnostic approach on suspicion of sepsis in the ED is the use of a highly sensitive screening test for sepsis [21]. In this regard, a cut-off ≥ 2 for the qSOFA score showed a very poor sensitivity to identifying patients with a high risk of short-term mortality. Although the qSOFA and GYM scores showed only a trend to statistical significance in the AUC for 30-day mortality, the GYM score was the best tool in terms of sensitivity. Lowering of the cut-off of the qSOFA score to ≥ 1 did not sufficiently improve the sensitivity to identifying patients at risk of poor prognosis, being similar to the SIRS criteria, which also had a low sensitivity in the identification of older high-risk infected patients.

Considering specificity, a qSOFA score ≥ 2 was the best test among the older patients evaluated. However, although specificity is useful for confirming a diagnosis suggested by other data, it is not the main characteristic required for a screening test [22]. Additionally, specificity is not included in the first diagnostic approach in the ED for the initial management of older high-risk patients [16].

It is also important to consider that a screening tool in the ED can be accessible at any level of care without technological requirements. Both the qSOFA and the GYM scores met this requirement and used clinical variables, which are easily obtainable in the first assessment. The criteria of alteration in mental status of the GYM score (Glasgow Coma Scale < 15) may be easier than that of the qSOFA score (Glasgow Coma Scale ≤ 13), since assessment of the level of consciousness with the Glasgow Coma Scale in this population is sometimes difficult. It is noteworthy that we found alteration in mental status to be the most important criterion related to a poor outcome. This is the great advantage of these two scores compared with the classical sepsis criteria, which require blood analysis in order to carry out the evaluation.

The present study has several limitations. First, we excluded patients with severe functional dependence, since this therapeutic limitation may play a very important role in risk stratification, and these patients are probably not susceptible to invasive therapies. Thus, our results should not be applied to this specific population. Third, the timing and type of treatment were not evaluated, and both of these aspects may condition disease outcome. However, local treatment guidelines for patients with this profile were applied during the study. Finally, for patients to be included in the study, they had to be clinically diagnosed with infection by the ED physician. Although this may represent a study bias, we felt that this approach was more similar to real life and the decisions made during the initial assessment of patients in the EDs.

In conclusion, the GYM score showed good accuracy in determining short-term mortality in older patients with infection. Compared to a qSOFA score ≥ 2 and ≥ 2 SIRS criteria, the GYM score cut-off ≥ 1 showed the highest sensitivity. Despite the high specificity of the cut-off ≥ 2 for the qSOFA score, the poor sensitivity of this score limits its use as a screening test in the ED. An important aspect of this study is that it was focused on older patients with acute infection, in whom risk stratification is complex.

Addendum

Collaborators. Members of the Infectious Disease Group of the Spanish Emergency Medicine Society (INFURG-SEMES) participating in this study: Pablo Herrero Puente (Hospital Universitario Central de Asturias), Rafael Rubio Díaz (Complejo Hospitalario de Toledo), Diana Moya Olmeda (Hospital Virgen de la Luz, Cuenca), Raúl López Izquierdo (Hospital Rio Hortega, Valladolid), José Andrés Sánchez Nicolás (Hospital Universitario Reina Sofía, Murcia), Ángel Aurelio Álvarez Hurtado and Carlos del Pozo Vegas (Hospital Clínico de Valladolid), José Gallardo Bautista (Hospital Universitario Virgen de la Macarena, Sevilla), Cecilia Yañez and Francisco Javier Candel (Hospital Universitario Clínico San Carlos, Madrid), Borja Cobanera Echebarria (Hospital Universitario de Basurto, Bilbao), Francisco Román and Pere Llorens (Hospital Universitario General de Alicante), Ramón Perales (Hospital General de Albacete), Javier Oñate (Hospital de Cruces, Baracaldo), Begoña Mora Ordoñez and Eva Fragero (Hospital Virgen de la Victoria, Málaga), Mar Ortega (Hospital Clinic, Barcelona), Carolina Gómez Hernández and Ana Darias Acosta (Hospital Universitario de Canarias, Tenerife), María Jesús Arranz and Ricardo Juárez (Hospital Nuestra Señora del Prado, Talavera), José María Ferreras Amez (Hospital Royo Villanova, Zaragoza), Eva Muro and Luis Pérez Ordoño (Hospital Universitario 12 de Octubre, Madrid), Jorge García Criado (Hospital Universitario, Salamanca), Andrés von Wernitz Teleki and Carmen del Arco (Hospital Universitario La Princesa, Madrid), Pablo Javier Marchena Yglesias (Parc Sanitari Sant Joan de Déu - Hospital General. Sant Boi de Llobregat, Barcelona. Hospital Moises Broggi Sant Joan Despí, Barcelona), Francisco Ruiz (Hospital Clínico Universitario, Zaragoza), Luis García Castrillo and Héctor Alonso (Hospital de Valdecilla, Santander), Manuel Salido Mota and Miguel Moreno Fernández (Hospital Regional Uniersitario de Málaga ), Belén Rodríguez Miranda (Hospital Rey Juan Carlos, Madrid), Martín Ruiz Grinspan (Hospital del Henares, Madrid), Josep María Guardiola Tey, Sergio Herrera Mateo and Miriam Mateo Roca (Hospital de Sant Pau, Barcelona), Francisco José Ezponda (Hospital de Zumárraga, Zumárraga), Elena Diez (Hospital de San Juan, Alicante), Francisco Gracia García (Hospital Reina Sofía, Córdoba), Itziar Huarte (Hospital de Donosti, San Sebastián), Sara Gayoso Martín (Hospital del Escorial, San Lorenzo del Escorial), María Cubillo Moreno and Silvia Juárez de Sosa (Hospital Universitario de Burgos, Burgos), Ana de la Chica (Hospital del Sol, Marbella), Carles Ferré Losa (Hospital Universitario de Bellvitge, L’Hospitalet de Llobregat), Rigoberto Del Rio Navarro (Hospital Universitario de Torrevieja, Torrevieja), Ángel Estella (Hospital de Jerez, Jerez), Marta Iglesias Vela and Luis Miguel Maestro Gilmartin (Hospital de León, León), Carlos Avellaneda Martínez (Hospital de Segovia, Segovia), Virginia Álvarez Rodríguez (Hospital Universitario de Getafe, Getafe), Manuel Fernández Rodríguez (Hospital San Eloy, Barakaldo), Alexandra María Guiu Marti (Hospital Son Espases, Palma), Octavio J. Salmerón Béliz (Hospital Universitario Fundación Alcorcón, Alcorcón), Silvia Flores Quesada and Osvaldo Troiano Ungerer (Hospital de Sant Pau i Santa Tecla, Tarragona), Tania Cano Rodríguez (Hospital Santiago Apóstol, Miranda de Ebro, Burgos), Luis Lapuerta Irigoyen (Hospital de Soria, Soria), Carmen Andonegui (Hospital de Mendaro, Mendaro), Nieves Cortes (Hospital Alto Deba, Mondragón), Gema Muñoz Gamito (Hospital Universitari Mútua Terrassa, Terrassa), Beatriz Valle Borrego (Hospital Severo Ochoa, Leganés), María Arranz Betegon and Ángel Sierra Moreno (Hospital de Viladecans, Viladecans), Manel R. Chanovas Borràs (Hospital de Tortosa Verge de la Cinta, Tortosa), Ainhoa Urraca Gamaury (Hospital de Zamora, Zamora), Jesús Cruzado Quevedo (Hospital General Universitario Santa Lucia, Cartagena), Verónica Carpio Muñoz (Hospital Juan Ramón Jiménez, Huelva), Cristóbal Marchante Tejada (Hospital Nuestra Señora de Valme, Sevilla), Bakarne Menchaca (Hospital de Bidasoa, Hondarribia), Josep María Mòdol Deltell (Hospital Universitario Germans Trias i Pujol, Badalona), Ángel Macías López (Hospital de Ávila, Ávila), María Ángeles Leciñera (Hospital Can Misess, Ibiza), Salvador Sarrà Moretó (Hospital del Vendrell, El Vendrell), Coral Suero Méndez and Salvador Pendón Fernández (Hospital de la Axarquía, Málaga), Raimundo Rodríguez and Miguel Ángel Cardenete Aljama (Hospital Universitario de Ciudad Real, Ciudad Real), Maite Fernández Rivas (Hospital del Bierzo, Ponferrada), Luis Amador (Hospital de Vigo, Vigo), José Eduardo Cobo Muñoz (Complejo Hospitalario de Jaén, Jaén), Elena Carrasco (Virgen de Alta Gracia, Manzanares), David Rodríguez Salazar (Hospital Comarcal d’Amposta, Amposta), Esther Rodríguez Adrada (Health Research Institute, Hospital Universitario San Carlos, Madrid).