Introduction

Perioperative bloodstream infections are critical surgical complications. The reported incidence and mortality rates of postoperative sepsis range from 0.29 to 3.84% and 7.3–45.9%, respectively; however, these rates are dependent on the type of surgical procedure [1]. Although surgical data sharing reportedly reduces postoperative complications and medical costs [2], this variability in mortality makes simple comparisons difficult. Hence, a predictive model is required to audit the outcomes of surgical management.

There are several prognostic severity scores for inpatients, including the Sequential Organ Failure Assessment (SOFA), quick SOFA (qSOFA), Acute Physiology and Chronic Health Disease Classification System II (APATCH II), Simplified Acute Physiology Score 3 (SAPS3), and systemic inflammatory response syndrome (SIRS) [3,4,5,6,7]. However, these indices require variable physical findings, such as respiratory rate, blood pressure, and altered mentation. Although these fluctuating variables are suitable for detecting the critical states of sepsis, they are inappropriate for retrospective assessments as it is difficult to obtain fluctuating values at the appropriate time retrospectively. Hence, this study aimed to develop and validate a prediction model for in-hospital mortality after bacteremia associated with gastrointestinal surgery using variables suitable for retrospective audits. In addition, a detailed distribution of bacteremia pathogens in gastrointestinal surgery is presented.

Materials and methods

Study design

This study was a single-institution, observational, retrospective analysis of patients with bacteremia during the perioperative period of gastrointestinal surgery, conducted according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines [8]. The study was approved by the institutional review board of Shizuoka General Hospital (SGHIRB#2020095). Due to the study’s retrospective nature, informed consent was not obtained from individual patients.

All patient data used in this study were obtained from the Department of Gastroenterological Surgery, Shizuoka General Hospital, Shizuoka, Japan. Blood culture samples were collected from patients with signs of fever, chills, shivering, or detection of anastomotic leakage as they were suspected of bacteremia. The antecubital vein was the first choice of sampling site; however, if collection from this site was difficult, other sites were used. The collection site was swabbed and disinfected with alcohol and twice with 10% povidone-iodine. Briefly, 10 mL of blood culture was placed in aerobic and anaerobic bottles. This technique was repeated twice from different collection sites. For patients with an inserted central venous catheter (CVC), one set was collected from the CVC after disinfecting twice with 10% povidone-iodine. Culture collection commenced for one week, and the time from collection to a positive result was recorded. The diagnosis of bacteremia was based on a positive blood culture.

Data collection and utilization

Surgical and bacterial culture databases were prospectively created based on gastroenterological surgeries performed between July 2006 and December 2021. Data from patients with bacteremia from the date of gastrointestinal surgery to the date of discharge or 30 days postoperatively, whichever was more prolonged, were extracted. If only one set of blood cultures was positive for coagulase-negative Staphylococcus (CNS), Micrococcus, Bacillus, Corynebacterium, Viridans Group Streptococci, or Propionibacterium acnes, it was excluded as a contaminant [9]. Only the initial patient information and positive culture results were analyzed in this study if the same bacteria were detected in the patient during the perioperative period. However, if different bacterial species were detected in a patient, the initial patient information and information on each bacterium were analyzed. Clinical findings, laboratory data, surgical procedures, and bacterial profiles were collected from databases and electronic medical records. Laboratory results were collected from the day of positive blood culture collection or the nearest three days before or after. Patients were excluded from univariable and multivariable analyses if laboratory data on lymphocyte count, albumin, or creatinine were not obtained within three days before or after the date of blood culture collection. After considering the exclusion factors, the entire cohort was randomly divided into a deviation and a validation cohort at a 2:1 ratio.

Definitions of variables

The Charlson Comorbidity Index (CCI) was used to assess the comorbidity burden of the study patients [10]. The surgery type was divided into the following categories: upper gastrointestinal tract (GI; esophagus to duodenum), lower GI (small intestine to anus), hepatobiliary pancreas (HBP), surface (inguinal hernia and abdominal wall), and others. The prognostic nutritional index (PNI) was calculated using the following formula: 10 × serum albumin level (g/dL) + 0.005 × peripheral blood total lymphocyte count (/mm3) [11]. Creatinine levels were measured according to the enzymatic method used at the Shizuoka General Hospital since 1991. Creatinine abnormalities were defined as those exceeding the upper standard limit of sex-specific serum creatinine levels (male/female, 1.07/0.79 mg/dL). CVC placement was defined as CVC in the patient at any time three days before the blood culture collection date. Appropriate antibiotics were defined as the empiric administration of a susceptible antibiotic within 24 h of blood culture collection. The primary endpoint of this study was in-hospital mortality, defined as death from any cause before the first hospital discharge after blood culture collection.

Statistical analysis

Continuous variables are presented as medians and interquartile ranges (IQRs), whereas categorical variables are presented as counts with the corresponding percentages. A logistic regression model was used to estimate the odds ratios (ORs) and 95% confidence intervals (CIs) for univariable and multivariable analyses. The logit transformation was applied to the dependent variable, and the logistic function was used to calculate the predicted probabilities. A linear assumption was verified for continuous variables. If a continuous variable did not fit the linear regression clinically, it was transformed into a categorical variable. All explanatory variables were included in the multivariable analysis, and variables with a P < 0.05 were retained in the final multivariable model. Explanatory variables with strong correlations were selected by including those with lower p-values. A receiver operating characteristic (ROC) curve was plotted to evaluate the model’s discriminative ability. A calibration plot was constructed to determine the calibration capabilities of the model. Model performance was evaluated using the area under the ROC curve (AUROC) and a calibration plot. The validation cohort was used to validate the model according to previously published guidelines [12]. A funnel plot was used to compare the outcomes of the validation cohort by type of surgery. All statistical analyses were performed using R software (version 4.3.1).

Results

Patients

A total of 20,637 operations were performed between July 2006 and December 2021. The rates of lower GI, HBP, upper GI, surface, and other surgeries were 40.4%, 27.4%, 17.5%, 12.5%, and 2.3%, respectively. Among them, 4,184 sets of blood cultures were obtained, with 398 patients showing positive blood cultures (Fig. 1). The median age of the patients with positive blood cultures was 72 years (IQR: 65–79 years), and 66.1% of them were male. Half of the infection sites were surgical sites, followed by the bloodstream, biliary tract, and urinary tract. Appropriate antibiotics were administered to 60% of the patients. After excluding nine patients with missing essential laboratory data, the remaining patients were divided into the deviation (259 patients) and validation (130 patients) cohorts. The patient backgrounds of both cohorts are presented in Table 1. The most common type of surgery was lower GI, with a total of 186 cases, and the addition of an ostomy with 68 cases (37%). Of the 398 patients, 77 (19.3%) died in hospital—the mortality rate of the deviation and validation cohorts was 19% and 21%, respectively.

Fig. 1
figure 1

Schematic diagram of the patient selection process in this study

Full size image
Table 1 Patient background of the deviation and validation cohorts
Full size table

Pathogens of bacteremia

Among the 398 patients with bacteremia, 510 genera of pathogens were identified (Table 2). The most common genus was Staphylococcus, accounting for 54% of S. aureus and 46% of CNS, followed by Bacteroides, Escherichia, and Enterococcus. The distribution of pathogens by species is presented in Supplementary Table 1. The composition of pathogens varied according to the type of surgery performed. The most common pathogens in the upper GI, lower GI, and HBP groups were Staphylococcus (18.5%), Bacteroides (17.1%), and Enterococcus (18.3%), respectively (Supplementary Table 2). Patients without CVC placement and within seven days of surgery had low infection rates with Staphylococcus, Enterococcus, and Candida (4.0%, 4.6%, and 1.1%, respectively). In contrast, patients with CVC placement eight days after surgery had high infection rates with the pathogens, as mentioned earlier (27.4%, 14.5%, and 15.3%, respectively; Supplementary Table 3).

Table 2 Pathogen Genera found in all bacteremia cases
Full size table

Univariable logistic regression in the deviation cohort

In the univariable analysis, the risk factors for in-hospital mortality were creatinine abnormality (P = 0.0001, OR = 3.60; 95% CI: 1.89–6.87), decreased PNI (P = 0.0002, OR = 0.90/unit; 95% CI: 0.85–0.95), decreased Albumin (P = 0.0003, OR = 0.33/g/dL; 95% CI: 0.18–0.60), and age ≥ 75 years (P = 0.0009, OR = 2.52, 95% CI: 1.40–4.55; Table 3). Although the lower GI was associated with a higher mortality rate than other types of surgery, it was not a significant risk factor. Additionally, appropriate antibiotic use was not a risk factor for in-hospital mortality in this study.

Table 3 Univariable and multivariable logistic regression results in the deviation cohort for predicting in-hospital mortality (N = 259)
Full size table

Multivariable logistic regression in the deviation cohort and prognostic modeling

In the multivariable analysis, albumin was excluded because of its high correlation with PNI and higher p-value. Creatinine abnormality (P = 0.0005, OR = 3.39; 95% CI: 1.71–6.71), decreased PNI (P = 0.0003, OR = 0.90/unit; 95% CI: 0.85–0.95), and age ≥ 75 years (P = 0.0026, OR = 2.89; 95% CI: 1.45–5.78) were independently associated with in-hospital mortality. The equation for the in-hospital mortality risk model, including these independent risk factors, is shown in Supplementary Table 4. The model showed good discrimination, with an AUROC of 0.781 (95% CI: 0.717–0.846) in the deviation cohort (Fig. 2A).

Fig. 2
figure 2

(A) Receiver operating characteristic (ROC) curve for in-hospital mortality in the deviation cohort (n = 259). (B) ROC curve for in-hospital mortality in the validation cohort (n = 130)

Full size image

Validation of the predictive model

The model also showed fair discrimination in the validation cohort, with an AUROC of 0.711 (95% CI: 0.602–0.820) (Fig. 2B). The calibration intercept was − 0.219, which indicates that the model’s predictions tend to be slightly overestimated (Fig. 3).

Fig. 3
figure 3

Calibration plot for the in-hospital mortality in the (A) deviation (n = 259) and (B) validation cohort (n = 130)

Full size image

Funnel plot of the in-hospital mortality rate of the validation cohort by type of surgery

The expected mortality and observed-to-expected (O: E) ratio for mortality according to the type of surgery are displayed in the funnel plot in Fig. 4. The predicted probability of in-hospital death for each type ranged from 7 to 33%. The outlier was superficial surgery, with only one case and no death. In contrast, the other O: E ratios ranged from 0.62 to 1.36, within the 95% confidence interval.

Fig. 4
figure 4

Funnel plot of in-hospital mortality rate according to the type of surgery. Upper, upper gastrointestinal tract; lower, lower gastrointestinal tract; HBP, hepatobiliary pancreas

Full size image

Discussion

In this study, we developed and validated a predictive model for in-hospital mortality in patients with bacteremia during the perioperative period of gastrointestinal surgery. This predictive model included the age, creatinine, albumin levels, and lymphocyte counts. We also provided a detailed list of the pathogens associated with bacteremia during the perioperative period of gastrointestinal surgery. Several studies have reported that the SOFA, qSOFA, APACHE II, and SIRS scores are good predictors of in-hospital mortality. The AUROC for in-hospital mortality of the SOFA, qSOFA, APACHE II, SAPS3, and SIRS scores are 0.63–0.96, 0.59–0.95, 0.77–0.85, 0.75–0.84, and 0.54–0.95, respectively [13,14,15,16,17,18,19,20,21,22,23]. However, these indices require physical variable findings, such as respiratory rate, blood pressure, and altered mentation, which vary throughout the day. Even if vital signs are recorded by monitors in intensive care units, complicated statistical processing is required to extract the data for statistical analysis [13]. Although these variable findings are suitable for detecting ongoing critical conditions such as sepsis, they are unsuitable for a retrospective audit because not all hospitals record these physical variable findings in a statistically processable form. In our cohort, the SIRS score could be retrospectively examined in 347 cases, and the AUROC for predicting in-hospital mortality was 0.56. Although SIRS is useful for detecting transient inflammatory reactions, it is assumed that patient background factors such as renal function and nutritional status are more related to in-hospital mortality. Therefore, in this study, we calculated the risk of in-hospital mortality using only the age, creatinine level, albumin level, and lymphocyte count; these values do not fluctuate significantly during the day. The deviation and validation cohorts demonstrated moderate discriminative power with AUROCs greater than 0.7 using these variables with slight diurnal variation. The funnel plot showed that the O: E ratios varied by type of surgery but were predictable with few outliers.

Several studies have demonstrated the positive effects of surgical audits on postoperative morbidity and medical expenses. Guillamondegui et al. reported that after initiating a 10-hospital collaborative effort to share surgical process and outcome data, a substantial reduction in some morbidities was observed, leading to an overall reduction in healthcare expenditures [24]. Henke et al. also reported that after establishing a statewide surgical quality consortium, morbidity was reduced from 15.8 to 13.8% (P = 0.02). They reported that the direct cause of lower morbidity was not a single factor but likely a combination of the effect of data feedback to practitioners, including the Hawthorne effect, that improves outcomes [25]. Similarly, using this model to evaluate the annual trends in our treatment outcomes revealed an annual improvement in in-hospital mortality, particularly after 2015. This rise was likely due to the introduction of guideline-directed prophylactic antibiotics [26], the establishment of the antimicrobial stewardship team, and consultation with infectious disease specialists. Conversely, the model presented in this study is not suitable for detecting or treating ongoing sepsis because risk factors such as age, nutritional status, and renal function are patient background factors that are difficult to modify. Guidelines for managing sepsis recommend a bundle of treatments, including control of blood pressure and glucose levels, prompt antimicrobial administration, use of crystalloid as first-line fluid, and appropriate ventilatory management [27]. Our model is suitable for cross-facility and cross-period audits to evaluate these treatments because the explanatory variables are simple and easy to collect retrospectively.

This study also reported detailed pathogens of perioperative bacteremia based on gastrointestinal surgical techniques. Despite the importance of knowing the bacterial profile for empiric antimicrobial therapy, there are few publications on pathogen-causing bacteremia associated with gastrointestinal surgery. Velasco et al. revealed the bacterial profile of 112 episodes of bacteremia in patients undergoing surgery for cancer in Brazil [28]. Our results are consistent with their crude in-hospital mortality rate of 19.6%. However, there were significant differences in the distribution of pathogens. These variations were probably owing to the difference in patient backgrounds, as there was a high proportion of patients with advanced disease (75%), a high surgical site infection rate (46%), and a lack of information on anaerobes. Abraham et al. also reported organisms isolated from the blood cultures of 72 surgical patients in the United States [29]. Their rate of resistant bacteria, such as multidrug-resistant Acinetobacter and methicillin-resistant Staphylococcus, differed markedly from the profile presented in this study. Therefore, several factors should be considered when selecting the appropriate empirical antibiotics, including regional factors, type of surgery, infection site, postoperative period, CVC placement, and antimicrobial susceptibility. Cai et al. reported that bacterial resistance to recurrent urinary tract infections is multifactorial, using artificial intelligence (AI) of neural networks, which predicted the clinical efficacy of empirical therapy with an AUROC of 0.867 [30]. Thus, the support of AI-trained models with a large dataset may be useful in determining the appropriate empirical antibiotics.

In this study, the items and formats of the explanatory variables were carefully selected. We dichotomized age and creatinine level because we considered dichotomization clinically appropriate. Although mortality risk increases with age, a linear relationship between the log odds of mortality risk and age cannot be assumed. There is no general age cut-off for the risk of death from sepsis, which varies from paper to paper from 65 to 80 years [31,32,33]. Therefore, we dichotomized age at 75 years, where the mortality risk was clearly divided. Acute kidney injury is a strong predictor of mortality in surgical sepsis [34]. Chronic kidney disease is also a risk for death from infection [35]. Although creatinine abnormalities are important for risk assessment, the normal range of creatinine levels varies according to sex and measurement method; the Jaffe method yields higher creatinine levels than the enzymatic method, particularly at low concentrations [36]. Accordingly, we dichotomized the creatinine level to the upper limit of the normal range by sex to be adaptable to other institutions. Malnutrition is associated with 30-day mortality in patients with sepsis [37,38,39]. Therefore, PNI was chosen due to its simplicity and objectivity, which were necessary for the audit. Although some studies have discussed the relationship between mortality in patients with sepsis and comorbidities such as heart failure, respiratory failure, and diabetes mellitus [40, 41], CCI was not a risk factor for in-hospital mortality in this study.

Inflammatory markers such as white blood cells (WBC) and CRP were not risk factors in this study. In particular, WBC as a continuous variable was not selected as a risk factor since WBCs may increase or decrease in the presence of infection. Therefore, a preliminary univariable analysis was performed by dichotomizing WBC into normal (3000–9000 /mm3) and abnormal (< 3000 or 9000< /mm3) groups; however, dichotomized WBC was still not a risk factor (P = 0.971, OR = 0.99; 95% CI: 0.51–1.92). Additionally, CRP was also not significantly predictive of mortality in this study. One potential reason is the time lag between the onset of inflammation and the rise in CRP. The serum CRP concentration rises after approximately six hours and peaks at approximately 48 h after inflammation [42]. However, in most cases, blood samples were collected at the onset of symptoms. Therefore, CRP may not have been elevated at the time of sampling. Moreover, due to the retrospective nature of this study, blood biochemistry samples for four patients (1%) were collected 2 or 3 days after blood culture collection. This time delay could have also affected the results. However, these 4 cases were mild, and blood cultures were taken as a precaution, which turned out to be positive. Since excluding these minor cases could have introduced selection bias into the study, these 4 cases were included in this study.

Appropriate antibiotic use was not a risk factor in this study. Staphylococcus was the most common pathogen in patients receiving inappropriate antimicrobial therapy, followed by Candida, Bacteroides, and Enterococcus. The cause of the inappropriate antimicrobial administration was the lack of coverage for resistant bacteria, such as methicillin-resistant S. aureus and methicillin-resistant CNS, as well as fungi and anaerobes. Critically ill patients tended to receive broad-spectrum antibiotics. On the other hand, empiric treatment of mild cases was often initiated with narrow-spectrum antimicrobials. Therefore, many cases in which the resistant pathogens were not covered by empiric therapy. This uneven distribution has been attributed to the fact that appropriate antimicrobial use is not a risk factor.

Nonetheless, this study had some limitations. First, the model was developed and validated using data from a single institution. Therefore, its generalizability using data from other hospitals remains unclear. Second, due to the lack of a dataset of physical findings, such as respiratory rate and altered mentation, the model could not be compared with those of the SOFA, qSOFA, APATCH II, and SAPS3. Third, because of the retrospective nature of this study, nine patients were excluded from the univariable and multivariable analyses due to a lack of laboratory data.

In conclusion, this study presented a model that predicts in-hospital mortality after bacteremia related to gastrointestinal surgery using age, creatinine and albumin levels, and the lymphocyte count. As these simple variables are easy to collect retrospectively, this model suits inter-hospital and inter-period audits. Our study also showed the detailed bacterial profile of perioperative bacteremia during gastrointestinal surgery, which will be helpful in the selection of empiric antibiotics for patients undergoing gastrointestinal surgery. Furthermore, this model will be helpful for the retrospective audits of bacteremia management associated with gastrointestinal surgery.