Introduction

Febrile neutropenia (FN) represents one of the most common complications of chemotherapy in cancer patients. It is a cause of considerable morbidity and carries a significant risk of mortality. Complications and mortality rates have been reported higher in bacteremic patients, especially those with gram-negative bacteremia [1]. However, the prediction of bacteremia or of absence of bacteremia at the onset of FN has proven to be difficult. Although factors such as shock, temperature higher than 40°C, and severe thrombocytopenia have been found to be associated with gram-negative bacteremia [2], a predictive model developed by Viscoli, on the basis of these parameters, was insufficiently discriminant when tested in a validation set [3]. In a more recent population-based study of patients with hematological malignancies, the risk of bacteremia varied greatly according to the type of malignancy; however, the mortality rates (overall cumulative mortality rate at 30 days of 32%) were similar for all underlying neoplastic diseases [4]. Bacteremia documentation being unavailable when treatment has to be initiated and to predict its occurrence appearing unreliable up to now, the further question for the clinician is: Do we really need the documentation of bacteremia? Or can we rely on other systems predictive of risk of complications without integrating bacteremic status?

Several predictive models have indeed been developed to identify patients at low risk of complications [5, 6]. The latter so-called Multinational Association for Supportive Care in Cancer (MASCC) scoring system has been internationally validated under various clinical conditions [7, 8] and has been widely accepted [9]. The use of the MASCC score also allows the selection of low-risk patients who can be safely treated with orally administered antibiotics and be, for at least some of them, successfully discharged early after a 24-h in-hospital observation [8]. However, there is still room for improvement of the MASCC score.

Therefore, as bacteremia is a known risk factor for the occurrence of complication, we hypothesized that the integration of this covariate, if known, could improve the discriminant ability of the MASCC score. Therefore, we were interested in (1) analyzing the MASCC score discriminant ability when stratifying the patients by bacteremic status, (2) assessing whether the baseline knowledge of bacteremia could improve risk assessment models targeted at the identification of low-risk patients, and (3) further exploring the ability to predict bacteremia at the time of treatment initiation in more recent series of patients.

Methods

We consecutively carried out two multicenter prospective observational studies from 1994 till 2005 involving patients treated with empiric antibiotic regimens for chemotherapy-induced FN. The first study led to the development of the so-called MASCC score, internationally validated, aimed at identifying low-risk patients (those defined as having a probability of at least 95% of fever resolution without developing a serious medical complication or death) at the onset of FN. The second survey was performed in order to look more closely at variables susceptible to have an impact on the neutropenia duration as it was hypothesized that the integration of such variables could improve the model. The results of this second survey are not yet published.

Data from 2,142 eligible patients (respectively, 1,139 and 1,003) were collected and pooled together for the present analysis. The full list of the participating investigators and institutions is given in Appendix.

Statistical methodology

The data were thoroughly inspected and explored using univariate analyses (contingency tables and descriptive statistics) as well as multivariate analyses (scatter matrices and logistic regression) to detect multicollinearity and outliers that were checked. The χ 2 test and Fisher’s exact test were used to test the statistical significance of differences in discrete data; Student’s t test and Mann–Whitney’s test were used to compare continuous variables. Homogeneity of the odds ratios (OR) was assessed using the Breslow–Day test. All reported P values were two-tailed, and we considered as significant a nominal P < 0.05, although multiple tests were done, considering that all the analyses that were carried out had an exploratory purpose.

To assess whether the integration of bacteremic status improves the MASCC score, the data were separated into a development set and a test set. The models were constructed using the development set (first study, 1994–1997, n = 1,139) and prospectively tested on the test set (second study, 1997–2005, n = 1,003). Confidence intervals at 95% of the area under the receiver operating characteristic (ROC) curves (AUCs) on the test set were calculated, and the method proposed by DeLong and colleagues was used to check for statistically significant differences in AUCs [10].

To address the problem of predicting bacteremia/absence of bacteremia, a logistic regression model was constructed. The factors considered as possibly associated with the diagnosis of bacteremia or gram-negative bacteremia, all available at FN onset, were: age (<60, ≥60), sex, underlying cancer (hematological, solid), treatment setting (adjuvant or neoadjuvant, first line, second line, bone marrow transplant), presence of a clinical site of infection (yes/no), controlled cancer (yes/no), granulopoiesis stimulating factors (yes/no), prophylactic antibiotics (yes/no), ECOG performance status (0 to 4), burden of illness (no symptoms, mild symptoms, moderate symptoms, severe symptoms, moribund), hospitalization status (inpatient/outpatient), temperature (<39°C, ≥39°C), fever duration at presentation (<24 h, ≥24 h), abnormality on the chest X-ray (yes/no), hypotension (yes/no), pulse (≥120 beats/min, <120 beats/min), respiratory rate (>24 breaths/min, ≤24 breaths/min), comorbidity (multiple choices possible: cardiac disease, chronic obstructive pulmonary disease, diabetes, confusion or alteration of mental state, blood loss, dehydration, weight loss of >5% within 1 month), previous FN (yes/no), previous fungal infection (yes/no), Talcott group (I to IV), hemoglobin level (≥8 g/dL, <8 g/dL), absolute neutrophil count (≥100/µL, <100/µL), platelet count (≥5,000/µL, <5,000/µL), creatininemia (≥2 mg/dL, <2 mg/dL), bilirubinemia (≥2 mg/dL, <2 mg/dL), albumin level (≥2.5 g/dL, <2.5 g/dL), and the MASCC score (<21, ≥21). Then based on forward–backward selection methods (using the statistical significance level of the differences in χ 2) and after checking for interactions among the factors and determining whether the model was linear in the logit for continuous factors, an input selection was made for the logistic regression models. The data were not separated into a development set and a test set, resulting in an overestimation of the predictive performance. The reason for doing this was the limited number of patients with gram-negative bacteremia: we wanted to have the largest possible number of such cases in order not to miss any possible prognostic marker.

Results

Patient characteristics

Median age was 52 years, the male-to-female ratio was almost 1:1. Fifty-seven percent (1,223 of 2,142) had hematological tumor, and 69% (1,488 of 2,142) of the patients were predicted at low risk (MASCC ≥ 21) with a 91% successful outcome rate (1,349 of 1,488) among them.

In 23% (499 of 2,142), bacteremia was present: 8% (168 of 2,142) single gram-negative bacteremia, 13% (283 of 2,142) single gram-positive bacteremia, and 2% (48 of 2,142) polymicrobial bacteremia. The overall serious complications rate including death was 14% (228 of 1,643) in non-bacteremic patients, 40% (68 of 168) in single gram-negative bacteremia, 25% (70 of 283) in single gram-positive bacteremia, and 35% (17 of 48) in polymicrobial bacteremia.

More details on the sociodemographic characteristics and pathogens causative for bacteremia were given in a previous publication [1] which looked at the outcomes and types of complications developed by patients with bacteremia.

Effect of MASCC score on complications with stratification by bacteremia status

Table 1 indicates patients’ outcome (complications and mortality) according to the MASCC score and stratifying for the bacteremic status. Predicted low-risk non-bacteremic patients had an overall 7% rate of severe complications including a mortality rate of 1%. On the other hand, in predicted high-risk patients, the corresponding rates were 32% and 9%. The rates of resolution without complication were statistically significant between the two groups (P < 0.001). Similar results were observed for single gram-positive and single gram-negative bacteremias: e.g., patients with gram-negative bacteremia and with a MASCC score <21 had a rate of complications of 60% including a mortality of 29%, while the corresponding rates in patients with gram-negative bacteremia and a MASCC score ≥21 were 20% and 6% (P < 0.001). No heterogeneity in the odds of having a successful outcome in both risk groups could be detected (P = 0.20) for the three strata of bacteremic status.

Table 1 Bacteremia versus final outcome stratified by the MASCC score and the type of cancer
Full size table

Since the negative predictive value of the MASCC score is better in patients with a solid tumor and bacteremia occurs more frequently in hematological malignancies, the analysis was further stratified according to tumor type with comparable results in both disease groups: patients with MASCC score <21 have poorer outcomes than patients predicted at a low risk of complications, regardless of tumor type.

As indicated in Table 2, the MASCC index score does not only predict differences when dichotomized. As expected, due to the construction of the score, the lower the score, the greater the risk of complications and death. This increased risk of complications and death is most significant in patients with a MASCC score below 15.

Table 2 Complication and death rate stratified by classes of the MASCC score and type of bacteremia
Full size table

The incidence of bacteremia in the sample population is related to the MASCC score (Table 3). Bacteremia was less often present in predicted low-risk patients (MASCC ≥ 21): 19% (290 of 1,488) compared to 32% (209 of 654) in predicted high-risk patients (MASCC <21) (OR, 0.52; 95%CI, 0.42–0.63, P < 0.001). Mainly single gram-negative bacteremia appeared less frequently in low-risk patients, i.e., only 5% (81 of 1,488) versus 13% (87 of 654) in non-low-risk patients (OR, 0.38; 95%CI, 0.27–0.52, P < 0.001).

Table 3 Incidence of bacteremia in the population stratified by the MASCC score
Full size table

Does the inclusion of bacteremia as a baseline covariate improve the MASCC score?

Or in other words, would knowledge—at the time of fever onset—of the presence of a bacteremia improve the identification of low-risk or high-risk patients? Three logistic regression models were constructed on the derivation set (first study, 1994–1997, n = 1,139) in order to predict a successful outcome (no serious complications) using as explanatory variables: (1) the MASCC score, (2) the MASCC score in combination with bacteremia, and (3) the MASCC score in combination with gram-negative bacteremia (all variables had P < 0.001). The ROCs from applying the three models (1), (2) and (3) to the test data (second study, 1997–2005, n = 1,003) are shown in Fig. 1. The AUC is significantly better when considering the model with the MASCC score and the knowledge of bacteremia, but this better performance is due to improvements occurring at low values of sensitivity. As our goal with a revised prediction rule would be to improve sensitivity while preserving the positive predictive value, there is no clinical relevance for the improvement. The AUCs and comparison tests were: (1) 0.756 (95%CI, 0.720–0.792), (2) 0.773 (95%CI, 0.738–0.807), (3) 0.755 (95%CI, 0.719–0.792); (1) versus (2) P = 0.02, (1) versus (3) P = 0.89. When the analysis was restricted to death prediction, bacteremia as well as gram-negative bacteremia were significantly associated with mortality when combined with the MASCC score (chi-square P = 0.03 and P = 0.01 on the development set). No difference in AUCs could be observed in the test data (see Fig. 2): (1) 0.778 (95%CI 0.715–0.840), (2) 0.790 (95%CI 0.729–0.851), (3) 0.791 (95%CI 0.729–0.854). The P values for the comparisons between the areas were (1) versus (2) P = 0.31 and (1) versus (3) P = 0.12.

Fig. 1
figure 1

ROC curves for predicting successful outcome (no serious complications) in the test population (N = 1,003)

Full size image
Fig. 2
figure 2

ROC curves for predicting mortality in the test population (N = 1,003)

Full size image

Prediction of bacteremia and gram-negative bacteremia

Although there are significant differences in the incidence of bacteremia according to the MASCC score, the score cannot be reliably used for predicting bacteremia: sensitivity of 42% (209 of 499), specificity of 73% (1,198 of 1,643), positive predictive value (PPV) of 32% (209 of 654), and negative predictive value (NPV) of 81% (1,198 of 1,488). The same holds for the prediction of gram-negative bacteremia: sensitivity of 52% (87 of 168), specificity of 71% (1,407 of 1,974), PPV of 13% (87 of 654), and NPV of 95% (1,407 of 1,488).

Therefore, a predictive model was built considering 28 characteristics for input selection as stated in the methodology section. Logistic regression analysis resulted in the delineation of an optimal set of variables independently related to the presence of bacteremia (Table 4). The predictive performance of this model was the following: area under the ROC curve of 0.720 (95%CI, 0.695–0.744), sensitivity 69%, specificity 64%, PPV 37%, and NPV 87% at a probability cutoff level of 0.24.

Table 4 Selected model for predicting, respectively, bacteremia and gram-negative bacteremia
Full size table

Similarly, a model was constructed in order to predict gram-negative bacteremia; for that purpose, the gram-negative bacteremias considered were the single gram-negative bacteremias and the polymicrobial bacteremias containing at least one gram-negative organism (Table 4). The area under the ROC curve was 0.742 (95%CI, 0.707–0.776), sensitivity 72%, specificity 66%, PPV 17%, and NPV 96% at a probability cutoff level of 0.09.

Discussion

We found a significant association between the predicted risk of complications and the bacteremic documentation: patients at a low risk of complications have also a lower risk of bacteremia than the other patients. Namely, gram-negative bacteremia is significantly more frequent (13%) in the high-risk patients compared to the low-risk group (5%).

The risk of complications and death was increased in bacteremic patients in both strata of predicted low- and high-risk patients—especially in patients with gram-negative bacteremia. However, we did not detect any interaction between the MASCC score and the bacteremic status, which led us to the conclusion that the MASCC score is discriminant in all strata. These results remain true if we are dividing patients into three risk groups rather than two.

In addition, the integration of the bacteremic status failed to increase the accuracy of a clinical prediction rule aiming at identifying patients at low risk of complications. The reverse situation would have led us to solve the problem of estimating the presence of bacteremia at fever onset, although we were not able to solve it. Indeed, our results confirm that the absence of bacteremia is unpredictable, at least with the covariates that we assess today systematically. As bacteremia is a heterogeneous entity, we also attempted to predict the absence of gram-negative bacteremia with the same conclusion that we failed to develop a satisfactory model. In the future, it is possible that we might have some laboratory parameters to be used as markers for bacteremia like C-reactive protein, interleukin, or procalcitonin levels; but—up to now—these markers were not demonstrated specific enough, specially at the start of a febrile episode [11–13], and were most often studied in single-institution studies with small sample sizes and heterogeneous patient populations. Multicenter trials with larger numbers of patients are needed to clarify the role of these markers and their potential to be useful early enough for the prediction of serious complication which is in fact a clinically important event, above the knowledge of bacteremia. Nevertheless, early identification of bacteremia remains an interesting theoretical issue as well as a practical issue for targeting the choice of the antibiotherapy (for instance if the absence of gram-negative bacteremia could be predicted) and a potential area for further research.

We observed that patients with a low MASCC score and a gram-negative bacteremia have a very high rate of complications and death (60% and 29%, respectively). Predicting such high-risk patients is another very interesting issue that was not really addressed by the clinical prediction rule derived from the MASCC score. This subgroup of patients fortunately represents a low proportion of febrile neutropenic patients, and any attempt to develop a rule for predicting such a high-risk population would need a further study with a sample size larger than the one of the data base we were presently able to analyze.

Our data indicate that, whatever the MASCC score is, the mortality in patients with gram-positive bacteremia is not different from that of non-bacteremic patients, confirming previous observations about the relatively indolent course of gram-positive infections in neutropenic patients [2, 14], except in those with a very low (<15) score, in whom gram-positive bacteremia can be a very serious event.

Finally, our conclusions were not different in patients with hematological malignancies and in those with solid tumors, although the negative predictive value of the MASCC score is slightly better in solid tumor patients than in hematological patients; this means that the MASCC score is able to capture, using other covariates, the predictive value that the underlying disease carries and, contrary to the bacteremic status, the underlying disease is not an independent prognostic factor of outcome which is against a common belief that a patient with a hematological underlying disease is always at high risk of complications and death during an episode of FN [15, 16]. We also confirmed that the MASCC score is not only helpful with a dichotomization into patients at predicted low and high risk. The MASCC score is also able to identify patients at very high risk of complications, and this was further discussed in a previous publication [1]. Developing specific therapeutic approaches for this very high-risk group is another very interesting topic for further research.