Introduction

Scoring systems have been developed in intensive care medicine to predict outcome of patients admitted for severe medical or surgical illnesses. According to the results of the Second European Consensus Conference [1] in intensive medicine, these systems are appropriate to predict mortality in groups of general intensive care unit (ICU) patients, are useful in clinical trial settings and help in improving resource allocation. Nevertheless, their low accuracy does not allow adequate individual prognosis.

Principally, three scoring systems have been validated: the Acute Physiology and Chronic Health Evaluation (APACHE) [15], the Simplified Acute Physiology Score (SAPS) [16] and the Mortality Probability Model (MPM) [18]. They should be reflective of the severity of organ dysfunctions in ICU patients. These systems have been constructed in general ICU populations, with mathematical models integrating the most relevant co-variates associated with mortality. However, these systems have not always proven their validity in specific populations, such as septicemic patients, HIV-positive patients admitted to medical ICU or presenting with Pneumocystis carinii pneumonia and cardiac patients [6, 8, 19, 20, 22] or patients with neoplastic diseases [4, 13]. In a previous study [21], we compared APACHE II and SAPS II in critically ill cancer patients admitted to a specific oncological medical ICU. We observed that the prognosis for hospital mortality was first determined by acute physiologic changes induced by the medical complication, as assessed by the severity scores. No major difference was found between the two scoring systems. However, they were not accurate enough to allow making individual decisions.

Because of the lack of effectiveness in prediction of mortality in specific populations, new models were designed. In a multicentre study [10], a preliminary ICU Cancer Mortality model (ICM) was developed from 1,483 cancer patients and validated on 230 additional patients. Using multiple logistic regression models, a score including 16 variables, available at the ICU admission, was constructed. It provided a good estimation of a cancer patients’ probability to die in hospital when admitted to an ICU. Nevertheless, the authors have not compared their score to the most frequently used prognostic scores.

The aim of this study was to evaluate the effectiveness of the ICM scoring system in predicting the hospital mortality of cancer patients admitted to our ICU for a medical problem, to compare this new model to the APACHE II and SAPS II scores and to determine their prognostic roles in comparison to variables related to neoplastic disease.

Patients and methods

Between January 1999 and June 2000, all cancer patients admitted to the medical ICU of a cancer hospital for an acute complication were prospectively registered. This registration database was used to select all patients eligible for inclusion in this study. The acute complication had to be a medical problem requiring intensive care. Patients with a scheduled admission such as surveillance of an anti-cancer treatment or for elective surgery, as well as those with non-neoplastic diseases, were not considered. In case of multiple admissions of a given patient during the study period, only the first episode requiring ICU admission was taken into account. We considered as a cancer patient any person with a pathologically proven malignant neoplasm, whether the disease was active or in remission.

The collection of data needed to calculate the scores was retrospectively performed by one of the authors (TB) according to the methodology described in the initial publications. The recorded clinical and biological variables are described in Table 1. Age, type of cancer, cancer extent (loco-regional versus metastatic), cancer phase (diagnostic, curative, controllable but no more curable, pivotal or palliative) [11] and cancer status (complete remission without or under active anti-cancer treatment, partial remission, stable disease, progression, disease under induction therapy) were prospectively recorded, as well as the cause of admission. Definitions for response were the standard World Health Organisation (WHO) criteria. Mortality was measured during the hospitalisation and ICU stays as a binary variable. We also considered ICU mortality as a secondary binary endpoint to look at the stability of our results.

Table 1 Clinical and biological variables recorded for the calculation of the scoring systems. APACHE II Acute Physiology and Chronic Health Evaluation, SAPS II Simplified Acute Physiology Score
Full size table

The three severity scores investigated were calculated as described in the original publications, as well as the predicted risk of death according to the published logistic equations; linear correlation between the ICM and APACHE II or SAPS II scores was assessed by the Pearson correlation coefficient. Goodness of fit and discrimination of the three scores were calculated with calibration curves and the Lemeshow-Hosmer method [17] and by ROC curves [12] and classification tables respectively. Statistical methods are detailed in our previous study report [21].

ICU mortality was defined as the number of patients dying during ICU stay from the admission day until discharge from the ICU either to a general ward or home. Hospital mortality was defined as the number of patients dying during hospital stay, including deaths in ICU. Univariate prognostic factors analysis was not performed. A backward stepwise logistic regression was used to perform a multivariate analysis on in-ICU and in-hospital mortalities [14] using a few a priori chosen covariates: ICM, APACHE II and SAPS II scoring systems, cancer phase, type of cancer and cancer status. All variables with a p value >0.05 were excluded from the final models.

Results

Between January 1999 and June 2000, 247 patients were admitted for a severe medical complication and included in this study. Their main characteristics are described in Table 2. Immunosuppression according to the APACHE II definition was observed in 68% of the cases. The principal causes of admission were cardiovascular or respiratory problems. Mainly they respectively consisted in arrhythmia or thrombotic events and in pulmonary infections, thoracic neoplastic infiltration or COPD decompensation.

Table 2 Patient characteristics
Full size table

Observed mortality during the hospital and ICU stays were respectively 34% (83/247; 95% CI: 28–40%) and 19% (46/247; 95% CI: 14–24%). A total of 164 patients were discharged alive from the hospital. Among them, 61 died during further follow-up. Median survival after discharge was 380 (95% CI: 229–536) days.

The rate of missing data ranged from 2 and 11.7%, except for oxygen and arterial pH measurements that were not available in 34.4% of the cases. Incorporation of missing data in gravity scores calculation was made according to the recommendations of Groeger et al. for ICM. The same methodology was used for Apache II and SAPS II. Mean (SD) APACHE II, SAPS II and ICM were respectively 18.5 (7.8), 37.3 (14.5) and −1.19 (1.35) with ranges from 4 to 51, 6 to 100 and −3.47 to 4.24. The predicted in-hospital mortality rate was respectively for the APACHE II, SAPS II and ICM 32%, 24% and 28%. The scores were significantly different between hospital survivors and non-survivors whether considering ICM (mean: −1.65 versus −0.26; p< 0.001), APACHE II (mean: 16.3 versus 23.0; p< 0.001) or SAPS II (mean: 33.16 versus 45.63; p< 0.001). ICM was moderately, although statistically, significantly correlated with the APACHE II and SAPS II scores, whether the overall population (respectively r=0.41 and r=0.52; p<0.001), hospital non-survivors (respectively r=0.38 and r=0.47; p<0.001) and hospital survivors (respectively r=0.18, p< 0.01 and r=0.30, p< 0.001) were considered.

Calibration curves for the three scoring systems showed that the prediction was quite similar for the three gravity scoring models (Fig. 1). Table 3 describes, for probability intervals of 10% (or larger intervals in case of few expected events), the number of predicted and observed deaths in each scale. Comparison (as proposed by Hosmer-Lemeshow) between the observed and expected contingency tables using homogeneity chi-square tests provides a significant p value for APACHE II (p<0.001) and SAPS II (p<0.001). The p value was not significant for ICM (p=0.06), although of borderline significance. Thus, the goodness of fit was seriously inadequate except for the oncologic specific severity scoring system.

Fig. 1
figure 1

Comparison of the calibration curves for Acute Physiology and Chronic Health Evaluation (APACHE II), Simplified Acute Physiology Score (SAPS II) and the ICU Cancer Mortality Model (ICM)

Full size image
Table 3 Evaluation of the goodness of fit of the Acute Physiology and Chronic Health Evaluation (APACHE II), the Simplified Acute Physiology Score (SAPS II) and ICU Cancer Mortality models (ICM) (hospital mortality) by the Hosmer-Lemeshow method
Full size table

Two distinct methods were used to assess the discrimination between the three scoring systems. Comparison of the area under the ROC curves revealed a slightly better observed fit in favour of ICM (area 0.79) versus the SAPS II (area 0.72) and APACHE II (area 0.65). For each of the scoring systems, the logistic equation and the derived individual predicted probability of death were used to construct a decision criterion. Classification tables for the three systems using decision criteria from 10 to 90% by intervals of 10% showed that the best overall correct classification rate was 72% for APACHE II with a threshold of 60%, 74% for SAPS II with a threshold of 50%, and 77% for ICM with a threshold of 40%. A comparison of the discrepancies between the three scoring systems by cross-tabulating their predictions at their optimal threshold has shown that for survivors to hospital discharge, APACHE II and SAPS II were better at predicting outcome than ICM (respectively p=0.05 and p=0.04). Correct classification rates were respectively for APACHE II, SAPS II and ICM 96, 96 and 91%. For non-survivors, ICM was better than APACHE II (p=0.002) and SAPS II (p=0.008). Correct classification rates were respectively for APACHE II, SAPS II and ICM 25, 29 and 42%. A summary of the comparisons between the three scoring systems is presented in Table 4.

Table 4 Comparison of the scoring systems’ performances to predict hospital mortality. APACHE II Acute Physiology and Chronic Health Evaluation, SAPS II Simplified Acute Physiology Score, ICM ICU Cancer Mortality model
Full size table

A multivariate prognostic factors analysis for ICU and hospital mortalities was performed, including ICM, APACHE II and SAPS II considered as continuous variables, the cancer status (responding versus induction patients versus others), cancer phase (diagnostic and curative stages versus others) and type of cancer (solid tumour versus haematologic tumour or lymphoma) (Table 5). The variables predicting ICU mortality were ICM (odds ratio [OR]=2.24; 95% CI 1.60–3.13; p<0.001) and APACHE II scores (OR=1.10; 95% CI: 1.04–1.16; p<0.001). SAPS II score (p=0.35), type of cancer (p=0.20), cancer status (p=0.09) and cancer phase (p=0.27) were not statistically significant and were not included in the model. Two independent variables were predicting the risk of death during hospitalisation: ICM (OR=2.31; 95% CI 1.69–3.16; p< 0.001) and SAPS II scores (OR=1.05; 95% CI 1.02–1.08; p<0.001). No statistically significant difference in the risk of death was associated with the type of cancer (p=0.95), cancer status (p=0.18), cancer phase (p=0.42) and APACHE II score (p=0.09).

Table 5 Multivariate prognostic factors analyses for ICU and hospital mortality, and survival after discharge from the hospital. APACHE II Acute Physiology and Chronic Health Evaluation, SAPS II Simplified Acute Physiology Score, ICM ICU Cancer Mortality models, OR odds ratio
Full size table

Discussion

The new specific oncological scoring system (ICM) is equivalent to the general systems APACHE II and SAPS II at predicting hospital outcome of cancer patients admitted to an ICU for a medical complication. In-hospital mortality was essentially predicted by the acute physiologic changes induced by the complication as assessed by the scoring systems and not by variables related to the cancer disease. Further, ICM models such as APACHE II and SAPS II scores were not accurate enough to allow adequate individual prognosis.

Only a few studies have assessed general gravity scores for cancer patients [2, 3, 5, 7, 9, 13, 23], and only one has compared two of them, APACHE II and SAPS II, without significant differences between the two systems [21]. Because of large variations in their design (type of patients, statistical and data analyses), it is difficult to perform meaningful comparisons between them. The general conclusion of this literature was a lack of accuracy of these different systems in order to predict outcome of cancer patients admitted into the ICU and the need for new prognostic models.

Recently, Groeger et al. developed an ICU Cancer Mortality Model in a group of patients with haematological and solid malignancies [10]. They concluded that their model provided an accurate estimation of cancer patient’s probability of hospital mortality after ICU admission, although, as for the previous discussed scoring systems, correct classification remained sub-optimal. We compared this ICM model to APACHE II and SAPS II in a population including both solid (mainly metastatic) and haematological malignancies. We found ICM to be equivalent to predict individual outcome in cancer patients admitted in an ICU for a medical complication. This model was slightly superior (Table 4) in any comparison except for prediction of survivors. This potential advantage could be explained by some specificities of the ICM model, including variables related to the neoplastic disease and design—thanks to an analysis focusing on a specific population of cancer patients. Nevertheless, the retrospective design of the study, the limited number of patients and the moderate differences among the three scoring systems do not allow us to conclude as to the superiority of the ICM model. Further, the ICM model is not able to better predict individual outcome than are the APACHE II and SAPS II scoring systems.

Interestingly, the in-ICU and in-hospital mortalities were essentially determined not by the variables related to the neoplastic disease but by the acute physiological changes induced by the medical complication leading to ICU admission, as respectively assessed by the ICM and APACHE II or ICM and SAPS II severity scoring system. In a previous publication [21], we reported for the first time that APACHE II and SAPS II were, in multivariate analysis, the only significant variables predicting in-hospital and ICU mortalities. Populations for that previous study and for the present one were similar, as was in-hospital mortality. This was corroborated by further publications [3, 5, 7, 23].

The interpretation of these data is important in terms of decision for ICU admission of a cancer patient. Indeed, they mean that, except when the patient is at the palliative care phase—a criterion of ICU refusal in our study—the hospital outcome of the patient with a medical complication requiring critical care is related not to the variables of the neoplastic disease but to the acute physiological changes induced by the complication as mainly reflected by the gravity scores, which do not assess the status of the neoplastic disease. The characteristics of the cancer disease influence in fact the prognosis after the complication resolution. It is thus inadequate to refuse intensive care for this type of patients on the basis of the variables of their neoplastic disease by considering that they will predict critical care failure.

A potential bias of our study might be due to its retrospective design with consequently missing data, although the patients were prospectively registered in our database, meaning that our study does not suffer from selection bias. According to the recommendation of Groeger et al. [10], for variables for which no specific data were recorded, a response of no or never or normal was imputed if the variable was categorically scaled and a value within normal limits if it was continuously scaled. The number of missing data was relatively low except for oxygen and arterial pH measurements. It seems unlikely that these variables had an important impact because, despite our retrospective design, our data fits reasonably predictions by the ICM scoring system, although goodness of fit was worse than in the prospective Groeger’s validation study with respective p values for the Hosmer-Lemeshow statistics of 0.06 and 0.31. Further, the area under the ROC curve was 0.79 compared to 0.81 in Groeger’s paper. Moreover, in the Groeger study, no information was available on the number of missing data. Thus it seems unlikely that the differences in the design could have importantly biased the results of our study.

In conclusion, a new oncological scoring system (ICM) appears to have similar effectiveness in predicting hospital outcome as general scales such as APACHE II and SAPS II for cancer patients admitted to an ICU for a medical complication. It is, for general gravity scores, an independent prognostic factor for in-ICU and in-hospital mortality. The particularity of the ICM score is to include both physiological parameters and neoplastic characteristics reflecting essentially cancer complications but not assessing the stage or the evolution of the tumoral disease. Nevertheless, its clinical impact in routine practice remains minimal. New scoring systems designed in specific population are needed to determine more accurately the prognosis of ICU patients. Our study confirms that the hospital outcome of a patient with a medical complication requiring critical care is mainly related to the acute physiological changes due to the complication and not to the characteristics of the underlying neoplastic disease.