Introduction

Although there is a strong rationale supporting the centralisation of critically ill children to paediatric intensive care units (PICUs), currently available evidence in favour of this strategy is not overwhelming [1]. The most significant studies in this field were published in the 1990s. A study including 436 patients comparing paediatric intensive care in 3 tertiary and 71 non-tertiary hospitals in the U.S. showed that the tertiary centres had a better severity-adjusted mortality [2]. A similar issue was addressed by a study involving all the ICUs that routinely provided paediatric intensive care in the Netherlands, but the results did not clearly support the greater effectiveness of tertiary centres [3]. Moreover, methodological criticisms were raised in the accompanying editorial [4]. Curiously, according to the results of a large multicenter study carried out in the U.S., severity-adjusted mortality was higher in teaching hospitals than in non-teaching hospitals. The same study, however, demonstrated that the presence of a paediatric intensivist improved mortality odds ratios [5]. Finally, in a region (Victoria, Australia) with highly centralised paediatric intensive care, the severity-adjusted risk of death in critically ill children was sharply decreased compared to a region were children where cared for in non-dedicated ICUs (Trent, UK) [6].

In the early 1990s, the inhomogeneous level of paediatric intensive care delivery in the U.K. was well known [7]. In the following years a strategy for centralisation of critically ill children to PICUs with high standards of care within specific geographic areas (regionalisation) was designed [7–9]. However, criticisms about the weakness of the evidence produced to support the benefits of centralisation were raised [8, 10, 11]. The effect of centralisation of paediatric intensive care in the area of Birmingham was studied at the end of 1990, demonstrating an increased admission rate to PICUs but no conclusive answer on the outcome of critically ill children [12].

In 2000, although no further evidence was available, the American Academy of Pediatrics and the Society of Critical Care Medicine recommended the implementation of regionalisation in the U.S. [13]. In addition to the studies mentioned above, evidence concerning centralisation of high-risk pregnancies and perinatal care was advanced to support these conclusions [14]. Although studies that investigated the organisation and features of PICUs in Europe have been carried out [15, 16], no comparisons with the outcome of children managed in adult ICUs are yet available.

In Italy, the lack of a homogeneous distribution of PICUs and of a shared policy concerning centralisation of critically ill children where PICUs are available yields a jeopardised situation. The consequences of a partial centralisation of critically ill children is still undetermined.

The objective of this study is to evaluate the outcome of critically ill infants and children who were admitted to 124 Italian adult ICUs from 2003 to 2007 (our study group), and compare with an historical control group of paediatric patients admitted to 26 Italian PICUs and ICUs with paediatric dedicated beds between 1994 and 1995 [17].

Materials and methods

Data source

Data concerning the 2003–2007 paediatric study group were stored in the Project Margherita database. This project was launched in 2002 by the GiViTI (Italian Group for the Evaluation of Intervention in Intensive Care Medicine), for continuous quality assessment and improvement of Italian ICUs; the group has now been joined by 225 units but only by 5 PICUs. All participating ICUs collect detailed information on patients’ demography and clinical history, data concerning ICU admission, ICU stay, and ICU and hospital outcome.

Most data collected for the study group were also available from a previous study, carried out by the GiViTI group about 10 years earlier, that we used as historical control group [17].

Study group: data collection and patients

All patients younger than 15 years admitted to the participating adult ICUs (excluding cardio-surgical and transplant units) were included. The data collection was carried out by a trained physician involved in the patient’s care. The information prospectively collected included demographics; admission diagnosis; comorbidities, defined as the co-existence of a disease state that was present before the admission to the ICU (involving the respiratory, cardiovascular, gastrointestinal, hepatic, renal, endocrine or other organ system or malignancies); surgical status; source and reason for admission; PRISM variables [18, 19]; failures and diseases that occurred during the stay; major intervention/procedures performed in the ICU; and ICU and hospital outcome.

We excluded patients with a hospital stay of less than 24 h, as this was the selection criterion adopted for the historical control group. For the sake of comparison, we also excluded children who were transferred to a PICU, as they were obviously absent in the historical control group of children admitted to PICUs.

Because the study was fully observational and no information that could identify patients was collected, informed consent was waived.

Project Margherita database quality control

A call centre was active to answer any questions during the study. The software for data collection provided investigators with an online definition of each item to collect. A multitude of validity checks were done concurrently with data entry. The system allowed saving inconsistent or implausible data, but marked the record as problematic. Data were further reviewed by the coordinating centre and all doubts were discussed and resolved with the single ICU.

To avoid selection bias, all patients admitted in months with more than 10% incomplete or inconsistent records were excluded.

Statistical analysis

Proportion was used as a descriptive statistic for categorical and ordinal variables, median and interquartile range (IQR) for ordinal and continuous variables, mean and standard deviation for continuous ones. Categorical variables were compared with the chi-squared test, and p < 0.05 was considered statistically significant. The trend over time of the proportion of eligible paediatric patients was assessed by means of the Cochran-Armitage test for linear trend [20].

The ratio between the actual and expected number of ICU deaths (standardised mortality ratio, SMR) with its 95% confidence interval (95% CI) [21] was computed according to the PRISM score for the current PRISM study group and compared with that of the historical control group.

To compare calibration across classes of risk between the study and control group, we used traditional calibration plots without applying the formal Hosmer-Lemeshow statistics since it gives controversial results when the number of covariate patterns is few compared to the sample size [22]. Since data needed for the Cox analysis of calibration [23] were not available for the historical control group, this analysis was not performed.

Results

From January 2003 to December 2007, after applying the exclusion criteria described in the “Materials and methods” section and summarized in Fig. 1, 1,265 patients from 124 adult ICUs formed the study group. The proportion of this subgroup of children out of the total ICU admissions was 0.47% in 2003, 0.52% in 2004, 0.55% in 2005, 0.61% in 2006 and 0.57% in 2007 (test for trend p = 0.026). Between July 1994 and June 1995, 14 PICUs and 12 ICUs with dedicated paediatric beds admitted 1,887 patients. After the application of exclusion criteria, 1,533 patients remained and formed the historical control group [17].

Fig. 1
figure 1

Flow chart of the study

Full size image

Table 1 reports the clinical characteristics of the ICUs and the patients forming the PRISM study and the historical control groups.

Table 1 ICU and patient characteristics
Full size table

The overall ICU mortality rate (which is the objective of the PRISM prediction) in the study group was 7.4 and 11.4% in the historical control group.

The great majority of patients were clustered in the first risk class (0.0–0.1) both in the study group (86.8%) and in the historical control group (75.5%). According to the PRISM score, the SMR was 1.11 (95% CI: 0.91–1.31) in the study group and 1.04 (95% CI: 0.88–1.19) in the historical control group.

In both samples, traditional calibration plots showed higher than predicted mortality in the low classes of risk, especially the first one, where the great majority of the patients clustered (Fig. 2). Deviations in higher risk classes were difficult to interpret given the very low number of patients in each risk subgroup.

Fig. 2
figure 2

Calibration plots for the study and historical control groups. The table below each plot reports the number of patients in each decile

Full size image

In the study group, there were more children older than 1 year (34.1%, p < 0.001), 16.1% more surgical emergencies, 15.4% fewer medical patients (p < 0.001), 26.2% more trauma patients (p < 0.001) and 4.1% more patients transferred from other hospitals (p = 0.001).

Discussion

The adequacy of care provision to critically ill children is still an open matter [1]. There is a strong plausibility underlying the strategy of centralisation and regionalisation of paediatric intensive care, but available evidence is limited to a few, non-recent studies, sometimes producing controversial results. The GiViTI Project Margherita has been joined by very few PICUs, hampering the possibility of direct comparison between intensive care delivered by specialist and non-specialist ICUs. For this reason we decided to compare the SMRs based on the PRISM score of children treated in adult ICUs with those of patients treated in PICUs about 10 years earlier.

A trend toward mortality overprediction over time was reported for the PRISM score, as for other prognostic models, to a large extent (up to 50% over 15 years) [24–26]. Hence, our hypothesis was that the study group would have a lower SMR compared to the historical control group. The SMRs, instead, were substantially the same for the two paediatric samples. These results suggest that the quality of care provided to paediatric patients in adult ICUs is comparable to that offered by PICUs 10 years earlier. Moreover, it is reasonable that Italian PICUs have improved their results over time and would be able to offer a higher level of care to critically ill children compared to 10 years ago. This is supported by a recent large study on Italian PICUs that reported a good calibration of the recently updated PIM 2 score (SMR 0.89; 95% CI 0.71–1.09) indicating a level of care comparable to international standards [26, 27].

Thus, our study strongly suggests that the level of care provided by Italian adult ICUs to paediatric patients would be below the standard offered by Italian PICUs, if a direct comparison was made.

It is also notable that in this study the percentage of children admitted to adult ICUs increased significantly over the 5 years of our study, indicating that the management of critically ill children by non-specialised personnel in Italy is still a well established habit.

It is clear that these results call for further investigations in this field, which would require a tight collaboration between permanent adult and paediatric registries. Unfortunately, at the moment in Italy only a few PICUs have joined Project Margherita and other registries are missing, thus the evidence provided by this study is the best available.

The PRISM score we used for SMR calculation is an obsolete tool for the assessment of expected mortality as it was developed from a cohort of patients admitted to PICUs between 1980 and 1985 [28], and this could be viewed as a major limitation of our analysis. However, when two samples are compared using the same score, although absolute SMR values may not be trustworthy, their difference is indicative of a different performance according to that score. This is the approach we followed which, far from addressing the predictive ability of PRISM (as external validation studies do), uses the same standard to compare ICU performance between different samples.

The consistency of our analysis (i.e. using the same prognostic score not to make exact predictions but to measure different performances) holds if three conditions are met: First, the prognostic model must have been correctly built [29]. Second, the case mix of study and control groups should be similar [23]. Third, the prognostically important variables, not related to care improvement over time, should have been the same over the 10-year period [23, 30]. The first two conditions can be granted, as the score underwent a rigorous internal validation procedure when it was created [18, 19], the pattern of clinical and demographic features of the patients was similar in the study and the control groups, and the differences in the study and control groups mostly concerned variables that the model adjusted for (i.e. age, neurological failure, surgical status). The third condition, on the other hand, is more difficult to verify. However, since the case mixes are similar, we have to assume that it is improbable that a new variable (i.e. prognostically relevant in the study sample but not in the historical control group) may significantly affect the value of coefficients included in the prognostic model. Moreover, the samples were from the same country, which is an important condition to exclude the possible differential influence of unmeasured context variables [23, 30].

The use of a score that considers the worst value over the first 24 h in ICU could be considered a limit of our study since a worst-in-24-h score may “reward” units providing a low level of care to their patients (e.g. patients who rapidly deteriorate because of mismanagement will have high scores) [31]. However, it is reasonable to think that the lower level of care was delivered by adult ICUs. In this case the expected mortality in these ICUs could have been overrated, reducing the SMR value. Thus, we can assume that our evaluation of the low level of care provided by adult ICUs was, at the worst, conservative, not substantially affecting the conclusions of our study.

In the end, a suboptimal outcome of children in general adult ICUs in Italy is suggested by this analysis, but only further studies adopting the most appropriate score and including a direct comparison with the results of PICUs will be able to give a definite answer to this hypothesis. The findings of such a study, which we hopefully will be able to run in the near future, could give relevant indications for the health care organisation in Italy and in other countries where the number of PICU beds is undersized, PICUs are unevenly distributed, and centralisation is not or only partially applied.