Introduction

Late-onset neonatal sepsis (LOS), defined as neonatal sepsis occurring after 3 days of age, is an important cause of morbidity and mortality in preterm infants [2, 3, 19]. Early and correct diagnosis of LOS is a challenging task. Particularly in preterm infants, the presenting signs are often very subtle and non-specific. Furthermore, as microbiological culture results are not available within 48 h, early identification of a genuine sepsis is a major problem. Considering the possibly devastating consequences of missing LOS, providers often have a low threshold for starting antibiotic therapy. However, unnecessary use of empirically started broad-spectrum antibiotics should be minimised for reasons of growing resistance against antibiotics and possible harmful effects on gastrointestinal immunity and allergy [4, 12].

Although many authors state that clinical signs are unreliable in the diagnosis of LOS in neonates [8, 14], good quality studies addressing the value of clinical signs are sparse, especially in preterm neonates [5, 9, 11, 14, 18]. Furthermore, most studies relied on blood culture-proven sepsis as a definite outcome measure. However, sepsis-like episodes with false-negative blood cultures (i.e. clinical sepsis) are frequently encountered in preterm infants because of the limitation in number of blood cultures taken and quantity of blood drawn [6, 13].

In the era of sophisticated laboratory techniques, much emphasis has gone to the value of haematological and biochemical markers in the diagnosis of LOS [10, 15, 16]. Hence, clinical judgement might unjustly be undervalued. Still, it remains of great importance to investigate the diagnostic value of clinical judgement.

The aim of this study was to evaluate the value of various clinical signs in identifying both blood culture-proven as well as clinical LOS in preterm neonates in a neonatal intensive care unit (NICU) setting, without the use of laboratory investigations. In addition, we wished to develop a nomogram, consisting of clinical signs, to assist in decision-making for treatment in preterm neonates suspected of LOS.

Methods

Patients and data collection

A prospective cohort study was undertaken at our level III NICU in Zwolle, The Netherlands, from July 2005 until November 2007. Eligible patients included all patients with a postconceptional age <34 weeks and more than 72 h postnatal age and not on antibiotic therapy for the last 24 h. Patients were followed until a corrected gestational age of 35 weeks or until discharge to other hospitals before 35 weeks.

An episode of suspected infection was defined as a clinical suspicion of infection by the attending neonatologist. The clinical suspicion of infection ranged from very mild to very severe. Each patient with mild to severe suspicion of infection was included irrespective of the prescription of antibiotics. In this way, inclusion of a wide spectrum of disease severity was aimed for in order to minimise the chance of overestimating the diagnostic accuracy of the clinical signs. In case of a very mild suspicion of infection, where no antibiotics were started nor blood cultures were taken, the episode was evaluated for the occurrence of LOS over the following 3 days. We reasoned that in case of withholding antibiotics, without further aggravation of clinical symptoms, no clinically relevant bloodstream infections would be missed.

At the onset of each episode, data on clinical signs were assessed in a standardised way. Before the start of antibiotic treatment, blood for blood cultures (1–3 ml), C-reactive protein (CRP), and full blood count were drawn.

Clinical signs and symptoms

Based on the literature [5, 7, 8, 10] and clinical experience, a total of 14 clinical signs were assessed: pallor or grey skin colour, capillary refill time >2 s [20], dyspnoea (grunting, nasal flaring and/or chest retractions), tachypnoea (respiratory rate >60/min during >1 h), need for increased respiratory support (intensifying the modus, i.e. low flow, CPAP or endotracheal ventilation and/or degree of respiratory support), increasing need for supplemental oxygen, tachycardia (pulse >180/min during >1 h), temperature instability (difference in body temperature >0.5 °C within 24 h), hyperthermia (rectal temperature >38.0 °C), hypothermia (rectal temperature <36.0 °C), feeding difficulties (vomiting or gastric aspirates >50 % of feed volume), increasing frequency of apnoea, bradycardia and/or cyanotic spells, lethargy and irritability. The clinicians and research nurses prospectively assessed these signs at the onset of each episode using a standardised form. Furthermore, the following seven risk factors were noted: gestational age at birth, birth weight, sex, central venous catheter (CVC) or removal of a CVC in the preceding 24 h, mechanical ventilation, actual weight and postnatal age.

Laboratory investigations

Blood samples for C-reactive protein, leucocytes with differential count and blood culture were taken at onset of clinical symptoms signalling suspected infection. C-reactive protein was measured by immunoturbidimetry on a Roche modular P instrument using the C-reactive protein latex Tina-quant® assay (Roche Diagnostics). CRP >10 mg/l was judged to be indicative for sepsis. Leukocytes and differential count were measured by flow cytometry on a Cell Dynn 4000 machine (Abbott). Leucocytosis was defined as leucocyte count ≥25 × 10−E9/l and leucopenia ≤5 × 10−E9/l.

Blood cultures were drawn before the start of antibiotic therapy (1–3 ml in 40 ml culture vials: Bactec™). A positive blood culture with organisms regarded as commensals (predominantly coagulase-negative Staphylococcus) was defined as contamination. However, a positive blood culture with skin commensals was defined as proven sepsis when the same organism was found in at least two blood cultures and/or signs of catheter-related sepsis were present (i.e. inflammation of the skin at the site of line insertion).

Final diagnosis of LOS

The outcome of each episode was classified in three mutually exclusive categories: blood culture-proven sepsis, clinical sepsis and rejected sepsis. The classification was made by the researchers based on the course of the episode after the start of antibiotics and laboratory values (CRP, full blood count). Blood culture-proven sepsis was defined as an episode with positive non-contaminated blood culture. Infants were classified as having clinical sepsis in case of a strong clinical suspicion for infection despite negative blood cultures as defined by the attending neonatologist or in case of raised CRP (>10 mg/l), leucocytosis or leucopenia or haematological markers. Rejected sepsis was defined as an episode with negative blood culture and/or an episode with a favourable course where no blood culture was done and no antibiotics were started in case of low CRP and normal haematological markers. In all our analyses and results, LOS was defined as blood culture-proven and/or clinical sepsis.

Statistical analysis

We used logistic regression to examine the association between clinical signs and the presence or absence of LOS. Some patients (n = 38) experienced more than one episode of suspected infection. To avoid inaccurately extra weighing of risk factors in these patients, we evaluated patient-specific risk factors (gestational age, birth weight and sex) only for the first episode. Odds ratios (OR) and 95 % confidence intervals (95 %CI) were used to quantify the strength of these associations.

Our variable selection and modelling approach were based on the following steps. Simultaneously fitting all 14 clinical signs and 7 risk factors in a single model would lead to overfitting (ratio of variables to number of events = 1:4) and unpredictable results [17]. Therefore, we classified all signs into four clinically coherent groups, so that the signs within each group shared common features related to pathophysiology. These four groups were: respiratory, circulatory, general symptoms and risk factors.

Within each group, we applied backward logistic regression to select only those signs that were significantly associated with LOS using a p value < 0.1 as a criterion to stay. We then built a multivariable model with the selected signs from each group. Model evaluation consisted of receiver operating characteristics analysis (AUC) and Hosmer–Lemeshow goodness-of-fit-test.

Bootstrapping was used to correct for possible overoptimistic results of the final model. Bootstrapping is an internal validation technique, where many repeated samples are drawn with replacement from the data set at hand. Bootstrapping generates an estimate of how well the model might fit in a new study population. In other words, bootstrapping estimates the expected optimism in model performance or shrinkage of the model [17].

Finally, a nomogram was developed to visualise the predictive strength of the different clinical signs and risk factors in a single diagram. This nomogram allows readers to calculate an expected risk of LOS based on the specific profile of a patient. The number of points for each predictor was based on the regression coefficients of the reduced multiple regression model. The total numbers of points derived by the presence or absence of all predictors were used to calculate the expected probability of LOS. Analyses were performed using the statistical package SPSS (PASW) version 18.0.

Consent and ethical approval

The study was approved by the local medical ethical committee of our hospital. Written informed consent was obtained from the parents.

Results

During the 2-year study period, a total of 319 eligible patients were admitted to our NICU of whom 142 experienced one or more episodes of suspected infection. A total of 187 episodes of suspected infection occurring in 142 patients were evaluated. Basic characteristics of included patients are presented in Table 1; inclusion and classification of episodes are presented in Fig. 1.

Table 1 Characteristics of the study population: patients with suspected infection. Data from patients who experienced more than one episodes of suspected infection are from their first episode
Full size table
Fig. 1
figure 1

Inclusion and classification of episodes of suspected late-onset sepsis

Full size image

Final diagnosis

A final diagnosis of LOS was made in 89 (48 %) out of the 187 episodes of suspected infection. Twenty-six percent (n = 50) of the episodes were classified as proven sepsis and in 21 % (n = 39) clinical sepsis was judged to be present (Fig. 1 and Table 2). Of the 39 episodes of clinical sepsis, in 46 % (n = 17), laboratory values were normal (low CRP and haematological markers) and the diagnosis of clinical sepsis was made on the basis of the severity and the course of clinical signs alone. Median CRP in the clinical sepsis group was 11 mg/l (range 0–86).

Table 2 Clinical signs and risk factors in episodes of suspected infection and associated definite outcome
Full size table

A positive blood culture was found in 56 episodes, of which six were considered contaminated. Among the 50 episodes of proven sepsis, Staphylococcus epidermidis was the most common isolate (50 %), followed by Bacillus cereus (28 %), Staphylococcus aureus (12 %), gram negatives (6 %), Streptococcus (2 %) and Candida (2 %).

Clinical signs and risk factors for late-onset sepsis

Table 2 shows data on clinical signs and risk factors for the different outcome groups. Clinical signs and risk factors associated with LOS at the p = 0.05 level were: weight at the episode, CVC, respiratory insufficiency, lethargy, capillary refill, pallor/grey skin and increased oxygen requirements. Clinical signs that showed no significant association with LOS were: temperature instability, apnoea, tachypnoea, tachycardia, dyspnoea, hyper- and hypothermia, feeding difficulties and irritability.

Results of multiple regression analysis

Selection of variables

After backward elimination within each of the four clinically coherent categories separately, the following variables remained significantly associated with LOS within the four categories: respiratory signs: increased respiratory support; circulatory signs: capillary refill time and pallor/grey skin; general signs: lethargy; and risk factors: weight at episode and a central venous catheter.

Multiple regression model

The remaining variables were entered in a multiple regression analysis and the corresponding results are presented in Table 3. Performance of this “full” model for predicting LOS was good, with AUC 0.80 (95 %CI 0.74–0.87; p < 0.001) and Hosmer–Lemeshow goodness-of-fit-test p = 0.438. The expected optimism in model performance evaluated by bootstrapping was small (e.g. a decrease in AUC from 0.80 to 0.71, a shrinkage factor of 0.89).

Table 3 Results from multivariable logistic regression analysis
Full size table

In the next step, we applied backward elimination (p < 0.1 as criterion to stay) to see whether variables could be excluded without a relevant loss in performance. Four variables remained significantly associated with LOS: increased respiratory support, capillary refill time, pallor/grey skin and a central venous catheter (Table 3). Labelling infants as low risk for sepsis when all four factors are absent, which was the case in 36 infants, would miss three infants (8.3 %) with clinical and proven sepsis and one infant (2.8 %) with proven sepsis. Sensitivity of the presence of one or more of these four factors for LOS was 97 % and specificity 37 %.

A nomogram was constructed based on the reduced model and is shown in Fig. 2. The value of each predictor corresponds to a score. The scores for all predictors are summed to a total score, which is then translated into a probability for LOS.

Fig. 2
figure 2

Nomogram for prediction of LOS in preterms suspected of infection with use of clinical signs and risk factors. LOS is defined as clinical and/or blood culture-proven sepsis. Instructions: Determine how many points the patient receives for each feature using the upper part of the nomogram. Sum the points for all features. Locate this sum score on the “total points” axis. Draw a straight line to the lower axis “probability sepsis” to find the estimated probability of that patient having LOS (clinical and/or proven LOS)

Full size image

Difference between clinical sepsis and proven sepsis

When the analysis approach was repeated but now for the associations with solely proven sepsis, instead of clinical and proven sepsis, the same clinical signs stayed in the final model, except for pallor and/or grey skin (Table 3). Performance of these models for predicting proven sepsis was also good with AUC 0.84 (95%CI 0.78–0.90; p < 0.001) and Hosmer–Lemeshow goodness-of-fit-test p = 0.319 for the full model and AUC 0.83 (95 %CI 0.76–0.89; p < 0.001) and Hosmer–Lemeshow goodness-of-fit-test p = 0.174 for the reduced model.

Discussion

Principal findings

This study shows that most clinical symptoms in isolation have only moderate predictive value for identifying LOS in preterm infants suspected for infection. The strongest predictive signs were: increased respiratory support, capillary refill time >2 s, pallor or grey skin colour and a central venous catheter in 24 h preceding the onset of suspected infection.

Combining several clinical signs in a nomogram augments the predictive value for identifying LOS. This nomogram allows users to calculate an expected risk of LOS in an individual patient with suspected infection, based on the specific profile of the patient.

Clinical signs that are too aspecific to be useful in excluding or diagnosing LOS were temperature instability, apnoea, tachypnoea, tachycardia, dyspnoea, hyper- and hypothermia, feeding difficulties and irritability. Even when two or more of these aspecific signs occur simultaneously, the risk of neither clinical nor proven sepsis is hardly changed (data not shown).

Lack of clinical relevance of body temperature in diagnosing LOS in preterms might be attributable to the use of incubators in this specific patient population. When changes in body temperature are observed, the environmental temperature in the incubator will be manipulated before serious hypo- or hyperthermia can occur. Probably, the need and magnitude of temperature adjustment of the incubator are more valuable items to measure.

The marginal diagnostic value of general respiratory signs (apnoea, dyspnoea and tachypnoea) might be explained by the high prevalence in preterms of non-infectious respiratory problems due to lung immaturity or bronchopulmonary dysplasia. The same might hold true for the minor clinical relevance of other general symptoms such as feeding intolerance and irritability.

Another interesting finding of this study is the difference in observed frequency of clinical signs between clinical and proven sepsis. In this study, especially pallor and/or grey skin colour was strongly associated with clinical sepsis compared to blood culture-proven sepsis. A grey skin colour is obviously considered a serious sign of sepsis by the medical team. The extent to which this assumption of medical workers is correct, however, cannot be answered by this study. Future studies looking at, for example, viral cultures might possibly further elucidate the issue of blood culture-negative sepsis in preterm infants.

Strengths and weaknesses

The strength of our study is that the data were prospectively collected in a population of solely preterm infants. The latter is important because clinical symptoms in preterm infants may have different clinical relevance compared to term infants [8]. Several presumed signs of LOS can also be caused by prematurity itself, such as temperature instability, apnoea and feeding intolerance. Temperature problems, for example, will be more indicative for infectious disease in term infants than in preterms.

Another strength of our study is that we did not only evaluate the predictive aspects of blood culture-proven sepsis, but also clinical blood culture-negative sepsis. Certainly, in clinical practice, clinical sepsis is frequently encountered and cannot be simply ignored.

There are certain limitations of our study. One of the major problems is the definition of clinical sepsis. The clinical signs we have evaluated were to a certain extent all contributing to the final diagnosis of clinical sepsis. This situation results in what is named incorporation bias. Since the symptoms and the diagnosis of clinical sepsis will be positively correlated in our study, misclassification may have resulted in overestimation of accuracy [1].

A serious threat when using multiple regression models, as we did in this study, is overfitting. Overfitting results in overly optimistic models. When the model is used in new patients, the performance is often worse than expected. To minimise the problem of overfitting, an adequate sample size, with enough events (i.e. LOS) compared to the potential predictors is very important. Taken to the extreme, if the number of predictors equals the number of events, the model will fit perfectly, even if all predictors are entirely unrelated to the outcome variable [10]. In general, it is assumed that a minimum of 10 to 15 events per predictor variable will allow good estimates. Finally, six predictors were used for multiple logistic regression, which seems fairly adequate, considering the 89 observed events of LOS (ratio 1:14).

Clinical and research implications

Most important predictive signs for identifying LOS in preterm infants are: increased respiratory support, capillary refill, pallor/grey skin and a central venous catheter in the 24 h preceding the episode of suspected infection. Our nomogram based on a combination of these clinical signs may predict LOS in preterms suspected of infection, even before ordering additional laboratory investigations. Clearly, we would like to emphasise that this model needs external validation in new patient groups. Still, the nomogram might be helpful in deciding on duration of antibiotic therapy; for example, in situations were no blood culture is available. Moreover, the start of antibiotics could be postponed in case of low risk, under close monitoring of clinical symptoms. Furthermore, further research evaluating the reliability and inter- and intra-observer variation for the more subjective symptoms is warranted.