Introduction

The incidence of invasive fungal infections (IFI) among adult high-risk hematology patients, i.e., patients with acute leukemia and/or allogeneic hematopoietic stem cell transplant recipients, ranges from 5 to 19 % [1–5], which makes this group of patients one of the most susceptible to contracting this type of infection. IFI in hematology patients is associated with high morbidity and mortality, and early diagnosis and treatment is crucial for a positive outcome [6, 7]. The problem is that the gold standard for diagnosis of IFI, identification of fungi by culture or histopathology, is limited by the low sensitivity of cultivation [8–10] and by the invasive technique needed to obtain tissue from a sterile site, a procedure that may be risky in these frequently thrombocytopenic patients.

The polysaccharide 1,3-β-D-glucan (beta-glucan; BG) is a component of the cell wall of fungi that can be detected in the sera of patients with IFI. The detection of BG in serum was included as a mycological factor for probable IFI in the revised diagnostic criteria of the European Organization for Research and Treatment of Cancer/Mycoses Study Group (EORTC/MSG) in 2008 [11]. Despite this, no recommendations have been formulated regarding the optimal cut-off levels or the number of measurements required for a positive test, which was also commented in the last recommendations on the diagnosis of IFI from the European Conference on Infections in Leukemia (ECIL) [12].

Previous studies have examined systematic BG screening in high-risk patients in hematological settings [13–16]. However, these studies are quite diverse as to the characteristics of the patient populations and control groups, sampling protocols, types of fungi included, and definitions of true-positive IFI (i.e., inclusion or exclusion of possible IFI), and the results are not readily comparable. Furthermore, the cut-off level for the BG Glucatell/Fungitell® assay recommended by the manufacturer was established based on two studies mainly involving non-neutropenic patients [15, 17]. A meta-analysis of the performance of the BG assay for the diagnosis of IFI in six cohorts of patients with hematological malignancies revealed that two consecutive positive samples determined according to the manufacturer’s cut-off level had high specificity (99 %) but low sensitivity (50 %) [18].

Another issue that complicates the interpretation of BG findings in the hematologic setting is the occurrence of positive reactions of BG in the absence of IFI, referred to here as “confounding factors of BG reactivity”. This may be due to reaction of the assay with glucans derived from a different source than invasive fungal elements, such as plants or bacteria [19] or products contaminated with fungal elements. Factors that are commonly described as potential confounding factors of BG reactivity include: intravenous antibiotics [20, 21], immunoglobulin, albumin, and coagulation factors [22–26], possibly due to contamination with fungal elements or contamination with BG derived from cellulose-filters used in the manufacturing process; bacteremia [27, 28]; surgical gauzes [29, 30]; and hemodialysis using cellulose-containing dialyzer membranes [31–33]. However, other studies investigating positive reactions in the absence of IFI present contradictory results [34–37].

The aims of this retrospective cohort analysis were to evaluate the diagnostic performance and optimal cut-off level of the BG assay and potential confounding factors of BG reactivity in the serum of high-risk adult patients with hematologic malignancies and allogeneic hematopoietic stem cell transplant recipients.

Materials and methods

Study design, patient enrollment, and assessment of IFI

All adult patients with hematologic malignancies or undergoing hematopoietic allogeneic stem cell transplantation, admitted to the Department of Hematology at Sahlgrenska University Hospital, Göteborg, Sweden, during the period 2009–2011 were screened for IFI by serum BG testing once to twice a week as a clinical routine. We retrospectively enrolled those patients that had at least two consecutive BG assays performed during this period. The complete medical journals were reviewed by an infectious disease specialist (H.H.), and assessment of IFI was performed based on the revised 2008 EORTC/MSG criteria [11], modified to exclude BG as a mycological criterion and categorized as proven, probable, possible, or no IFI. The BG results were available to the reader at the time of IFI assessment. Patients with proven or probable IFI were considered true-positive cases of IFI. Patients with possible IFI were excluded from the subsequent analysis. The remaining patients were considered true-negative cases of IFI. True-positive and true-negative patients were used to assess diagnostic performance. A subgroup analysis was done to investigate potential confounding factors of BG reactivity amongst patients without IFI. This subgroup consisted of patients from the true-negative group as described above who had at least one BG value above the detection level of 50 pg/ml. Figure 1 gives an overview of the patient selection procedure.

Fig. 1
figure 1

Patient selection procedure. BG beta-glucan. *According to the 2008 European Organization for Research and Treatment of Cancer/Mycoses Study Group (EORTC/MSG) criteria modified to exclude BG as a mycological criterion

Full size image

BG test

The Glucatell assay kit was used for measuring the levels of BG in serum samples. The serum samples were immediately frozen at −20 °C at the laboratory of Clinical Microbiology at Sahlgrenska University Hospital. The analyses were then successively performed by two certified laboratory technicians who were blinded to the clinical data of the patients. The sera were pretreated with heat at 75 °C for 10 min and analyzed in duplicate according to the manufacturer’s instructions. The laboratory is accredited by the Swedish Board for Accreditation and Conformity Assessment (SWEDAC) in accordance with International Standard ISO 15189:2007. The test results were reported as numerical values between the lowest and highest levels of detection of 50 and 400 pg/ml, respectively.

Data collection and statistical analyses

Due to the non-systematic BG testing in this retrospective study, the time intervals of test sampling differed somewhat between the patients. To overcome this, a mean BG value for each patient was used for the statistical analyses of BG diagnostic performance. To calculate the mean values, we used: for the no IFI group, the values of all performed BG tests during the study period; for the IFI group, the values of all performed BG tests during a predefined time period of one week before and one week after fulfilment of EORTC/MSG criteria (1–3 samples), representing the period of IFI diagnosis. BG levels in patients with and without IFI were compared by the Mann–Whitney test. A receiver operating characteristic (ROC) curve, based on the mean BG values, was generated to assess the overall diagnostic performance using GraphPad Prism 5.01 software (GraphPad Software Inc.). The sensitivity, specificity, and positive and negative predictive values (PPV and NPV, respectively) at different cut-off levels for BG were calculated by ordinary two-by-two tables in order to find an optimal cut-off level for this BG assay. All mean BG values in the dataset of the ROC curve analysis were used as possible cut-off levels. The PPV and NPV at the optimal cut-off level obtained from our previous calculation were compared in scenarios of different IFI prevalence and in scenarios of one single BG value versus two consecutive BG values as the cut-off. BG samples with a maximum interval of 14 days were considered to be consecutive samples. To enable mathematical calculations, the highest and lowest detectable levels of the Glucatell assay kit at our laboratory were converted from <50 and >400 pg/ml to 50 and 450 pg/ml, respectively. In the subgroup analysis, the medical records of patients with no IFI but with at least one sample with detectable BG levels were analyzed and the presence or absence of specific variables, previously described as potential confounding factors of BG reactivity, were recorded at each BG sampling time. The statistical analysis was carried out in two steps. First, each variable was analyzed by bivariate analysis of variance including the mean BG level and study subject as factors. Those variables showing a significant difference in the mean BG level in this first calculation were then included in the second calculation, where multiway analysis of variance was used. A p-value <0.05 was considered statistically significant.

Results

Patients and BG levels

In total, 127 patients who had at least two consecutive BG assays performed during the study period were enrolled in the study. The demographic data and baseline diagnoses are shown in Table 1. Fungal prophylaxis was given as follows: fluconazole to all allogeneic stem cell transplant recipients during the first 3 months, and fluconazole or posaconazole to all other patients according to the doctor’s discretion. As shown in Table 2, 13 cases of proven or probable IFI, representing true-positive cases, were found. Eighteen cases of possible IFI were excluded from the subsequent analyses. The remaining 96 patients with no IFI were considered true-negative cases. Twelve out of the 13 true cases of IFI had a mean BG level considerably higher than the 80 pg/ml cut-off value recommended by the manufacturer at the time point of diagnosis, as well as during the 2 weeks following diagnosis. The single case with BG levels below the cut-off was a case of mucormycosis.

Table 1 Baseline characteristics of the study patients
Full size table
Table 2 Clinical data of patients with proven or probable IFI
Full size table

Reliability and optimal cut-off level for BG in the diagnosis of IFI

To evaluate the overall diagnostic performance and the optimal cut-off of BG in a hematologic population, we performed an ROC curve analysis and evaluated the sensitivity, specificity, PPV, and NPV at different cut-off levels for BG, using a mean BG value for each patient. As described above, these calculations were based on 13 true-positive and 96 true-negative patients, giving a prevalence of IFI in this study group of 12 %. The median value and interquartile range for BG was 273 (242–450) pg/ml in the IFI group and 51.8 (50–66.1) pg/ml in the no-IFI group. This gave a significant difference in median BG levels, p < 0.001 (Fig. 2a). As shown in Fig. 2b, the overall diagnostic performance of BG in this setting was very high, with an area under the ROC curve of 0.979 [95 % confidence interval (CI) = 0.944–1.0]. The sensitivity, specificity, PPV, and NPV with 95 % CI at some different cut-off levels for the BG test are presented in Table 3. The optimal cut-off level, with the corresponding highest sensitivity and specificity, was 158 pg/ml. The PPV and NPV at this cut-off level were 0.75 and 0.99, respectively.

Fig. 2
figure 2

Diagnostic performance of beta-glucan (BG). a Serum BG levels in patients with proven or probable invasive fungal infection (IFI) and without IFI. Each dot indicates the mean BG level for each patient and the horizontal lines represent the median BG value. The groups were compared using the Mann–Whitney test. b Receiver operating characteristic (ROC) curve analysis of serum BG levels for the diagnosis of IFI. A mean BG level for each patient was used in the analysis. The 2008 EORTC/MSG criteria, modified to exclude BG as a mycological criterion, were used to establish IFI diagnosis. The mean values of the BG assays performed in a 2-week window of positive diagnosis were used for the IFI group (n = 13). The mean values of all performed BG assays were used for the no IFI group (n = 96)

Full size image
Table 3 Sensitivity, specificity, and positive and negative predictive values for the diagnosis of invasive fungal infections at different cut-off levels of BG
Full size table

Predictive values at different IFI prevalences and with two consecutive BG samples

Since predictive values depend on the prevalence of the disease in question, we also wanted to assess the predictive values of the BG assay in scenarios with different possible IFI prevalence rates. Furthermore, it is recommended that two or more consecutive serum samples should be used to increase the diagnostic accuracy of the BG test [13, 15, 38]. We, therefore, also calculated the PPV and NPV comparing a single BG value above 158 pg/ml and two consecutive BG values above 158 pg/ml at different prevalence rates of IFI. Eight out of 109 patients did not have two consecutive BG tests performed during the defined IFI period and were excluded from this analysis. Table 4 shows the PPV and NPV in scenarios with different hypothetical IFI prevalences and when using different strategies of BG testing to define IFI. The PPV at the 12 % prevalence found in our study group did not differ markedly when using the mean BG value or two consecutive BG tests to define IFI (0.75 and 0.71, respectively). However, when using only one BG test to define IFI, the PPV was only 0.33. With two consecutive BG tests and at a hypothetical IFI prevalence of 20 %, the PPV was increased to 0.82, while it was decreased to 0.49 at a 5 % IFI prevalence. The NPV remained high (between 0.96 and 0.99), regardless of IFI prevalence or strategy for BG testing.

Table 4 Positive and negative predictive values at different hypothetical IFI prevalence rates and with different strategies for BG testing, based on the previously suggested optimal cut-off level of 158 pg/ml
Full size table

Confounding factors of BG reactivity

We conducted a subgroup analysis of 59 patients with no IFI but with at least one sample with BG above the lowest detection limit of 50 pg/ml. This resulted in 677 BG samples. The majority of the samples (93 %) had BG levels below the new suggested cut-off level of 158 pg/ml. The mean BG levels were significantly higher among patients with the following three variables present: admission to the intensive care unit (ICU); treatment with blood products, such as plasma, albumin, or coagulation factors; and treatment with the chemotherapeutic drug pegylated asparaginase (Table 5).

Table 5 Possible confounding factors of BG reactivity
Full size table

Discussion

In this study, all 13 patients with proven or probable IFI had BG values that were considerably higher than the 80 pg/ml cut-off level recommended by the manufacturer, except for the one case of mucormycosis, which is known to be associated with no or low levels of BG [39]. The lack of false-negative cases further supports the use of BG as a diagnostic or screening tool for IFI in the hematologic setting. The overall diagnostic performance for the BG test in the present study setting (presented as the area under the ROC curve) was very high, and, to our knowledge, the highest shown in any study, including the recent meta-analysis of Lamoth et al. [18]. Previous studies on the diagnostic performance of BG show diverging results, with sensitivities and specificities for the recommended 80 pg/ml cut-off level ranging from 60 to 100 % and 45 to 95 %, respectively [13, 14, 17, 40–43]. Differences in the characteristics of the patient populations and the control groups, sampling protocols, types of fungi included, and definitions of true-positive IFI, i.e., inclusion or exclusion of possible IFI, may explain the diverse results. Furthermore, the occurrence of environmental laboratory contamination as a possible reason for BG reactivity may also differ in different study settings. The low number of IFI cases and the fact that the clinicians and the reader of the reference standard were not blinded to the BG test result are limitations that must be considered when interpreting the results on the diagnostic performance of BG testing presented in this study.

Our results showed that, by raising the cut-off level of BG from 80 to 158 pg/ml, the specificity of the test was improved, without loss of sensitivity. At a prevalence of IFI of 12 % found in our study group, the PPV at the recommended cut-off level of 80 pg/ml was 0.52, whereas it increased to 0.75 when a cut-off level of 158 pg/ml was used. If using the BG test with a cut-off level of 158 pg/ml in a scenario of 20 % IFI prevalence, the PPV would increase to 0.85, while it would be only 0.54 in a setting with a low IFI prevalence of 5 %. The NPV was high at all cut-off levels and at the different putative prevalence rates of IFI. Previous studies evaluating diagnostic performance at similar cut-off levels of around 150–200 pg/ml in hematologic patients also show high specificities, but considerably lower sensitivities, ranging from 45 to 89 % [16, 44, 45].

It has previously been suggested that, when using BG for the diagnosis of IFI, more than one positive sample increases the diagnostic performance of the assay [13, 15, 38]. In analogy to the findings of Odabasi et al. [15], we found a significantly increased PPV and a remaining high NPV when using two consecutive BG levels rather than a single BG level as the cut-off. In contrast, Racil et al. [38] and Ellis et al. [13] showed only a slight increase in the PPV at a low cut-off level (60 pg/ml), but no increase in the PPV at a higher cut-off level (100 pg/ml) when comparing a single sample with two consecutive samples. With the strategy of using only one positive BG test as the cut-off, the PPV did not reach values above 0.48, despite a relatively high hypothetical IFI prevalence of 20 %.

False-positive BG tests are a well-known difficulty when evaluating BG findings. We found a significantly higher mean BG level in serum samples collected from patients with acute lymphoblastic leukemia who were treated with the chemotherapeutic drug pegylated asparaginase. To our knowledge, this has not been shown in any previous study. This might be explained by bilirubinemia or triglyceridemia, which are known side-effects of pegylated asparaginase treatment [46–48], and may interfere with BG reactivity [49]. In agreement with other studies [23, 26, 27], we also found that patients admitted to the ICU and patients that had recently received treatment with plasma, albumin, or coagulation factors had significantly higher BG levels than patients not admitted to the ICU and patients not having received such treatments. Several possible reasons for positive BG tests in patients admitted to the ICU may be proposed, one being the exposure to cellulose-containing filters in mechanical ventilators. However, further studies addressing this issue are warranted.

Similarly to other studies rejecting the previously proposed correlation between intravenously administered antibiotics and false-positive BG levels [20, 34], we did not find any significant correlation between high BG levels and treatment with any kind of intravenous antibiotics. This may be due to improved manufacturing of antibiotics, and, presumably, rejection of lots containing high levels of fungal cell wall components. Mennink-Kersten et al. [28] suggested that bacteremia gives rise to elevated BG levels, which has not been confirmed by others [36, 37], nor by us. Furthermore, although other studies have shown BG reactivity as a result of ongoing dialysis with cellulose filters [31–33], we found no such association.

To conclude, we found that BG testing had high overall diagnostic performance for the diagnosis of IFI in adult hematologic patients at high risk for infection if two consecutive serum samples and a higher cut-off level than that recommended by the manufacturer is used. A novel finding was that treatment with pegylated asparaginase gives rise to positive BG tests in the absence of IFI.