Retraction Note to: Support Care Cancer (2017) 25:1279–1289

https://doi.org/10.1007/s00520-016-3522-6

The Editor-in-Chief has retracted this article [1]. Following publication, concerns were raised with respect to data and data analyses present in the article. Some of the concerns were voiced in a Letter to Editor [2] and subsequently further concerns were received by the journal. Upon post-publication review of the raw data provided by the authors, the reviewer was unable to replicate the findings and conclusions of the article. The Editor-in-Chief and the article’s original Handling Editor agreed that the data reported in this article are therefore unreliable and retraction of the article is appropriate.

A summary of the re-analyses of the data can be found below.

The authors do not agree to this retraction.

Background

The study by Hansen et al., was a small randomized trial comparing bismuth to placebo tablets for the prevention of toxicity in hematological patients receiving chemotherapy (final reported n = 50). The two subgroups of patients enrolled were multiple myeloma (n = 26) and lymphoma (n = 24). The toxicities of interest consisted of stomatitis, infection, diarrhea, febrile neutropenia (FN) and cytopenia. The investigators concluded that in patients with bismuth prophylaxis, there was a reduced incidence and duration of grade 2 stomatitis, infections, duration of cytopenia and the incidence of FN. In patients with lymphoma, bismuth significantly reduced the duration of diarrhea. Gender related differences were also reported in myeloma patients with respect to the severity of stomatitis, the incidence of febrile neutropenia, infection as well as the duration of cytopenia. Gender related differences were also reported in the incidence of stomatitis in lymphoma patients.

Data Reanalysis

The Journal contacted the authors of the paper to request the original data; the authors complied and provided electronic copies and print copies where now electronic version was available. An independent biostatistician was asked to assess the article and the data provided, and to consider the following:

  • The appropriateness of the analytical approach used by the authors, and presented in the article.

  • To analyze the data using the authors’ approach to see if, in fact, it was done and reported correctly.

  • To analyze the data using an alternate approach if the methodology (design, implementation and analysis) by the authors was inappropriate.

Data, in various formats (i.e. Excel files, scan documents) was provided to the Journal. The independent biostatistician then attempted to rebuild the database for the reanalysis. The total sample size was 59 patients, but according to the Hansen et al., publication, 9 patients were excluded for various reasons. A request to the authors was made for the list of patients who were excluded but resulted in only 8 patients. Therefore, the final dataset for reanalysis consisted of 51 patients. Within the database and publication, there were no definitions of study endpoints for infection and FN. Therefore, as assumption was made that an infectious event or a FN event occurred in cases where the grade was ≥ 2. For the time endpoints (i.e. days of stomatitis, days of diarrhea, days infection), some were reported in minutes instead of days, with the range being 2 to 13 min. There are 1440 min in one day. Using 1440 as a denominator, the reported cases in minutes would be <  = 0.09 days. In those cases where minutes were reported to be less than 0.1, a 0 was entered in the new database for that event. With respect to the statistical analysis of the new dataset, essentially the same tests were used as in the Hansen publication (i.e. Fisher’s exact test for categorical data, t-test for continuous data and the Mann–Whitney test for non-parametric data).

Results of Reanalysis

The findings of the reanalysis suggested several discrepancies. For instance, it was stated in the paper that there were 33 male and 17 female patients. However, in the current database, the number of male and female patients were 32 and 19 respectively.

Figure 1 of the paper (Stomatitis in myeloma patients): Upon assessment of stomatitis grade in myeloma patients, there was only a single grade 2 event. Therefore, it was not possible to replicate any of the myeloma outcomes in Figure 1 of the paper, such incidence of grade 2 stomatitis, duration of stomatitis, severity of stomatitis, as well as gender differences. The results could not be replicated because there were not enough events.

Figure 2 of the paper (Febrile Neutropenia in myeloma patients): The independent biostatistician was not able to replicate any of the results in Figure 2 of the paper. As an illustration, it was stated in the paper that the incidence of febrile neutropenia in myeloma patients was 40% and 23% in the placebo and bismuth groups respectively (p = 0.0005). However, the calculated difference from the new data was 33.3% vs. 14.3% (p = 0.25). In addition, there were no significant gender differences in the incidence of FN. The mean days of FN were also numerically difference between the recalculated and original reported results.

Figure 3 of the paper (Infections in myeloma patients): Upon reanalysis, there were no significant differences between groups in the incidence of infections, the grade of infections and the duration of infections. As an illustration, the paper states: “In six out of ten (60%) placebo-treated myeloma patients, infection occurred after high-dose melphalan treatment. As shown in Fig. 3a, bismuth could significantly decrease (two tailed p = 0.0001) the incidence of infections in myeloma patients, since 5 out of 13 (38%) receiving bismuth displayed infection.” However, upon reanalysis, the incidence of infection in the placebo and bismuth groups were 58.3% and 42.9% (p = 0.43). There were also no gender differences in infection rate in myeloma patients between placebo and bismuth.

Figure 4 of the paper (Duration of cytopenia’s in myeloma patients): The independent biostatistician was not able to replicate any of the duration of cytopenia data presented in Figure 4 of the paper. Nor were there any differences within gender between bismuth and placebo.

Figure 5 of the paper (Duration of diarrhea in lymphoma patients): The difference in mean diarrhea duration was significant and shorter in the bismuth group. However, the independent biostatistician was not able to replicate the mean values. The reported medians in the bismuth group was also difference than what was calculated.

Evaluation of Gender Differences in Lymphoma Patients: The independent biostatistician was not able to identify any significant gender differences in grade 2 stomatitis, infections or febrile neutropenia between the bismuth and placebo groups. This contrasts with what was reported in the paper.

Conclusions

Upon reanalysis of the data provided by Hansen et al., almost all of study findings could not be replicated. Indeed, most of the key study endpoints (stomatitis, infection, diarrhea, febrile neutropenia and cytopenia) were not statistically different between patients randomized to receive bismuth and placebo.

References

[1] Hansen, P.B., Penkowa, M. Bismuth adjuvant ameliorates adverse effects of high-dose chemotherapy in patients with multiple myeloma and malignant lymphoma undergoing autologous stem cell transplantation: a randomised, double-blind, prospective pilot study. Support Care Cancer 25, 1279–1289 (2017). https://doi.org/10.1007/s00520-016-3522-6

[2] Dalgaard, P., Ekstrøm, C.T. & Kronborg, D. Bismuth adjuvant and adverse effects of chemotherapy: issues with statistical analyses. Support Care Cancer 26, 1685–1686 (2018). https://doi.org/10.1007/s00520-016-3522-6