Intraoperative radiotherapy (IORT) is widely used in all continents to deliver precise radiotherapy at the time of cancer surgery [1]. Initial pioneering experiences in Japan and the USA were documented in the late 1970s and since then, a significant body of knowledge on methodology, result analysis in terms of cancer site and development projects has become available in the medical literature [2–4].

Systematic literature reviews of IORT studies have documented feasibility, tolerability and high local control promotion using IORT as a boost component of escalated radiotherapy; including similar specific results for primary rectal, recurrent rectal and pancreatic cancer [5–7]. Overviews and editorials on IORT practice and results have identified technological diversity among the procedures performed (electron beams, kilovoltage X-rays and brachytherapy with isotopes) and consistent methodology in terms of reporting clinical results emphasizing local control and toxicity [1]. Expert IORT institutions have demonstrated particular commitment to evaluating clinical outcomes observed in cancer location- or type-oriented IORT-containing treatment approaches [8]. Nevertheless, evaluating scientific quality is a difficult problem to which there is no standard solution. Prior to publication, experts in a specific field estimate the importance and relevance of scientific articles (peer review; [9]). Alternative methods (statistical and mathematical data), such as citation rates and journal impact factors, measure the quantity and performance (i.e. quality) of published science, as well as the connectivity between scientific fields, research departments or authors [10].

In terms of the bibliographic metrics found in medical databases during a recent period, this paper focuses on an extensive analysis of the available literature reporting IORT data. This information provides authors and editorial journals with a reflection of the preferences and policies relating to the transfer of scientific knowledge generated in studies with IORT as a cancer treatment modality.

Materials and methods

We reviewed the publication pattern of scientific journal articles and analyzed the bibliometry of IORT-related papers published in PubMed during a certain time period. As a quantity indicator, we used the number of articles published in PubMed between 1997 and 2013. As a quality indicator, we used the 2012 Science Citation Index Impact Factor in order to evaluate the impact of the research on the scientific community. Furthermore, we stratified the identified publications by study type, scientific topic reported, tumor site, journal specialty and the year of publication.

Statistical analysis

We performed a one-way analysis of variance (ANOVA) to determine any significant differences between the means of the analyzed groups. In order to determine which specific group was significantly different from the others we performed a post-hoc test (Tukey’s test).

Results

Between 1997 and 2012, a total of 972 journal papers from 534 different first authors (281 different institutions from 32 countries) were published in the PubMed database (mean journal impact factor of 2.73, standard deviation ± 2.62).

Study type

The majority of articles identified are retrospective cohort reports (n = 622, 64 %), followed by review articles (n = 204, 21 %; p < 0.001). Only 6 % (n = 55) of the reported articles are phase II-III trials. The highest mean impact factor was related to prospective publications (p < 0.001; ◉ Fig. 1a).

Fig. 1
figure 1

Publication quantity (total rate; solid bars) and quality (journal Impact factor; black line) stratified by a study type and b journal specialty

Full size image

Journal specialty

From a total of 207 journals, IORT articles were significantly more frequently reported in surgery (n = 399, 41 %) and radiotherapy journals (n = 273, 28 %; p < 0.01). The highest impact factor was achieved by radiotherapy and medical oncology journals (p < 0.01; ◉ Fig. 1b).

Primary scientific topic

Clinical cancer outcome reports represented the most frequently published primary topic (n = 661, 68 %; p < 0.001). Clinical reports and technology were associated with the highest mean impact factor (p < 0.01; ◉ Fig. 2a).

Fig. 2
figure 2

Publication quantity (total rate; solid bars) and quality (journal impact factor; black line) stratified by a primary topic and b tumor site

Full size image

Tumor sites

Gastrointestinal tumor reports were most frequent (n = 456, 47 %; p < 0.001). The mean superior impact factor was related to breast and gynecologic cancer publications (p < 0.01; ◉ Fig. 2b). Of the 28 tumor sites reported, the most commonly mentioned sites were breast (n = 191, 19.7 %) pancreatic (n = 183, 18.8 %) and colorectal cancer (n = 179, 18.4 %).

Year of publication

The year of highest productivity was 1997 (n = 102, 11 %; p < 0.001) and the year with the highest mean impact factor was 2008 (p < 0.001; ◉ Fig. 3).

Fig. 3
figure 3

Publication quantity (total rate; solid bars) and quality (journal impact factor; black line) stratified by year of publication

Full size image

Discussion

Used either alone or in combination, specialized radiotherapy techniques are developed to improve the therapeutic index of conventional cancer therapies [11]. Brachytherapy, radiosurgery, intensity-modulated radiotherapy (IMRT) and image-guided radiotherapy (IGRT) are all examples of radiation technology-driven developments aimed at assuring high cancer local control rates while minimizing toxicity [12].

Bibliometrics is a set of methods (statistical and mathematical) generally used to measure and analyze the quantity and quality of articles and publications [10]. Quantity indicators measure the productivity of a particular researcher: in the present analysis, we have used the simplest method available to asses this (i.e. the number of articles published in PubMed). Although this count is a very straightforward indicator that can be easily calculated, one must be very careful when using it to compare different groups. While the number of publications does reflect productivity, it does not address the quality of the articles [10]. By analyzing the article methodology, this limitation can potentially be overcome (the impact of a case report is obviously not equivalent to that of a multicenter randomized controlled clinical trial). Additionally, the number of publications is also influenced by cohort size [10]. Beyond productivity, the level of quality of the publication can be identified by using performance indicators [10]. In the present analysis we used the Journal Citation Index Impact Factor as a surrogate measure of quality.

Bibliometrics of highly specialized radiotherapy techniques have never been reported; either because they have evolved into the development of a specific journal devoted to this particular scientific area (brachytherapy or radiosurgery), or because the information was spread due to its multispecialty nature (IORT). The evaluation of metrics for cancer treatment techniques is relevant to understanding the patterns of medical practice result reporting, as well as to evaluating the sensitivity of the peer review system within the broad spectrum of oncology journals.

The main findings of the present analysis of metrics in IORT publications during the period 1997–2013 are: the limited availability of phase III trials specific to IORT [2]; the balanced reporting of IORT data in surgical- and radiation oncology-based journals; the progressive incremental increase in impact factor over time and the increasing tendency to report data from early-stage breast cancer studies [13, 14]. Additionally, the fact that IORT is a cancer treatment modality able to contribute with competitive results for a variety of cancer sites and histological types is extensively documented; with most reports published in the first quartile of the specific specialty journals. Due to the retrospective nature of IORT clinical reports, it is identified that the relevance of editorial interest in the published data is related to the potency of the analysis performed (number of patients and long-term follow-up maturation; [10]).

Pooled analysis strategies for interinstitutional data sharing and reporting have found a particularly positive status in terms of the quality of their metrics [11, 14]. This effect on interinstitutional pooled analyses has recently been confirmed in soft tissue and renal cancer models [12, 16, 17].

Nonrandomized surgical and radiotherapeutic clinical cancer control results have an intrinsic limitation to the attainment of metric publication standards. This fact affects the discrimination of knowledge. Additionally, local cancer therapies have methodological and intrinsic restrictions in terms of generating hypotheses to be tested in randomized oncology trials. This fact contributes to the marginal success of designed and completed randomized trials for IMRT/IGRT, brachytherapy and radiosurgery.

Designed IORT trials employing dose-escalation strategies and using radiotherapy precision techniques have indicated that improved cancer control with equiv­alent normal tissue toxicity is a realistic scenario [2]. Reports are consistent with improved promotion of local control in pancreatic [18], gastric [19] and gastroesophageal cancer [20], as well as in locally advanced [21, 22] and locally recurrent rectal cancer [23]; while decreased normal tissue toxicity (severe enteritis) has been proven in the retroperitoneal sarcoma model [24].

New hypotheses have also been tested with IORT in studies searching for noninferiority or equivalence of outcomes using single-fraction IORT compared to conventional radiotherapy in early stage breast cancer [13, 14]. Results are questionable regardless of the technology employed (electrons or kilovoltage X-rays).

Finally, of relevant potential for the understanding and reporting of IORT contributions to combined cancer management is the analysis of risk factors associated with local relapse and identification of the effect of adverse features such as close or positive surgical margins, multiple involved surgical specimens, extensive nodal involvement, radioresistant disease, etc. The cancer models including this kind of analysis in IORT-containing radiosurgical strategies are oligorecurrence [17, 23–26] and resectable pancreatic cancer [18]. Technical reports of IORT implementation involving physical science developments (e.g. in vivo dosimetry or quality assurance; [3]), treatment planning modified for surgical features and radiation equipment design are scarce in the medical literature and, with occasional exceptions, published in the latter quartiles of impact-factor rated specialty journals [27].

Conclusion

We identified a consistent and sustained scientific productivity of international IORT expert groups. Journals with surgical and radiooncological content showed the dominant publication frequency [28–30].The highest impact factor was achieved by medical oncology-based journals. Quantity and quality indicators could potentially help researchers to select the journals to which they submit their manuscripts and organizations to make decisions regarding appointments, promotions and funding of particular research, individual researchers or a research group. Moreover, they could help to determine the geographic origin of research and detect growth of a country’s scientific impact. These indicators could also help to measure research focus, detect misallocation of resources, predict research success and orient research funding to optimize support of research policy.