Introduction

Judah Folkman, father of angiogenesis research

The field of angiogenesis research began in the late 1960s with an attempt to delineate the role of neovascularization in tumor growth (Folkman et al. 1963). Over the past 50 years, angiogenesis research has become a very active and broad field of biomedicine. This new field of biology was initiated and pioneered by the American medical scientist Judah Folkman (1933–2008), who is unanimously considered the father of angiogenesis (Ribatti 2008). Over the years, Folkman developed new and ground-breaking concepts and methodologies for the study of angiogenesis, and his laboratory trained many scientists who have become leaders in the field. Folkman introduced the concept of “anti-angiogenesis” as a potential novel anticancer therapy and his discovery is considered a paradigm shift in cancer therapy (Cao and Langer 2008). More importantly, angiogenesis inhibition therapy is the focus of a worldwide scientific research effort and a priority of the National Cancer Institute. As a result, more than 1.2 million patients worldwide are now receiving antiangiogenic therapy (Bischoff and Griffioen 2008).

Dr. Folkman received more than 150 awards and honors from 11 nations for his distinguished research. He was also elected to the National Academy of Sciences, to the Institute of Medicine and appointed to the President’s Cancer Advisory Board (Klagsbrun and Moses 2008). Yet, during the first 20 years of his career, his ideas were met with a lot of skepticism, criticism and rejection (Folkman 2008a). Folkman liked to reflect on the resistance he faced from his pairs by saying: “If your idea succeeds, everybody says you’re persistent, if it doesn’t succeed you’re stubborn” (Bikfalvi 2016).

Delayed recognition in scientific literature

The early rejection of Folkman’s ideas by the scientific community leads to the hypothesis that his work suffered from “delayed recognition”. Delayed recognition is a phenomenon where papers do not achieve recognition in terms of citations until a few years after their original publication (Garfield 1989a, b, 1990; Glänzel et al. 2003; Van Calster 2012). Associated analyses also refer to terms such as “resisted discoveries” (Barber 1961), “premature discoveries” (Stent 1972), “late-bloomers” (Merton 1988), “Mendel syndrome” (Costas et al. 2011). In today’s scientometrics literature, such papers are generally called a “Sleeping Beauty” (SB) (Van Raan 2004), a publication that goes unnoticed for a long time, and then, almost suddenly, is awakened by a “Prince” (PR), attracting a lot of attention from there on in terms of citations.

The quantitative criteria that scientometrics offers for studying delayed recognition can be very useful in understanding the dynamics of scientific change. However, any analysis of this kind must be examined critically by qualitative approaches such as historical or sociological analyses.

Quantitative criteria for studying delayed recognition can be summarized as being of three kinds: average-based criteria, quartile-based criteria and parameter-free criteria. One example of average-based criteria is van Raan’s (2004). Van Raan defined a SB as an article that goes unnoticed (‘sleep’) for long periods of time before almost suddenly receiving a lot of attention (‘is awakened by a prince’). He defined three variables for SB (a) depth of sleep (cs), that is, average citations per year received in the sleeping period since publication, with cs ≤ 1 standing for deep sleep and 1 < cs ≤ 2 for less deep sleep; (b) length of sleep (ns), that is, duration of sleeping period, often lasting between 5 and 10 years; and (c) awake intensity (Cw), amounting for instance to 20 citations per year over a period of 4 years.

The quartile-based criterion is a measure presented by Costas et al. (2010). They described three categories of publications by identifying the “Year 50%” as the year in which an article received at least 50% of its citations for the first time. Delayed documents, which include SBs, are papers that have not received 50% of their citations when 75% of other documents in their fields have already received 50% of their citations (“Year 50%” > “P75”). Apart from delayed documents, they identified “flashes in the pan” (“Year 50%” < “P25”), which are publications that are quickly but briefly well cited, and publications that followed a more common citation life (“P25” ≤ “Year 50%” ≤ “P75”).

A parameter-free index was proposed by Ke et al. (2015): “The Beauty coefficient” (B). B quantifies the extent to which a paper could be considered a SB by adding up differentials between the citation curve of the publication and a reference line calculated between the year of publication and year of maximum citations. Applying their criteria to their database, Ke et al. found that the top 1000 SBs in their database correspond to papers with B ≥ 317.93.

SBs are relatively rare (< 0.1%) (Ke et al. 2015) but they have been identified in numerous medical or research fields such as physics (Van Raan 2004), chemistry (Ke et al. 2015), ophthalmology (Ohba and Nakao 2012), paediatrics (Završnik et al. 2016), psychology (Ho and Hartley 2017), or radiology (Gorry and Ragouet 2016), and popularized among the wider scientific community (Cressey 2015). The reasons for SB pattern of citations may be linked to paradigm shift in the research field (Van Raan 2004) or, social recognition through Nobel Prize, for example (Du and Wu 2016). However, the explanations for sleeping beauties are under explored.

In line with the fairy tale, a “Sleeping Beauty” must be awoken by a Prince. The prince (PR) is usually the author of the first citing paper. However, Du and Wu (2016) propose that candidates for PR paper should fulfill additional criteria such as: (1) the PR paper should be published at the time when the SB began to be highly cited; (2) the PR paper should be a highly cited paper itself; and (3) PR paper should be co-cited with the SB.

Aim

Judah Folkman initiated the concept of anti-angiogenesis in the 1970s as a way to control vascular growth. His work paved the way to the first clinically used angiogenesis inhibitor in breast cancer (Bischoff and Griffioen 2008). Despite the success of this breakthrough therapy and the academic peers’ recognition, science historians as well researchers in the angiogenesis field and Folkman himself, reported that the hypothesis on anti-angiogenesis suffered skepticism for over 20 years (Bikfalvi 2016; Folkman 2008a; Ribatti 2008).

Therefore, the present work aims to explore whether Judah Folkman’s scientific work and his hypothesis on tumor angiogenesis suffered from resistance to discovery (Barber 1961) through scientometrics analysis, and whether the author was right in feeling scientific resistance to his hypothesis (Bikfalvi 2016).

Resistance to discovery could conduct to delayed recognition of papers which do not achieve recognition in terms of citations until a few years after their original publication. Thus, we analyzed Folkman’s scientific production and the citation life of all of his publications looking for SBs, specifically. Then, we attempted to identify the relevant PRs, and tried to understand the reason for delayed recognition and awakening mechanisms.

Methods

To collect the publications of Judah Folkman, we used the Scopus® database, and a collection of 510 publications were extracted with metadata. Citation data for Folkman’s publications were also extracted from the Scopus® database, and a corpus of 116,703 citations was harvested through 31 December 2015. Descriptive bibliometric analyses were conducted using the built-in functions of Scopus and exported in CSV format to Excel, which was employed for further statistics and calculation. Analysis of co-authorship relations between researchers was conducted using a network visualization tool VOSviewer® (http://www.vosviewer.com) (van Eck and Waltman 2010).

Using Ke et al.’s criteria (2015), the “Beauty coefficient” (B) was calculated for all of Folkman’s papers in order to identify top SBs. B quantifies the extent to which a paper may be considered a SB by adding up differentials between the citation curve of the publication and a reference line calculated between the year of publication and the year of maximum citations.

Following Costas et al. (2011), we also used the quartile-based criteria. We calculated the “Year 50%” for all articles of the same year of publication, organized them in ascending order in terms of their percentiles, and recorded those falling on percentiles 25 as “P25” and those falling on percentiles 75 as “P75”.

CitedReferencesExplorer (Thor et al. 2016) was used to conduct a Reference Publication Year Spectroscopy (RPYS) in order to identify the PR paper. The numbers of co-citations between the SB paper and each candidate PR paper were calculated using Gephi® an open-source network analysis software package (https://gephi.org/). Each citing article is represented by a unique Scopus identifier (EID). The number of co-citations is equal to the number of citation duplicates detected by Gephi®.

Finally, all the bibliometrics analyses were complemented by a historical approach of the Folkman biography and the history of angiogenesis using sources with criticism.

Results

Judah Folkman’s scientific production

Folkman published his first paper in 1953 while he was enrolled at Harvard Medical School in 1957, on a method of obtaining abdominal hemostasis. After graduation from medical school, he became a surgical resident at Massachusetts General Hospital. His residency was interrupted when he enlisted in the US National Naval Medical Center in Bethesda, Maryland from 1960 to 1962 where he conducted research on artificial blood substitutes that could be stored for a long time. He then returned to complete his residency at Massachusetts General Hospital. In 1967, he was appointed Surgeon-in-Chief at the Children’s Hospital, and the year after that Professor of Pediatric Surgery at Harvard Medical School, at the unprecedented age of 36. He stepped down from that position in 1981 in order to work exclusively on angiogenesis research.

During the 41 years that he worked at the Boston Children’s Hospital, he was the author of 396 articles, 41 reviews, and 76 publications of other forms (conference proceedings, editorial, letters, chapters and book) (Fig. 1, boxes). Throughout his scientific career, he published in various disciplines (multidisciplinary, medicine, surgery, oncology, pathology, biochemistry) in more than 160 different journals (26 articles in P.N.A.S, 16 in Science, 14 in Nature, etc.) (Gorry and El Aichouchi 2017). He also collaborated with a large number of researchers (more than 150) during his long and prolific career. VOSviewer software was used to draw a co-publication graph to explore Folkman’s scientific collaborations (Gorry and El Aichouchi 2017): his main collaborators were his fellows or technicians, Yueng Shing, Evelyn Flynn, Robert D’Amato, and Michael O’Reilly (23, 20, 19, and 18 co-publications respectively). Regarding his main research interest, he published his first paper on tumor growth in 1963, and used the word “angiogenesis” for the first time in two publications in 1971. In 2015, his work was cited more than 116,700 times (Fig. 1, black dashed line).

Fig. 1
figure 1

Folkman’s publications and citations. Blue box, number of Folkman’s publications by year; black box, publication year of Folkman’s landmark paper; black line, dotted line, cumulative citations for all Folkman’s publications by year. (Color figure online)

Full size image

A single sleeping beauty

The calculation of the “Beauty coefficient” for all Folkman’s papers revealed the existence of one extreme case of delayed recognition, with a high B coefficient B = 1052.17 compared to the rest of the papers (the second most delayed paper has a B = 175.04) (Fig. 2). It is also the paper with the highest citation count: 6279 citations in 2015 (Table 1). This paper entitled “Tumor Angiogenesis: Therapeutic implications” is a review which has been published in 1971 (Fig. 1; black box), in a top journal (99th percentile rank): the New England Journal of Medicine (Folkman 1971).

Fig. 2
figure 2

Citations and B index of Folkman’s publications through time. The diameter of each circle is proportional to its corresponding B index

Full size image
Table 1 Top 10 Folkman delayed papers ranked by decreasing B index
Full size table

This landmark paper has averaged 139.53 citations per annum, amounting to 6279 citations up to the 31st December 2015. During its first 23 years, however, the paper was cited only 113 times (Fig. 3; red line). This suddenly rose to 18 citations in 1995, 33 citations in 1996, and 65 citations in 1997. Since 1995, the paper averaged 293.6 citations per annum. This clearly exemplifies a sleeping beauty, although it does not fit exactly into Van Raan’s definition. With its 4.7 citations per year during the sleeping period, the paper qualifies as a “light sleep” (Van Raan 2004; Gorry and Ragouet 2016); it slept for 23 years instead of five to ten. Also, the average number of citations over the 4 years following its awakening was 51 per annum, which is a very high awake intensity by Van Raan’s definition (minimum of 20 citations per year for a 4-year period). In addition, using Costas et al.’s criteria (2010), Year 50% = 39 (which corresponds to the year 2009), while P75 = 34.5. This means that Folkman’s paper is classified as a paper with delayed recognition.

Fig. 3
figure 3

Citation history of Folkman’s Sleeping Beauty and Prince Papers, and angiogenesis publication trends. In color, number of citations per year for Sleeping Beauty (red); PR Paper #1 (blue); PR Paper #2 (green); PR Paper #3 (orange); Black dashed line, number of publications on angiogenesis per year; black dotted line, reference line lt; dotted dashed line, distance dt maximizing the awakening time; vertical line, awakening time. (Color figure online)

Full size image

Identifying the Prince

In order to determine the PR paper, the year of the awakening was calculated using Ke et al.’s criterion (2015) and was revealed to be ta = 1995. In addition, a Reference Publication Year Spectroscopy analysis was applied to the dataset of the citing references of Folkman’s landmark paper from 1972 to 2000. The plot revealed two notable peaks: the first peak is situated in 1971, which is the publication year of Folkman’s landmark paper. The second peak is situated in 1995, the awakening year of the SB (Fig. 4, red line). In order to take into account the delays between submission dates and publication dates (Li 2014), a 3-year window around 1995 was chosen in which the top publications citing Folkman’s main paper were identified. The PR paper was likely to have been published in a top rank journal, to be among the first highly cited citing articles, and to share the largest number of co-citations with the SB paper (Du and Wu 2016). In order to rank the PR candidate papers, we propose to rescale respectively the citation and co-citation numbers between 1 and 10 by calculating Pcit(i) and Pcocit(i), which are respectively the citation and co-citation rankings of a PR candidate paper i using the equations below:

$$P_{\text{cit}} \left( i \right) = 9\left( {\frac{{x\left( i \right) - x_{ {\rm min} } }}{{x_{ {\rm max} } - x_{ {\rm min} } }}} \right) + 1$$
(1)
$$P_{\text{cocit}} \left( i \right) = 9\left( {\frac{{y\left( i \right) - y_{ {\rm min} } }}{{y_{ {\rm max} } - y_{ {\rm min} } }}} \right) + 1$$
(2)

where xmin and xmax are respectively the smallest and largest citation numbers in the list of PR candidate papers, ymin and ymax are respectively the smallest and largest co-citation numbers in the list of PR candidate papers, and x(i) and y(i) respectively the citation and co-citation numbers of a PR candidate paper i.

Fig. 4
figure 4

Reference Publication Year Spectroscopy applied to the dataset of the citing references of Folkman’s SB. Red line: number of cited references; Blue line, deviation from the 5-year median. (Color figure online)

Full size image

Using Eqs. (1) and (2), a unique ranking is calculated, the P coefficient:

$$P = P_{\text{cit}} \times P_{\text{cocit}} .$$

The ranking of the PR candidates enabled us to distinguish three main papers that stand out in terms of their number of citations and their number of co-citations (Table 2). The first-ranked paper (PR paper #1) is an article published by Judah Folkman as corresponding author in October 1994 in Cell (O’Reilly et al. 1994) (Fig. 3, blue line). It is the most cited article citing Folkman’s paper and the most co-cited paper with the SB in the list. The second article (PR paper #2) is a review published in December 1995, authored by Folkman, and published in the New England Journal of Medicine (Folkman 1995) (Fig. 3, green line). The third article (PR paper #3) was published in February 1995 in Nature Medicine by Lars Holmgren, Michael O’Reilly, and Judah Folkman (Holmgren et al. 1995) (Fig. 3, orange line). It is noteworthy that this article was submitted and accepted by Nature Medicine in late 1994, and was already cited by O’Reilly and colleagues in PR paper #1 as a “submitted paper” (O’Reilly et al. 1994). We must emphasize that the three PR candidates are all self-citations by Folkman himself.

Table 2 Top candidate Prince Papers citing Folkman’s Sleeping Beauty around the awaking year
Full size table

In addition to their high co-citation numbers with the SB (Table 2), the three papers have high co-citation numbers with each other. Indeed, PR paper #1 was co-cited 420 times with PR paper #2 and 237 times with PR paper #3. Moreover, PR paper #2 was co-cited 121 times with PR paper #3.

Beside the citation of Folkman’s SB by the three PR papers, the awakening can be attributed to the discovery of his pioneering work by a whole research community. It is worthy of note that the awakening citations of Folkman’s paper matched the annual rate of publications entitled or indexed (abstract, keywords) for the keyword “angiogenesis” (Fig. 3, dotted line).

Discussion

Delayed recognition of the angiogenesis-dependent tumor growth hypothesis

Despite the fact that Judah Folkman is recognized today as the father of angiogenesis research, his landmark paper suffered from delayed recognition. Folkman’s SB first presented the hypothesis that “tumor growth is angiogenesis-dependent” (Folkman 1971). In this rather theoretical article, Folkman showed preliminary evidence that tumors could not enlarge beyond a microscopic size of 1–2 mm3 without recruiting new capillary blood vessels. In the same article, Folkman introduced the term “antiangiogenesis” to mean the prevention of new vessel sprouts from being recruited by a tumor. Folkman also reported the isolation of a tumor angiogenesis factor (TAF) and speculated that antiangiogenesis may provide a form of cancer therapy by producing an antibody against TAF.

Many skeptics challenged the hypothesis that tumor growth was angiogenesis-dependent (Kerbel 2000), and many scientists argued that the search for an angiogenesis inhibitor was a “fruitless exercise” (Folkman 2008b). This was attributed to the lack of bioassays for angiogenesis, the inability to culture endothelial cells in vitro, and the absence of molecules that regulate angiogenesis (Folkman 2008a). However, the 1980s witnessed the discovery of the first molecules that did mediate angiogenesis. New proangiogenic molecules such as acidic and basic fibroblast growth factor (Shing et al. 1984) and vascular endothelial growth factor (Ferrara and Henzel 1989) were isolated from tumors. In addition, the first angiogenesis inhibitors were discovered: low concentrations of interferon α in vitro were found to specifically suppress migration of endothelial cells in vitro (Brouty-Boye and Zetter 1980). By 1988, daily low dose interferon α was successfully used to treat cancer in a teenager dying of progressive pulmonary hemangiomatosis of both lungs. This was the first recorded anti-angiogenic therapy. All these experimental and clinical advances throughout the 80s were not sufficient to trigger the awakening of Folkman’s paper. It was not until the mid-nineties that it received the attention and recognition it deserved.

Alternatively, scientific controversy might explain delayed recognition of the SB (Gorry and Ragouet 2016), and Folkman’s landmark paper might have suffered from controversies surrounding the financing of his lab and its partnership with Monsanto during that sleeping period (Hess 2006). However, the controversy born of the difficulties of replicating Folkman’s results by other laboratories in the mid-nineties did not affect the awakening of his landmark paper by the PR papers. Angiogenesis research started to spread to many laboratories and became a burgeoning field with hundreds of papers per year (Fig. 3, Black dashed line).

The Prince’s kiss of life

Since the late 1970s, Folkman’s laboratory conducted a long-term effort to prove the existence of angiogenesis inhibitors. This effort was fruitful in that the laboratory reported eleven molecules between 1980 and 2005 (Folkman 2008a).

In 1991, Michael O’Reilly came to Folkman’s laboratory as a postdoctoral fellow (Folkman 1996). His mission was to attempt to uncover the mechanism of suppression of metastatic growth by a primary tumor. Noel Bouck and colleagues had recently reported that the emergence of tumor angiogenesis was the result of a shift in balance between positive and negative regulators of angiogenesis in a tumor (Rastinejad et al. 1989). This led Folkman to suggest that a primary tumor might suppress growth of its distant metastases by releasing an angiogenesis inhibitor into the circulation. Indeed, the discovery of Angiostatin came as a result of an attempt to test this hypothesis.

The discovery and publication of Angiostatin in 1994 resulted in a fundamental evolution in the field of angiogenesis (Soff 2000). The scientific community finally appreciated the real importance and relevance of endogenous angiogenesis inhibitors (Folkman 2004), and consequently recognized the major importance of Folkman’s (1971) founding paper. While PR paper #1 uncovered an important link between tumor angiogenesis and metastasis, PR paper #2 proposed a new hypothesis to explain tumor dormancy by making the link between angiogenesis and another “hallmark of cancer”: apoptosis. Indeed, PR paper #2 suggested for the first time that angiogenesis inhibitors control metastatic growth by indirectly increasing apoptosis in tumor cells. In light of the recent experimental breakthroughs, PR paper #3, was the occasion to review and discuss the clinical relevance of angiogenesis research in general.

Ke et al.’s criterion for identifying the awakening time (Ke et al. 2015) and Reference Publication Year Spectroscopy both pointed to 1995 as an awakening year for Folkman’s SB. This means that the PR paper was published a priori in 1995. However, taking into account publication delays by considering a 3-year awakening period revealed O’Reilly et al.’s (1994) paper on Angiostatin to be the main PR paper, a result supported by our historical analysis. Nevertheless, the awakening process of Folkman’s SB involved three successive and highly related PR papers authored or coauthored by one PR: Judah Folkman himself.

Conclusion

This paper has attempted to apply methodology of citation analysis to explore whether Judah Folkman’s scientific work suffered from delayed recognition, and whether the author was right in feeling scientific resistance to his tumor angiogenesis hypothesis (Bikfalvi 2016). It has shown that Folkman’s landmark paper is indeed a SB based on the calculation of the B coefficient and according to Costas’ criteria. Although it does not fulfill Van Raan’s SB publication criteria, this is due to the arbitrary thresholds on the sleeping period and on awakening intensity (Gorry and Ragouet 2016; Li and Fred 2016). However, the idea that papers with delayed recognition show the highest impact in their fields is supported by our case study (Costas et al. 2010).

However, Folkman’s SB paper is categorized as a “review” paper, and there is no consensus so far as to the inclusion or exclusion of this type of document in citation analysis. Although the definition of a review might vary across fields, journals and time, it usually presents an overview of recent research advances and highlights results inconsistencies. Following Kuhn’s epistemology of science (Kuhn 1970), it is reasonable to say that Folkman’s review hypothesizing that “tumor growth was angiogenesis-dependent” led to a profound paradigm shift in cancer research. Indeed, traditional strategies for treating cancer focused mainly on targeting cancer cells. Thanks to Folkman’s (1971) hypothesis, it is now accepted that the microvascular endothelial cell recruited by the tumor is a new target (Cao and Langer 2008).

On the other hand, this paradigm shift was the result of a long and slow process of accumulation of knowledge and evidence. This process took place during the sleeping period of Folkman’s SB. Since its publication in 1971, Folkman’s landmark paper slept for more than 20 years until the discovery of Angiostatin in 1994. This period witnessed, among others, the development of bioassays for angiogenesis, the discovery of angiogenic molecules, and the establishment of relationships between angiogenesis and other processes (e.g. metastasis). This accumulation of knowledge and experimental evidences weakened the early skepticism and encouraged more investigators to work on tumor angiogenesis and develop anti-angiogenic drugs. It also led researchers in many other fields beyond cancer biology and ophthalmology to study the angiogenic process (Folkman 1996).

At the scientometric level, the identification of the PR(s) and the relevant PR papers is a difficult task. The awakening of a SB is a complex phenomenon. Singling out the first citing paper does not capture this complexity, especially in a case like Folkman’s SB paper. In line with existing literature (Hartley and Ho 2017; Ohba and Nakao 2012; Teixeira et al. 2017) multiple PR papers for a single SB should be considered. We suggest to choose a 3-year window around the awakening year as an appropriate period for studying the awakening of the SB. Also, a new coefficient “P” that ranks the PR paper candidates based on their citation scores and their co-citations with the SB was introduced. However, the role and impact of citations as well as co-citations in the awakening process is not fully understood yet, and remains a challenging question for future research.

Interestingly, the fact that Judah Folkman was one of the co-authors of the three identified PR papers exemplifies the self-awakening phenomenon (Ohba and Nakao 2012; Teixeira et al. 2017), and challenges the practice of excluding self-citations when conducting such bibliometric analyses. If self-citation is believed to sustain self-promotion, it may be justified by the cumulative nature of science (Popper 1959), when authors refer to previous hypotheses, methods or results. The fact that Folkman kept citing his own paper after all those years (n = 15 before the awakening year 1994) demonstrates his phenomenal persistence and belief in the importance of his theory. It should certainly not be seen as a means of artificially inflating citation rates in order to strengthen his position in the community (Glänzel 2008). During the sleeping period, the NIH turned down Folkman’s grant proposal, and he was able to continue his research program by securing funding with private companies (Hess 2006). Constancy and continuity in a research field are important components that ensure development of new research subject areas (de Solla Price and Gürsey 1976; LaBonte 2014). In contrast, lack of continuous financing and research could hamper the development and growth of a new research area.

It is true that Judah Folkman was already a recognized researcher during the awakening of his SB in the mid-nineties. However, delayed recognition in terms of citations can touch any researcher’s body of work, which raises questions about the relevance of short-term citation-based metrics for the evaluation of research impact.

Finally, if scientometric analysis is key to identify the occurrence and awakening of SBs, qualitative approaches such as historical and sociological analyses play an important role in challenging and validating scientometric results.