Introduction

In recent years, bibliometric indicators have been increasingly used to evaluate and manage research activities, provoking controversy and attempts to limit or regulate their application from both within and outside the field of Information Science & Library Science (IS&LS) (Bornmann & Marx, 2018; Leydesdorff et al., 2016; Petersohn & Heinze, 2018). Some high-profile examples include the Declaration on Research Assessment (DORA) and the Leiden Manifesto for Research Metrics (Hicks et al., 2015; San Francisco Declaration on Research Assessment [DORA], 2012).

Since it first emerged from a meeting of the American Society for Cell Biology in December 2012, the DORA declaration has been adopted by numerous organizations and institutions. Its main intent is to halt the use of impact factors and other citation-based metrics for individual researcher assessment. This general recommendation was backed up by a series of specific recommendations directed toward funding agencies, institutions, publishers, organizations that supply metrics, and researchers involved in the generation and dissemination of scientific knowledge. The overarching guideline is to consider the content of the documents and to be as explicit as possible with regard to the assessment criteria used.

As for the Leiden Manifesto, this was conceived during the Science and Technology Indicators Conference, celebrated in Leiden (Netherlands) on 3–5 September 2014. This document addressed the growing concern among experts in scientometrics, social scientists and research managers about the improper and increasingly generalized use of bibliometric indicators for evaluating research performance. The manifesto consists of a compendium of best practices, complementing some aspects dealt with in DORA that are particularly important in the area of Social Sciences and the Humanities. Specifically, it emphasizes the relevance of qualitative assessments in these areas of knowledge, publications in languages other than English, and research topics with a mainly local or regional impact.

At the same time, the publication of bibliometric studies in multidisciplinary and other journals outside the field of IS&LS has multiplied. This expansion outside the bounds of the core discipline reflects the wider interest in bibliometrics as a research methodology (Ellegaard, 2018). This phenomenon has been largely overlooked, although it could be associated with an improper or abusive use of bibliometric methodology or indicators, the performance of studies that do not contribute anything novel or original to the field, and the lack of even minimal reflection on the theoretical scaffolding that has been constructed around the discipline over the past century (Johnson, 2011).

One of the most prominent papers that have analyzed bibliometric publications outside the IS&LS field is the study by Jonkers and Derrick (2012), who identified 3852 publications in the Web of Science (WoS) from 1991 to 2010, using authors’ institutional affiliations to classify them as experts or non-experts in IS&LS and bibliometrics. Results showed exponential growth in published bibliometric studies by both experts and non-experts starting in 2004, although this growth was much more pronounced among non-expert teams. There was also slow and steady growth in contributions from mixed teams. Although the studies published by experts presented a slightly higher citation degree than those by non-experts, the difference was not statistically significant.

Similarly, Larivière (2012) analyzed bibliometric publications indexed in the WoS from 1972 to 2011, finding a dramatic surge in publications from the mid-2000s and a growing interest in metric studies in different fields, especially medicine but also natural sciences, particularly physics. As a result, the proportion of bibliometric studies published in IS&LS journals declined from 80% in the 1980s to 40% in 2008.

Ellegaard and Wallin (2015) identified 2854 bibliometric studies published from 1964 to 2013 on the WoS, in the areas of natural sciences, technical sciences, and health sciences. These were dichotomized based on their publication in journals included or not in the IS&LS category of the Journal Citation Reports and compared in terms of output and citation. Results were in line with those reported by Jonkers and Derrick (2012): the authors observed substantial growth in scientific production in both document categories starting in 2004, with a slightly higher number of mean citations in the IS&LS documents, although this was more pronounced for methodological or theoretical IS&LS studies. The analysis of the non-IS&LS documents showed that the greatest number of bibliometric studies outside the IS&LS categories were concentrated in the health sciences, although papers were also present in the other fields studied (applied sciences, multidisciplinary sciences, physical sciences, and life sciences). Additionally, the non-IS&LS documents were widely spread out among a large number of journals, and the citations they received were mainly intradisciplinary, with few references to IS&LS bibliometric studies.

Ellgaard (2018) identified 4215 documents published in the IS&LS category and 4637 in other fields of science in the WoS from 1964 to 2016. A detailed analysis of the citation patterns of the 200 most cited articles in both categories showed that IS&LS articles received more mean citations (13.7) than non-IS&LS articles (8.8). Moreover, there were important differences among the topic categories, with a widespread presence of bibliometric studies in the health sciences and in disciplines like business economics, engineering, management, and public administration, and relatively few studies in other fields. Finally, this study corroborated the low degree of cross-citations between IS&LS and non-IS&LS literature.

Published studies on bibliometric publications outside the IS&LS field have focused on comparative analyses of scientific production, characteristics of the cited literature, and cross-referencing between IS&LS and non-IS&LS literature. However, the rapid expansion of bibliometrics to all areas of knowledge production raises the need for an up-to-date vision of the features of bibliometric studies published outside the field, disaggregated by the field in which they were produced. Our study, moreover, incorporates a coauthorship analysis in order to study the positions and interactions established between IS&LS and non-IS&LS research communities, an aspect building on the preliminary analyses of Jonkers and Derick (2012). Finally, we attempt to advance some research lines proposed in previous studies. Namely, Jonkers and Derrick (2012) called for an in-depth content analysis of non-IS&LS literature and an assessment of IS&LS researchers’ participation in it, while Ellegaard and Wallin (2015) signaled the interest of considering individual types of analysis performed and moreover determined the extent to which an “integrative and interdisciplinary research approach” had been produced, as opposed to disciplinary dispersion. Indeed, Glänzel and Schoepflin (1994) already pointed out the relevance of this aspect decades ago.

The objective of the present study is to describe the characteristics of scientific output from researchers, from both within and outside the field of IS&LS, who use bibliometric methods.

Specifically, we aim to assess the differences between documents and authors linked to different areas of knowledge with regard to the following:

  • Evolution of the number of papers published, plus document type and length.

  • Authorship, degree of collaboration between authors, interaction between investigators in different areas, and position occupied in the social structure of the coauthorship network.

  • Participation of authors in more than one area of knowledge.

  • Citation of the authors according to their output and integration in the coauthorship network.

  • Reference to theoretical underpinnings of the discipline, the process of knowledge generation, and types of studies performed.

Methods

The methodological process proceeded as follows.

Identification of the dataset (documents under study)

To identify the studies employing bibliometric methods, our search strategy included all of the terms used by Jonkers and Derrick (2012), Ellegaard and Wallin (2015), and Ellegaard (2018). This strategy has two advantages: first, it makes use of all the core terms that precisely delineate the area analyzed; and second, it creates a dataset that is comparable to these previous studies, which are the principal references that have analyzed bibliometric publications outside the IS&LS field.

Search strategy: bibliometric* OR scientometric* OR webometric* OR altmetric* OR informetrics* OR “citation analysis” OR “citation study” OR “scholarly productivity” OR “publication analysis” OR “scholarly impact” OR “patent citation”.

Search field: Topic (title, abstract, key words, and key words plus).

Database: Web of Science Core Collection.

Period: 1964–2019.

Date of search: 20 July 2020.

Although initially no restrictions were imposed on document type, as is common in most bibliometric studies, the analysis included only articles, reviews, letters, and proceedings papers. In the Harvard Dataverse repository, the documents comprising the dataset on which the study was based were identified (https://doi.org/10.7910/DVN/NE8CXE).

Review of data homogeneity and quality, classification of documents, and assignment of authors to an area of knowledge

Once the records were retrieved from the search, the bibliographic information from the documents was downloaded. All the retrieved documents were categorized into one or more areas of knowledge, based on the WoS subject category assigned to the journal of publication. Each of these categories are linked to five broader branches of knowledge, and results are presented according to these:

  • Arts & Humanities.

  • Life Sciences & Biomedicine.

  • Physical Sciences.

  • Social Sciences.

  • Technology.

Given our study objectives, we considered it of interest to create a separate category, pooling IS&LS (the core discipline for bibliometric studies). The documents published in journals under categories linked to computer science but not IS&LS were assigned to the category “Technology”. The category for Multidisciplinary Science, which includes the documents published in journals like Science or Nature was also specifically differentiated in order to analyze the presence of bibliometrics in generalist scientific journals. This does not imply and should not introduce any confusion about the fact that these studies are based on multidisciplinary approaches; normally the papers published by these journals are based on disciplinary approaches (Stasa, 2020).

Table 9 shows the list of WoS categories and their correspondence to the branches or areas of knowledge described. For multi-assigned journals, we made a single assignment to each, albeit only 10.04% (n = 2014) of the documents analyzed were assigned to two or more areas of knowledge.

In relation to the authorships, authors’ names were reviewed to correct typos and identify homonyms (referring to two or more authors with the same signature) or variations of single authors’ names. We manually reviewed all the questionable author signatures, analyzing agreement for discipline, topic, institutional affiliations and coauthorship. For example, “Davis, B” refers to two different authors, Brennan Davis and Brian Davis, linked to different institutions and with different collaborators. On the other hand, “de Moya-Anegon, F” (36 signatures), “Moya-Anegon, F” (27 signatures), “Anegon, FD” (12 signatures), “De-Moya-Anegon, F” (4 signatures), and the typo “Moya-Anegon, M” all represent variations of a single author’s name: Félix de Moya-Anegón.

Calculation of indicators

Defining features of the “disciplinary dispersion” of bibliometric studies from outside the IS&LS area

We analyzed the characteristics of the documents published according to area of knowledge, based on the disciplinary classification of WoS journals. For journals with multiple assignments to disciplines linked to several areas of knowledge, we used a complete assignment to each area, as described. We considered the following aspects:

Analysis of scientific output and general characteristics of publications by area of knowledge

We obtained different, overall indicators by area of knowledge in order to analyze the dissemination and evolution of scientific output for bibliometric studies produced outside the IS&LS disciplines and to determine the general characteristics of these publications in terms of document type, length, and number of bibliographic references included.

The indicators obtained were:

  • N total documents published (and grouped by five-year periods).

  • N documents by document type.

  • Mean N pages.

  • Mean N references.

  • N authors.

  • Transience index (% authors with a single publication). The concept of “transience” refers to the authors who have only contributed to a single paper, and it is related to the distribution of authors of scientific publications according to their productivity, as popularized by Lotka (Pulgarin, 2012). Different studies have determined that the proportion of “transient” authors, that is, with a single publication, can reflect the state of development of the scientific area. Thus, determining the transience index is of interest in order to analyze the degree of consolidation among the bibliometrics research community, both in IS&LS and in other areas of knowledge under analysis (Patra & Mishra, 2006; Pulgarin, 2012).

  • N authorships.

  • Average N authors per document.

Document citations by area of knowledge

We determined the citation degree of the publications by areas of knowledge to obtain an overview of the impact generated by the bibliometric studies. The indicators obtained were:

  • N citations.

  • Mean citations per document.

  • % uncited documents.

Topics addressed and theoretical basis

We analyzed the topics addressed in the documents (publication output, authorship/collaboration, and citation) and the theoretical framework of the studies, as assessed by their reference to models, theories, or laws, thus deepening the line of analysis initiated by Derrick et al. (2012). To characterize the topic of the documents and establish the extent to which the research areas were theoretically grounded in bibliometric science, we determined the number of documents whose title, abstract, or key words referred to one or more of the three main topics covered by bibliometrics, namely the analysis of scientific output, authorship/collaboration, and citations. Documents that addressed more than one of these topics were multi-assigned. Similarly, we identified the documents that mentioned models, theories, or laws, as these are three essential elements in the process of knowledge generation and the theoretical basis of any discipline. Multi-assignment was likewise applied to papers that addressed more than one of these aspects. The indicators obtained were:

  • N documents that analyze aspects related to publication output.

  • N documents referring to authorship/collaboration.

  • N documents that analyze citation.

  • N documents that mention laws.

  • N documents that refer to theories.

  • N documents that consider models as part of the analysis undertaken.

Analysis of the coauthorship network, determination of the degree of specialization in the academic community and of the interdisciplinary approach of the research

In order to obtain a comprehensive vision of the structure of the field, the position occupied by researchers, and the relationships established between them—particularly regarding the links between the authors’s of different areas of knowledge (interdisciplinary links)—we generated a coauthorship network. Given the high transience index observed, the network was limited to the authors who published at least two documents in the dataset analyzed.

The indicators obtained were:

  • N components.

  • N and % of authors in the giant component and in other components.

  • N multidisciplinary components.

  • N interdisciplinary links.

  • Average yearly citation per document of authors in the giant component and in other components. In order to determine the influence of being part of the giant component on the citation degree received, we calculated the average citations received by the authors according to the number of documents published and the date of their publication.

In the terminology associated with graph theory, which forms the basis for social network analysis, a component is the set of nodes or vertices connected through ties or arcs, either directly or through intermediaries (graph), but which make up a sub-graph differentiated and disconnected from the rest of the network components. The “giant component” or “largest component” is the component that brings together the highest number of authors, as there is a link connecting each pair of points in this graph. The network also encompasses a number k of disconnected graphs (G), constituting components of authors that do not have collaborative ties to authors in the giant component or in other components (Scott, 2000; Wasserman & Faust, 1994). The study of components and the size of the giant component in the scientific coauthorship networks enables a characterization of the connectivity and degree of integration and cohesion among the research community of a given scientific discipline. The analysis also elucidates the research community’s stage of development in terms of the extent to which it resembles a small world network (González-Alcaide et al., 2015; Liu & Xia, 2015).

To analyze the extent of specialization of the researchers and the interdisciplinary collaboration links established between different areas of knowledge, each author was assigned to a single area of knowledge, following a method similar to that used in the study by Abramo et al. (2018), which identified and analyzed multidisciplinary teams. We considered researchers’ main area of knowledge to be where the plurality of their scientific activity was concentrated, that is, the area in which they published the greatest number of documents. In that regard, we used the WoS assignment of subject areas as a reference. If an author had participated equally in more than one area, they were considered a “multi-assigned” researcher. In the network generated, we color-coded the nodes corresponding to each author according to the assigned area of knowledge. We analyzed the distribution of the most productive authors (> 9 documents) according to the degree of specialization (concentration of all or most of their documents in a single area of knowledge) or of diversification of their scientific production between different areas of knowledge, following the terminology used by Abramo et al. (2018).

For each component identified, we determined whether all authors were assigned to a single area of knowledge. Otherwise, the component was considered as having an interdisciplinary nature (“multidisciplinary components”). To determine the scope of the interdisciplinary collaboration links established, we calculated the number of links established between authors assigned to different areas of knowledge in relation to the total number of existing links, both in the case of the giant component and of the rest of the components in the network. Pajek software for network analysis was used to generate the network and analyze the links established between areas of knowledge and researchers.

Results

Defining features of the “disciplinary dispersion” of bibliometric studies from outside the IS&LS area

Scientific output and characteristics of documents published, by knowledge area

The search strategy yielded 23,028 documents, of which 20,064 were included in the analysis. These comprise 14,993 articles, 546 documents classified as both articles and proceedings papers, 2435 proceedings papers, 1775 reviews, and 315 letters (Table 1).

Table 1 Documents published on bibliometrics, distributed by area of knowledge and document type
Full size table

The distribution of the documents by areas of knowledge (Table 1) shows that a plurality (39.19%) of papers originate within the IS&LS area. However, the proportion of papers from social sciences (26.76%) and life sciences & biomedicine (23.17%) is also very noteworthy. Although articles are the primary medium for research dissemination in all areas of knowledge, reviews also had a prominent presence in the life sciences and biomedicine as well as in technology. Proceedings papers were also observed frequently in the area of technology.

Regarding other document types, these were not included in the analysis of indicators; however, we did observe a high number of editorials in the area of Life Sciences & Biomedicine and Social Sciences, along with a relevant role for books in the areas of IS&LS and Social Sciences, and for meeting abstracts in Life Sciences & Biomedicine.

The evolution of the number of documents published by five-year period (Fig. 1) illustrates the growing relevance of bibliometrics as a research field, with a boom in these types of studies starting in the 2000s. Indeed, the last 15 years have seen a 12-fold increase in the number of documents, from 863 in the 2000–2004 period to 10,155 in 2015–2019. Moreover, this growth is not restricted to papers in the IS&LS area. In the last five years of the study period, about 3,300 documents were published in journals specializing in IS&LS, but there were also around 3000 documents each in journals in both Social Sciences and Life Sciences & Biomedicine. Papers in the area of Technology, Multidisciplinary Science, and the Arts & Humanities also showed marked growth, which is particularly noteworthy in the latter area, as its scope is traditionally far removed from bibliometrics (Table 10).

Fig. 1
figure 1

Evolution of bibliometric literature in different areas of knowledge

Full size image

Remarkably, in 2019 the proportion of bibliometric papers published in journals in the Life Science & Biomedicine (32.69%) and Social Sciences (31.09%, excluding the IS&LS category) exceeded that from journals in IS&LS (25.27%).

With regard to the length of the documents and the number of bibliographic references contained therein (Table 2), there were important differences between areas. The papers published in Arts & Humanities and Social Sciences are much longer than those in Life Sciences & Biomedicine and in Multidisciplinary Science. As for the references, papers in Social Sciences, Physical Sciences and Technology have the highest average number of references per document, while IS&LS has the lowest value.

Table 2 Length and number of references in publications on bibliometrics
Full size table

The analysis of these two variables, disaggregated by document type, logically show that reviews and articles were longer and used more references compared to letters and proceedings papers (Table 11).

Table 3 shows the characteristics of authorships in the research communities analyzed. There were important differences with regard to output and collaboration between knowledge areas. The proportion of transient authors was 73.44% in the IS&LS area but exceeded 83% in Social Sciences and Life Sciences & Biomedicine. These values gradually ascended even further in other areas, reaching 94.40% in the case of Arts & Humanities. The highest average authors per document was in Life Science & Biomedicine (4.10), while much more modest values were observed in IS&LS (2.41) and Social Sciences (2.68). These two areas, together with Arts & Humanities (1.94), showed the lowest levels of average authors per document.

Table 3 Size, transience index, and collaboration in the research community responsible for documents published on bibliometrics
Full size table

Document citations by area of knowledge

The documents published in Multidisciplinary Science journals present the highest mean citations per document (22.66), followed by journals in IS&LS (16.44) and Social Sciences (15.22). The rest of the areas analyzed have much lower citation values, particularly Arts & Humanities. Life Sciences & Biomedicine is the area with the smallest proportion of uncited publications (14.93%), followed by Multidisciplinary Science (17.46%) and IS&LS (18.21%). The remaining areas show notably higher values (26% to 37%) on this indicator (Table 4). Table 12 shows the distribution of the citation per document by knowledge area and document types.

Table 4 Citation degree of published documents on bibliometrics, by knowledge area
Full size table

Topics addressed and theoretical basis

With regard to the topics that the documents address (Table 5), analysis of citations is the most important subject studied in all knowledge areas. Publication output attracts the most interest among Life Science & Biomedicine authors, while authorship/collaboration receives the most attention from researchers in IS&LS and Multidisciplinary Science. Generally, the concepts and theories that underpin knowledge generation in the field are only modestly represented, albeit with significant variations between areas: 6.61% to 23.49% of the documents discuss models; 2.39% to 10.67%, theories; and 0.99% to 3.29%, laws. Life Science & Biomedicine is the area with the fewest publications mentioning theories, and a low proportion discusses laws and models. On the other hand, and despite its modest contributions to the field of bibliometrics, Arts & Humanities stands out for its good coverage of laws and theories, a phenomenon that also extends to the area of Social Sciences. Finally, although Technology and Physical Science are the areas concentrating the most publications on models, this topic is also well covered by IS&LS and Social Sciences.

Table 5 Topics addressed and theoretical basis for knowledge generated in documents on bibliometrics
Full size table

Analysis of the coauthorship network, determination of the degree of specialization in the academic community and of the interdisciplinary approach of the research

The generation of the coauthorship network shows a high degree of atomization, as only 15.95% of the authors who have published more than one document are found in the giant component. This value is 31.67% if all authors are considered.

The giant component (Fig. 2, n = 1282 authors) is comprised mainly by investigators from the IS&LS area (66.46%). Other knowledge areas have less representation within the giant component: 15.45% of the authors are from Life Science & Biomedicine; 7.56% are from Technology; 5.77%, from Social Sciences; and 0.08% to 1.01%, from the other areas considered (Table 6).

Fig. 2
figure 2

Giant component of the coauthorship network for scientific publications on bibliometrics. Nodes denote authors, and the lines between them, coauthorship links (thicker lines = more coauthorships). Turquoise: Information Science & Library Science; yellow: Social Sciences; green: Life Sciences & Biomedicine; dark blue: Physical Sciences; red: Technology; pink: Multidisciplinary Science; white: Arts & Humanities; brown: author assigned to more than one area

Full size image
Table 6 Investigator participation in the giant component, by knowledge area
Full size table

Regarding the remaining 1230 components, the main finding is that most are made up of authors from a single knowledge area. Interdisciplinary links are present in just 24.8% (n = 305) of them and in only 26 components with more than 10 authors. There are also numerous components comprised of a large number of authors from Life Sciences and Biomedicine (71 components with 5–38 authors). Table 13 shows the distribution of the components by knowledge area. The quantification of interdisciplinary links illustrates the scant cooperative practices between authors belonging to different areas of knowledge. These account for just 7.28% (n = 1365) of the total links established (n = 17,394) among the authors of the giant component, an aspect that can be corroborated visually in the coauthorship network. The constituent authors that have established interdisciplinary links generally occupy peripheral positions in the network and isolated connections with other authors. The interdisciplinary links among the authors in the other components present an even lower value, representing just 5.87% (n = 1901) of the total collaboration links established (n = 30,482).

Authors in IS&LS and Life Sciences & Biomedicine tend to produce research in their own knowledge area. Among the top producers, 82% (n = 175) of those in IS&LS, and 77% (n = 54) in Life Sciences & Biomedicine, have published over 70% of their papers in their own area. On the other hand, the most productive authors in Social Sciences participate in documents from other areas much more frequently (Table 7).

Table 7 Concentration of scientific output among top producers (> 9 documents) in the main area of knowledge in which they participated
Full size table

Regarding the citation of authors according to output levels (Table 8), generally the authors within the giant component present a much higher citation degree than those outside it. The only exception is the top producers (> 9 documents) in Life Sciences & Biomedicine, whose citation degree within the giant component (1.99 citations per document) is notably lower than in top producers outside it (3.09). In addition, the Social Sciences authors and the top producers in IS&LS present the highest citation degrees. However, the rest of the IS&LS authors (2–5 documents and 6–9 documents) actually show a lower citation degree than authors from other areas.

Table 8 Average yearly citation per document stratified by research output and participation in the giant component
Full size table

Discussion

The results of the present study confirm the growing and generalized interest in bibliometrics as a research methodology to be used across all areas of knowledge, albeit with important differences in output levels between them. Bibliometric studies are particularly strong in areas like Life Sciences & Biomedicine, where these methods have figured prominently for some time (Ellegaard, 2018; Ellegaard & Wallin, 2015; Larivière, 2012). However, bibliometric publications are also emerging in areas that do not have traditional links with this methodology, for example the Arts and Humanities (Hammarfelt, 2016; Zuccala, 2016). Several factors may contribute to this boom, which started becoming apparent in the mid 2000s. For one, bibliometrics has been increasingly integrated into researcher assessment and research activity management. It has also become easier to access a large volume of bibliometric data, and a myriad of bibliometric indicators has been made available through the principal providers of scientific literature, university rankings, and initiatives monitoring the scientific output of research institutions. Methodological advances in the field, including the development of software for analyzing and visualizing bibliometric data, together with the spreading of “metric-wiseness” (Rousseau & Rousseau, 2017) could also contribute to the diffusion of bibliometrics to all areas of knowledge (Egghe, 2005; Zuccala, 2016). The prominence of bibliometric studies published in multidisciplinary journals, combined with the concentration of the greatest citation degree in this area, reflects the general interest in bibliometrics and the relevance that the discipline has acquired for the wider research community. This trend has been favored by the transversality of some of the topics covered by bibliometric studies, like the analysis of the characteristics of scientific communication processes or the funding and evaluation of research. Furthermore, the papers that have analyzed the role played by multidisciplinary journals generally agree in highlighting that they favor the dissemination of the studies to a wider audience and facilitate the translation of these ideas to different disciplines. This tendency may also have contributed to the disciplinary dispersion of bibliometrics observed in the present study (Ackerson & Chapman, 2003; Stasa, 2020).

However, the bibliometric research growth has emerged in a widely dispersed and spontaneous way, with the emergence of different, autonomous research components, characterized by scant intellectual or personal relationships with the mainstream IS&LS researchers who laid the theoretical foundation for the discipline (Hood & Wilsom, 2001; Patra et al., 2006; Yang et al., 2017). The high transience indexes observed outside the area of IS&LS, particularly in areas like Arts and Humanities, respond without a doubt to the fact that bibliometrics has only been recently incorporated as a research method and there is not yet a specialized scientific community dedicated to performing this type of study, as also noted in relation to other research areas (Patra & Mishra, 2006; Pulgarin, 2012). However, in other areas showing the persistent existence of a high number of relatively unproductive authors and high transience indexes, despite the fact that researchers have been performing bibliometric studies for decades, it is worth asking why there is no evolution toward the consolidation of a specialized research community or a greater integration of researchers from different areas.

Previous studies that have analyzed the citation practices of bibliometric studies outside the IS&LS field have highlighted the weak intellectual links with bibliometric knowledge produced within other areas of knowledge. In particular, authors observed few citations among papers from different disciplines and especially to bibliometric studies published in IS&LS journals (Ellegaard, 2018; Ellegaard & Wallin, 2015). Our results enable a more in-depth examination of this aspect: documents published outside of IS&LS journals follow their own pattern of development, more in line with those seen in their own field. For example, the collaboration indexes in Life Sciences & Biomedicine are much higher—and in the Arts & Humanities much lower—than those in IS&LS (Glänzel, 2002). Likewise, the length of the papers and the mean bibliographic references cited therein vary by the area of knowledge in which they were published. The longest papers are from journals specializing in Arts & Humanities and Social Sciences while the highest average number of bibliographic references per document has been observed in Social Sciences, in consonance with the conventions in these fields (Haustein et al., 2015).

The analysis of coauthorships shows the high level of atomization and independence observed among the community of non-IS&LS bibliometricians. The presence of non-IS&LS authors was limited in the giant component, dominated by IS&LS authors and with the sporadic involvement of interdisciplinary groups linked to more than one area of knowledge. This practice could be further promoted or encouraged in light of the benefits that interdisciplinary authorship has in terms of fostering the visibility, creativity, adoption of new approaches, and dissemination of theoretical and methodological principles from the area (Chang, 2018). The giant component emerges when the number of collaborative links increases to the point where there is a critical mass of connectivity, in which a large number of authors are tied together through collaborations. The existence of a giant component is one of the features of a coauthorship network that has been used to characterize the evolution of research communities in different areas of knowledge. In the mature phase of development (“normal science”) and in the presence of a giant component, research communities are characterized by their cognitive and social unity along with prominent cooperative activity that facilitates cohesion and the flow of knowledge between community members. However, the size of the giant component observed in the present study (32% of the authors) is far below that seen in other areas considered to be “healthy,” for example the 84% seen by Kumar and Markscheffel (2016). Thus, the coauthorship network for scientific publications on bibliometrics seems to be atomized and dispersed and cannot be defined as a small world network (Bettencourt et al., 2009; Liu & Xia, 2015).

In addition, the notion of “isolated researchers” or “isolated components,” as laid out in graph theory and used in social network analysis, is associated with certain research gaps (Yu et al., 2020), especially in cases such as ours. Indeed, isolated authors and groups are ubiquitous in the research community under study, with numerous peripheral, fragmented, and dispersed lines of research, characterized by the absence of interactions with the research community that is tackling the main research questions of the discipline (Savic et al., 2014; Yu et al., 2020). Here, this isolation carries negative connotations associated with “traditional” research: old-fashioned methods and subject matter; stagnated bodies of knowledge; lack of innovation and reflection; spend little time on research; and one-sided, repetitive, simplified, or self-focused research (Ochsner et al., 2013).

Based on the literature and our results, it seems that IS&LS and non-IS&LS researchers performing bibliometric studies work autonomously, as if their research were not anchored to a common theoretical and methodological basis. This split undoubtedly constitutes an important barrier that limits progress in knowledge generation. One consequence may be the disregard for the extensive theoretical foundation undergirding the discipline, as evidenced by the scant mention of models, theories, and laws, especially in papers from Life Sciences & Medicine. That said, our results show a slightly greater consideration for these foundational concepts than reported by Derrick, Jonkers & Lewison (2012), especially in the Social Sciences and Arts & Humanities. Those authors found that just 0.27% of non-IS&LS publications published from 2004 to 2010 referred to bibliometric theory or laws.

This finding can be understood in the context of Schubert’s (2002) study, which pointed to the lack of evidence indicating an increase in the hardness of the field. Different factors may explain this situation, for instance, the absence of standards in the discipline (Bornmann & Marx, 2018; Glänzel, 1996; Rousseau, 2002) or the appeal of the field to academic opportunists who adopt a few basic methods or notions in order to quickly generate publications, to the detriment of the theoretical, interpretative, and critical foundation on which any bibliometric study should be based. The concept of “opportunism” refers to the consistent attitude of taking maximum advantage of the circumstances for the purpose of obtaining the maximum possible benefit. This concept is used in fields like biology to explain the behavior of certain species or in the business setting in reference to taking advantage of business opportunities. However, it is associated with negative connotations when people put personal interests or profits above desirable principles like integrity or formalized or socially accepted ethical norms. The studies that have analyzed the explanatory factors and the contexts associated with opportunistic human behaviors have highlighted that in addition to personal features like integrity or moral values, opportunistic behaviors are strongly conditioned by the permissibility of groups or organizations, and they are favored in contexts in which it is extremely difficult to be consistent in one’s behaviors. This may be because of the absence of external controls and the incapacity to identify and penalize inappropriate behaviors, generating a sensation of impunity in which opportunistic behaviors fail to produce enough adverse consequences in comparison with the potential benefits that they may reap (Strange, 2008; Wathne & Heide, 2000). In the academic sphere, Major (1998) analyzed the figure of the professor seen through the lens of fictional literature. The author highlighted the existing duality between their consideration as admirable figures without any ambition for power (whether economic, personal or political), comparing these to the disdainful professors who use their position to seek personal advantage, coining the term “academic opportunists.”

With regard to scientific publications and bibliometric studies, Strange’s (2008) association between opportunistic behavior and abuses in authorship is notable. Abramo et al. (2011) also used the concept of opportunistic behavior to define potential alterations in behavior that some researchers made to comply with the guidelines established for evaluation processes. More recently, Abramo et al. (2019) analyzed gift authorship practices in Italian universities as one form of opportunistic behavior. Moreover, Zagonari (2019) refers to tactical or opportunistic citation and publication behaviors to describe questionable behaviors by authors and editors; these can be identified using different bibliometric indicators that analyze researchers’ publication records.

The science historian López Piñero (1972) argued decades ago that “the use of bibliometric methods offers enormous possibilities for self-promotion in those who, with a handful of statistical recipes, try to save themselves the work of assimilating all the advances made up to now in history, philosophy, sociology, economics and other sciences”. These possibilities are amplified by the ease of access to bibliometric indicators or computer programs that enable simple and speedy processing of bibliographic data and the generation of a diverse array of maps and other representations.

Several papers have pointed to the potential harm associated with the expansion of “bad bibliometrics,” (Aguillo, 2015) which is characterized by the publication of superficial studies with an unclear approach, objective, or critical interpretation of the results (Johnson, 2011; Wallin, 2005) and that may respond more to opportunistic behaviors in order to gain favor in performance evaluations or academic promotions. These shortcomings can be found both within and outside the IS&LS field. In that sense, Derrick (2012) reported that “Analysis of a field or topic” had a larger weight among non-LIS research papers (25.7% of the documents from 2004 to 2010, compared to just 14.1% among LIS documents). In contrast, we observed an increased weight in the category of “publication output” in IS&LS (which encompasses the category mentioned by Derrick), reaching 19.3%. This finding could respond to the dissemination of bibliometric methods among researchers linked to other topic areas in the IS&LS area or to opportunistic behaviors.

The improvement in the quality of research, along with the publication of studies that meaningfully contribute to advancing knowledge, are essential pillars sustaining the development and maturity of a scientific discipline. Egghe (2005) highlights the importance of publishing high-quality bibliometric papers, that is, those “that present good mathematical (probabilistic) models and explanations of informetric regularities (in the broad sense) and/or papers in which interesting and important data gathering is presented”.

Our results show the widespread use of bibliometric methods for research in all areas of knowledge. The resulting studies reflect the development patterns and dynamics of the research communities from which they emerged, or which “appropriated” the methods for their own use. Future studies should explore these patterns with more granular detail, analyzing these authors’ fields of specialization and the role that bibliometrics plays in their body of work, particularly among the most productive non-IS&LS authors who have established themselves as bibliometric references in their own areas of knowledge. Another potential line of research would be the visibility of bibliometric studies in non-IS&LS and multidisciplinary journals, and the value that these studies add to the area of knowledge in which they were published.

Although analyses like ours, based on the subject area categorization of the scientific journals, are one of the most widely used approaches for studying the phenomenon of interdisciplinarity, this is a complex topic, both from the conceptual point of view and in relation with the different units of analysis and methodological focuses used for its study (Sugimoto & Weingart, 2015). In that sense, the present study has focused on analyzing interdisciplinarity from the perspective of disciplinary diversity (integration and relationships established between authors linked to different areas of knowledge), following the conceptual approach proposed by Rafols and Meyer (2010). Future studies could study network coherence, another of the conceptual approaches proposed by those authors in relation to the study of interdisciplinarity, analyzing the position that the researchers occupy in the coauthorship network and the structures that characterize it (Rafols & Meyer, 2010).

Finally, it would be of great interest to see qualitative analyses of the quality of individual studies and the fulfillment of ethical criteria governing publishing processes, for instance with regard to aspects like avoiding salami or redundant publications, or meeting authorship criteria, among others.

In light of this study’s results and of our personal experience as author, reviewer, and editor who regularly participate in publication processes for bibliometric studies (both in IS&LS and non-IS&LS journals), it appears that bibliometric research has reached a point of no return. That is, it is no longer possible to exercise any control, at least from within the IS&LS area, of methodological rigor or of which lines of research are most important to elevate. For the sake of the discipline, however, several actions may be beneficial:

  • Embrace a pedagogical role, both through the production of manuals or theoretical books, which may be more acceptable and useful to researchers in the Social Sciences or Arts & Humanities, and in the form of editorials or training articles in journals dedicated to natural sciences.

  • Raise awareness among non-IS&LS and multidisciplinary journal editors who do not specialize in the publication of bibliometric studies that these papers should conform to the standards expected of any other research work: originality, innovation, advancement of knowledge, and methodological rigor. Bibliometrics has always had a social and applied component that should not be neglected. Although the science is based on quantitative indicators, these are meaningful only to the extent that they can be applied in practice or shed light on the underlying social phenomena that give rise to them.

  • Foster interdisciplinary collaborations and research groups made up of both IS&LS and non-IS&LS investigators and the participation of IS&LS experts in multidisciplinary journals or journals from other areas of knowledge. Such an endeavor may be realized through diverse types of initiatives, for example the celebration of scientific conferences with cross-cutting themes, the publication of bibliometric journals or specific journal sections outside the scope of IS&LS but including IS&LS specialists.

  • Promote a new declaration or manifesto to capture the essential elements of any study based on bibliometric methods. Indeed, the great impact that DORA and the Leiden manifesto have had should stimulate a similar statement supporting the quality of bibliometric research.