Abstract
The Bibliometric-enhanced Information Retrieval workshop series (BIR) was launched at ECIR in 2014 [19] and it was held at ECIR each year since then. This year we organize the 10th iteration of BIR. The workshop series at ECIR and JCDL/SIGIR tackles issues related to academic search, at the crossroads between Information Retrieval, Natural Language Processing and Bibliometrics. In this overview paper, we summarize the past workshops, present the workshop topics for 2020 and reflect on some future steps for this workshop series.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Motivation and Relevance to ECIR
Searching for scientific information is a long-lived user need. In the early 1960s, Salton was already striving to enhance information retrieval by including clues inferred from bibliographic citations [23]. The development of citation indexes pioneered by Garfield [11] proved determinant for such a research endeavour at the crossroads between the nascent fields of BibliometricsFootnote 1 and Information Retrieval (IR) — BIR. The pioneers who established these fields in Information Science — such as Salton and Garfield — were followed by scientists who specialised in one of these [30], leading to the two loosely connected fields we know of today.
The purpose of the BIR workshop series founded in 2014 is to tighten up the link between IR and Bibliometrics [20]. We strive to get the ‘retrievalists’ and ‘citationists’ [30] active in both academia and the industry together, who are developing search engines and recommender systems such as ArnetMiner, Dimensions, Google Scholar, Microsoft Academic Search, and Semantic Scholar, just to name a few.
These bibliometric-enhanced IR systems must deal with the multifaceted nature of scientific information by searching for or recommending academic papers, patents, venues (i.e., conferences or journals), authors, experts (e.g., peer reviewers), references (to be cited to support an argument), and datasets. The underlying models harness relevance signals from keywords provided by authors, topics extracted from the full-texts, co-authorship networks, citation networks, and various classifications schemes of science.
BIR is a hot topic with growing recognition in the community in recent years: see for instance the Initiative for Open Citations [25], the Google Dataset Search [5], the Indian JNU initiative for indexing the world’s literature in full-text [22], the increasing number of retractions [4], and massive studies of self-citations [13, 27]. We believe that BIR@ECIR is a much needed scientific event for the ‘retrievalists’, ‘citationists’ and others to meet and join forces pushing the knowledge boundaries of IR applied to literature search and recommendation.
2 Summarzing the Past BIR Workshops
The BIR workshop series was launched at ECIR in 2014 [19] and it was held at ECIR each year since then. As our workshop lies at the crossroads between IR and NLP, we also ran BIR as a joint workshop called BIRNDL (Bibliometric-enhanced IR and NLP for Digital Libraries) at the JCDL [7] and SIGIR [9] conferences. All past workshops had a large number of participants (between \(\sim \)30 and \(\sim \)60), demonstrating the relevance of the workshop’s topics.
In the following, we present an overview of the past BIR workshops and keynotes at BIR (Tables 1 and 2). All pointers to the workshops and proceedings are hosted at sites.google.com/view/bir-ws. Many of the presented workshop papers appeared in extended form in one of our four BIR-related special issues (2015 [20], 2018 [8, 18], 2019 [2]).
3 Workshop Topics
The call for papers for the 2020 workshop (the 10th BIR edition) addressed current research issues regarding 3 aspects of the search/recommendation process:
-
1.
User needs and behaviour regarding scientific information, such as:
-
Finding relevant papers/authors for a literature review.
-
Measuring the degree of plagiarism in a paper.
-
Identifying expert reviewers for a given submission.
-
Flagging predatory conferences and journals.
-
Information seeking behaviour and HCI in academic search.
-
-
2.
Mining the scientific literature, such as:
-
Information extraction, text mining and parsing of scholarly literature.
-
Natural language processing (e.g., citation contexts).
-
Discourse modelling and argument mining.
-
-
3.
Academic search/recommendation systems:
-
Modelling the multifaceted nature of scientific information.
-
Building test collections for reproducible BIR.
-
System support for literature search and recommendation.
-
4 Target Audience
The target audience of the BIR workshops are researchers and practitioners, junior and senior, from Scientometrics as well as Information Retrieval and Natural Language Processing. These could be IR/NLP researchers interested in potential new application areas for their work as well as researchers and practitioners working with, for instance, bibliometric data and interested in how IR/NLP methods can make use of such data.
5 Peer Review Process and Workshop Format
Our peer review process is supported by Easychair. Each submission is assigned to 2 to 3 reviewers, preferably at least one expert in IR and one expert in Bibliometrics or NLP. The accepted papers are either long papers (15-min talks) or short papers (5-min talks). Two interactive sessions close the morning and afternoon sessions with posters and demos, allowing attendees to discuss the latest developments in the field and opportunities (e.g., shared tasks such as the CL-SciSumm [12] at the BIRNDL joint workshop, see Sect. 2). These interactive sessions serve as ice-breakers, sparking interesting discussions that usually continue during lunch and the cocktail party. The sessions are also an opportunity for our speakers to further discuss their work.
6 Next Steps
Research on scholarly document processing has for many years been scattered across multiple venues like ACL, SIGIR, JCDL, CIKM, LREC, NAACL, KDD, and others. Our next strategic step is the First Workshop on Scholarly Document Processing (SDP)Footnote 2 will be held in November 2020 in conjunction with the 2020 Conference on Empirical Methods in Natural Language Processing. This workshop and initiative will be organized by a diverse group of researchers (organizers from BIR, BIRNDL, Workshop on Mining Scientific Publications/WOSP and Big Scholar) which have expertise in NLP, ML, Text Summarization/Mining, Computational Linguistics, Discourse Processing, IR, and others.
Notes
- 1.
- 2.
References
Atanassova, I.: Beyond metadata: the new challenges in mining scientific papers. In: Cabanac, G., Frommholz, I., Mayr, P. (eds.) Proceedings of the 8th International Workshop on Bibliometric-Enhanced Information Retrieval (BIR 2019) Co-located with the 41st European Conference on Information Retrieval (ECIR 2019), Cologne, Germany, 14 April 2019. CEUR Workshop Proceedings, vol. 2345, pp. 8–13. CEUR-WS.org (2019). http://ceur-ws.org/Vol-2345/paper1.pdf
Atanassova, I., Bertin, M., Mayr, P.: Editorial: mining scientific papers: NLP-enhanced bibliometrics. Front. Res. Metrics Anal. (2019). https://doi.org/10.3389/frma.2019.00002
Beel, J., Dinesh, S.: Real-world recommender systems for academia: the pain and gain in building, operating, and researching them. In: Mayr, P., Frommholz, I., Cabanac, G. (eds.) Proceedings of the Fifth Workshop on Bibliometric-enhanced Information Retrieval (BIR) Co-located with the 39th European Conference on Information Retrieval (ECIR 2017), Aberdeen, UK, 9th April 2017. CEUR Workshop Proceedings, vol. 1823, pp. 6–17. CEUR-WS.org (2017). http://ceur-ws.org/Vol-1823/paper1.pdf
Brainard, J., You, J.: What a massive database of retracted papers reveals about science publishing’s “death penalty”. Science (2018). https://doi.org/10.1126/science.aav8384
Brickley, D., Burgess, M., Noy, N.: Google dataset search: building a search engine for datasets in an open Web ecosystem. In: The World Wide Web Conference on - WWW 2019, pp. 1365–1375. ACM Press (2019). https://doi.org/10.1145/3308558.3313685
Cabanac, G.: In praise of interdisciplinary research through scientometrics. In: Mayr, P., Frommholz, I., Mutschke, P. (eds.) Proceedings of the Second Workshop on Bibliometric-Enhanced Information Retrieval Co-located with the 37th European Conference on Information Retrieval (ECIR 2015), Vienna, Austria, 29th March 2015. CEUR Workshop Proceedings, vol. 1344, pp. 5–13. CEUR-WS.org (2015). http://ceur-ws.org/Vol-1344/paper1.pdf
Cabanac, G., et al. (eds.): BIRNDL 2016: Proceedings of the Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries co-located with the Joint Conference on Digital Libraries, vol. 1610. CEUR-WS, Aachen (2016)
Cabanac, G., Frommholz, I., Mayr, P.: Bibliometric-enhanced information retrieval: preface. Scientometrics 116(2), 1225–1227 (2018). https://doi.org/10.1007/s11192-018-2861-0
Chandrasekaran, M.K., Mayr, P. (eds.): BIRNDL 2019: Proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries co-located with the Joint Conference on Digital Libraries, vol. 2414. CEUR-WS, Aachen (2019)
Chandrasekaran, M.K., Mayr, P. (eds.): Proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2019) co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019), Paris, France, 25 July 2019, CEUR Workshop Proceedings, vol. 2414. CEUR-WS.org (2019). http://ceur-ws.org/Vol-2414
Garfield, E.: Citation indexes for science: a new dimension in documentation through association of ideas. Science 122(3159), 108–111 (1955). https://doi.org/10.1126/science.122.3159.108
Jaidka, K., Chandrasekaran, M.K., Rustagi, S., Kan, M.Y.: Insights from CL-SciSumm 2016: the faceted scientific document summarization shared task. Int. J. Dig. Lib. 19(2–3), 163–171 (2018). https://doi.org/10.1007/s00799-017-0221-y
Kacem, A., Flatt, J.W., Mayr, P.: Tracking self-citations in academic publishing. Scientometrics (2020). https://doi.org/10.1007/s11192-020-03413-9
Koolen, M.: Bibliometrics in online book discussions: lessons for complex search tasks. In: Mayr, P., Frommholz, I., Cabanac, G. (eds.) Proceedings of the Third Workshop on Bibliometric-enhanced Information Retrieval co-located with the 38th European Conference on Information Retrieval (ECIR 2016), Padova, Italy, 20 March 2016. CEUR Workshop Proceedings, vol. 1567, pp. 5–13. CEUR-WS.org (2016). http://ceur-ws.org/Vol-1567/paper1.pdf
Labbé, C.: Trends in gaming indicators: on failed attempts at deception and their computerised detection. In: Mayr, P., et al. [17], pp. 6–15. http://ceur-ws.org/Vol-2080/paper1.pdf
Leydesdorff, L., Milojević, S.: Scientometrics. In: Wright, J.D. (ed.) International Encyclopedia of the Social & Behavioral Sciences, vol. 21, 2nd edn, pp. 322–327. Elsevier, Amsterdam (2015)
Mayr, P., Frommholz, I., Cabanac, G. (eds.): Proceedings of the 7th International Workshop on Bibliometric-enhanced Information Retrieval (BIR 2018) Co-located with the 40th European Conference on Information Retrieval (ECIR 2018), Grenoble, France, 26 March 2018, CEUR Workshop Proceedings, vol. 2080. CEUR-WS.org (2018). http://ceur-ws.org/Vol-2080
Mayr, P., et al.: Special issue on bibliometric-enhanced information retrieval and natural language processing for digital libraries. Int. J. Dig. Lib. 19(2–3), 107–111 (2018). https://doi.org/10.1007/s00799-017-0230-x
Mayr, P., Schaer, P., Scharnhorst, A., Larsen, B., Mutschke, P. (eds.): BIR 2016 Proceedings of the 1st Workshop on Bibliometric-enhanced Information Retrieval co-located with the 36th European Conference on Information Retrieval, vol. 1143. CEUR-WS, Aachen (2014)
Mayr, P., Scharnhorst, A.: Scientometrics and information retrieval: weak-links revitalized. Scientometrics 102(3), 2193–2199 (2014). https://doi.org/10.1007/s11192-014-1484-3
Pritchard, A.: Statistical bibliography or bibliometrics? [Documentation notes]. J. Document. 25(4), 348–349 (1969). https://doi.org/10.1108/eb026482
Pulla, P.: The plan to mine the world’s research papers. Nature 571, 316–318 (2019). https://doi.org/10.1038/d41586-019-02142-1
Salton, G.: Associative document retrieval techniques using bibliographic information. J. ACM 10(4), 440–457 (1963). https://doi.org/10.1145/321186.321188
Schenkel, R.: Integrating and exploiting public metadata sources in a bibliographic information system. In: Mayr, P., et al. [17], pp. 16–21. http://ceur-ws.org/Vol-2080/paper2.pdf
Shotton, D.: Funders should mandate open citations. Nature 553(7687), 129 (2018). https://doi.org/10.1038/d41586-018-00104-7
Teufel, S.: Do “future work” sections have a purpose? citation links and entailment for global scientometric questions. In: Mayr, P., Chandrasekaran, M.K., Jaidka, K. (eds.) Proceedings of the 2nd Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2017) Co-located with the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2017), Tokyo, Japan, 11 August 2017. CEUR Workshop Proceedings, vol. 1888, pp. 7–13. CEUR-WS.org (2017). http://ceur-ws.org/Vol-1888/paper1.pdf
Van Noorden, R., Singh Chawla, D.: Hundreds of extreme self-citing scientists revealed in new database. Nature 572(7771), 578–579 (2019). https://doi.org/10.1038/d41586-019-02479-7
Wade, A.D., Williams, I.: Personalized feed/query-formulation, predictive impact, and ranking. In: Chandrasekaran, M.K., Mayr, P., [10], pp. 6–7. http://ceur-ws.org/Vol-2414/paper1.pdf
Webber, B.: Discourse processing for text analysis: recent successes, current challenges. In: Chandrasekaran, M.K., Mayr, P., [10], pp. 8–14. http://ceur-ws.org/Vol-2414/paper2.pdf
White, H.D., McCain, K.W.: Visualizing a discipline: an author co-citation analysis of Information science, 1972–1995. J. Am. Soc. Inf. Sci. 49(4), 327–355 (1998). b57vc7
Wolfram, D.: Bibliometrics, information retrieval and natural language processing: natural synergies to support digital library research. In: Cabanac, G., et al. (eds.) Proceedings of the Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL) co-located with the Joint Conference on Digital Libraries 2016 (JCDL 2016), Newark, NJ, USA, 23 June 2016. CEUR Workshop Proceedings, vol. 1610, pp. 6–13. CEUR-WS.org (2016). http://ceur-ws.org/Vol-1610/paper1.pdf
Acknowledgement
We organizers wish to thank all those who contributed to this workshop series: The researchers who contributed papers, the many reviewers who generously offered their time and expertise, and the participants of the BIR and BIRNDL workshops. Since 2016, we maintain the Bibliometric-enhanced-IR Bibliography that collects scientific papers which appear in collaboration with the BIR/BIRNDL organizers.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cabanac, G., Frommholz, I., Mayr, P. (2020). Bibliometric-Enhanced Information Retrieval 10th Anniversary Workshop Edition. In: Jose, J., et al. Advances in Information Retrieval. ECIR 2020. Lecture Notes in Computer Science(), vol 12036. Springer, Cham. https://doi.org/10.1007/978-3-030-45442-5_85
Download citation
DOI: https://doi.org/10.1007/978-3-030-45442-5_85
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-45441-8
Online ISBN: 978-3-030-45442-5
eBook Packages: Computer ScienceComputer Science (R0)