Keywords

1 Introduction

Scientific evaluation, especially peer evaluation, of scholarly works is among the cornerstones in academia. Awards for scholars and their works can be seen as one particular form of scientific evaluation. For centuries, academic awards in different forms have been used to publicly recognize and honor individuals and their contributions, thus encouraging further scientific discoveries [1]. Awards for academic publications are worth noting because they signal what kinds of contributions are valued by the scientific community [2]. Understandably, award-winning papers are usually recognized as exemplary works.

Despite the long-established efforts in analyzing LIS literature, a limited number of studies have analyzed award-winning papers in LIS [3, 4]. By analyzing the award-wining LIS papers from Emerald Publishing, this study aims to provide researchers, professionals, and other interested stakeholders with an updated view of the main spheres of LIS.

In this study, we assume that award-winning papers stand out as high-quality ones that represent qualified cases for analyzing the status quo and the intellectual development of LIS. This paper addresses the following research questions:

  • RQ1. What are the authorship patterns of the award-wining LIS papers from Emerald Publishing?

  • RQ2. What is the dominant paper type (theoretical article, empirical research, or literature review) within the award-wining LIS papers from Emerald Publishing?

  • RQ3. What topics have been frequently discussed in award-wining LIS papers from Emerald Publishing?

  • RQ4. Do the award-winning papers have higher citations than the non-award papers?

2 Literature Review

2.1 Analysis of LIS Literature

There has been long-standing scholarly interest among LIS researchers to utilize bibliometric methods and content analysis to understand the temporal and evolutional landscape of diverse aspects of research and practice in the field. Many studies indicate that scientific collaboration is common in LIS. Blessinger and Frasier [5], for example, pointed out that approximately 54% of 2200 journal articles published between 1994 and 2004 were co-authored by at least two people. Similarly, according to Aharony [6], co-authored works accounted for more than 70% of the publications by 10 LIS journals in 2007 and 2008, with North American and European scholars playing a leading role.

There have been multiple investigations of methodological issues in LIS. For example, Tuomaala et al. [7] analyzed LIS research articles in 2005. They reported that empirical research strategy (76%) was the most frequently used, followed by conceptual research design and other research strategies (e.g., literature review). In terms of types of analysis, quantitative analysis (58.4%) was dominant, while qualitative analysis was used in 14.1% of the sampled LIS articles in 2005. Ullah and Ameen [8] maintained that empirical, descriptive, and quantitative research methodologies have been used in the majority of LIS research. Compared with qualitative and quantitative research, mixed methods research accounts for a small portion of LIS research [9]. Fidel [10] pointed out that 5% of the LIS research applied mixed methods designs during 2005 and 2006.

In terms of topical areas of LIS literature, the classification system originally created by Järvelin and Vakkari [11] has been widely used [7, 12]. There are several main classes in the classification system, including the professions in library and information services, library history, publishing, education in LIS, methodology, analysis of LIS, library and information service activities, information storage and retrieval, information seeking, scientific and professional communication, and other aspects of LIS. Recently, Blessinger and Frasier [5] grouped LIS journal publications from 1994 to 2004 into five major topical categories: library operation (33%), research in library and information science/users (20%), library/information science profession (18%), technology (18%), and publishing/publishing studies (11%).

2.2 Analysis of Award-Winning Papers

There have been a few studies focusing on award-winning articles across disciplines. For example, Tackney et al. [13] conducted a content analysis of 40 best papers from Management Spirituality and Religion Interest Group. Focusing on the history of the Association for Information Systems best publication awards, Ghobadi and Robey [14] developed a framework involving contribution characteristics, demographic patterns, and citation histories of award-winning papers. In addition, some studies have specifically compared the differences between award-winning papers and other papers in citation counts. Sen and Patel [15] examined the citation rates of articles selected for the most prestigious awards of American Society of Civil Engineers. They found that nearly 25% of the award-winning papers were never cited, and over 30% were cited only once. Similarly, Coupé’s [16] findings indicated that the papers that won “best paper” prizes in economics and finance journals were rarely the most cited. Wainer et al. [17] compared the citations of award-winning papers and random papers from different computer science conferences, pointing out that award-winning papers had a higher probability to receive more citations. From a different perspective, Mubin and co-authors [18] examined the readability of award-winning papers at the ACM Conference on Human Factors in Computing Systems (also known as the CHI Conference) in comparison to their non-award counterparts. They found that award-winning full papers had a lower readability.

Research analyzing award-winning papers in LIS is, however, relatively scarce. Brooks [3] examined 28 best articles published in the Journal of the Association for Information Science and Technology (JASIST) from 1969–1996, reporting that best papers tended to be lengthy and single-authored, and were cited and self-cited more often than average articles. In a similar vein, Zhang et al. [4] expanded the period and compared the citation counts of 45 award-winning articles published between 1969 and 2013 with average papers. They found that most best papers belonged to the top 50% stratum, and there was a wide range of citations among the Best JASIST Papers. It should be noted that these two papers have only investigated citation counts of award-winning papers from a single journal (JASIST). To have a rather comprehensive view, this study will analyze different aspects of award-winning papers in addition to citation count, including collaboration type, paper type, and topic.

3 Methodology

3.1 Data Collection

There exist various best paper awards in LIS journals (e.g., JASIST, Knowledge Organization) and conferences (e.g., iConference, the Joint Conference on Digital Libraries Conference). However, information about award-winning papers is often scattered without being organized in one place. Notably, Emerald Publishing, one of the major global academic publishers, has been updating a gallery of award-winning papers on its website since 2008. 16 LIS journals are currently managed by Emerald Publishing, from 15 of which 106 publications were granted outstanding paper awards between 2008 and 2019. Considering the relatively easy data access, we decided to use award-winning papers from Emerald Publisher as a sample. The full list of those 106 publications was first manually created, and the full text of each publication was downloaded to compose the study corpus for further content analysis. Admittedly, the limited dataset of award-winning papers from one publisher may result in bias in terms of reporting or interpreting findings (Table 1).

Table 1. LIS journals published by Emerald Publishing
Full size table

In addition to full-text files, information of citation counts of both award-winning articles and average paper was retrieved from Scopus. In response to each award-winning paper, citation counts of the remaining average papers published concurrently in the same journal were also kept. In total, citation counts of 826 average papers were collected.

3.2 Data Analysis

The retrieved award-winning papers were examined in four key aspects: collaboration type, paper type, topic, and citation count. The coding schemes for research topic and paper type were developed by the authors based on previous studies [5,6,7, 19, 20]; they were further refined during the coding procedure.

Specifically, collaboration type was analyzed in accordance with Qin and colleagues’ [21] classification of scientific collaboration consisting of no collaboration, collaboration within a department, collaboration between two or more departments within an institution, collaboration between two or more institutions within a country, and international collaboration. Author and institution information of each award-winning paper was manually recorded in a separate spreadsheet for analysis.

This paper analyzed general paper types instead of specific research methods. On the one hand, existing listings and classifications of research methods in LIS follow varying criteria [7, 22, 23]. On the other hand, the authors often found it challenging to assign methods to some research articles due to their implicit method/methodology statements where research methods were not clearly specified. Last but not least, not all the award-winning papers qualify as research “carried out, at least to some degree, by a systematic method with the purpose of eliciting some new facts, concepts, or ideas” [24, p. 251]. Due to the aforementioned concerns, the authors decided to reveal the methodological issues from a macro perspective [7, 19]. Specifically, each article was assigned to one of the following paper types: theoretical paper (i.e., argument and perspective paper, concept analysis), empirical research (i.e., qualitative research, quantitative research, mixed methods research), and literature review.

The coding scheme for analyzing topics comprises the following categories: professions and information institutions; methodology; education in LIS; information systems and technologies; information service activities; information organization and retrieval; information management; information ethics, policy, and security; user studies; scientific communication; analysis of LIS publications; and other aspects. When an article involved many topics, the coders attempted to “identify its main topic” [7, p. 1449]. Meanwhile, the author keywords were recorded to assist in visualizing the topics covered in the study corpus.

Two co-authors coded each publication independently after a close reading of the full text. A trial coding of 20 articles that were randomly sampled was conducted to assess the intercoder reliability. Both simple agreement (also known as percent agreement) and Cohen’s kappa were used. The inter-coder reliability test indicated that the coding process was reliable [25] (Table 2). The two coders discussed and resolved conflicts and continued to code the remaining articles on their own. When all articles were coded, the two coders compared and finalized the coding results.

When analyzing citation counts, we compared the citation count each award-winning paper received with those of average papers published on the same issue. To have a better understanding of the citation counts of both award-winning paper group and average paper group, we also provided the five-number summary for both groups as well as a corresponding box plot with outliers.

Table 2. Inter-coder reliability test
Full size table

4 Results

4.1 Authorship

Excluding the article Research Data Management as a “wicked problem” [26] by 23 authors, the average number of authors per paper was 2. Of the 106 articles, 49 (46%) were single-authored, 27 (25%) were written by two authors, and 17 (16%) were written by three authors. In total, there were 232 authors involved in producing the 106 award-winning articles, among whom five authors were awarded twice for their works: Andrew K. Shento, Kenning Arlitsch, Kerry Wilson, Reijo Savolainen, and Sheila Corrall. The authors across all papers were from 31 different countries; 78 (34%) were from the U.S., 60 (26%) were from the U.K., and the rest were from China, Australia, Germany, Canada, and other countries (Fig. 1). Moreover, it has been found that in addition to faculty members of colleges or universities, practitioners from libraries and other institutions are also key participants in LIS research. More than 55 (24%) authors of award-winning papers were librarians, many of whom were academic librarians. Additionally, more than 35 (15%) authors from other disciplines (e.g., computer science, media studies, sociology, business, and engineering) contributed to the award-winning papers.

Fig. 1.
figure 1

Countries of affiliation of authors

Full size image

In terms of collaboration, intradepartmental and inter-institutional collaborations were relatively commonplace, accounting for 45% of all types of collaboration. International collaboration occurred in 8% of all cases. Comparatively, collaboration among two or more departments within an institution only accounted for 1% (Fig. 2).

Fig. 2.
figure 2

Distribution of collaboration patterns

Full size image

4.2 Paper Type

Sixty-five (61.3%) articles were coded as empirical studies, among which qualitative research was dominant, followed by quantitative research and mixed methods research. Notably, no quantitative studies were awarded, while seven qualitative papers were awarded in 2009; no mixed methods research papers were given awards between 2013 and 2016. Theoretical papers accounted for 32.1% of the corpus and mainly consisted of argument/position papers. It is worth mentioning that only four articles, three of which were given awards in 2016, were coded as concept analysis/development articles. Seven literature review papers comprised the smallest proportion of the sample. Figure 3 shows the frequency distribution of the award-wining paper types by award year (Table 3).

Fig. 3.
figure 3

Frequency distribution of the award-wining paper types

Full size image
Table 3. Paper types of award-wining papers
Full size table

4.3 Topic

As shown in Table 4, 29 articles concerned information service activities, accounting for the largest proportion (27.4%). The most popular topic within the category of information service activities was user education, and there were ten papers focusing on information literacy. Other popular topics related to information services included digital information resources and reference services. The topic of professions and information institutions accounted for 23.6%, of which more than two in five articles were about library development strategies. For user studies, papers on human information behavior were dominant, and there was notable attention to online communities [27, 28]. Figure 4 shows the frequency distribution of the award-winning paper topics by award year.

Table 4. Topics of award-winning papers
Full size table
Fig. 4.
figure 4

Frequency distribution of the award-winning papers topics

Full size image

In terms of keywords, there were originally 538 keywords in total. After data cleaning, 377 distinct keywords were identified, 69 of which appeared twice or more. Figure 5 shows the average number of keywords per paper. Before 2012, the average number of keywords was around 4. It peaked in 2013. In terms of common keywords, academic libraries (19 times), information literacy (13 times), public libraries (12 times), and digital libraries (10 times) were used most frequently. Additionally, the Internet, library instruction, search engines, user studies, and social media appeared more than 5 times (Fig. 6).

Fig. 5.
figure 5

Average number of keywords per paper

Full size image
Fig. 6.
figure 6

Word cloud of keywords

Full size image

4.4 Citation Count

Compared with their average counterparts, 32 award-winning papers were the most cited papers of the journal issues where they got published respectively; the 67 award-winning papers had a higher mean and median in citation counts than the average papers concurrently published by the journals. According to Fig. 7, the citation counts of award-winning paper were generally higher than average papers. However, it is worth noting that some award-winning papers were rarely cited. Specifically, six award-winning papers were never cited, and five were cited only once.

Fig. 7.
figure 7

Citation counts of award-winning papers and average papers

Full size image

5 Discussion

5.1 Pervasive Collaboration in LIS Publications

As the findings indicated, more than half of the award-winning papers were coauthored, resonating with the prevalence of collaboration in LIS that has been widely noted in previous studies [5, 29,30,31]. Not surprisingly, authors from North America (mainly the U.S.) and Europe (mainly the U.K.) contributed to a large portion of the corpus under study, which was in line with previous findings [6, 32]. The dominance of American and European scholars in LIS publications might be related to the long-standing academic tradition and the developed higher education systems in developed countries. It has also been found that in addition to LIS faculty and students as the core force, practitioners (mainly librarians) and authors from other disciplines also contributed to many award-wining papers. In fact, contributions to LIS publications by practitioners have been recognized by previous studies [33, 34]. Taking advantage of their work experience, practitioners are more likely to offer insightful thoughts on certain topics, such as reference services, cataloging practices, and library user studies. Undeniably, the degree of interdisciplinarity in LIS relies on the participation of authors from other disciplines [35]. Authors from other disciplines help extend the scope of LIS research and might provide promising techniques and perspectives to re-examine certain existing topics. In short, scientific collaboration and communication through geopolitical and disciplinary borders are of great importance in supporting the development of one domain, and such practices are currently being nurtured in LIS [31, 36, 37].

5.2 Co-existence of Various Methodological Paradigms in LIS

Reflecting on methodological issues is essential in all disciplines. As previously mentioned, no specific methods were coded for each paper because of the conflicting classification systems of methods in LIS; rather, the present study characterized the sampled papers based on their paper design and associated types of analysis from a macro perspective [7, 19].

In social science research, quantitative research, qualitative research, and mixed methods research are usually seen as three major paradigms [38]. According to the findings, qualitative research accounted for the vast majority of award-winning research papers, echoing the fact that qualitative research is being increasingly recognized and used in LIS studies [12, 39]. It should be noted that qualitative research is not prioritized over quantitative research, and vice versa. Both qualitative and quantitative research have their own advantages in different research scenarios. Generally, qualitative research helps understand complex cultural and social contexts and explore people’s lived experiences, while quantitative research is appropriate for examining quantifiable measures of variables mainly by means of descriptive and inferential statistical methods [40,41,42]. The large number of papers focusing on information service activities and human information behavior in the award-winning papers might explain why qualitative research accounted for a high proportion in the papers to some degree.

Mixed methods research is a relatively young paradigm synthesizing both qualitative and quantitative methods [38]. Although mixed methods research has been discussed and applied in many disciplines [43,44,45], Fidel [10] found that “mixed methods” was not a familiar term in LIS research, and only 17% of the empirical research articles he surveyed used mixed methods. Similarly, 18% of the empirical award-winning papers in the current study were mixed methods studies. Though Fidel [10] suggested that it is not always easy to identify mixed methods research because of a lack of explicit method statements in many studies, some of the award-winning mixed methods papers in the present study stated clearly that both qualitative and quantitative methods were applied [46,47,48,49]. To achieve transparency and replicability, a clear and detailed method statement is expected for all types of research. For instance, Anfara Jr. et al. [50] regard the public disclosure and openness of methods and research processes as one potential way to promote the research quality and rigor in the qualitative research community.

5.3 Diverse yet Consistent Topics in LIS Literature

LIS is an interdisciplinary field that encompasses diverse topics [51]. As shown in the literature review, there have been continuous attempts to categorize topics or areas in LIS [6, 7, 11, 52, 53]. 377 distinct keywords were revealed, with more than 300 of them appearing only once, such as tattooing, HIV, and democracy. The average number of keywords indexed in each article per year could reflect the scope of its topic(s) to some degree [54]. The significant increase in the number of keywords per paper after 2012 might indicate that the award-winning papers are involving more topics. To have a clear understanding of what keywords mean, it is always important to put them in specific contexts. For example, Sundberg and Kjellman [55, p. 18] examined “how tattoos can be considered documents of an individual’s identity, experiences, status, and actions” based on the tattoo practices of Russian/Soviet prisoners.

According to Pawley [56], four models dominate LIS research and teaching, namely, science/technology, business/management, mission/service, and society/culture, providing us with a simplified way to examine these diverse topics in LIS. It has been found that the top three topics of the award-winning papers, information service activities, professions and information institutions, and user studies, are all related to the mission/service model. For example, Arlitsch [57] discussed the disruptive implications of the Espresso Book Machine on library user services, collection development, and special collections; Curry [58] explored the potential of makerspaces to function as new learning spaces within academic libraries in higher education.

The results show that some keywords concerning information technology, such as the Internet, search engines, and social media, were also frequently used. There have been many previous studies demonstrating the importance of information technology. For example, Davarpanah and Asleki [32] claimed that a high number of articles were related to communications and information technology. Liu and Yang [59] pointed out that the most popular research topics in LIS were closely related to social media, data, and information retrieval in the most recent decade. In short, LIS has constantly maintained a focus on the aspects of mission/service and science/technology in the changing social and technological environment. With regard to human and technological dimensions in LIS, Cibangu [60] argued that emphasizing human dimensions does not mean undervaluing the importance of technologies, and LIS scholars are expected to be humanities-prone thinkers, thus making LIS a human science, which provides a potential direction for LIS.

5.4 Complicated Relationship Between Awards and Citation Counts

Awards and citations are often referred to as qualitative and quantitative assessment of quality or impact of publications, respectively. Best paper awards are generally used to acknowledge publications based on their knowledge contribution, paper structure, the rigorousness of the argument/analysis, and paper writing and presentation [61]. However, the processes of selecting award-winning papers are unavoidably subjective; the lack of transparency is also an issue. As for citations, an author may have different reasons when citing other documents, including but not limited to providing background information, giving credit for related works, and criticizing previous works [62, 63]. Previous research shows that compared with positive and neutral citations, negative citations account for a relatively small portion of total citations [63, 64]. Some scholars believe that “even a negative citation makes it clear that the referenced work cannot be simply ignored” [65, p. 2].

Due to the varying nature of awards and citation counts, it is not possible to simply determine that awards lead to high citation counts. Instead, there seems to be a complicated relationship between publication awards and citation counts [66]. We found that the award-winning papers under study did have a higher citation count on average than their average counterparts; however, not all award-winning papers were cited frequently, and 11 of them were either never cited or only cited once. There might be several potential reasons to explain the lower citation counts of some award-winning papers. First, it is likely that one award-winning paper as a “sleep beauty” are so innovative that no one cites it for years to come [67]. Second, people don’t cite award-winning papers that merely summarize existing literature and are lacking in interesting or groundbreaking findings. Third, other criteria other than “quality” might be considered during the evaluation processes. There remain other possibilities for sure. The underlying reasons or mechanisms are understudied.

6 Conclusion

This study is descriptive in nature. Instead of arguing what topic and paper type should be given awards, our primary goal is to reveal the characteristics of the award-wining LIS papers under study, and to provide implications for researchers to understand the recent trends in LIS. We found that the authors of more than 60% all of the outstanding papers were from the USA or the UK, 54% of the award-winning papers were co-authored, and intradepartmental collaboration and inter-institutional collaboration within one country accounted for the largest proportion. Furthermore, empirical research accounted for the vast majority of award-winning papers, more than half of which are qualitative. Additionally, there seems to be a wide variety of topics in the LIS literature, with information service activities, professions and information institutions, and user studies topping the list, indicating that the LIS discipline has maintained a focus on users and services. It is also worth noting that LIS has been evolving dynamically, and its foci go beyond the traditional aspects of storage, organization, and use of information, and special attention has been paid to areas such as information literacy, artificial intelligence, and the humanities. We also found that on average, the award-winning papers under study did have a higher citation count than their average counterparts; however, not all award-winning papers were cited frequently.

Admittedly, this study could only partially reflect the disciplinary landscape and development because of the restrained sample of LIS journal publications from Emerald Publishing, which limits the generalizability of the research results. Expanding the scope of research data could help improve the generalizability of the research results. For future research, scholars might be interested in understanding the understudied relationship between publication awards and citation counts.