Introduction

Measuring research impact using citation analysis has a long tradition in scientometrics (Zahedi & Haustein, 2018). With the advent of ICT and various social networking sites, authors are starting to get broad platforms to share their research publications. Techniques to assess the social effect and public interaction of scientific research have led to the introduction of alternative ways of citation analysis. Readership seems to be a good indicator of an impact compared to other social web activities, such as tweeting (Journals, 2014). Mendeley is a prominent reference management tool and a rich source of readership (Zahedi et al., 2014). It seems to be a significant source of altmetrics because 63% of Web of Science articles from 2005 to 2011 had at least one Mendeley reader by April 2013 (Zahedi, Costas, & Wouters). However, Mendeley is a widely used reference management tool. It also has the features of the academic and social networks, which provide the facility of creating profiles to the users through which they can search, access, share research works and ideas, perform collaboration, information management etc. among the academic community without country or language barrier (Savithry, 2014). Users can create and maintain their profiles in Mendeley, which comprises the details of the users, including their discipline, research interests, biographical information, contact details, and publications. These user profiles are complemented with readership counts. The readership count indicates how many Mendeley users have added the author’s article to their personal research library (Schlögl et al., 2014).

Coverage is a crucial factor in using a statistic as a research tool. With the increase of users from different categories, locations, etc., there is a growth of representation from the categories or locations. Mendeley has more excellent coverage regarding the number of articles with non-zero scores than the other altmetrics sources, including other reference managers, such as BibSonomy (Borrego & Fry, 2012; Costas et al., 2015; Thelwall et al., 2013).

Some previous studies assessed the relationship between readership and citation and traced a significant correlation between the two (Bar-Ilan, 2012; Costas et al., 2015; Maflahi & Thelwall, 2015; Ravikumar & Dohtdong, 2018; Zahedi et al., 2014). The variation in correlations between Mendeley readership count and citation received for different types of readers suggested that the meaning of Mendeley readership count depends upon the readers’ occupations (Ravikumar & Dohtdong, 2018). This paper discusses the correlation between the Mendeley readership score and how different user categories use Mendeley.

ICT-based products and services bring a robust change in library and information. With these products and services, the libraries are overcoming the problem of isolation and lack of access to information. The emergence of these technologies has changed the scenario in librarianship. It provides a global network among the research peers (Parvez, 2011). Punchihewa (2018) reveals that most of the libraries in Sri Lanka adopted the web2.0 tools in their libraries. The study found that the libraries use only the basic features of web2.0 applications except for social networking sites.

As the implementation of the library 2.0 application is gradually increasing in Sri Lanka, this will also impact the dissemination of the research articles of the Sri Lankan authors. This study portrays the effect of Mendeley readership (one of the most popular new media) in the citation of the articles and review articles of Sri Lankan authors.

Literature review

Mendeley is the most popular and trusted reference management tool with similar characteristics to citations in terms of their distribution across fields allowing its users to create a reference list of articles for their study and view and communicate with other users of Mendeley (Nath et al., 2020; Thelwall & Wilson, 2016; Zahedi et al., 2017). Costas et al. (2015) pointed out that Mendeley has high scientific papers coverage rates of more than 60% or even more than 80% for WoS papers depending on the area.

ICT and social networking sites provide broad opportunities for scholars to access and disseminate research papers and traditional facilities. Altmetric is a new media tool which is emerging as an alternative or a complement to traditional citation counts that aim to measure Web-driven scholarly interactions (Pooladian & Borrego, 2017; Williams, 2017).

Syamili and Rekha (2017) opined that the kind of impact measured by the altmetric score, whether social or research or societal impact, is still an open question. Ravikumar (2018) stated that though the altmetric variables are nascent, the new media tools are penetrating the traditional citation area, which was considered the tool for evaluating scientific literature. Bornmann (2014), Robinson-García et al. (2014), Roemer and Borchardt (2015) and Williams (2017) also discussed the effectiveness of altmetric tools as an impact assessment tool from different angles.

Readership count is a strong impact indicator (Journals, 2014; Maflahi & Thelwall, 2015), and Mendeley is a good source of readership data (Costas et al., 2015; Zahedi et al., 2014). Previous studies found significant correlations between the Mendeley readership counts and the citation counts (Bar-Ilan, 2012; Ravikumar & Dohtdong, 2018; Thelwall, 2017; Zahedi et al., 2014). The correlations between Mendeley readership and citation scores are higher than the correlations between citations and other altmetric indicators, and the Mendeley readership score is notably helpful in identifying highly cited papers (Costas et al., 2015; Thelwall et al., 2013; Zahedi et al., 2014, 2017).

Thelwall and Sud (2016), Thelwall and Wilson (2016) conducted subject-specific studies on 45 different subjects of the medical domain and observed a strong correlation between Mendeley readership statistics and citation in almost all the subject fields.

Zahedi et al. (2014) assess the impact of publications saved by different types of Mendeley users. The core area is to investigate the impact of publications saved by the different users in Mendeley to explore how their readership counts correlate with their citation indicators. Studies indicate that correlation tends to decrease slightly if the readership of the students not counted (Thelwall & Wilson, 2016; Zahedi et al., 2014).

Journal articles are considered the primary source of scholarly research communication as they have gone through a rigorous screening process. These are widely accessible to the readers (Lee, 2015; Lowry et al., 2004). Because of the accessibility and citation in the articles, most of the impact is researches on Journal articles. In the digital era, DOI is necessary to identify and easily access journal articles. The reader counts will be more comprehensive for articles with DOIs (Thelwall & Sud, 2016).

Subject-specific, user-specific studies were carried out on Mendeley readership score and its association with the traditional impact indicator citation count. However, no country-specific work was noticed during the literature review. This study intended to do a country-specific study, i.e. a South Asian country Sri Lanka, to study the correlation between Mendeley readership score and citation count. It will help to know whether the correlation between Mendeley readership and citation in the country-specific study is in the same existing trend.

Research questions

The research question for the study are

  1. i.

    If we conduct a country-specific study, will the correlation of the Mendeley readership score and the citation count be similar to the earlier studies or not?

  2. ii.

    Is there any impact of the number of indexed articles of the WoS subject category on the correlation?

  3. iii.

    What is the correlation pattern among the different Mendeley user categories?

Objectives

The objective of the study is

  1. 1.

    To study the correlation between the Mendeley readership score and the Web of Science citation counts for Sri Lankan authors.

  2. 2.

    To identify the most occurred WoS subject category among the articles and study their correlation between readership and citation.

  3. 3.

    To study the correlation pattern among the different Mendeley user categories.

Data and methodology

Empirical data was gathered using quantitative data collection methods. The articles of Sri Lankan authors with citations of more or equal to 5 were downloaded from the Web of Science database in August 2021. Nine thousand one hundred thirty articles were downloaded from the WoS database, but articles without a DOI were not considered for the study. For the remaining 8547 articles (93.61% of the total article), Mendeley readership data was collected using its DOIs. Code was written in Python to extract Mendeley readership using Mendeley API. Descriptive statistics are computed for all data under study. R statistical software was used for the present study (Table 1).

Results and discussion

Correlation between Mendeley readership score and citation counts

A significant correlation value of 0.901 exists between citation count and Mendeley readership score at a 0.01 level of significance. The top 10 articles carrying the highest readership score have an average readership of 5295.1 and a citation count of 4167.7. The following ten articles have a 2372.8 and 1667 as the average readership and citation count. Likewise, the following ten articles carry 1564.6, and 1051.8 average readerships and citation counts.

This dataset's observed correlation value of 0.901 is for 8547 articles, including 32 (0.37%) articles having zero Mendeley readership score. To assess the impact of the articles with zero readership scores, we have excluded these articles and calculated the correlation. It found that no significant changes were there in the correlation value. We have two more filterings done in the dataset to make this clearer. In the first case, articles with a citation count of less than ten were excluded. The correlation was calculated; similarly, in the second case, articles with a Mendeley readership score of less than ten were excluded from the dataset to calculate the correlation value. In both cases, it's implied that a data set with minor citation and readership does not significantly influence the correlation trend.

Thelwall and Wilson (2016) also conducted a study on the Mendeley readership altmetrics based on a sample of 332,975 articles from 2009 in 45 medical fields in which 0 Mendeley readership score was traced in 22% of the sample. A strong correlation between Mendeley readership counts in almost all fields. It was also found that correlations were similar whether the articles had readership scores.

Thus, our observation is also in line with the previous studies. Mendeley readership score carries a significant correlation with citation count. The readers from Mendeley can be potential citers, and seems to be a direct cause-effect relationship, but it is not always applicable.

The 32 articles with zero Mendeley readership score carry an average citation count of 9.78. It indicates that though Mendeley readership score has a significant correlation with citation count and is helpful to get citations, it is not the cause of the citation.

WoS subject category wise analysis of the correlation

In this particular analysis, it is observed that most of the WoS subject category carries a strong significant correlation between their Mendeley readership score and citation count. A few categories like religion, social issues, archeology, cell and tissue engineering, etc. did not have sufficient data to calculate the correlation value.

WoS category with higher readership score

In Fig. 1, the correlation of citation count with overall readership score and different user categories for five subjects with higher readership is presented.

Fig. 1
figure 1

Correlation of citation count with overall readership score as well as with different user categories for 5 randomly selected subject categories with higher readership. Correlation is significant at the 0.01 level (2-tailed)

Full size image

It shows that, though the correlation (Pearson correlation, r-value) between the overall readership and citation counts for the subject with higher readership varies within the range 0.841 to 0.994, but user category-wise the ranges vary. e.g., for the researcher, the overall correlation range is 0.837 to 0.992, whereas, for the nonspecific user Bachelor and Masters category, the ranges are 0.666 to 0.990 and 0.784 to 0.993 respectively.

It can be stated from Fig. 1 that WoS subject categories with higher readership scores carry a strong correlation between Mendeley readership score and citation counts and significantly show a similar kind of trend among the different subject categories under the study.

WoS category with medium readership score

In Fig. 2, the correlation of citation count with overall readership score and different user categories for five WoS subject categories with low to medium readership scores is presented.

Fig. 2
figure 2

Correlation of citation count with overall readership score and different user categories for WoS subject category having medium readership score. Correlation is significant at the 0.01 level (2-tailed)

Full size image
Fig. 3
figure 3

WoS subject category with the highest occurrence. Double axis graph in Fig. 3. Correlation is significant at the 0.01 level (2-tailed)

Full size image

From Fig. 2, we can say that a subject with medium readership scores and a strong correlation between overall readership and citation count carries a strong correlation in most of its user categories, e.g.: crystallography. Accordingly, subjects with medium readership scores and a weak correlation between overall readership and citation count were weakly correlated in most of its user categories, e.g.: dentistry, health policy and services. Again, subjects with medium readership scores and moderate correlation between overall readership and citation count carry a weak to moderate correlation in most of its user categories, e.g.: development studies.

WoS category with a readership score of less than 200

Thelwall and Sud (2016) reveals that Mendeley is ignored in some discipline which causes the low readership score. It is shown that Business, Management and Accounting shows a more gradual decrease in correlations from about 2005.

In this study, Area Studies, Law, Mathematical & Computational Biology, Metallurgy & Metallurgical Engineering, Mineralogy, Nursing; Obstetrics & Gynecology, Otorhinolaryngology, Peripheral Vascular Disease, Robotics, Primary Health Care, Women's Studies are some subjects with readership score less than 200 and negatively correlated with citation count.

It shows that the use of Mendeley in all the disciplines of the universe of knowledge is not uniform. In the cases where the readership score is significantly less i.e. below 200, then subjects tend to show a negative correlation between readership and citation i.e. though they have very less number of readership score but their citations are not such less, in most cases it is relatively observed as very high.

WoS subject category with the highest and least occurrence

From the analyzed data it is observed that the highest number of articles belong to the WoS subject category Chemistry, Public, Environmental & Occupational Health, Engineering, Environmental Science and Physics with 574, 514, 458, 433, and 393 articles respectively.

From Fig. 3, it is visible that WoS category with the highest occurrence has less readership and citation count than that of the other categories. It is also observed that the correlation between readership and citation is highest for Environmental Science which is not the WoS category with the highest occurrence.

For the subject categories like Audiology & Speech-Language Pathology, Behavioral Sciences; Zoology, Biotechnology & Applied Microbiology, Cell & Tissue Engineering, Nursing, Oceanography, Optics, Paleontology, Architecture etc. with most minor indexed articles i.e. less than ten. It is observed that among these articles, some are negatively correlated, whereas some are not. There are 88 subject categories in our study that have less than ten articles in the Mendeley database and there is a variety in the citation count and readership score of the articles. For example, Oceanography, Nutrition & Dietetics, Physiology and Paleontology have only one article in Mendeley and their readership scores are 115, 195 and 13, respectively, but the citation counts are 2097, 758,662. Sport Science is one of the subjects with seven articles in Mendeley with a 2149 readership score and 264 citation count. Astronomy & Astrophysics has eight articles in the Mendeley database with a 464 readership score and 452 citation count.

Thus, the number of indexed articles does not affect the readership or citation. Any article from any discipline with moral relevance and quality can get readers and citations from the different user categories.

Readership score, citation count study based on the age of the article.

From Table 2, it is clear that no. of WoS indexed articles of Sri Lankan authors is increasing with time. DOI was introduced in 2000 and there is a significant increase in the no. of WoS indexed articles in the years 2001 to 2005. Readers through Mendeley have also increased in a high proportion for the articles published from 2011 onwards.

Table 1 Summary of citation count and Mendeley readership count of the analyzed data
Full size table
Table 2 Summary for the WoS indexed articles of Sri Lankan authors with a minimum 5 citation count published from 1991 to 2021
Full size table

The correlation between citation count and readership score is highest for the articles of the age category B, i.e. the articles published from 2011 to 2015.

Use of Mendeley by the readers based on their user category

From Fig. 4, it is seen that all the variables, i.e. the citation count, readership score, user category-wise readership, etc., are more or less positively correlated. There are nine significant reader categories among Mendeley users observed in the study, i.e. Lecturer, Researcher, Associate Professor, Professor, Masters, Post Graduates, Bachelors, PhD Students, and Doctoral students. Among these users, Researchers, PhD students and Masters students carry the highest mean readership score, i.e. 13.55, 12.46 and 11.94, respectively. It reflects that Mendeley is more prevalent among Researchers, PhD students and Masters students.

Fig. 4
figure 4

Correlation matrix graph between citation and altmetric variables. Dark blue circles indicate a strong correlation value between the variables. Light blue circles indicate a weak correlation value between the variables. (Color figure online)

Full size image

Researchers, PhD students and Masters Students have a correlation value of 0.98, 0.896 and 0.872, respectively, with a significant level of 0.01. The user categories having the least Mendeley readership score, i.e. Lecturer and Associate Professor, also possess a similar kind of strong correlation.

Findings

From this study the following findings are derived.

Finding 1

In some earlier papers, correlation studies are limited to a few subjects or the subjects covered by the sample journals, indicating an overall positive correlation in the subject categories (Haustein & Larivière, 2014; Maflahi & Thelwall, 2015; Parabhoi et al., 2019). This paper thoroughly investigated the correlation of the Sri Lankan articles of all the subjects published from 1991 to 2021 and found that the Mendeley readership score strongly correlates with citation count and helps identify highly cited articles. However, readership is not the causation of the citation. Subjects have a higher Mendeley readership score positively correlated with the citation. It is also true for different user categories of those particular subjects. On the other hand, in subjects with medium readership scores, the correlation between readership and citation varies from strong to weak. Subjects having a Mendeley readership score of less than 200 tend to show a negative correlation.

Finding 2

Parabhoi et al. (2019) found that the increase and decrease in readership of the articles depend on the quality of the paper and relevance to the subject domain. Subjects having more or less indexed articles in WoS does not mean that it will have likewise more or less readership and citation. Mendeley indexes only DOI-based articles, which has a significant role in capturing the readership data.

Finding 3

Many authors mentioned the popularity and use of Mendeley by different user categories, especially the students. Nevertheless, during our study, we did not find much literature discussing the correlation pattern of readership score and citation in different user categories of Mendeley. In this study, the articles of Sri Lankan authors were primarily accessed by the Researchers, PhD students, and Masters students. The subjects with an overall low or medium correlation between citation and readership count negatively correlate in some user categories. However, when generalized, all the user categories carry a more or less positive correlation between readership and citation.

Practical implication

This study highlights some facts about the potential applications and uses of Mendeley readership score in science communication. Observed that though Mendeley readership score shows a strong correlation with citation count and helps identify highly cited articles, it is not the causation of the citation. An article with a higher readership score is more likely to carry a higher citation, but this is not always true. Moreover, all the subject categories do not have much-indexed articles in Mendeley, i.e. Mendeley is not widely used by the users of all the subject categories. Among the user categories, Mendeley is used by the students from higher education only, i.e. Researchers, PhD students, and Master's students.

The following statement was made by considering all the facts revealed during the study:

Mendeley readership score used as a tool in the impact assessment of the articles in the future, but at present, it is in a very nascent stage. Now they are widely used in all the subject categories and the user categories in the present context. Thus, Mendeley readership score cannot be used as an alternative to the citation. Mendeley readership score indicates the citation, but it does not have the potential to apply and be used for policymaking or science communication in the current scenario.

Thus, this paper provides a concrete decision on using Mendeley readership scores as an alternative to the citation count. It also fulfills the need for country-specific work related to readership statistics and citation count.

Conclusion

The World Wide Web and web 2.0 are indispensable in the current era of information explosion. Library 2.0 applications are used widely across the globe. In such a situation, studying the impact of the altimetric tools becomes imperative. Mendeley is a popular reference management tool that impacts the dissemination of articles and rising readership. The present study observed a strong positive correlation between the Mendeley readership score and traditional impact assessment tool citation, except for the articles having less than 200 readerships. The articles with less than 200 readerships negatively correlate with citation counts. It contradicts the earlier results and the finding, so it's very early to consider Mendeley readership count as an absolute alternative to the citation count. Though authors and publishers use to share links to their publications on social media like Facebook, Twitter, etc., it rarely seems to lead to articles to read (Niu et al., 2010; Tenopir et al., 2012). Other altmetric tools do not have the identical readership as Mendeley, a social reference management system launched in 2008 and taken over by Scopus' operator Elsevier in 2013 (Askeridis, 2018). From the review of the users' profiles in our sample, it is clear that researchers, PhD students, and masters students are widely using Mendeley reference management tools compared to other user categories. However, to make this index more reliable, Mendeley is expected to be familiar and used by all the user categories like a citation. More micro-level studies like subject-specific, country-specific, institution-specific etc., will be required in the future to test and trace the consistency of Mendeley in impact assessment.