Introduction

Traditionally, most scholars have primarily disseminated their work through journal articles, books, book chapters and conference presentations. In line with this, the impact of a scholar’s work has often been assessed by, or with the aid of, the number of citations to their publications from other scientific publications, often extracted from the Web of Science (WoS) or Scopus. Nowadays, however, scientists may also disseminate their research through the web in various ways, for example by listing their publications, skills or achievements in the web or in social websites. This web publicity seems to be a public good in the sense that it provides extra channels for others to find out about relevant research. Hence it seems important to assess the extent to which scholars are using the web and the social web to disseminate their research, and to find out whether particular groups are more successful at this than others.

A number of new approaches to measure research impact on the web have been proposed. Counting the number of web citations to offline publications can give evidence of research impact, since web citations correlate with traditional citations (Smith 2004; Vaughan and Shaw 2004, 2005). Download data can also give evidence of the use of articles, not just by scholars but also by other readers, such as students and practitioners (Brody et al. 2006; Neylon and Wu 2009; Watson 2009). More recently, there has been much interest in altmetrics, which are metrics derived from social web services, such as the number of registered Mendeley readers for a publication or the number of tweets about it (Bar-Ilan et al. 2012; Priem et al. 2010; Wouters and Costas 2012). It is also possible to assess the impact of web pages that are not traditional scholarly publications. For example, counts of web links or URL citations (mentions of the URL of a page in another page) can be used as an indicator, assuming that a scholar’s web presence is linked to or mentioned because it has had some type of influence (Kousha and Thelwall 2006).

Despite the apparent importance of disseminating research on the web and a number of studies that have assessed individual web or social web metrics, no studies so far have assessed the extent to which influential scientists use the social web to disseminate their research or the impact of any such attempts. The present study partly fills this gap by identifying a range of different social web presences for a group of successful researchers, highly cited researchers working at European institutions, and by assessing the impact of the social web presences found.

Literature review

Traditional approaches for measuring scholars’ impact

For many years, the WoS and its predecessors, from Thomson Reuters/Institute for Scientific Information (ISI), was the only major international multidisciplinary database of publications and citations. Since 2004 Scopus (from Elsevier), Google Scholar and Microsoft Academic Search have provided alternatives and, since then, many studies have analyzed and compared these databases. Both WoS and Scopus only allow access to subscribers and both also ignore citations from publications not indexed by them. WoS mainly indexes a large collection of academic journals, which is a disadvantage for disciplines, like computer science, where other types of publications, such as conference proceedings, are important ways to disseminate research (Bar-Ilan 2010; Goodrum et al. 2001; Zhao 2005). Nevertheless, since September 2008 WoS has included two proceedings citation databases, covering proceedings from 1990 to the present (Bar-Ilan 2010). To cover citations from books and monographs, Thomson Reuters also introduced the Book Citation Index in 2011 (see http://wokinfo.com/products_tools/multidisciplinary/bookcitationindex/, August 2013).

Scopus has a larger number of publication sources than WoS (Moed and Visser 2008) and it includes books series and conference proceedings in addition to journal articles (Elsevier 2012; Jacso 2005); however, it only offers full citation data from 1996 (Bar-Ilan et al. 2007). Google Scholar has broader coverage (Couto et al. 2009) and a wider variety of sources than WoS and Scopus, but collects its data partly by automatically crawling the web without any quality control (Aguillo 2012). It seems to be a useful tool to complement other sources, but the quality control issues undermine its value for using it alone to evaluate the impact of research (Bakkalbasi et al. 2006; Kousha and Thelwall 2007; Torres-Salinas et al. 2009). Formal citations have also been extracted from digitised books in Google Books rather than from journal articles (Kousha and Thelwall 2009).

Although citation-based metrics have limitations, such as slowness (it takes time for a publication to obtain citations) or different motivations and factors affecting citations (Brooks 1986; Cronin 1984), so that some unread papers are cited (Cronin, 1982) and some relevant or used articles are uncited (Cronin 1982; Vinkler 1987), they seem to be established as the best method for assessing research impact. However, the advent of the web has changed scholars’ behaviors. Nowadays, scientists use the web not only to communicate with each other (Shingareva and Lizárraga-Celaya 2012), but also to collect and publicize scientific information (Chen et al. 2009; Pitzek 2002; Polydoratou and Moyle 2009). A scholar’s publications could therefore appear in different web channels, such as personal websites, research group websites, institutional repositories, research blogs, disciplinary repositories (e.g. ArXiv, RePEc), or social web sites focused on academics (e.g. Mendeley, Academia.edu, ResearchGate). In this new social context, scholars also can add tags and comment on publications through the web (Neylon and Wu 2009), promoting participation and interactions between researchers. This has created the need to assess the extent to which the social web has successfully been used by scholars for research dissemination.

Adoption and use of the web in scholarly communication

The advent of the web has generated the need to understand how scholars are changing their scholarly communication practices. Whilst web use may not be extensive among scientists in developing countries (Ynalvez et al. 2005); in developed countries overall scholars are taking advantage of this new platform to collect and spread research through the web, although they use it in different ways (Barjak 2006; Jamali and Nicholas 2010; Mas-Bleda et al. 2014; Rowlands et al. 2011). There are also disciplinary differences in uptake. For instance, chemists are less prone to adopt the Web than other scientists (Brown 2007). Scholars’ web presences have been investigated to some extent, often through personal websites (Barjak et al. 2007; Dumont and Frindte 2005; Mas-Bleda and Aguillo 2013), but also through research group websites (Barjak and Thelwall 2008) and research blogs (Shema et al. 2012).

Attitudes towards social web initiatives have also been investigated. A survey of one Finnish university found that almost 40 % of 126 respondents considered Web 2.0 tools useful for scholarly communication although few used them (Gu and Widén-Wulff 2011). Other surveys have also found the adoption of social web services amongst scholars to be modest or low, including one of 1,477 UK scholars and PhD students (Procter et al. 2010). A study focused on researchers’ opinions about Web 2.0 tools (Ponte and Simon 2011) and advertised a survey in several ways (European R&D Newsroom, authors’ research group website, blog and Twitter accounts and open academic mailing lists). They found that the adoption of web-based tools by the 345 respondents who filled out the survey completely was modest or low, with less than 50 % of them using wikis (42 %), blogs (39 %), social networks (35 %), social bookmarking (26 %) and microblogging (18 %). A study of an international self-selecting convenience sample of nearly 2,000 social web using researchers (Rowlands et al. 2011) also found low use of some of these social web-based tools; however, respondents using social media associated these tools with several benefits, especially its ability to communicate quickly. This article also claimed that social web-based tools were likely to have more impact next few years.

New approaches to measure new impacts through the web: from web citations to altmetrics

Changes in scholars’ behaviors as a consequence of the emergence of the Internet have caused the need to develop new approaches to assess web research impact that might be used to complement traditional metrics. Conventional citation analyses have been attempted on the web, in the sense of counting citations to scholarly publications from web resources, finding that web citations can correlate with traditional citations (Smith 2004; Vaughan and Shaw 2004, 2005) and that they can be frequent enough to validate online impact assessment (Kousha et al. 2010). This method has been extended to specifically focus on citations from types of document that were previously not used for impact assessment, for instance, analyzing the impact of articles in teaching through assessing citations from online syllabuses (Kousha and Thelwall 2008) or from online PowerPoint presentations (Thelwall and Kousha 2008).

A second approach for assessing research impact in the web is based on usage statistics, which provide information about how many times a publication has been viewed or downloaded. Usage measures are earlier indicators of impact than citation metrics since they can start to become meaningful soon after an article has been published online whereas citations may take a few years to accumulate (Bollen et al. 2009). They have some limitations, however, such as commercial publishers being unwilling to share their usage data (Haustein and Siebenlist 2011), making usage analyses impossible except for limited collections of articles within specific repositories (e.g., arXiv). Moreover, when a reader downloads an article it is not possible to know if they eventually read it (Neylon and Wu 2009). Some studies have found that an early download counts for articles correlate with their later citations (Brody et al. 2006; Watson 2009), which validates their use for impact assessment. Conversely, it has also been shown that citations can increase the number of downloads (Moed 2005).

A third method uses web links, mentions or URL citations (see below) with the assumption that any of these can be indicators of some type of impact (Kousha and Thelwall 2006). Early studies collected web links (inlinks, outlinks, interlinks) through commercial search engines or personal web crawlers to assess the impact of academic websites by counting links pointing to them (Thelwall and Harries 2004; Wilkinson et al. 2003). These studies mainly focused on university interlinking (Bar-Ilan 2004; Ortega and Aguillo 2009; Ortega et al. 2008; Thelwall and Smith 2002; Thelwall and Zuccala 2008) and departmental interlinking (Li et al. 2005). However, hyperlink counting using the major commercial search engines has become impossible because they do not allow hyperlink searches any more (Google is a partial exception but its hyperlink search is too limited to be useful). In response, title mentions and URL citations were proposed as an alternative to hyperlinks (Kousha and Thelwall 2014; Thelwall et al. 2012). A title mention is the inclusion of a title (for example the title of a publication) in a webpage, with or without a hyperlink, while URL citations are “mentions of a specific URL in the text of a web page, whether hyperlinked or not” (Kousha and Thelwall 2014).

URL citations to journal articles have been investigated (Kousha and Thelwall 2006), detecting a low but significant correlation between the average numbers of URL citations to Library and Information Science open access journal articles and the average numbers of ISI citations for the journals in 2000. Chung and Park (2012) also used web co-mentions of 576 communication researchers to identify their web presences.

A fourth method focuses on creating impact indicators from social web-based tools, known as altmetrics. It uses information such as the number of social web readers, tags, bookmarks, comments, tweets or blog posts about publications to assess their impact (Bar-Ilan et al. 2012; Priem et al. 2010). Altmetrics can evaluate impact in a much shorter time period than conventional measures, and can also help to identify the “impact of influential but uncited work, and impact from sources that aren’t peer-reviewed” (Priem et al. 2010). Social media tools allow large-scale studies by providing open APIs or Application Programming Interfaces (Priem et al. 2011). Bibliometricians see some value in altmetrics, especially download metrics (Haustein et al. 2013) and there is already evidence that a range of altmetrics associate with traditional citations counts, with Mendeley (Li et al. 2012; Haustein et al. 2013; Zahedi et al. 2013) and Twitter (Eysenbach 2011; Thelwall et al. 2013) seeming to be the most promising sources.

There are now some websites, such as Altmetric (altmetric.com) and ImpactStory (impactstory.org), that provide a range of altmetrics for publications (Adie and Roe 2013; Priem et al. 2012). Priem and Hemminger (2010) provided a list of Web 2.0 tools classified in eight categories (bookmarking, reference managers, recommendation systems, comments on articles, microblogging, Wikipedia, blogging and other sources), recommending that they are included as part of a systematic evaluation strategy rather than in isolated form. Wouters and Costas (2012) described 16 web-based tools, providing information about their characteristics, limitations and possibilities for research evaluation, and concluding that they can be useful for scholars, but “present serious limitations that do not yet allow them to be considered systematically in research evaluation” (Wouters and Costas 2012, p. 42). Figueiredo and other authors (Figueiredo et al. 2013) assessed the quality of four textual features (title, tags, description and comments) related to four social web applications. Nevertheless, although altmetrics tools have some limitations for use in broad research evaluation at present, some are promising for measuring scientific impact (Bar-Ilan et al. 2012; Haustein and Siebenlist 2011; Haustein et al. 2013; Li et al. 2012; Mohammadi and Thelwall 2013; Taraborelli 2008; Wouters and Costas 2012).

Research questions

The purpose of this work is to assess the deployment of, and impact of, different social web presences for a specific group of scientists, highly cited researchers working at European institutions. This study follows up a previous investigation into the traditional web presences of the same group of researchers (Mas-Bleda et al. 2013). The questions guiding this research are as follows.

  1. 1.

    What proportion of European Highly Cited (EHC) scientists has a profile in Google Scholar, Microsoft Academic Search, Mendeley, Academia and LinkedIn or any content in SlideShare?

  2. 2.

    Do EHC scientists’ web presences have a measurable impact?

  3. 3.

    Are there disciplinary differences in the answer to the above questions?

Methods

In this study we first identified the highly cited scientists working at European institutions. Second, we systematically identified different web presences for these researchers. Third, we assessed the impact of these web presences.

Selection of highly cited scientists working at European institutions

The EHC scientists were taken from a previous study (Mas-Bleda et al. 2013). Highly cited researchers working in 45 European countries were extracted from the ISIHighlyCited.com database and the subsequent online directory (http://researchanalytics.thomsonreuters.com/highlycited/) which replaced it, created by the Institute for Scientific Information (ISI)/Thomson Reuters. This database contained the 250 most highly cited researchers during 1981–2008 in each of 21 disciplines. Highly cited researchers were found in only 22 of the 45 selected countries (see “Appendix”).

The ISI disciplines were grouped into five broad areas: engineering (computer science, engineering, geosciences, materials science), physical sciences (chemistry, mathematics, physics, space sciences), health sciences (clinical medicine, immunology, microbiology, neuroscience, pharmacology), life sciences (agricultural sciences, biology and biochemistry, ecology/environment, molecular biology and genetics, plant and animal science) and social sciences (economics/business, psychology/psychiatry, general social science).

There are several sample selection limitations. The discipline bias of the database used (ISIHighlyCited.com) is likely to cause the social sciences to be underrepresented; the humanities are also completely excluded. There is also a language bias in the underlying WoS data towards English speaking countries, especially the United States (Van Leeuwen et al. 2001). Although ISIHighlyCited.com provides the 250 most highly cited researchers in each of 21 disciplines, the total number of researchers in these disciplines is unknown, and so the representation of highly cited researchers may have additional disciplinary biases, even within the physical sciences. A fourth limitation is that the study is restricted to highly cited scientists working at European institutions, which represent less than a quarter of the highly cited researchers identified by ISI/Thomson Reuters.

The number of women in the population was very low (5 % of the EHC researchers) which may be due to systematic bias inherent in the method used. In order to partially correct for this source of gender bias Microsoft Academic Search (http://academic.research.microsoft.com/) was used to increase the percentage of women, since it was the only citation database that offered rankings of scientists ordered by total citations received. Deceased researchers were removed on the assumption that their web presences may not be maintained, a total of 64. The final population for this study thus consisted of 1,517 living EHC scientists, 1,360 (90 %) men and 157 (10 %) women (of which 91 were selected from Microsoft Academic Search).

Identification of scientists’ web presences

We identified two types of web presences (see Table 1): institutional web presence (personal websites and research group websites) and social web presence (profiles in Google Scholar, Microsoft Academic Search, Mendeley, LinkedIn, Academia.edu and content in SlideShare).

Table 1 Types of web presences identified and indicators used to assess their impact
Full size table

A personal website is a website (one or more web pages) in the web domain of an academic institution and created by or for a researcher. Previous studies have shown that scholars often use this to provide information about their research or teaching (Antelman 2004; Barjak 2006; Barjak et al. 2007; Björk et al. 2010; Dumont and Frindte 2005; Ponte and Simon 2011). In contrast, a research group website is a website in the web domain of an academic institution that focuses on a research group or laboratory. Scientists from some fields use this instead of personal websites (Mas-Bleda and Aguillo 2013).

Google Scholar and Microsoft Academic Search (MAS) are databases of publications and citations. Both are useful sources of information about academics’ publications but have limitations that make it difficult to use them for comprehensive research evaluation (Aguillo 2012; Wouters and Costas 2012). Authors must opt to create their own Google Scholar profiles, whereas MAS automatically creates profiles for them. Authors creating Google Scholar profiles can choose to make them public or private, but only public profiles can be found by others.

Mendeley is a free reference manager that also allows authors to list their own publications and to include PDF full text copies (Priem and Hemminger 2010). In August 2013 it had almost 2,500,000 members. It seems to be a useful source for altmetrics (Bar-Ilan et al., 2012) and is more popular than similar tools, like CiteULike (Priem et al. 2011), and its data could be used to complement citation metrics for scholarly impact assessment in some disciplines (Haustein et al. 2013; Li et al. 2012; Mohammadi and Thelwall 2013).

LinkedIn is apparently the biggest professional social network, with more than 225 million users (in August 2013). It can be a useful source of information about a researcher’s education and experience and to help contact them, but it seems that few scholars list their publications in the site. Despite not being created for academic purposes, it seems to be a popular social tool among scholars (Bar-Ilan et al. 2012; Rowlands et al. 2011).

Academia.edu seems to be the largest social network tool for academics to share research, with more than 3,900,000 academics signed up (data obtained in August 2013) in the platform since it started in 2008. Nevertheless, few studies have investigated this tool (e.g., Menendez et al. 2012; Thelwall and Kousha 2014).

ResearchGate is another important social network focused on academics but we decided to include only Academia.edu because it started earlier and had a larger number of members than ResearchGate.

SlideShare is advertised as the world’s largest platform for sharing presentations and is amongst the most visited two hundred websites in the world. In addition to presentations, registered users can also upload documents, PDFs, videos and webinars. Although authors can create SlideShare profiles, we did not seek EHC scientists’ profiles because we expected to find very few. Instead we checked if each researcher had at least one presentation (or other contents) in this tool. SlideShare was selected because presentations are part of scholars’ activities (for example for conference talks, seminar talks or teaching), but they are not usually taken into account in traditional research evaluation or they are not evaluated adequately (Kousha and Thelwall 2014).

We used Google searches to manually check whether each selected EHC scientist had a traditional web presence in the form of a personal website or a research group website. We also used searches in Google and within each site to check whether the scientists had a profile in Google Scholar, Microsoft Academic Search, Mendeley, LinkedIn and Academia, and whether they had at least one slide in SlideShare. This list was chosen to cover the most popular of a range of different social web tools: citation databases (Google Scholar and Microsoft Academic Search), reference managers (Mendeley), social networking tools (LinkedIn and Academia.edu) and platforms for sharing presentations (SlideShare). The manual searching for social web presences may have missed some, especially for academics with common names. The searches were conducted from November 2012 to March 2013.

The searches for profiles in LinkedIn, Academia and SlideShare were carried out automatically using Webometric Analyst 2.0 (http://lexiurl.wlv.ac.uk/) and then the results were checked manually.

The following steps were taken to maximise the chances of identifying all scholars’ web presences.

  1. 1.

    We searched using different versions of researchers’ names:

    • Version 1 (Name1 Surname/s). Example: Walter Fiers

    • Version 2 (Name1 Name2 … Surname/s). Example: Walter Charles Cornelius Fiers

    • Version 3 (Name1 Initials Surname/s). Example: Walter C C Fiers

    • Version 4 (Initials Surname/s). Example: W C C Fiers

  2. 2.

    Before searching for EHC scientists’ profiles, we tested each tool with existing profiles in them to discover how to search effectively within them.

  3. 3.

    We combined different versions of names with disciplines (specific and broad) and institutions (in local language and in English).

When we were not sure whether a profile belonged to a specific scientist (because there was not enough information to identify them) that profile was ignored because it would be unlikely to be useful if the scientist could not be identified from it.

Impact measures for scientists’ web presences

The impact measurement was carried out in May 2013. We counted URL citations to scientists’ web presences, using Webometric Analyst (http://lexiurl.wlv.ac.uk/) Bing API searches. For instance, if a web page has the text “Especially interesting is Hornik’s homepage (statmath.wu.ac.at/~ hornik/), which provides several publications about this topic”, then it is a URL citation for the URL statmath.wu.ac.at/~ hornik/. The results obtained from the Bing API seem to be fewer than matches returned from the online Bing web interface and so the results for this part are likely to be underestimates.

The principal drawback of URL citations counts is that “including URLs in the visible text of webpages seems to be unnatural, and it is not clear that they are a reasonable source of online impact evidence, except perhaps in special cases like articles” (Thelwall and Sud 2011, p. 1489). In spite of this limitation, URL citations were the only way to assess the impact of personal websites and research group websites, since we were not able to access other kinds of information, such as number of visits received or information about downloads. We used other indicators to assess the impact of the additional web presences, as summarized in Table 1.

Google Scholar followers were used, which was the number of people following the scientist’s profile. We assumed that a person follows the other person’s profile because an interest in their work.

Mendeley provides information about the number of readers that a specific publication has (that is, people who have added that publication to their libraries), so a way to assess scholar’s impact in this tool is the total of readers for all of a scholar’s publications. We manually searched for EHC researchers’ profiles in this tool using different versions of author names (as explained above) using the online interface.

In Academia.edu three indicators were used: followers, profile views and document views. As above, followers was the number of people following the scientist’s profile, profile views was the number of times that the scientist’s profile had been seen, and document views was the times that the scientist’s publications included on the scientist’s profile had been seen, or zero if no documents were included in the profile.

SlideShare provides useful information for impact assessment: views, likes and shares. Views counted the times that the content (slide or publication) had been seen, likes counted the number of times that someone had pressed the ‘like’ button, and shares was the number of times that that content had been shared through Twitter or LinkedIn. SlideShare lets users recommend a slide or publication on Google (using the publicly recommend on Google option), but does not report numbers of recommendations. It is necessary to have a Twitter or LinkedIn account to share a slide or publication, and a Google account to recommend it. For researchers with several slides or publications, all were totaled when calculating the indicators. LinkedIn did not provide any information to help measure the impact of a specific profile.

Results

EHC scientists’ web presences

Most of the scientists in all of the broad disciplines had a personal website (see Table 2). A smaller proportion of health and life sciences researchers had a personal website than in the other disciplines, and a larger proportion had a research group site, which suggests a higher level of cooperation among researchers in the health and life sciences. This result was also found in a previous study (Mas-Bleda and Aguillo 2013).

Table 2 EHC scientists with web presences for each discipline
Full size table

EHC researchers had a low or very low presence in social sites (see Table 2), except for Microsoft Academic Search, for which it was very high. The high presence of scientists in Microsoft Academic Search was expected, since it automatically creates profiles for researchers. Because authors’ profiles were created automatically in MAS, it will not be discussed further. Less than half of scientists in all disciplines had a presence in any of the social websites (Google Scholar, Mendeley, Academia.edu, LinkedIn, SlideShare). This was higher in the social sciences (49 %), engineering (43 %) and health sciences (40 %) than in the life sciences (30 %) and physical sciences (28 %). No EHC researcher had a presence in all of the sites and only about one-third had both an institutional web presence and a social site. This proportion was higher in the social sciences (40 %) and engineering (38 %) than in the health sciences (26 %), physical sciences (25 %) and life sciences (23 %). Of the 65 scientists with some content in SlideShare, 49 % had one or more slides, 48 % had a paper, book, book chapter or report, and 8 % provided other contents, especially CVs. This content might have been provided by the EHC researcher or by another person, such as a co-author.

LinkedIn was the social web site most used by EHC scientists. Maybe the most surprising result was the very low use of Academia.edu, especially for researchers from physical sciences, health sciences and life sciences. Concerning disciplines, social scientists had the most personal websites (82.7 %), but fewest research group websites (1 %), suggesting less importance for research groups in this discipline. Social science also had the largest proportion in all social sites, except in LinkedIn.

EHC scientists having one type of web presence were more likely to have another in many cases, especially in life sciences and engineering. For example, engineers with LinkedIn profiles also tended to have a personal website and profiles in Google Scholar and Academia.edu, and those with profiles in Google Scholar also tended to have profiles in Mendeley and LinkedIn. Physical scientists with Google Scholar profiles also tended to have personal websites and a profile in Mendeley. Health scientists with personal websites tended to also have Google Scholar and LinkedIn profiles. Life scientists with profiles in Mendeley also tended to have profiles in Google Scholar, LinkedIn and SlideShare; and those with profiles in LinkedIn also tended to have a personal website and profiles in Mendeley, Academia and SlideShare. Social scientists with profiles in Google Scholar also tended to have profiles in Academia.edu and LinkedIn.

Independent samples median tests were conducted to see if the average age of the scientists having a web presence differed from the average age of the scientists not having a web presence. EHC researchers with a personal website or a profile in Google Scholar or LinkedIn tended to be younger than those who did not (p = 0.000 for the first two and p = 0.006 for LinkedIn). The difference in medians was not large, however, only 3 years in most cases except for personal websites (6 years) and Google Scholar profiles (6 years).

Impact of EHC scientists’ web presences

The proportion of EHC scientists with evidence of impact for their online profiles (when present) was high in most cases, at least in the sense of having a non-zero impact score according to the different impact metrics used (Table 3). The exceptions are URL citations to personal websites and SlideShare likes and shares. In the case of SlideShare, the views metric is non-zero for all researchers and so the personal website is the only type of web presence investigated that does not have an associated metric which is at least non-zero for the majority of EHC researchers having that type of presence. The relatively scarcity of URL citations does not indicate that personal websites have less impact than the other types of web presence, however, because URL citations probably require far more human labour to produce than do the other metrics and are also less natural than the other metrics because hyperlinks could be created instead of URL citations.

Table 3 Proportion of web presences for EHC scientists that have a non-zero impact score
Full size table

Chi-square tests were used for every metric to seek evidence of differences between disciplines in the impact of their web presences. For URL citations, data related to researchers with at least 1 URL citation were used, finding some evidence of disciplinary differences in the proportion of personal websites with at least one URL citation (p < 0.05). For the other metrics, because the samples are small, a Chi square test was used to look for disciplinary differences between the proportion above and below the median (to maximise the power of the test) but there were only significant disciplinary differences (p < 0.05) for Google Scholar. The lack of significant results is probably due to the low numbers for the other cases.

Of the 970 EHC researchers having a personal website, 364 (37.5 %) received at least one URL citation. In terms of disciplinary differences, about half of the physical scientists and engineers received at least one URL citation, but only a quarter or less of the scientists in health sciences and life sciences obtained any URL citations. Figure 1 shows that the disciplinary differences found with the above Chi square test probably extend to higher numbers of URL citations. Overall, only 43 (4.4 %) researchers received at least 20 URL citations, 13 (1.3 %) received at least 50 URL citations and only 3 (0.3 %) researchers obtained 100 or more URL citations.

Fig. 1
figure 1

URL citations to EHC scientists’ personal websites

Full size image

Of the 143 EHC scientists having a Google Scholar profile, 101 (71 %) had at least one follower; however, only 17 (12 %) had ten or more followers and 4 (3 %) twenty or more. Social scientists were most likely to have at least one follower, although engineers were slightly more likely to have at least 20 followers, confirming the disciplinary differences found with the above Chi square test (see also Fig. 2). No relationship was found between the number of publications provided in the scientists’ profiles and the number of followers (p > 0.05) for any discipline.

Fig. 2
figure 2

Followers of EHC scientists’ Google Scholar profiles

Full size image

Only 22 EHC scientists had an Academia.edu profile, but almost all had at least one follower, half (54 %) had ten or more and a quarter (27 %) had twenty or more followers. Almost all scientists with an Academia.edu profile had at least one profile view, three quarters (68 %) had 250 or more and a less than half (41 %) had 500 or more. About half (12 or 54 %) of the 22 EHC scientists with an Academia.edu profile provided at least one publication. Almost all of them had at least one document view, half of them had at least 250 document views and a quarter had at least 1000 document views.

Mendeley provides the number of readers for a specific publication, but no indicators of the popularity of a scholar’s profile, so scientists’ impact was assessed by totaling the readers of all their publications (or their first 250 publications if EHC researchers included more). From the 93 EHC scientists having a profile in Mendeley, only 12 (13 %) included at least one publication. Eight (67 %) EHC scientists had 150 or less readers (all of them provided less than 10 publications), while the other 4 (33 %) had more than 700 readers. The scientist (from health sciences) with the highest impact had 17,935 readers, and the second one (from life science) had 8,379 readers.

All scientists with SlideShare content had the total views for that content being least one, while more than half (57 %) had total views of at least five hundred. Nevertheless, only 8 (12 %) of the scientists had content marked with a like and only 10 (15 %) had shared content.

Discussion

This investigation focuses on highly cited scientists, who are successful in traditional publishing, and it is important to know if they are also successfully exploiting the social web. This is the first study to measure the possession of, and impact of, social web presences for highly cited scientists; however, the methods used have a number of limitations, including discipline biases and language biases of the database used (ISIHighlyCited.com). ISI/Thomson Reuters has recently modified the methodology used for the identification of highly cited researchers in order to correct some of its biases and has added new scientists (Thomson Reuters 2012), so future work focused on this sample should use the new list. In addition to sample biases, there may be some mistakes in the manually collected results, such as missing websites or profiles of some scholars and incorrectly identified websites or profiles of others, particularly if they have common names. Another limitation is that the web and web use changes over time and so at some stage the results of the current study will be obsolete. Finally, some of the researchers may be long term sick, retired, in a senior management position or maybe otherwise inactive, and hence not want to maintain any kind of web profile.

Social web presences

Although EHC scientists tended to have a personal website, whether created by themselves or by their institution, few researchers had a profile in any of the social sites investigated (excluding the automatically generated MAS profiles), in contrast to a previous study of all researchers (Rowlands et al. 2011). It is known that physical scientists often collect and disseminate information using the web, especially through disciplinary repositories, such as ArXiv (physics and mathematics) or SAO/NASA Astrophysics Data System (Goodrum et al. 2001; Shingareva and Lizarraga-Celaya 2012), except for chemists (Brown 2007). Hence, a high percentage of scientists in this area might be expected to use social web sites to publicize their research. However, since only 28 % of EHC scientists used any of social sites analyzed (Google Scholar, Mendeley, Academia, LinkedIn, SlideShare), it seems that the leading European scientists have fallen behind in social web use. It would be interesting to study whether other leading non-European scientists and other non-leading researchers have better social web presences.

Disciplinary differences in web presences

Social scientists were the most likely to have both personal websites and social web presences. This finding may be an artefact of the small proportion of them in the total population of EHC scientists; however, a previous study (Barjak 2006) found that economists and computer scientists were more reliant on the web than scientists from several other disciplines, supporting the pattern found here to some extent.

Preferences for social web sites

LinkedIn was the most popular social web presence for EHC researchers (excluding MAS), perhaps because it is at least 2 years older than the others (2002). Nevertheless, a higher proportion of researchers had a LinkedIn profile in another study: 70 % of the 57 presenters at the 2010 Leiden STI Conference (Bar-Ilan et al. 2012). LinkedIn was the second most used social tool in a larger sample (Rowlands et al. 2011), but the proportion of sampled researchers using it was not reported. Only 9 % of EHC scientists had a public profile in Google Scholar. In contrast, 23 % of the 57 presenters at the 2010 Leiden STI Conference had one (Bar-Ilan et al. 2012). Classifying these as social scientists, however, then the proportion is similar to that obtained for EHC social scientists (24 %). Very few EHC researchers had profiles in Mendeley and Academia.edu. A previous study (Menendez et al. 2012) found that more users of Academia.edu (for their dataset) worked in research institutions in North America (38 %), than in Europe (21 %; more than half in the UK). This suggests that Academia.edu is not so popular for European scholars, maybe with exception of these working in United Kingdom. Very few EHC researchers (4 %) used SlideShare. Overall, then, LinkedIn is the most popular website for EHC researchers and their social web use overall may be below the average for publishing researchers.

URL citation impact of web presences

Using URL citations, the findings suggested that over a third of the personal websites of researchers had some impact in the form of at least one URL citation, and those from the physical sciences, engineering and social sciences had a higher impact than those from health sciences and life sciences. Researchers’ social web profiles did not receive any URL citations, suggesting that when someone wants to highlight a scholar or the scholar’ work, they tend to cite or mention the scholar’s institutional personal website rather than any social web presences. A possible reason might be that social web tools mainly provide a list of publications (or scholars’ presentations in the case of SlideShare), perhaps with a little extra information, but personal websites can also include other information, such as a full CV, class materials or specific software. Hyperlink counts might be a more appropriate way than URL citations to assess the Web impact (Thelwall and Harries 2004), but these cannot be calculated easily anymore.

Other impact evidence for social web presences

Indicators such as readers, followers, profile views and document views suggested that the various social web presences have had some impact. These indicators are more fine-grained than URL citations since they require less effort from the visitor to create. With the exception of Google Scholar, almost all of the social web profiles had some evidence of uptake, if not impact. Google Scholar was an exception because it only reports followers and people may use Google Scholar without registering to follow anyone. Nevertheless, these indicators can be easily manipulated and the value of each reader, follower or viewer is unclear—for instance how many viewers, readers or followers in each social website would indicate the same level of impact as a single traditional citation? The answer probably varies by site and the type of impact indicated probably also varies by site, with some perhaps being closer to traditional academic impact (e.g., Academia.edu document views for physical scientists, assuming that these views are mainly from postgraduates or researchers) and others perhaps indicating a type of educational impact (e.g., Mendeley for social scientists, assuming that the main users are students).

Finally, contents provided in SlideShare by EHC scientists were viewed many times, but few were shared or liked. It is difficult to interpret these results. Maybe contents were not liked because they were not considered good enough by users, but maybe they were considered good by users but they did not click like for other reasons, such as users not noticing the like option not having an account in order to complete the action. Similarly, the lack of sharing may be because many users did not have the necessary Twitter or LinkedIn account or did not want to log into an account in order to share the resource. In any case, it is not clear whether scholars tend to use like and share functions in the social web for academic resources.

Conclusions

Scholarly communication has been changing for decades due to the introduction of computing and the web (Liu 2003). These changes have included faster and more widespread scholarly publication, including online publication without peer review, with consequent problems evaluating the reliability of information (Kalay 2008). In this context, it seems particularly important that the most influential researchers should be well represented in the web and the social web so that their work is effectively publicized and easily accessible. The findings suggest that whilst EHC researchers tend to have a traditional web presence and an automatically generated MAS profile, they tend not to have generated any social web presences themselves, with the partial exception of LinkedIn profiles. The results also suggest that the influential scientists may lag behind other researchers in this respect. These conclusions should not be generalized beyond Europe, however. For example, Web visibility may be a particular problem in developing countries (e.g., Russell et al. 2012), and so their highly cited scholars may be less successful.

Although it is difficult to be sure of the impact of social web profiles because there is no clear way of assessing the statistics that are available for them (e.g., followers, views), it seems reasonable to suggest that they have some impact, and perhaps particularly on the new generation of researchers for whom the social web may be a natural part of their life. Assuming that it is in the interest of science to promote influential researchers as much as possible, it seems important to rectify this situation, for example by universities or scholarly organizations providing support to key figures so that they are included in the social web.

In terms of future work, the current study highlights the need for investigations into the value of the various social web indicators (e.g., followers, viewers) and to check in concrete terms, if possible, the impact that the social web is having on scholarly communication.