Introduction

Increasing demand for greater public accountability have induced notable changes in the science policies of many European countries. New public management implies that national funding agencies are exercising “market-like mechanisms” mainly focused at cost-effectiveness of higher education and scientific research (Himanen et al. 2009). Government institutions have shifted from controlling the research process and its input, to the monitoring of output and outcome indicators. Although these indicators can include different sets of qualitative and quantitative data, they are mostly based on the number of publications (output) and citations (outcome). Such bibliometric indicators have proved to be very useful in developing internal quality control systems which serve as a strong political and societal foundation for basic scientific research (Moed 2009). In many European countries, bibliometric methods have provided a solid base for performance-based research funding systems and the evaluation of universities as national innovation centers (Geuna and Martin 2003; Hicks 2012b). However, constant pursue for higher research productivity has caused significant changes in the global scientific production, as well as the publishing and citing behavior of individual researchers.

Recent changes in the global scientific production

Both the effects and the effectiveness of new scientific policies should be analyzed in the context of several important changes in scientific production that have occurred during the first decade of the twenty-first century. The first was the global exponential increase in the number of scientific publications available on the Internet. The question that often arises is whether this is due to the genuine increase in scientific productivity or simply the consequence of the enhanced online visibility of publications, particularly those from non-English speaking countries (Collazo-Reyes 2013). The emergence of Elsevier’s Scopus and Google’s Scholar in 2004 have disrupted the absolute dominance of the Thomson Reuters citation indices both in the means of providing information on the latest scientific research and offering the platforms for research evaluation. Scientific production of many countries that were underrepresented in Thomson Reuters databases has suddenly become “internationalized” and more accessible globally. Web of Science (WoS) has notably expanded its coverage which was particularly evident in the area of social sciences where the number of indexed journals has grown by 42 % in the 2005–2010 period. If we consider Eastern European (EE) countries, this growth was even more imposing. EE countries were represented by 438 journals in SCI and SSCI editions of 2012 Journal Citation Reports (JCR), three times more compared to 2005. In the JCR edition for social sciences, the coverage of EE journals has grown more than six times in the 2005–2012 period. On the other hand, this is still far less than 977 EE journals currently indexed by Elsevier’s Scopus.

The second obvious trend is the growing incidence of various academic misconducts, i.e. publication and citation misbehavior. The most obvious indicator of such a trend is the constantly growing number of JCR title suppressions. There were 9 suppressed titles in 2007, 20 in 2008, 26 in 2009, 34 in 2010, 51 in 2011 and as much as 65 in 2012. Two main reasons for title suppressions are large percentage of journal self-cites and so-called citation stacking, an intensive citation exchange between recipient and donor journals. Some of the journals were not only suppressed from JCR, but were also denied further indexing in Thomson Reuters databases. Among the latest examples are two journals from Bosnia and Herzegovina which were practicing citation stacking and charging large fees for publishing articles without a proper reviewing process (Šipka 2012). However, despite the strict policy of title suppressions in WoS, more than a half of journals indexed by SSCI still have no formal misconduct policy, which is usually reflected in their lower IF values (Resnik et al. 2010). Furthermore, the fact that some of the journals suppressed from JCR (and WoS) are continuingly being indexed by Scopus and/or Scholar produces additional confusion and information bias for the research evaluation process. This is particularly alarming and may cause serious fluctuations, if bibliometric indicators are based on the so-called prestige metrics (Mañana-Rodríguez 2014).

The third important phenomenon related to the increased emphasis on bibliometric indicators is the growing urgency, motivation or the opportunity of social scientists to publish more frequently in scientific journals. Publication behavior in social sciences is known to differ from that in (basic) sciences, notably because it does not rely mainly on journal articles (Nederhof 2006). However, some recent studies show that the importance of scholarly journals as a medium of research results dissemination in social sciences and humanities (SS&H) is constantly growing, particularly in the “bordering” disciplines such as psychology and economics (Archambault et al. 2006; Engels et al. 2012; Larivière et al. 2006). Still, we need to know that social sciences were, until recently and more than any other scientific field, politically and culturally divided between the East and the West. Social scientists from the EE countries are in particularly hard situation, being torn between the local and international scientific regulations that are sometimes conflicting (Pálné et al. 2012). In some manner, usual modes of communication in SS&H are devaluated by the new research evaluation policies which emphasize the importance of “globalization”, particularly by favoring the international periodicals covered by WoS and Scopus. As Hicks (2012a) well noted, we are witnessing the co-evolution of national research evaluation policies and publishing patterns in the field of social science and humanities.

Research evaluation in Eastern European countries

Although, until recently, poorly represented in international databases, EE countries largely build their scientific evaluation policies on the data available from Thomson Reuters’ and Elsevier’s databases. Hungary has the longest tradition in using bibliometric method for research evaluation. Information Science and Scientometrics Research Unit (ISSRU) of the Hungarian Academy of Sciences, established in 1978, was a pioneering European institution in scientometrics, along with the Centre for Science and Technology Studies from Leiden and Science and Technology Policy Research from Sussex. The initial evaluations of Hungarian research institutes performed by the ISSRU and based on WoS data have demonstrated that bibliometric methods are more objective than local peer review assessments, but should be used responsibly because the inappropriate use of indicators can be equally misleading and irresponsible as not using quantitative indicators at all (Vinkler 2000).

The evaluation of research units in Poland is based on four comparison criteria: scientific achievements, scientific potentiality and tangible and intangible benefits of the scientific activity (Koczkodaj et al. 2014). Polish evaluation model presumes the possibility of different weights for each criteria but initially more than 60 % of variance is attributed to the C1 criteria, i.e. the number of scientific publications. In this category, the largest number of points is awarded for the articles published in high-impact journals indexed by WoS. A paper published in a journal with a high Impact Factor can grant a researcher up to five times more points than a paper published in a journal included in ERIH database.Footnote 1 However, the need for a special treatment of national SS&H journals is marked by the efforts to create the Polish citation index (Fenrich et al. 2013). The main purpose of the Pol-Index would be to enable production of the Polish Impact Coefficient, more appropriate for the evaluation of the so-called B journals—National SS&H journals not indexed by any major international database.

Serbia is one of the few countries with the fully functional national citation index, highly integrated into national research evaluation policy. Serbian Citation Index (Šipka 2005) covers the majority of Serbian scientific journals and provides data for their annual categorization using the large set of bibliometric indicators. However, although national impact factor is used for journal evaluation, the rules of the Serbian Ministry of Education, Science and Technological Development highly favor journals from the Thomson Reuters Master Journal List. The differences in the “values” of journal articles are huge. A paper published in an international journal ranked in the top 30 % of its category is worth as much as eight papers published in the lowest ranked national journal. Furthermore, an article published in a leading international journal is worth more than a national monograph. Current rules stipulate that from one to three articles published in WoS journals are necessary for tenure or promotion at Serbian universities and scientific institutes. One specificity is that national scientific boards are allowed to categorize a maximum of two national journals as “international” for each social science field, thus making them worth equally as a lowest ranked WoS journal in a corresponding field. Such decisions should be based on the national impact factor values, but very often those “privileged” journals are not the highest ranked national journals.

Systematic evaluation of research institutions in the Czech Republic has started in 2004, while performance-based research funding system started with the implementation in 2009 (Vanecek 2014). Current evaluation methodology is largely based on quantitative indicators which assign the largest weights to patents and articles published in non-zero Impact Factor journals. The differences among publication values are even more pronounced than in the previous examples. For example, an article published in Nature or Science is worth more than a national patent. In some cases, an article published in a top international journal is assigned 50 times more points than an article published in a journal of the “national” fields (Fiala 2013). “National” fields are those more focused on local topics and relying more on publications written in Czech language, basically most of the social sciences and humanities. Similarly to most of the other countries, articles published in journals covered by Scopus, ERIH and other major international databases are also valuated, but are worth several times less than journals from the Master Journal List of Thomson Reuters.

Slovenian Research Agency defines rules on the quality assessment and financing of researchFootnote 2 which are based on the different weight number of points for different categories of publications. Articles published in WoS journals are usually worth 20–80 points, while the articles published in other bibliographic databases, such as MEDLINE or PsycINFO, are worth ten points. An article published in a national journal not covered by any relevant international database is worth only five points. Journals covered in WoS and Scopus are treated equally in the evaluation process. The highest (A) category is awarded to articles published in the top 5–10 % of journals in the corresponding WoS or Scopus discipline field, according to the Impact Factor or the Source Normalized Impact Per Paper (SNIP). Such evaluation methodology has proved to be very useful. Recent evaluation of project proposals in Slovenia has shown that the peer review system is more reliable the more it is convergent with the scientometric indicators and in some cases, those indicators were even able to detect the conflict of interest (Južnič et al. 2010).

Croatian National Council for Science has passed several modifications of laws and rulebooks regarding the evaluation of research output during the 2005–2013 period.Footnote 3 The criteria and procedures for university promotions and tenures are largely based on the number of articles published in WoS journals (A1), but differ significantly among fields. Researchers in the field of physics and chemistry are obliged to publish 3–10 articles in the above median Impact Factor journals. In social sciences and humanities, the required number of articles in WoS journals is three times lower. In addition, articles published in journals covered by other relevant international databases (A2) are also accepted. The particular score for each researcher is calculated by using both the Impact Factor and the fractional contribution of the author, so that the first author receives more credit than the other co-authors. Current rules are obviously intended to discourage improper publication patterns of some researchers, since one of the articles states that the number of papers published in the same journal cannot surpass 50 % of the total number of papers required for the tenure, except if it is a journal from the first quartile of a particular WoS subject category. It means that Croatian researchers cannot easily grant a promotion by publishing articles in a single, lower-ranked (national) journal.

Most of the other EE countries also base their science evaluation policies on the data provided by Thomson Reuters. Bibliometric indicators in Russia are currently used somewhat superficially, focusing on the global state of science, rather than the evaluation of basic research (Varshavskii et al. 2011) However, Russian Federal Agency of Scientific Organizations has recently adopted a “roadmap” in order to improve the effectiveness of education and scientific research institutions.Footnote 4 New guidelines impose financial incentive mechanisms intended to enhance the publication activity of researchers, particularly publishing in peer-reviewed international journals covered by WoS and Scopus. As a main research performing national institution, Bulgarian Academy of Sciences is also practicing evaluation based on quantitative criteria. The evaluation was carried out by the European Science Foundation (ESF), an international association based in Strasbourg. It consists of 72 organizations from 30 European countries and focuses on R&D policies and scientific cooperation in Europe. The ESF reportFootnote 5 on research performance of the Bulgarian Academy of Sciences states that publishing in English language and in international journals are highly encouraged, but the quality improvement of national journals is also set as a priority. Romanian Research Assessment Exercise is performed under the subsidy of the National University Research Council. It is based on several principles of quality, but the publications visible in WoS and other international databases usually account for the minimum of 60 % of the total score. The effects of the Estonian Research and Development Strategy 2002–2006 were manifested through the rapid increase in the number of articles published in international journals which have grown by almost 10 % a year.Footnote 6 Research evaluation methodology of the Lithuanian Ministry of Science and Education implies the use of Thomson Reuters indices for the evaluation in sciences but not in the social sciences and humanities.Footnote 7 Similar rules are being applied by the Ministry of Education, Science, Research and Sport of the Slovak Republic, stressing the importance of publishing in journals with Impact Factor, particularly in physics and environmental sciences, but allowing the use of national publication counts in the fields which are poorly covered by international databases.Footnote 8

The effects of the new research evaluation policies in Eastern European countries

National research evaluation policies have obviously remodeled publishing behavior of scientists from the non-English speaking countries, particularly towards publishing in WoS journals (Leite et al. 2011; Ossenblok et al. 2012). EE countries have significantly (and rapidly) improved their scientific productivity and visibility in international databases in the 2000–2010 period (Vanecek 2014). However, they still lag behind the average world citation impact (Kozak et al. 2013) and lack a solid international communication network of citing journals (Pajić and Jevremov 2014). In the group of newly indexed WoS journals from EE countries, as much as 36 % of IF variation comes from the so-called “citation circles” of the other regional and national journals (Teodorescu and Andrei 2014). Some research shows that the increase in global productivity is not substantial if only top journals are considered, but are merely the effect of the authors’ increased motivation to publish in “right” journals, i.e. those with the lower Impact Factor in order to fulfill the requirements set by the national science policy institutions (Allik 2013; Mali 2011; Segalla 2008). Researchers in the SS&H fields are experiencing a serious conflict of values, being forced to fulfill formal evaluation criteria by producing articles which are endorsed neither by the humanities community nor by the wider society (Lõhkivi et al. 2012).

Although a significant breakthrough in the visibility of the EE scientific production is apparent, it would be rush to say that EE social sciences are truly globalizing. Even if the internationalization is indeed gaining momentum, one should ask what the real benefits of such a process are. The increasing use of English language only further emphasizes the core-periphery dichotomy of the global scientific area where already dominant regions of North America and Western Europe are favored even more and the autonomy of the other regions declines as their dependence on the dominant scientific centers increases (Heilbron 2013). As a consequence, scientists from EE countries are now reluctant to publish their best papers in national journals, even if they are covered by WoS or Scopus (Jokić et al. 2009). All of the above largely devaluate the credibility of bibliometric indicators based on the data available in international databases, particularly the evaluation of individual researchers in SS&H fields.

Purpose of the study

This study aims at exploring the main features and possible effects of the globalization of Eastern European social sciences, particularly the genuineness and sustainability of this process. The problem is analyzed from the perspective of diversity of Eastern European scientific production in the two major international databases: WoS and Scopus. A new bibliometric indicator called Journal Diversity Index is suggested in order to explore the differences in productivity and publication patterns of researchers from different countries. The main research problem was explored through four specific research questions:

  1. 1.

    How did international scientific production of EE countries in the field of social sciences and humanities change in the past 10 years?

  2. 2.

    What is the status of EE national journals covered by international databases and how important are they for the globalization process and the national scientific productivity?

  3. 3.

    What is the possible (rebound) effect of the current publication practice of the EE social scientists on the sustainability of globalization process and reliability of the current evaluation practice?

  4. 4.

    What are the differences in coverage of WoS and Scopus as the two major scientific information sources and what is the possible impact of these differences on the validity of research evaluation?

Data and method

The analysis has covered 20 EE countries. Data were taken from Web of Science and Scopus. The search in WoS was limited to the Social Sciences Citation Index (SSCI) and Arts and Humanities Citation Index (A&HCI). The search in Scopus was limited to the Social Sciences & Humanities subject area. All publications with the name of each country in the author’s affiliation were extracted (CU field in WoS and AFFILCOUNTRY field in Scopus). Since most of the EE countries have introduced new evaluation policies between 2004 and 2008, the year 2004 was selected as the starting year. Total number of publications, number of published articles, average number of citations per article, number of indexed journals and the number of unique journals in which authors publish articles were calculated for each country. All indicators refer to the 2004–2013 period and are based on data as of July 2014.

Using the number of published articles and the number of unique journals in which those articles appeared, a new indicator called Journal Diversity Index (JDI) was calculated for each country using the following formula:

$$ \text{JDI} = 100 \cdot \sqrt {V \cdot B} $$

JDI is basically the geometric mean of two attributes of diversity: variety (V) and balance (B) (Stirling 2007) and it can range from 0 to 100. Variety is calculated as the ratio between the number of unique journals in which country’s publications appeared (CNoJ) and the total number of “available” journals (TNoJ):

$$ V = \frac{{\text{CNoJ}}}{{\text{TNoJ}}} $$

In this formula, the value of TNoJ should offer a suitable context for interpretation. The number of “available” journals could be defined as a total number of journals in a certain database, field or country. For the purpose of this analysis, TNoJ is defined as the number of unique WoS SS&H journals in which papers from all 20 analyzed countries have appeared during the 2004–2013 period (TNoJ = 4954). Variety (V) for a county which is represented by at least one article in each of those journals will have a value of 1.

Balance indicates how uniform is the distribution of published articles across journals. Maximum value of balance should be 1 if the same number of articles are published in each journal. This is the situation when 50 % of country’s articles are published in 50 % of journals in which all of the country’s publications have appeared. JDI is calculated using the estimation of balance based on the number of journals that account for 50 % of a country’s published articles. Balance is simply a variance of proportions rescaled to range from 0 to 1:

$$ B = \frac{{{\text{CNoJ}}_{0.5} }}{\text{CNoJ}} \cdot \left( {1 - \frac{{{\text{CNoJ}}_{{{\text{o}}.5}} }}{\text{CNoJ}}} \right) \cdot 4 $$

where CNoJ0.5 stands for the number of journals that account for 50 % of published articles by the authors from a particular country and CNoJ is the total number of source titles in which the authors from a particular country have published their papers. Hence, Journal Diversity Index can be calculated as:

$$ {\text{JDI}} = 200 \cdot \sqrt {\frac{{{\text{CNoJ}}_{0.5} }}{\text{TNoJ}} \cdot \left( {1 - \frac{{{\text{CNoJ}}_{0.5} }}{\text{CNoJ}}} \right)} $$

Since bibliometric distributions are known to be highly skewed (Seglen 1992), Spearman’s rank coefficient of correlation Rho was used as a measure of relationships among JDI and other bibliometric indicators. Another important issue related to the skewness of distributions is the fact that it is hard to expect that any entity will have a maximum value of JDI (JDI = 100). However, since it is basically possible to calculate JDI for entities other than countries, it is also possible that with an appropriate selection of TNoJ and a different type of evaluated subjects (e.g. institutions), JDI range will approach its theoretical limits.

In order to further explore the diversity of scientific production and the importance of national and regional journals for the representation of EE countries in WoS, a map of countries and the most relevant publication sources for each country was generated using the Pajek visualization tool (Nooy et al. 2011) by implementing the force-directed Kamada–Kawai separate components algorithm. A class of force-directed algorithms was introduced in 1980s as an intuitive, physics-based model to visualize network structures (Kamada and Kawai 1989). After assigning different weights to nodes and different “spring tensions” to edges, the algorithm iterates to reach the optimal, minimum energy layout. In the fields of scientometrics and information science such algorithms can be used in many ways, from mapping the structure of science (Boyack et al. 2005) to the visualization of information retrieval results (Pajić 2014).

Results

Data on the scientific production of EE countries in the field of social sciences and humanities (SS&H) during the 2004–2013 period are presented in Fig. 1. When two 5-year periods are compared, it is evident that all countries have managed to substantially increase their scientific productivity. The number of publications in WoS journals by the authors from EE countries has grown by more than 140 % on average (median 116 %) in 2009–2013 when compared to 2004–2008. In the case of the Scopus database, the increment is almost 190 % on average (median 117 %). The boost of EE scientific production is impressive even when considered in the context of the global scientific growth and general improvement of journal coverage in international databases. For example, the number of publications in WoS journals by the authors from Germany has grown by “only” 51 % in the same period, from France by 62 % and from the Netherlands by 81 %. The only comparable growth was that of Spanish authors who have published 120 % more papers in WoS journals in 2009–2013 when compared to 2004–2008. However, when we consider the absolute numbers, EE countries still lag far behind most of the Western European countries. As another example, in 2004–2008, the total number of publications by the authors from all EE countries taken together was merely 41 % of those by German authors. In the 2008–2013 period, this percentage has grown to 63 %, which supports the general impression that social scientists from EE countries are gradually improving their productivity or, at least, their global visibility.

Fig. 1
figure 1

Number of publications in the SS&H fields by authors from the Eastern European countries in 2004–2013 in two major bibliographic databases

Full size image

Eastern European countries differ largely from each other in scientific production. Social scientists from Russia and Poland have published the largest number of publications in journals covered by WoS and Scopus, followed by the authors from the Czech Republic, Croatia, Hungary and Romania. On the other side, those values are practically incomparable to the low productivity of Moldova and Montenegro who are barely visible in international databases. It is in this respect that the largest relative increases in the number of publications are those of the countries that were poorly represented in WoS and Scopus during the first analyzed period. The most obvious example is that of Albania, which has increased the number of publications in Scopus by more than eight times, mainly due to the inclusion of a single regional journal. The effects of database coverage changes are also apparent in the cases of Ukraine, Romania and Serbia, but this issue will be discussed later. In addition to the growth of general scientific output, the proportion of journal articles within the total number of publication has also grown for most of the analyzed countries (Table 1). In the first analyzed period, meeting abstracts have represented a significant share of EE publications in international databases, while in 2009–2013 the proportion of scientific articles have increased significantly to the median value of 72 % in WoS and 73 % in Scopus.

Table 1 Percentage of journal articles within the total number of publications by authors from Eastern European countries in SS&H fields
Full size table

Both the growth of the total number of publications and the increase in the share of journal articles may be attributed to the WoS and Scopus coverage expansions that have occurred during the 2005–2010 period. As it was said before, this expansion was particularly in favor of regional EE journals and especially in the case of the Elsevier’s database. Data presented in Fig. 1 show that scientific productivity of EE countries is systematically larger if the Scopus database is used as an information source. However, countries’ rank positions are almost exactly the same regardless of whether the WoS or Scopus data are concerned. The rank correlation between the number of publications in WoS and the number of publications in Scopus is very high and significant (ρ = 0.98, p < 0.01). It seems that the indexing policies of Elsevier and Thomson Reuters towards EE countries are very similar. The quantitative difference is simply the effect of the larger number of national journals covered by Scopus (Table 2). It is as if the presence of EE countries in WoS is almost linearly improved in Scopus. Basically, Scopus covers most of the national journal already indexed in WoS and a significant number of additional source titles. The correlation between the number of national journals indexed by WoS and those indexed by Scopus in the group of 20 analyzed countries is submaximal (ρ = 0.97, p < 0.01). However, it should be pointed out that the absolute difference between the productivity indicators based on the data from the two major providers, which was evident in 2004–2008, is further accentuated in the 2009–2013 period.

Table 2 Scientific productivity and journal diversity of EE countries in the field of SS&H (2004–2013 period)
Full size table

In the group of 20 analyzed EE countries, the number of national journals included in SSCI and A&HCI databases correlates highly with the number of published articles (ρ = 0.92, p < 0.01). The same is true for the Scopus data (ρ = 0.93, p < 0.01). The variety of (available) publication sources seems to be a relevant precondition for the increased productivity at the national level. By broadening the circle of more accessible journals, scientific productivity of a country also improves. Data presented in Table 2 reveals that the number of unique journals (CNoJ) in which social scientists from EE countries are publishing (or are able to publish) highly correlates with the number of published articles (ρ = 0.92, p < 0.01), but also with the number of indexed national journals (ρ = 0.88, p < 0.01). However, if we take into account only the relative increase in productivity, the relationship between the number of journals and scientific productivity is not so straightforward. Although not significant, the correlation between the relative increase in productivity and the number of national journals indexed in SSCI and A&HCI is negative (ρ = −0.38, p = 0.09). Some EE countries have obviously benefited more than the others from the fact that they are now substantially represented in international databases with their national journals. As we can see from the Scopus data presented in Fig. 2, the growth of national journals coverage differs largely among EE countries. Croatia, Poland, Romania and the Czech Republic have had the sharpest increase in the number of national journals indexed by Scopus. However, while the Romanian authors have improved their scientific productivity by more than six times in the 2008–2013 period, Croatian, Polish and Czech authors have managed to merely double the number of published articles, compared to the previous 5 years. On the other hand, the number of articles by the authors from Serbia has grown more than six times, despite the fact that representation of Serbian SS&H journals in Scopus didn’t improve substantially compared to the other EE countries. Although these differences could be attributed to a number of different factors, such as the voluminosity of a journal or the publications language, it seems that the social scientists from different countries are developing different publication strategies when choosing the appropriate journal to publish an article. This issue is further analyzed by calculating the new bibliometric indicator called Journal Diversity Index (JDI) based on the WoS data.

Fig. 2
figure 2

The growth of the number of SS&H journals indexed in Scopus during the 2000–2013 period (numbers in brackets denotes the percentage growth of the number of publications in 2008–2013)

Full size image

Journal Diversity Index represents the number of journals that account for 50 % of published articles, corrected for the total number of unique journals. As we can see from Table 2, JDI and the number of journals that account for 50 % of published WoS articles (CNoJ0.5) largely differ among EE countries. Unlike the total number of unique journals (CNoJ) in which social scientists from EE countries are publishing articles, JDI correlates significantly neither with the number of published articles (ρ = −0.01, p > 0.05) nor with the number of national journals indexed in WoS (ρ = 0.08, p > 0.05). However, it does correlate positively and significantly with the number of citations per article (ρ = 0.67, p < 0.01). As a measure of diversity of publication sources, JDI is obviously more related to the impact or the effectiveness of scientific production, rather than to the simple activity of authors measured by the number of published articles. This also confirms the theory that social scientists from different countries are developing different publication strategies when choosing the appropriate journal to publish their articles. Some of them are directed towards more heterogeneous group of (higher impact) journals, while some largely rely on a smaller group of “available”, or more easily accessible journals, usually with the lower values of Impact Factor. In order to examine this assumption, journal titles that account for the majority of publications were extracted for each country and the proportions of articles published in each of those journals were calculated. Results are displayed graphically in Fig. 3.

Fig. 3
figure 3

Scientific productivity (circle size) and Journal Diversity Index (circle color) of the Eastern European countries and the most relevant WoS journal titles for each country in the fields of social sciences and humanities

Full size image

The map in Fig. 3 represents indirect connections among 20 EE countries through the most relevant WoS journals in which social scientists from those countries most frequently publish their articles. Orange nodes depict countries. The size of the node reflects country’s relative productivity, i.e. the number of published articles compared to the total productivity of all EE countries. Red color hue range indicates the JDI value. Countries with the larger JDI values are colored red (darker) and those with the lower JDI values are yellow (brighter). Green nodes denote journals, whose size also varies depending on the total number of published articles by the authors from all EE countries in the 2004–2013 period. Edges connect each country with the minimum of three most relevant WoS journals. Since the force-directed algorithm was used, the resulting map indirectly shows the mutual proximity of countries. It means that two countries are close to each other if the authors from those countries frequently publish articles in the same journals (not necessarily as co-authors). The width and saturation of the edges represent the relevance of each journal for the scientific productivity of a particular country, expressed as the proportion of the total number of articles by the researchers from that country. If an edge which connects a country with a journal is more salient, it means that the authors from that particular country rely more on the specific WoS journal when publishing their papers.

The visualization of the productivity indicator and JDI of analyzed countries has revealed several important issues related to the diversity of scientific production of EE countries. First, the most relevant WoS journals for each country are actually national journals indexed in WoS or the journals published in nearby and culturally close countries. In that sense, some countries are in a better position if they have more national journals covered. At the same time, some of the countries without their own indexed journals are also “privileged”, but only if they share the common language and/or same cultural context with those countries that have ten or more journals included in WoS. Such relations are evident in the cases of Belarus and Russia, Moldova and Romania and Bosnia and Herzegovina and Croatia. The obvious exception is Albania without such a “supporting” country, being rather distant, both culturally and historically, from the other, mostly Slavic, Balkan countries. Consequently, Albania has the largest JDI in the group of countries without a WoS indexed national journal, since the Albanian authors are obviously directed towards publishing in a wider range of foreign, English language journals. The importance of the cultural, historical and regional proximity for the international production in the field of social sciences and humanities is most evident in the case of six countries of the Former Yugoslavia, as well as the dyad formed by the Czech Republic and Slovakia. However, it seems that language is not necessarily the crucial clustering factor, since, for example, the connections among Croatia, Serbia, Montenegro and Bosnia and Herzegovina are established mainly through several regional journals which publish articles exclusively in the English language.

The second result that emerges from Fig. 3 is that Journal Diversity Index is not a measure of scientific productivity, but rather an estimate of the veracity of globalization and the sustainability of international production of EE countries. For example, Ukraine has the lowest JDI value which means that the absolute number of articles published by Ukrainian authors can be a misleading indicator of their international scientific production since almost 70 % of articles were published in a single national journal indexed in SSCI. On the other hand, the most genuine international productions are those of Hungary and Poland. Although the most relevant journals for those countries are also of national origin, they account for only a small fraction of the total number of published articles. Furthermore, Hungary is practically the only country which is somewhat isolated from the other EE countries. This can probably be attributed to the aforementioned long tradition of using bibliometric indicators and focusing on a genuine international affirmation of national (social) sciences. This supports some previous research results showing that Hungarian journals have strong international orientation, unlike WoS indexed journals from Croatia, Slovenia and Serbia that are predominantly oriented towards domestic authors and local languages (Sambunjak et al. 2008). If we analyze the color and the size of nodes simultaneously, we can see that the largest rank differences between the scientific productivity, i.e. the number of published articles and JDI are those of Russia and Croatia. Although ranked very high in the productivity among other EE countries, Russia and Croatia are among the lowest ranked countries when JDI is considered. For example, more than 50 % of articles by Croatian authors are published in just four national journals, with more than a third of articles published in a single journal.

The third very important issue concerns the status of dropped (national) journals and their influence on the credibility and validity of the current evaluation practice in EE countries. By focusing on a limited number of indexed national journals, the authors largely devaluate the genuineness and sustainability of the country’s international scientific production. Again, the most prominent example is Ukraine, whose production was almost exclusively based on a single national journal Actual Problems of Economics. The alarming fact is that this journal was dropped from the Thomson Master Journal List in 2013, after what Ukraine will probably be barely visible on the international scene and ranked much lower among the other EE countries. Croatian Collegium Antropologicum, that accounted for 37 % of Croatian and 34 % of Bosnian articles in SSCI and A&HCI, was also dropped in 2014 and this will largely affect the international visibility of the science originating from those two countries. Other similar examples, although not as radical as the ones before, are Romanian European Journal of Science and Theology, Polish New Educational Review, Estonian Eesti Rakenduslingvistika Ühingu aastaraamat and Slovenian Slavistična Revija. Yet another dropped journal visible on the map should amend this list. Nigerian African Journal of Business Management, along with Actual Problems of Economics, has accounted for 31 % of Serbian articles in the field of economics.

Another important issue related to the fact that not a small number of national journals were dropped from further indexing in Thomson Reuters databases, is the use of Scopus as a data source for evaluation purposes. Although the international affirmation of EE journals primarily depends on editorial behavior and management, database indexing policies can also influence both their global reputation and the validity of local evaluation practice. Elsevier obviously has slightly different indexing policy compared to Thomson Reuters and is focused more on the massive indexing of local and regional journals. This raises the question of differences in the quality criteria of two database providers. Most of the aforementioned journals, although no longer included in WoS, are continuingly being indexed by Scopus. The most obvious examples are Ukrainian Actual Problems of Economics and Croatian Collegium Antropologicum. Table 3 presents the list of journals that account for the largest shares of articles published by the authors from EE countries in Scopus database. As it was mentioned before, the most impressive relative increase of the Albanian scientific productivity can be attributed to the inclusion of a single regional journal of dubious quality.Footnote 9 In the case of countries which have based their evaluation methodology on the Scopus data, this can cause a serious bias in bibliometric indicators and send a negative message to the local academic community. This is probably the reason why most EE countries still do not treat Scopus and WoS journals equally or do not take Scopus journal list into account in the evaluation process.

Table 3 The list of Scopus journals that account for the largest percentage of articles published by the authors from EE countries in the fields of SS&H in the 2009–2013 period
Full size table

Discussion and conclusions

The main purpose of this study was to explore the proportions, sustainability and genuineness of the growth in international scientific productivity of EE countries in the field of social sciences and humanities. There are three possible conclusions, aimed at different stakeholders, which can be derived from the presented results. The first one concerns national science policy institutions and their current evaluation practice. Although the evaluation process based on bibliometric indicators is believed to be more adequate the more it is formal and stimulating (Moed 2009), it seems that the policies of (some) EE countries are too formal in stimulating futile publication behavior, aimed primarily at quantity, rather than quality. We have confirmed some previous research results showing that the countries which profit most in terms of percentage of published documents, tend to show a decline in their average citation rates (López-Illescas et al. 2009). By imposing the pressure on authors to publish in WoS journals, even larger pressure has been put on newly indexed national journals. And the pressure to publish means that reviewing process is not always sufficiently rigorous (Macdonald and Kam 2007). Instead of taking the opportunity to enhance the quality of national journals indexed by WoS and thus improve the true global visibility of EE social sciences, we are witnessing the segregation of EE national scientific productions which mostly rely on regional, language and cultural proximities. This supports some of our earlier findings that journals from EE countries are primarily being used for local promotions and formal fulfillment of policy rules, rather than the true promotion of national science, thus being only locally international, but globally national (Pajić and Jevremov 2014). Unfortunately, we have to agree with other authors from the region that the current evaluation practice of some EE countries is obviously hindered by the fact that the interests of various lobbing groups, in and outside of science, cannot be fully channeled (Mali 2011).

The second conclusion should concern the bibliographic database providers. In the lack of appropriate data sources, “scientifically small” countries were easily directed towards WoS as the main gatekeeper of scientific excellence. The long lasting and successful “marriage” between Thomson Reuters and bibliometric practice was only reinforced by the emphasis of national science policies on publishing abroad (Archambault et al. 2006). However, rather liberal indexing policy of new bibliographic databases and the growing number of journals dropped from WoS, suggest that the massive coverage expansion in the past 10 years is yet to show its true effects. The presented analysis has shown that at least two EE countries will substantially deteriorate their international rankings in scientific production as a consequence of Thomson Reuters’ indexing policy decisions. The first important question is whether the initial selection criteria were consistent and rigorous enough or whether some countries were favored and unduly overrepresented in WoS (Kosanović and Šipka 2013). The other question is how national policy institutions should handle the differences in coverage policies of Thomson Reuters and Elsevier, since those differences could be quite confusing when it comes to the bibliometric evaluation at the national level. Leading database providers should be aware that with the radical improvement of electronic publishing and self-archiving systems, their principal role is not recognized as the mere knowledge dissemination, but mainly as a quality control mechanism. Thomson Reuters seems to have more rigorous, but often inconsistent policy on this matter than Elsevier. The national scientific policy institutions should promote national journals by being more responsible and more involved in controlling their editorial practice, but it is hard to believe that science evaluation is nowadays possible without a solid and reliable international data.

The third conclusion may be of interest for the theory and practice of scientometrics. This article has contributed with a new indicator which could be used as a form of “corrected” measure of scientific productivity. It was shown that Journal Diversity Index is not related to the mere production nor the number of national journals indexed in WoS, but does have the potential to predict the average citation rate of articles. In this sense, it could be used as a measure of true globalization of scientific production of different entities and in this case, as a measure of sustainability of international production in EE countries. Additionally, the article has further explored the publication patterns in the field of social sciences and humanities, particularly the role of (inter)national journals covered by the major bibliographic databases. It seems that the aforementioned direction of social scientists (at least those from the EE countries) towards publishing in international journals is mostly the effect of the new national evaluation policies and the coverage expansion of international databases. The presented results have shown that, in the case of EE countries, only communication channels are internationalized, but not the communication itself. Social scientists from most of EE countries still rely heavily on the accessibility of national and regional journals. However, a decisive factor of accessibility is not the common language, but the cultural proximity or the ability to publish articles more easily. Although similar patterns of regional connections were already detected in the European region as a whole (Almeida et al. 2009), they are obviously more apparent in the case of EE countries and social sciences. It seems that social scientists from EE have two possible options as a response to the new research evaluation practice in their countries. The one is to focus on high-impact international journals. Apart from being the harder way to globalize the production, this option carries the risk that by preferring foreign journals the authors will move the focus on topics that are interesting for international auditorium and thus create a gap between academics and general public of their own countries (Lõhkivi et al. 2012; Schuermans et al. 2010). The other option is to (ab)use the existence of regional journals in international databases, but in this case the question arises what is the true quality and impact of such production and what is the true benefit of being visible, but not truly recognized internationally.