Introduction

Scientific performance is evaluated at essentially four different levels: the researcher, the scientific journal, the university or research organization, and the country. At each of these levels, performance indicators have been produced that are supposed to evaluate quality or “intending to” facilitate the tasks of evidence-based research assessment. Many studies [4,5,6,7, 9, 10] have compared national scientific productivity through publication success (i.e., number or proportion of papers published by authors affiliated with a given country) with the impact of these publications (i.e., number or proportion of citations received by those papers). In general, the rationale behind these studies is that whenever a country receives a larger share of citations (C) compared to the proportion of papers it publishes (P), it must produce better-than-average science or science that has a greater impact. According to the same citation-centric logic, a country for which P > C is producing sub-par science [12]. The number of citations a country’s publications receive is primarily driven by two factors: journal placement and citation performance. Research has shown that journal placement, or the quality of journal into which a paper is published, influences the visibility of the papers produced [12]. In certain journals, such as those with higher impact factors, often garner greater publicity and can attract many citations. However, less well explored is the extent to which the country’s journals have transformed the creation of knowledge and the assessment of scientific productivity.

Over the years, the number of journals indexed in citation databases has increased, although it varies significantly between countries. Indexation status matters for scholarly journal prestige and trust and the performance of journals at the international level is gauged through the global citation databases. Journals indexed by the Clarivate Analytics’ Web of Science (WoS) and Elsevier’s Scopus are considered to have a certain degree of quality and authority, and since the distribution of these databases are worldwide, journals cited in both databases can be said to have achieved a certain degree of “international visibility” [1]. The increasing proportion of indexed international journals of a country provides new venues for papers from that country to be seen by other researchers worldwide. The more internationally visible are the scientific output of a country, the higher citations can be expected. Those journals can also attract internationally authored papers to be published, which according to some previous findings [8, 11] usually attract more citations compared to domestic papers.

In this work, we evaluate publication success at the macro level by analysing the relationship between a country’s productivity and the quality of the papers it produces in Scopus during 2005–2014, and the indexing of a country’s journal in the citation database. Being indexed in a global citation database such as Scopus is a major attainment for journals worldwide and achieving this success brings with it not only a measure of country’s scientific performance, but also quality assurance of the journals to the scientific community and a number of important benefits for participating scholars that ultimately translate into greater scientific visibility, quality, and impact. The research questions to be addressed are:

  1. (a)

    What is the overall publication success, measured by the temporal evolution of a country’s indexed journals in Scopus, productivity and impact?

  2. (b)

    What correlations exist, if any, between a country’s publication success in Scopus and the quantity of the country’s journals, indexed in Scopus?

  3. (c)

    What correlations exist, if any, between a country’s publication success in Scopus and the quality of the country’s journals, indexed in Scopus?

Method

This study is based on Scopus data collected from the SCImago Journal and Country Rank (http://www.scimagojr.com). To obtain comprehensive data in terms of geo-coordinates [3] and performance metrics [14], we consider all countries that have at least one indexed journal in Elsevier’s Scopus, and published articles, reviews as well as conference papers in the period 2005–2014 in all Scopus subject areas. The full counting method was used [13] to attribute papers from the Scopus database to countries i.e. if a country appears in the affiliation field of paper, it is attributed to this country (with a weight of 1). The following information for each country for the 10 year period (2005–2014) were obtained, and we describe them as indicators for country’s publication success in Scopus:

  1. (a)

    Total number of publications (TP)

  2. (b)

    Total number of citations (TC).

  3. (c)

    Citations per paper (CPP).

  4. (d)

    Citedness rate (CR).

  5. (e)

    Proportion of international collaboration (IC).

We obtained the following information as indicators considered for the quantity of the country’s journals indexed in Scopus:

  1. (a)

    Number of indexed journals of each country in 2014 (TJ).

  2. (b)

    Number of documents published in indexed journals of each country for the 10 year period (TD).

It is important to note that the country’s publication success indicators are not just limited to that country’s Scopus-indexed journals. They also show the total performance of that country in Scopus database. For example, the total number of publications of each country includes all the publications of that country, either they have been published in domestic or foreign journals. However, the journal indicators are only limited to that country’s Scopus-indexed journals.

Measuring journal performance has always been the work of a single metric. However, since no two journals are alike, and neither are the tools that measure them, five metrics, driven by Scopus, were obtained and we consider them to be a proxy of the quality of the country’s journals. This offers a greater currency and flexibility in journal performance measurement than any single-metric method currently available. The metrics considered for the quality of the country’s journals indexed in Scopus are:

  1. (a)

    Mean citations per paper of indexed journals of each country (MCPP).

  2. (b)

    Mean SJRFootnote 1 of indexed journals of each country (MSJR).

  3. (c)

    Mean h-index of indexed journals of each country (MH-I).

  4. (d)

    Mean CiteScore of indexed journals of each country (MCS).

  5. (e)

    Mean SNIPFootnote 2 of indexed journals of each country (MSNIP).

The data related to country’s publication success and journal related indices are based on the mean value of total 10 years data. We are taking a strong assumption that Scopus has introduced indicators of the quantity and quality of scientific performance that are prestigious and can be considered for performance evaluation purpose. Data collection was conducted on June and July, 2016. Moreover, the Microsoft Excell and SPSS (version 18) was utilized for data analysis. The Pearson correlations (sig. (2-tailed)) analysis is performed to address research questions (b) and (c). When Pearson’s r is close to 1, this means that there is a strong relationship between the variables in the research questions. When r is close to 0, this means that there is a weak relationship between the two variables in the research questions. If the sig (2-tailed) value is greater than 0.05, we can conclude that there is no statistically significant correlation between the two variables in the research questions. That means, for example in research question (b), increases or decreases in a country’s publication success in Scopus do not significantly relate to increases or decreases in the quantity of the country’s journals, indexed in Scopus. If the sig (2-tailed) value is less than or equal to 0.05, it can be concluded that there is a statistically significant correlation between the two variables in the research questions. That means, for example in research question (c), increases or decreases in a country’s publication success in Scopus significantly relate to increases or decreases in the quantity of the country’s journals, indexed in Scopus.

Findings

The Overall Publication Success, Measured by the Temporal Evolution of a Country’s Indexed Journals in Scopus, Productivity and Impact

Table 1 gives a summary of the publication success of 102 individual countries, measured by the number of journals indexed in Scopus in 2014 (i.e. 22,581 journals). The journals are predominantly from Western Europe (48.9%) and North America (27.7%), with the United States and the United Kingdom dominate with a total of 11,522 (51%) journals. Even though the contribution from the peripheral countries is comparatively small, there are significant contributions from the South-East Asian countries (Malaysia 79; Thailand 26; Philippines 22; Indonesia 16). This infers that journals from these countries are international in terms of representing researches from this region. The increase of access and visibility through the national citation index system such as the Islamic World Science Citation Index (Iran) and those in Asia–Pacific countries such as the China Science Citation Index, China Social Sciences Citation Index, Indian Citation Index, Korean Citation Index System, as well as Thai Citation Index and Malaysian Citation Index has provided better chances of the country’s journals to gain indexation status in Scopus over time. Table 1 also shows the growth rate in the number of country’s journals indexed in 2014 compared to 2005. Over this period, the number of journals from Estonia has grown from just 2 in 2005 to 23 in 2014, making it the fastest growing country in terms of having indexed journals in Scopus, following by Iran (650%) and Malaysia (618%).

Table 1 Scientific performance of 102 countries, measured by the number of journals indexed in Scopus in 2014
Full size table

Figures 1a–e present the publication success of the top 20 countries based on five indicators of productivity, impact and collaboration: (a) total number of papers; (b) total number of citations; (c) citations per publication; (d) citedness rate; and (e) proportion of internationally collaborative publications during a 10 years time span (2005–2014). The trend in the top 20 most productive (Fig. 1a) and the most cited countries (Fig. 1b) show the presence of almost the same countries with slight variation in their position. US lead in terms of productivity (5,482,709), followed by China (3,134,879), UK (1,559,693), Germany (1,406,655) and Japan (1,225,808). Looking from the perspective of the total number of citations, the top countries slightly change the order as follows: US (69,027,397), UK (18,726,646), Germany (15,785,549), China (14,187,638) and France (10,667,946). With regard to the citations per publications (Fig. 1c), scientific output from Panama topped the list (16.8), followed by Island (15.5), Switzerland (15.1), Netherlands (14.4) and Denmark (14.2). However, in terms of the citedness rate (Fig. 1d) and the proportion of international collaboration (Fig. 1e), US and UK, as well as most productive and most cited countries do not appear in the top 20. The top 20 most internationally collaborative countries mainly come from those that are less developed or the periphery world, like Panama (88.5%), Mali (88%) and Madagascar (85.5%). This clearly indicates that countries that show high scientific performance in terms of productivity and impact tend to be less collaborative, vice versa.

Fig. 1
figure 1

a The top 20 most productive countries. b The top 20 most cited countries. c The top 20 countries with the highest citations per publications. d The top 20 countries with the highest citedness rate (%). e The top 20 countries with the highest international collaboration (%)

Full size image

Correlations Between a Country’s Publication Success and Quantity of the Country’s Journals Indexed in Scopus

To respond to the second research question, the association between country’s publication success indicators (TP, TC, CPP, CR, IC) and two quantity indicators of the country’s journals indexed in Scopus (TJ, TD) was investigated. Pearson correlation tests (significant 2-tailed) are applied to check the possibility of the relationship in both directions. The results of running a series of Pearson correlation tests (Table 2) indicated that TP has statistically significant and strong correlations with both TJ (r = .893) and TD (r = .912). Put it in another words, increases in a country’s total publications statistically significantly relate to increases in the quantity of the country’s journals indexed in Scopus. TC also exhibited strong correlations with both journals related indices TJ (r = .870) and TD (r = .891). Moreover, negative but statistically significant correlations were observed between IC and TJ (r = −.620) and TD (r = −.586). There are no statistically significant associations between other publication success indicators (CPP and CR) and TJ. These findings indicate that:

Table 2 Correlations between country’s publication success and the quantity of the country’s journals
Full size table
  1. (a)

    The total number of Scopus-indexed journals of each country (TJ) and the total number of documents published in Scopus-indexed journals of each country (TD) are strongly correlated with the total publications (TP) and total citations (TC) of countries.

  2. (b)

    No statistically significant associations were found between two quantity indicators of a country’s journals (TJ and TD) and citations per publications (CPP) and citedness rate (CR) of countries.

  3. c))

    Both quantity indicators of a country’s journals (TJ and TD) are negatively correlated with the proportion of internationally collaborative publications (IC).

Correlations Between a Country’s Publication Success and the Quality of the Country’s Journals, Indexed in Scopus

To respond to the third research question, we considered all the country’s publication success indicators (TP, TC, CPP, CR, IC) and five quality indicators of the country’s journals indexed in Scopus (MCPP, MSJR, MH-I, MCS, MSNIP). Table 3 presents the analysis of these ten indicators based on Pearson Correlations (2-tailed). Results of the study revealed statistically significant associations between four out of five country’s publication success indicators (TP, TC, CPP, CR) and all five quality indicators of the country’s journals indexed in Scopus. The proportion of international collaboration is the only publication success indicator with negative and not statistically significant associations with quality indicators of journals. Closer look to this table shows that MCPP indicated moderate correlations with TP (r = .560) and TC (.606) as well as weak correlations with CPP (r = .374) and CR (r = 0.289). The same trend can be seen in associations between MSJR, MH-I, MIPP, MSNIP and country’s publication success indicators. The findings indicate that:

Table 3 Correlation among the country’s publication success indicators and five indicators of the quality of the country’s journals
Full size table
  1. (a)

    All five indicators of the quality of the country’s journals have same (moderate statistically significant) relationships with both country’s publication success indicators total publications (TP) and citations (TC).

  2. (b)

    All five indicators of the quality of the country’s journals have shown positive but weak statistically significant associations with citations per publications (CPP) and the proportion of publications with at-least one citations (CR).

  3. (c)

    There are no statistically significant associations between five indicators of the quality of the country’s journals and the proportion of publications produced through international collaboration (IC).

Discussion and Conclusion

Our analysis to address the first research question on the overall publication success, measured by the temporal evolution of a country’s journals, productivity and impact, indicated that countries share of indexed journals varies significantly. Scopus-indexed journals published in the US, UK and the Netherlands are the most strongly overrepresented. This may not be surprising following the fact that some most important publishing companies are located in these countries, especially Elsevier, the owner of Scopus and one of the main scientific publishers, which is based in Netherlands, and has an obvious bias in its coverage of European journals. It was found that the growth in the number of Scopus-indexed journals in South-east Asia is encouraging, which infers that journals from these countries are international in terms of representing researches from this region. US and UK lead in terms of scientific productivity and citation impact as measured by the total number of citations received. These countries also had the highest number of documents published in their Scopus-indexed journals. However, countries with the highest citations per publications, citedness rate and internationally collaborative outputs mainly come from those that are less developed or the periphery world.

Our second question was “what correlations exist, if any, between a country’s publication success in Scopus and the quantity of the country’s journals, indexed in Scopus?” and we have analyzed the data of five indicators of each country’s scientific performance along with two indicators of the quantity of the country’s journals indexed in Scopus. Among the studied indices, we have found that total publications and total citations of 102 countries are correlated with both total number of Scopus-indexed journals and the total number of documents published in Scopus-indexed journals of countries. Accordingly, one can conclude that for a country, increase or decrease in scientific productivity and impact in time could be an artifact arising from change in country’s indexed journal in national and international levels. This finding is consistent with that of Basu [2] who reported a statistically significant relationship between India’s scientific publications in Science Citation Index (SCI) and its share of journals indexed in the Journal Citation Reports (JCR). On the other hand, no statistically significant associations were found between citations per publications and citedness rate of countries and the quantity indicators of country’s journals indexed in Scopus. The proportion of internationally collaborative publications was the only indicator of publication success that disclosed negative correlations with both quantity indicators of country’s journals.

Our third question was “what correlations exist, if any, between a country’s publication success in Scopus and the quality of the country’s journals, indexed in Scopus? and we have analysed the data of all the country’s scientific performance indicators and five indicators of the quality of the country’s journals indexed in Scopus. Among these studied indices, moderate statistically significant associations were found between all five indicators of the quality of the country’s journals and total publications and citations of countries. This finding can be explained by the fact that vast coverage and visibility of scientific developed country’s journals attract high quality research output from all over the world. As a consequence, the impact of most of these journals is high because of the large number of citations received. The associations between five quality indicators of the country’s journals and two other publication success metrics (citations per publication and citedness rate) were found to be weak but statistically significant. Moreover, no statistically significant relationship were not seen between internationally collaborated publications and indicators of the quality of the country’s journals.

Throughout the period of 10 years (2005–2014), we found that the world country’s scholarly productivity has increased and an additional set of their publications have acquired international visibility because the growth in the number of indexed journals of countries in citation indices like Scopus. Our division of publication success into two categories—the quantity of the country’s journals indexed in Scopus and the total performance of that country in Scopus database—provides a context with which the relative success of publications can be holistically assessed, yielding new insights into the scientific impact of individual countries and the performance of journals that they published. This research is not without any limitation. Further research may be required to study the publication success and journal performance of countries in other citation indices and using different indicators. Moreover, future research can explore the status of country’s publication success in different subject areas and categories.