Comparison of Research Results by Scientific Communities

Abstract—

In the article, the issues of normalization of scientometric indicators of the publication level were studied. An integrated approach to the evaluation of research results was formalized. The similarities and differences between professional and scientific communities were considered. The concept of a professional scientific community was introduced. Local and network subtypes of scientific communities were identified. A method has been developed for obtaining the values ​​of scientometric indicators normalized by both local and network scientific communities, the distinguishing feature of which is the use of linguistic variables and production rules of fuzzy logic. The testing of the proposed methodology was carried out based on the example of a scientometric database.

INTRODUCTION

In the context of global world processes, research centers are dispersed and financed unevenly. There is a tangible difference in the cost of funding research within individual countries, for example, in leading scientific powers and developing countries. This implies a significant difference in the levels of scientific potential in different parts of the globe. In order to reduce the influence of this factor on the development of science, when conducting a comparative analysis of the results of scientific research, in addition to the absolute indicators of the level of development of science, such relative indicators are used as the share of expenditure on science in the gross domestic product. Rationing of indicators for assessing the results of scientific research allows objective comparison of them.

A characteristic trend for modern science is its internationalism. New international projects are emerging, as well as projects executed with foreign support. While earlier, besides the scientific and educational organization, the state in which the research was conducted was considered as the business card of a publication, today, although country affiliation is still of great importance, new criteria are gradually being formed. Among them is the affiliation of the authors of the study to one or another community. Here the concept of “community” should be interpreted in the broadest sense, down to the group of users of social networks.

Reaching a consensus on the scientific significance of some work requires a mutual comparison of research results carried out in different environments and, as a rule, unequally scientifically recognized communities. Such a comparison is proposed to be carried out on the basis of a method whose distinctive feature is the use of fuzzy logic and fuzzy sets. The use of these mathematical tools should provide the adequacy and reliability of the comparison of research results by scientific communities.

Our proposed method of comparative analysis of research results, whose authors belong to different scientific communities, has methodological significance for selective examination of research works carried out in the framework of competitions, reporting, awarding prizes, and awarding academic ranks and degrees.

FORMALIZATION OF APPROACHES TO THE ASSESSMENT OF THE RESULTS OF SCIENTIFIC ACTIVITIES

The 20th century was characterized by the development of the metric sciences, which include scientometrics. Recent developments in this area include wikimetry, altmetry, and librametry [1].

In connection with requests from the state apparatus, as well as its various structures and departments responsible for the state of national science, an area of ​​scientometry has been actively developing that explores the economic aspects of scientific activity.

Financing of scientific research requires the improvement of scientometric methods and criteria for the examination of research results. Such criteria should allow evaluation of the effectiveness of the expenditure of funds allocated to finance science.

A financial approach to evaluating research results implies:

((1))

where In is income and C is costs.

However, many experts directly involved in scientific activities are guided by a bibliometric approach:

$$\left\{ \begin{gathered} NP \to \max \hfill \\ NC \to \max \hfill \\ \end{gathered} \right.,$$

((2))

where NP is the number of publications and NC is the number of citations.

In turn, critics of the “pure” bibliometrics suggest evaluating the results of scientific research in a different way:

$$QR \to \max ,$$

((3))

where QR is the quality of research.

Investors, as a rule, are guided by innovative considerations for evaluating the results of scientific research:

$$IP \to \max ,$$

((4))

where IP is the innovative potential.

Thus, when evaluating the results of scientific activity, for managers, one set of indicators are important, for the researchers themselves, others are important; therefore, it is advisable to use an integrated approach, the essence of which can be expressed as follows:

$$Im \to \max ,$$

((5))

where Im is the impact.

In formula (5), the impact should be understood as the cumulative effect obtained as a result of the influence produced or the effects on various spheres of life.

Why do statistics not work when assessing the effectiveness of scientific research? As a rule, these are macro-level indicators that are not suitable for the evaluation of smaller objects [2–5].

Some authors for each direction propose to use separate scientometric databases, citation standards, including those over time. These indicators can be used to compare similar assessment objects in more or less similar areas. The reduction of the indicator value to a value within a specified range is carried out using a conversion, which is usually called rationing or normalization. Normalized indicators can be used to determine whether an object shows a result above or below the average level in relevant areas [6]. The disadvantage of normalized indicators is that they involve the maintenance of special tables that need regular revision.

CRITERIA FOR THE EVALUATION OF RESEARCHERS, ORGANIZATIONS, AND JOURNALS BY THE PROFESSIONAL SCIENTIFIC COMMUNITY

A professional community should be understood as an association that has historically been established and institutionalized as a relatively self-regulating association of professionals with a specific ethical code that is relatively independent of public opinion or formal and informal norms that are strictly adhered to [7]. In the modern world, the professional community is a non-profit and non-governmental organization that unites people based on the same interests or needs [8].

The scientific environment is not just a set of scientists, including those with the highest achievements in research activities. It includes scientific foundations that fund research projects and specialized publications, which should be considered as an arsenal of opportunities for a scientist to broadcast the results of their own research. It is possible to diagnose the involvement of a scientist in the scientific community based on the analysis of their publication activities. The use of the potential of a social environment by a scientist is a prerequisite for productive research and obtaining good scientific results [9].

A professional community can be informal, but only to a certain extent, since the usual audience of readers of scientific literature, as well as users of electronic scientific libraries and scientometric databases cannot be called informal. At the present stage of the development of science, the professional community is still an elite, including a narrow circle of people on whom the program, financial, and legal support of state policy in the scientific sphere directly depends [10]. As an example, these are various expert groups and commissions, including:

• The National Higher Attestation Commission;

• free associations (like the Russian Association for Artificial Intelligence—RAII);

• national scientific communities and organizations (such as the Royal Society of London for Improving Natural Knowledge);

• expert mega-groups.

That is how the project “The Corps of Experts in Natural Sciences” was initiated in 2007 to promote the use of bibliometric indicators as a private tool in a well-thought out system of searching for and selecting scientific experts, as well as organizing assessments and competitive procedures [11].

It seems more appropriate to separate professional and expert scientific communities, although they may merge or intersect somewhere. In the latter case, we should talk about the professional scientific community. The recognition of a researcher as a member of the professional scientific community can be expressed in the form of incentives and can take the form of:

• appointment of additional scholarships;

• selection as a corresponding member of the Academy of Sciences;

• state and international awards;

• assignment of honorary positions, titles, etc.

Alternative metrics (altmetrics) are non-traditional scientometric indicators, which are evaluated based on the functionality of web-platforms. Indicators of conventional altmetrics based on social networks and indicators derived from them cannot and should not be regarded as the opinion of the professional scientific community. A research front was formed around the questions about the use of altmetrics to determine the measure of social influence of academic information. In other words, altmetrics became an actual topic; publications on it are actively cited [12].

The attractiveness of the use of altmetrics tools at the management level is due to the possibility of taking the social effect of research results and popularization of scientific knowledge among the general public into account [13].

The main altmetrics generators are:

• Altmetric—a service that offers both paid services to corporate subscribers and free ones for individual users;

• Impactstory—a free service for individual scientists that collects information in Google Scholar, ORCID, Figshare, and GitHub;

• Plum Analytics—a service where the widest range of both assessment objects and altmetrics is presented [14].

It should be noted that not only researchers–authors of works, but also specific results of scientific research, scientific achievements, from applications that have passed the competitive selection to discoveries that received the Nobel Prize, are awarded. At the same time, it is important to know in which organization a particular researcher worked, which country they represented, etc. Accordingly, statistics on organizations, countries, and other summary features are calculated. At the same time, various prizes, awards, and regalia are not equivalent. Thus, in the Russian Federation, a doctoral degree is valued above the degree of a candidate of science and cash payments in the form of bonuses, awards, and scholarships differ in size. Despite the fact that most studies have a source of funding, not every result receives special recognition in the form of awards, prizes, etc. Apparently, the professional community is not able to respond to all publications published in a particular field over a certain period of time.

Researchers use journal impact factors as tools in planning a publication to evaluate the likelihood of gaining benefits from placing it in a journal with a high impact factor without the risk of rejection [15].

The so-called “gray” scientific literature transmits some of the knowledge that is not only complementary to international journals, but has an independent value. The content of “gray” literature finds its way into the mainstream of knowledge through secondary and highly specialized sources, as well as through new search methods generated by computer technology [16]. Some of the studies published in the “gray” literature, cause public interest, which can be traced using altmetrics.

The Altmetric Attention Score rating of the ResearchGate scientometric platform is calculated using an automated algorithm and is a weighted total of the number of citations of a research result. This rating is useful in that it makes it possible to evaluate the results of research on the basis of attention to them, but it does not allow us to say anything about the quality of the results, although the discussions associated with it can help [17].

Due to the heterogeneity of groups of users of social networks, altmetrics assess the social, applied, or educational significance of a publication, but not the fundamental significance [18].

All other research results are only reviewed and are mentioned in the research reports. Clearly, such results constitute the majority, but their importance also needs to be somehow assessed using scientometric indicators, such as Hirsch-like indicators of citations, as well as indicators derived from them [19].

A METHOD OF COMPARATIVE ANALYSIS OF RESEARCH RESULTS

The metadata of the results of scientific work are characterized in accordance with the field of science and research format. Fields of science are grouped according to the classifier built into the scientometric platform. There are various ways to systematize the sciences; however, it is problematic to choose a single standard that is generally accepted by the professional scientific community among them. The research format is a type of result of intellectual activity supported by scientometric platform. The result of a study may be in the form of a publication specifying existing subtypes, they may be a patent, and they may also take a different form depending on the capabilities of the platform, as well as the structure of the database and its contents.

The external environment in which, due to circumstances, a researcher has to work has a significant influence on the choice of the community. This may be a scientific school, localized in a certain place, or some kind of infrastructure that creates opportunities for effective scientific activity in a certain territory. In addition to the geographical location, the community can be located at the network address of a resource that provides services for scientific activities, for example, at the network address of scientometric platforms that operate on the principle of “Web 2.0” (Science 2.0). Researchers are beginning to master wiki, blogs, and other Web 2.0 technologies, going beyond the usual research experience [20]. The scientific contribution of a study is expressed through an assessment of the significance of the results, taking the amendments to the environment in which these studies were carried out into account. Thus, the characteristics of a scientific community can be determined by the task of the science field and a certain address in the form of geographical coordinates or an electronic link.

Let us introduce an additional dimension z, the attitude of some scientific community, as borders of which any indicators can be taken, that is, logical ones, ordered lists, etc. To assess the results of research, a universal set R of all existing indicators is set, equipped with methods for performing calculations, describing the requirements for input and output data formats, as well as other explanatory information.

It is more convenient to use local communities Z_geo, built on the geographical principle: city–region–country. In the conditions of the information society, scientific activities go beyond the borders of one state and often becoming international. Articles published in collaboration with representatives of an international scientific team have the greatest chances of being highly appreciated by the professional community [21].

The local scientific community, as a rule, is represented by many researchers whose affiliation of publications points to the same locality or territorial entity. In the era of the development of scientific communications, a local community Z_geo can be replaced by an “online community” Z_net, i.e., a group of users of a web resource united by a common interest; this resource is endowed with the necessary functionality for social communication. The network scientific community is an association of users of a web resource built in accordance with the concept of Web 2.0 and the ideology of Science 2.0. Researchers can communicate on the basis of political, moral, cultural, economic, or other interests [22].

The indicator r ∈ R selected as baseline may be very different. However, we must understand what objects are subject to assessment. In this case, the results of scientific research are evaluated, i.e., publications. It is possible to build indicators R based on rankings for bibliometric indicators (for local communities), altmetrics (for network communities), and various indicators derived from them, which are widely studied and reflected in the literature on scientometrics [23]. If we add some “equalizing” (“rationing” or “normalizing”) corrections to them, then we obtain a new indicator r*, on the basis of which it is possible to draw conclusions about the effectiveness of the research in relation to the level of the nearest scientific community.

We will build the methodology for comparing scientometric indicators for scientific communities guided by the scheme shown in Fig. 1, starting at the bottom elements.

Fig. 1.

An objective tree for comparing indicators by scientific communities.

We propose to calculate the deviations of the values of indicators of a scientific research result from the average value of this indicator in the scientific community z:

$$\Delta = r - {{r}_{z}},$$

((6))

where Δ is the baseline deviation; r is the baseline value relative to the research result; and r_z is the average value of baseline r in the scientific community z.

In this case, it does not matter whether the scientific community is built on a geographic basis or is virtual. Deviation can be positive (Δ > 0) or negative (Δ < 0), i.e., better or worse.

There may be several near scientific communities. If we consider the scientific community as a multitude of individuals joining it, then the intersection of these communities is possible. In addition, in some cases, one community may be nested in another, which is indicated by the sign ⊂. Due to the fact that scientific communities are interconnected, they need to be somehow united.

In particular, the “part of the world” of a community z_pow is represented by nested country z_coun, regional z_reg and city z_city communities; here, z_city ⊂ z_reg ⊂ z_count ⊂ z_pow. In turn, the network community z_soc of all users of a social science network include limited user groups by interest z_com for this resource and z_com ⊂ z_soc.

Using the mathematical apparatus of fuzzy sets, it is possible to formally define inexact and ambiguous concepts. The theory of fuzzy sets includes the concept of a linguistic variable introduced by Lotfi Askar Zadeh. Its values words or sentences of a natural language containing fuzzy values [24].

Rating scale of the linguistic variable $\hat {\Delta }$ of deviation of value of scientometric indicator r from the average level r_z in the community z in general form consists of n levels:

$$\hat {\Delta } = \left\{ {{{A}_{1}}C{{F}_{{{{A}_{1}}}}},{{A}_{2}}C{{F}_{{{{A}_{2}}}}}, \ldots ,{{A}_{n}}C{{F}_{{{{A}_{n}}}}}} \right\},$$

((7))

where A_i is the name of the i-th rating scale; $C{{F}_{{{{A}_{i}}}}}$ is the confidence factor of belonging to the i-th scale of assessment of the linguistic variable of deviation of the scientometric indicator r from the average level r_z for the community z; i = 1, …, n

To obtain the linguistic variable $\hat {\Delta }$, first the deviations Δ_z in the z-th communities are fuzzified into linguistic variables:

$${{\hat {\Delta }}_{z}} = \left\{ \begin{gathered} {{\Delta }_{z}} \geqslant 0,{{\varphi }_{ + }}\left( {{{\Delta }_{z}}} \right) \hfill \\ {{\Delta }_{z}} < 0,{{\varphi }_{ - }}\left( {{{\Delta }_{z}}} \right) \hfill \\ \end{gathered} \right.,$$

((8))

where ${{\hat {\Delta }}_{z}}$ is the linguistic variable of deviation of the scientometric indicator r from the average value r_z in the community z; Δ_z is the deviation of the scientometric indicator r from the average value r_z in the community z; φ₊ and φ_– are the fuzzification conversion.

Fuzzification φ is carried out on the basis of production rules, where the membership functions IF Δ_z ≥ p_i$\wedge \,{{\Delta }_{z}} < {{p}_{{i + 1}}},$ THEN ${{\hat {\Delta }}_{z}}$ = $\left\{ {{{A}_{i}}{{\mu }_{i}}\left( {{{\Delta }_{z}}} \right),{{A}_{{i + 1}}}{{\mu }_{{i + 1}}}\left( {{{\Delta }_{z}}} \right)} \right\},$ have a triangular form. In addition, production rules may look like μ_i. At the same time, for the linguistic variable estimated by IF Δ_z ≥ p_n, THEN ${{\hat {\Delta }}_{z}}$ = $\left\{ {{{A}_{n}}C{{F}^{{\max }}}} \right\},$ scales, it is necessary to set n parameters p₁, p₂, …, p_n (by default believing that p₀ = 0), which act as boundaries of intervals in fuzzification diagrams, as can be seen in Figs. 2 and 3. Thus, for the boundaries of the intervals $p_{1}^{ + },$$p_{2}^{ + },$ …, $p_{n}^{ + },$$p_{{ + \infty }}^{ + },$$p_{1}^{ - },$$p_{2}^{ - },$ …, $p_{n}^{ - },$$p_{{ - \infty }}^{ - },$ we obtain $\left( {{{n}_{ + }} + 1} \right) + \left( {{{n}_{ - }} + 1} \right) = {{n}_{ + }} + {{n}_{ - }} + 2$ production rules of fuzzy logic.

Fig. 2.

A fuzzification diagram for conversion of φ₊.

Fig. 3.

A fuzzification diagram for conversion of φ_–.

Positive values of Δ corresponds to fuzzification conversion φ₊, set in a positive direction.

To obtain the boundaries $p_{i}^{ + }$ of the intervals of fuzzification diagram in Fig. 2, one can use the universal formula:

$$p_{i}^{ + } = {{d}^{{i - 1}}},$$

((9))

where $p_{i}^{ + }$ is the i-th interval boundary of the fuzzification diagram in the positive direction; d ∈ ℕ; i = 1, …, n₊.

Thus, at the output of the conversion of φ₊, we obtain the following linguistic variable:

$$\begin{gathered} {{\varphi }_{ + }}\left( {{{\Delta }_{z}}} \right) \\ = \left\{ { \ldots ,{{A}_{0}}{{\mu }_{0}}\left( {{{\Delta }_{z}}} \right),{{A}_{1}}{{\mu }_{1}}\left( {{{\Delta }_{z}}} \right),{{A}_{2}}{{\mu }_{2}}\left( {{{\Delta }_{z}}} \right), \ldots ,{{A}_{{{{n}_{ + }}}}}{{\mu }_{{{{n}_{ + }}}}}\left( {{{\Delta }_{z}}} \right)} \right\}, \\ \end{gathered} $$

((10))

where φ₊ is the fuzzification conversion in the positive direction; Δ_z is the baseline deviation from the average value in the community z; A_i is the i-th level of the scale of linguistic variable values; μ_i is the membership function of the i-th level of the linguistic variable values; n₊ is the number of intervals in the fuzzification conversion diagram φ₊; and i = 0, …, n₊

Negative values of Δ corresponds to fuzzification conversion φ_–, set in the negative direction. Accordingly, to obtain the boundaries $p_{i}^{ - }$ of the intervals of fuzzification diagram in Fig. 3, one can use the universal formula:

$$p_{i}^{ - } = - {{d}^{{i - 1}}},$$

((11))

where $p_{i}^{ - }$ is the i-th interval boundary of the fuzzification diagram in the negative direction; d ∈ ℕ; i = 1, …, n_–.

At the output of the conversion of φ_–, we obtain the linguistic variable:

$$\begin{gathered} {{\varphi }_{ - }}\left( {{{\Delta }_{z}}} \right) \\ = \left\{ {{{A}_{{{{n}_{ - }}}}}{{\mu }_{{{{n}_{ - }}}}}\left( {{{\Delta }_{z}}} \right),{{A}_{{{{n}_{ - }} - 1}}}{{\mu }_{{{{n}_{ - }} - 1}}}\left( {{{\Delta }_{z}}} \right), \ldots ,{{A}_{0}}{{\mu }_{0}}\left( {{{\Delta }_{z}}} \right), \ldots } \right\}, \\ \end{gathered} $$

((12))

where φ_– is the conversion of fuzzification in the negative direction; Δ_z is the baseline deviation from the average value in the community z; A_i is the i-th level of the scale of linguistic variable values; μ_i is the membership function of the i-th level of the linguistic variable values; n_– is the number of intervals in the fuzzification transformation diagram φ_–; and i = 0, …, n_–.

Next, the linguistic variables are merged, for which purpose the operation $\otimes$ of fuzzy multiplication is selected as the objective merge function. Thus, the linguistic variable $\hat {\Delta }$ may include the combination of linguistic variables ${{\hat {\Delta }}_{1}},{{\hat {\Delta }}_{2}}, \ldots$ of the nested communities z₁, z₂, … :

$$\hat {\Delta } = \otimes {{\hat {\Delta }}_{\alpha }},$$

((13))

where $\hat {\Delta }$ is the integrated linguistic variable of the baseline deviation from the average values in scientific communities; ${{\hat {\Delta }}_{\alpha }}$ is the linguistic variable of the baseline deviation from the average value in the scientific community; and $\otimes$ is the multiplication operation of linguistic variables.

Here, the multiplication operation $\otimes$ of linguistic variables is set in the following way:

$$\hat {X} \otimes \hat {Y} = \left\{ {{{A}_{i}}\frac{{CF_{X}^{{{{A}_{i}}}}CF_{Y}^{{{{A}_{i}}}}}}{{C{{F}^{{\max }}}}}} \right\},$$

((14))

where $\hat {X}$ and $\hat {Y}$ are linguistic variables; $\otimes$ is their multiplication operation; A_i is the i-th level of the linguistic variable values; $CF_{X}^{{{{A}_{i}}}}$ and $CF_{Y}^{{{{A}_{i}}}}$ are the confidence factors of belonging to the i-th level of values of the linguistic variable X and Y; CF^max is the maximum possible value of the confidence factor, as a rule CF^max = 100; and i = 1, …, n.

However, a result is possible when all the output factors CF will be equal to zero. In this case, as an exception, the linguistic variable $\hat {\Delta }$ should include the union of linguistic variables ${{\hat {\Delta }}_{1}},{{\hat {\Delta }}_{2}},$ … nested communities z₁, z₂, using the criteria min:

$$\hat {\Delta } = \mathop {\min }\limits_\alpha \left( {{{{\hat {\Delta }}}_{\alpha }}} \right),$$

((15))

In formula (15) the pessimistic criterion min is chosen as the most rational.

For the linguistic variables $\hat {A}$ and $\hat {B}$ we set the order relation:

$$\hat {A} > \hat {B} \Leftrightarrow \phi \left( {\hat {A}} \right) > \phi \left( {\hat {B}} \right),$$

((16))

where $\hat {A}$ and $\hat {B}$ are linguistic variables and ϕ is defuzzification conversion.

In order to improve the accuracy of calculations in the event that there is a significant difference in the values of linguistic variables of nested communities, we use the normalization operation, denoted as || ||:

$$\begin{gathered} \left\| {\hat {\Delta }} \right\| = \left\| {C{{F}_{1}},C{{F}_{2}}, \ldots ,C{{F}_{n}}} \right\| \\ = \left\{ {\frac{{C{{F}^{{\max }}}C{{F}_{1}}}}{{{{\Sigma }_{i}}C{{F}_{i}}}},\frac{{C{{F}^{{\max }}}C{{F}_{2}}}}{{{{\Sigma }_{i}}C{{F}_{i}}}}, \ldots ,\frac{{C{{F}^{{\max }}}C{{F}_{n}}}}{{{{\Sigma }_{i}}C{{F}_{i}}}}} \right\}, \\ \end{gathered} $$

((17))

where $\hat {\Delta }$ is the linguistic variable of deviations of the indicator r from the average values ${{r}_{{{{z}_{1}}}}},{{r}_{{{{z}_{2}}}}},$ … in the communities z₁, z₂, … nested in z; CF_i is the confidence factor of belonging to the ith level of the scale of the linguistic variable values, while CF_i takes a value in the interval [0, CF^max]; n is the number of levels in the scale of linguistic variable assessment; CF^max is some constant, as a rule CF^max = 100; and || || is the linguistic variable rationing conversion.

Defuzzification of the linguistic variable $\hat {\Delta }$ can be carried out in various ways, for example, by the center of gravity, median, and center of maxima methods [25].

As a result, one can obtain the indicator r* of “assessment in the eyes of the scientific community” z of some result of a study, to determine which it is necessary to carry out rationing and defuzzification:

$$r^* = \phi \left( {\left\| {\hat {\Delta }} \right\|} \right),$$

((18))

where r* is the value of the baseline r normalized by scientific communities; $\hat {\Delta }$ is the linguistic variable of the deviation of the baseline r from average values in scientific communities; || || is the linguistic rationing operation; and ϕ is the defuzzification conversion of the linguistic variable.

The method we proposed is quite flexible, but the accuracy of the results will be higher if we compare publications published in the same year and related to the same type of scientific community, while the baseline is identical. This will make it possible to make the indicator r* more sustainable over time, as with the appearance of new publications in the same field, even if the value of r remains, the indicator r* is subject to change.

In this way, it is possible to compare the assessments of various scientific results in different communities, adjusted for the scientific level of each community. Moreover, it is possible to compare local and network communities with each other. However, this is not a very transparent method, since some researchers may not be familiar with the theory of fuzzy sets. In addition, with respect to networked scientific communities, the developed methods and models lead to some technical difficulties, since the built-in functionality of existing platforms for scientific communication, unlike the Web Of Science scientometric base, does not necessarily provide an estimate of the average level of indicators (in this case, most likely, altmetrics) in sections of online user communities.

AN EXAMPLE OF COMPARISON OF RESEARCH RESULTS BY SCIENTIFIC COMMUNITIES

For comparison, we selected two English-language articles on fuzzy decision-making, published in 2017 by teams involving mainly Russian (A) and Vietnamese authors (B). As a baseline, a bibliometric indicator of the number of citations in the Web of Science scientometric database was used. Assessments of the indicators r_A and r_B, given in Table 1, were taken as of May 2019.

Table 1.   Number of citations of publications

Data on scientific communities were taken from the information about the authors. A publication belongs to the scientific community if at least one of the authors of the publication belongs to this scientific community. Thus, collective articles may relate to several communities at once. However, to simplify the calculations, we will consider only those scientific communities to which the author who is mentioned first belongs. Table 2 shows the average number of citations in Moscow and the Russian Federation of articles of domestic authors in Russian, published in 2017.

Table 2.   The average values of the number of citations of publications on scientific communities associated with the authors of publication A

Table 3 contains data on the average number of citations of foreign publications published in 2017, in the city of Hanoi and the country of the Socialist Republic of Vietnam.

Table 3.   The average values of the number of citations of publications on scientific communities associated with the authors of publication B

Table 4 shows the parameters of the fuzzification conversion of baseline deviations for scientific communities, obtained by substitution d = 5 in formulas (9) and (11).

Table 4.   The boundaries of the fuzzification conversion intervals for deviations from the average number of citations of publications by scientific communities

In Table 5 we present the results of fuzzification of citation number deviations for the publication A, using fuzzy logic production rules such as

Table 5.   The fuzzification of deviations of the number of citations of publications from the average values in scientific communities associated with the authors of the publication A

Substituting the data from Table 5 in formula (13) for the publication A, we obtain the integrated linguistic variable of the citation number deviation of the publication A from the average values in the scientific communities z_city and z_count:

$$\begin{matrix} {{{\hat{\Delta }}}_{A}}={{{\hat{\Delta }}}_{\text{city}}}\otimes {{{\hat{\Delta }}}_{\text{count}}}=\{`` \text{Above}\,\text{the}\,\text{average"} \\ CF=1.035,\,{\kern 1pt} ``\text{Almost}\,\text{high}" {\kern 1pt} CF=\text{80}\text{.535}\}. \\ \end{matrix}$$

((19))

According to formula (17) for the publication A, the citation index $r_{A}^{*}$ normalized by the scientific communities is calculated:

$$\begin{matrix} r_{A}^{*}=\phi \left( \left\| \left\{ `` \text{Above}\,\text{the}\,\text{average"} \right. \right. \right.CF=1.035, \\ \left. \left. \left. {\kern 1pt} ``\text{Almost}\,\text{high"} CF=80.535 \right\} \right\| \right) \\ =\phi \left( \left\{ `` \text{Above}\,\text{the}\,\text{average"} \right. \right.CF=1.269, \\ \left. \left. {\kern 1pt} ``\text{Almost}\,\text{high"}{\kern 1pt} CF=98.731 \right\} \right)=0.748. \\ \end{matrix}$$

((20))

Guided by the same fuzzification rules, in Table 6 we obtain linguistic variables of citation number deviations for publication B.

Table 6.   The fuzzification of deviations of the number of citations of publications from the average values in scientific communities associated with the authors of the publication B

Thus, we have an integrated linguistic variable of the citation number deviation of the publication B from the average values in the scientific communities z_city and z_count:

$$\begin{matrix} {{{\hat{\Delta }}}_{B}}={{{\hat{\Delta }}}_{\text{city}}}\otimes {{{\hat{\Delta }}}_{\text{count}}} \\ =\left\{`` \text{Below}\,\text{the}\,\text{average"}{\kern 1pt} CF=91 \right\}. \\ \end{matrix}$$

((21))

Accordingly, the citation index normalized by the scientific community for publication B is equal to:

$$\begin{matrix} r_{B}^{*}=\phi \left( \left\| \left\{ `` \text{Below}\,\text{the}\,\text{average"}{\kern 1pt} CF=91 \right\} \right\| \right) \\ =\phi \left( \left\{ `` \text{Below}\,\text{the}\,\text{average"}{\kern 1pt} \,CF=100 \right\} \right)\,\,\text{=}\,\,\text{0}\text{.375}. \\ \end{matrix}$$

((22))

From this it follows that the publication by domestic researchers has a greater number of citations and should be valued more highly than the publication of foreign colleagues, given the appropriate adjustment for the scientific communities to which the authors of the publications belong.

CONCLUSIONS

The formalization of approaches to the assessment of scientific activity allows us to explain the choice of indicators that are convenient in assessing the results of research work. The opinion of the professional scientific community is also important. At the same time, it is necessary to understand what each individual scientific community represents, since all of them differ significantly in both number and competence and may not be equivalent to each other, despite the respect that the scholarly profession enjoys virtually universally.

Comparison of the performance indicators of research regarding the level of scientific communities would be useful when evaluating publications, including dissertation papers written in different places or at different periods of time. In addition, the need for such a comparison appears when deciding on the composition of participants in conferences and other scientific events when the presence of representatives of approximately the same level from different local communities is desirable.

Professional scientific communities can be guided by the formulated criteria for evaluating research results, scientific publications, and research organizations. This will make it possible to move away from the total dominance of representatives of the most reputable scientific communities in achieving the highest scientific contributions.

The method of comparative analysis of research results in scientific communities opens up new opportunities for determining the degree of compliance of Russia’s scientific priorities with the world’s scientific mainstream, as well as for monitoring our country’s international scientific cooperation in priority scientific areas, thereby contributing to the formation of an effective system of scientific organizations, an increase in their role in the socio-economic development of the country, and an increase in the prestige of national science.