Keywords

1 Introduction

In developed countries, scientific endeavors are primarily supported by expenditure in the business sector. When arranged by country, the share of the total amount of funding in 2016 accounted for by such expenses was 78% in Japan, 74.7% in China, 65.6% in Germany, and 64.2% in the USA [1]. Conversely, funding in research, development and elaboration projects in Russia is accounted for mainly by state investment and in 2016 accounted for 68.2%, of such expenses. By comparison, the share of science funding in that year by the entrepreneurial sector amounted to 28.1%. Irrespective of these figures, the principal mechanism to support critical research initiative projects in Russia comes mainly from competitive funding efforts by way of scientific foundations [2, 3].

One should note the dissimilarity between the kinds of fundamental research which is funded by grants from scientific foundations, and the practical orientation research that is supported by other sources, e.g., state investment programs. As a rule, prospective fundamental research is motivated by natural ingenuity and the progress of material developments in a field and, in this way, provides an incentive towards the expansion of scientific knowledge. In 2016 in Russia the share of grants from foundations that support scientific, technical and innovative research accounted for only 2.4% [1]. By contrast, in the USA the same sector share is about 4%Footnote 1. It is evident that in Russia the foundations grants share should be significantly increased in the coming years. This relates to the significant enhancement in scientific funds regulations and activities, as well as the improvement in the general mechanisms for competitive project selection [5].

Nowadays, in scientific foundations, in order to make decisions concerning the allocation of fundamental research support funds, multiple models of professional expertise are used. Various models can be utilised in several separate competitions within the same foundation. In this regard, one of the essential tasks of a scientific foundation is to choose and organize the best form of projects selection relative to the competitive financing system used. One of the most common approaches to solving this problem is an anonymous public review (peer review) carried out by scientists in expert roles in that particular field of science [6,7,8,9]. As a rule, the review is carried out via an electronic system by several independent experts who are selected for the review procedure at random. An alternative process is to adjudicate on each individual project success or failure by means of an expert council – a collegial group permanently composed of reputable scientists. In this case, if the identities and other information about the scientists involved in the peer-review process is hidden, the information about the expert council members is open and publicly available.

The chosen approaches are single-stage procedures, however, in reality, mixed (hybrid) multi-stage combinations of methods are used. Hybrid multi-stage methods are necessary for the purpose of increasing and varying the representation and independence of scientific project evaluations. For example, the final decision concerning project support can be made by the expert council based on the anonymous peer review results. In this case, experts can be assigned ranks that reflect their qualifications. It should be noted that regardless of the evaluation methods and models used in scientific foundations, the examination of projects is carried out by qualified members of the scientific community. The main actors are scientists, and based on their set of opinions, the ultimate decisions concerning funding support or refusal is formed. Such scientific foundation entities, in turn, provide a platform for scientists to organize expert evaluation processes and carry out the distribution of funds, and act as an agent between the state and the scientific community (who may collectively be considered to represent ‘science’ as a field of endeavor).

Questions regarding the development of scientific foundation entities and the identification of appropriate independent research expertise have been raised repeatedly in the economic literature [10,11,12,13]. Several researchers have also pointed out the shortcomings of the examination methods used in the research projects, in particular those applied in the widely used anonymous public review method [14,15,16,17,18,19]. These authors have emphasized that one of the reasons for criticism of this method is its failure to take into account the individual economic and psychological characteristics of the adjudicating scientists. For an analysis of this hypothesis, the authors of this composition propose to use agent-based modeling tools [20, 21]. Such tools allow for a flexible approach to the construction of dynamic models of socio-economic systems functioning as a result of an individual economic agent’s behavior. Examples of other classes of simulation models where we can see the impact of a participant’s psychological characteristics on project result are presented in [22].

In a research project funded by the Russian Science Foundation (project No. 17-78-30035), the authors built an agent-oriented model of professional expertise and decision-making focusing on the support of scientific research projects, which takes into account economic and psychological characteristics of participants of this process. The authors use general data and provide results based on experiential simulations.

2 Professional Examination and Decision-Making in the Appraisal of Research Projects: Model Assumptions

The structure of the scientific community in Russia is heterogeneous. It is conditioned, on the one hand, by natural-geographical factors (particularly the spatial extent of the country), and on the other hand, by the intrinsic institutional features inherent in the scientific community itself. Depending on the interests and position of the observer, one can identify the following within the landscape of the scientific community: (a) institutional macroeconomic agents such as the Russian Academy of Sciences (RAS), Ministry of Science and Higher Education of the Russian Federation, scientific foundations (RFBR, RSF); (b) microeconomic level agents – universities, institutes of the Russian Academy of Sciences, private educational institutions, publishing houses of scientific literature; (c) agents of nanoeconomic level – individual scientists and their small groups (associations). At the same time, agents, regardless of what level of the economy they belong to, are endowed with a wide range of characteristics that influence their decision-making processes.

In this paper, which is devoted to the analysis of competitive financing mechanisms and professional examination of research projects, the authors focus on modeling the behavior of agents at the nanoeconomic level. Herewith we do not consider the relationship between the activities of agents of various levels in real life, (see for details on this aspect: [23]). Thus, in the proffered research model, there are two main types of objects: researchers (scientists) and research projects (works). Below is a description of the characteristics of these objects.

The first type of object are the researchers who make up the scientific community. Each researcher can either play the role of the producer of research projects (that are subject to decision-making by the scientific foundation on the question of financing), or the role of a reviewer – an expert who evaluates research projects submitted to the competition. The scientific community is heterogeneous: there are scientific schools in it, uniting both some performers and some reviewers. The results of project reviews may be influenced by the affiliation of the reviewer and the producer with one particular scientific school.

The basic economic and psychological characteristic of this type of object, whose influence on the examination processes of research projects we are interested in, is the oppositional binary “individualism – collectivism”. Accounting for this characteristic in the model is carried out by assigning researchers to one of two disjointed sets: the class of dependent (collectivism) or the class of independent (individualism) researchers.

It is assumed that researchers in the first class (dependent) are influenced by the scientific school. Reviewers, in examining the research projects of producers related to the same scientific school, provide these producers with additional support, and overestimate their merits. Researchers belonging to the second class (independent), on the contrary, carry out a fair examination of the research projects of other researchers, regardless of the class in which the latter are included. In other words, if the reviewer is not a member of the scientific school, his rating does not depend on the affiliation of the project’s producer to the scientific school.

Each researcher also possesses certain specific qualifications, has their own reputation, and feels a certain satisfaction or dissatisfaction from their activities. Qualification (qualification) affects the ability of the researcher to produce and examine research projects; reputation (reputation) reflects the degree of respect for this expert and the credibility of his opinion from the scientific community; and, satisfaction (satisfaction) shows the perception by the researcher of his position within the scientific community and his attitude to his reputation. These characteristics are randomly distributed among experts in the range from 0 to 1.

The second type of objects – research projects – are tied to the first type of objects – researchers. Each research project is characterized by two features: the value of the idea behind the project (idea) and the quality of its execution (quality). The value of the idea is set randomly in the range from 0 to 1. The quality of the execution of a research project is determined by the underlying idea and qualification of the researcher representing the project for the competition. Qualification determines his ability vis-a-vis the implementation of this idea: quality = idea ∙ qualification. The quantity of research projects, both submitted for the competition and already carried out by researchers, is limited and does not exceed 50% of the total number of researchers. This provision reflects the necessity of taking into account the order of receipt of projects in the scientific foundations. Each researcher can submit only one project to the competition.

In the model proposed below, the process of research projects rating by dependent and independent experts is simulated. The duration of the simulation is limited to 100-time cycles, each of which is identified with a year. The population of researchers is not updated; the life expectancy of each researcher is assumed to be 100-time cycles. Choice of researchers submitting research projects for the competition, out of the total number of researchers, each tact takes place randomly, taking into account the restriction on the number of projects examined. Research projects that won the competition are executed during 3-time cycles.

3 Mechanisms for the Rating of Scientific Projects by Researchers, Realization of Feedback, and Conceptual Model Scheme

As mentioned above, the entire category of researchers in any individual time slot is divided into two equal parts – the producers of research (performers) and the review researchers. A researcher becomes a performer if his research project succeeds in the competitive process. In other words, some researchers are engaged in the preparation and implementation of research projects, while the other part rates the projects participating in the competition. The rating procedure is carried out in the following way: each project applying for funding is assigned a team of three randomly selected reviewers. As the researchers are divided into two classes – dependent and independent – we get four possible combinations of pairs within the category “reviewers of scientific project”, each of which has its own assessment rules (see Table 1).

Table 1. Rules for examining research projects depending on the “class affiliation” of reviewers and producers of research projects.
Full size table

The assessment of the first pair is implemented in the following way. The dependent reviewers seek to support the research projects of other researchers who are fellow members of their scientific school. Each dependent reviewer forms his opinion (opinion) on the project of another dependent researcher, with a value which is randomly set from 0 to 1. At the same time, the dependent reviewer cannot increase the final assessment of the research project by more than 50%, therefore, taking into account the opinion of the dependent reviewer, the assessment of the dependent researcher is formed in accordance with the formula rate = quality ∙ qualification ∙ (1 + 0.5 ∙ opinion). Thus, the component representing his personal attitude (subjectivity) is added to the reviewer’s basic assessment (objectivity). A unified assessment system operates for the three remaining pairs: a research reviewer assesses the quality of a scientific project, based on his qualification rate = quality ∙ qualification. If the final score of a scientific project rate by the reviewer is ≥0.6, then the variable result (which represents the decision to support the research project) is given 1 additional point. A scientific project is deemed to be supported in a case where the variable result has a value of ≥2, i.e. two reviewers out of three consider the project to be reasonably supported.

The mechanisms for changing the status of participants in the assessment after each slot are implemented through changes in the reputation and satisfaction of the researchers (see Table 2).

Table 2. Changing the satisfaction and reputation of researchers depending on winning or losing the competition.
Full size table

In cases where a dependent producer wins the competition by means of the support provided by a dependent reviewer, the reputation of the scientific project producer increases by 0.3 points. If a scientific project wins the competition without additional support from dependent reviewers, then the reputation of the project producer increases by 0.5 points. For all researchers whose projects win the competition, satisfaction increases by 0.3 points, and for those whose projects were not supported, the reverse process takes place and the satisfaction value decreases by 0.1 points. At the same time, the reputation of the dependent reviewer is subject to change. If the scientific project of the dependent producer wins in the competition, then the reputation of the reviewer is reduced by 0.1 points, which represents the loss of confidence in him on behalf of the scientific community.

We also note that the reputation and satisfaction of researchers cannot fall below zero. In this case, the researcher is considered satisfied if the parameter for satisfaction is >0.

Accounting for the psychological characteristics of agents not only allows for the modeling of the project rating results in a more accurate manner, but also the discovery of new psychological structures in the scientific community. These structures arise in connection with a change in the estimated indicators of the researchers’ activity, and adds to the characterization of their reputation (and the satisfaction of the current situation in the scientific community) after each slot. Based on these indicators, the researchers are arranged in small groups that can provide the most pleasant psychological atmosphere and comfort for the researcher. This matter is concerned with the formation of peculiar groups of “friends” (“neighbors”) surrounding each researcher in the “satisfaction – reputation” space. Initially, such groups (numbering between four to nine researchers) are formed randomly. Depending on the situation, the researcher may try to choose another environment for himself and leave his group.

If a researcher is satisfied with his current position in the scientific community, his satisfaction value is >0, and he maintains his membership in this group. If a researcher is not satisfied with his current position in the scientific community, then his satisfaction value = 0, and he conducts an analysis of the participants reputation in his surrounding group. In cases when the total reputation of group members belonging to the same dependency class (independence) as the researcher is more than the total reputation value of the participants from another group, the researcher compares his reputation with the average reputation of all members of his environment. If the average reputation of the environment is lower than that of the unsatisfied researcher, his satisfaction increases by 0.1 points. Otherwise the researcher leaves his group and is seen to exist at large in the scientific space (whereby he finds other neighbors), and his satisfaction also increases by 0.1 points. In a situation where the total reputation of the neighbors belonging to the same class is less than the total reputation of the neighbors from another class, the researcher carries out a class change, i.e. the transfer from a set of dependent to a set of independent neighbors or vice versa, after which the satisfaction of the researcher also increases by 0.1 points. The scheme of the described examination model and the decision-making on the support of scientific projects is presented in Fig. 1.

Fig. 1.
figure 1

Scheme of the model of examination and decision-making on the support of scientific projects.

Full size image

4 Software Implementation of the Model and Analysis of the Results of Experimental Simulations

The tool for implementing the agent-based model is the Net Logo software package (environment), which is freely distributed and developed by Northwestern University (Northwestern University). More details about the advantages of this environment and its potential application in building models of natural, social and technical multi-agent systems can be found in the academic literature [24,25,26].

The interface of the model is presented in Fig. 2. In the left part of the interface, the control elements of the model are presented, the visualization of the virtual world is presented in the center, and the graphical information for analysis is on the right hand side.

Fig. 2.
figure 2

Interface of professional expertise and decision-making model on the support of research projects in the NetLogo environment. (Color figure online)

Full size image

The user has two available control levers, which are used to set the total number of researchers in the scientific community (number) and the proportion of researchers in the entire group belonging to the category of independents (percent_independent).

The virtual world is limited; agents cannot penetrate beyond its borders. The researchers (scientists) are presented in the form of large circles. Each researcher takes a specific place (cell, spot) in the virtual world, and two researchers cannot be in one place at the same time. The dependent class has a blue color; the independent class has a brown color. Researchers create research projects that take the form of small circles. Scientific projects that submitted for the competitions are marked in yellow, scientific projects that won the competition are in red.

The graph (scientists) represents the number of dependent and independent researchers in the scientific community; the graph (winners) shows the number of projects that won the competition. The information windows under the graphs (dependent, independent, winners) duplicate this information.

In order to obtain the data for analyzing the simulation results, we use the R programming language, connecting the RStudio development environment with Net Logo [27, 28]. Since the initial characteristics of the agents in the model are set randomly, we will analyze the influence of the independent researchers’ share in the scientific community (percent_independent) on the number of dependent researchers (dependent) and on the number of scientific projects that won the competition (winners).

To form the corresponding range model for the diagrams, we are going to implement 10 model simulations with the following initial settings: the scientific community consists of 1000 researchers, and 50% of researchers submit projects for the competition and carry them out. Further, the simulation duration is limited to 100-time cycles, the share of independent researchers varies from 10 to 90%, and the step of its change is 10%. Figure 3 shows the number of dependent researchers as a function of the share of independent researchers according to the results of one simulation model with the conditions specified above. The results of the range diagrams construction are presented in Figs. 4 and 5.

Fig. 3.
figure 3

Graph of the number of dependent scientists as a function of the proportion of independent researchers in the model.

Full size image
Fig. 4.
figure 4

Box plot of the number of dependent scientists as a function of the proportion of independent researchers in the model.

Full size image
Fig. 5.
figure 5

Box plot of the number of research projects that won the competition, as a function of the proportion of independent researchers in the model.

Full size image

According to Fig. 4, with initial values of the independent researchers representation spanning 10% to 40% of the total number of researchers, there is a tendency for independent researchers to become dependent. Upon completion of 100-time cycles, between 95% and 100% of researchers belong to the dependent class. A similar situation is observed with initial values of the independent researchers share of representation equal to from 60% to 90%. In this case, the reverse trend is visible - dependent researchers gravitate toward the independent category. By the completion of 100-time cycles, between 90% and 100% of researchers belong to the independent class.

Most compelling in these interactions is the range where the representation of dependent researchers and independent researchers equal to 50%. We identify that when starting from equal positions in terms of representative share, dependent researchers have an advantage over independent researchers. The median value indicates that upon the completion of 100-time cycles, 70% of researchers are in the class of dependent ones, with total fluctuations from 45% to 90%. Accordingly, it can be concluded that there is no parity between dependent and independent researchers, despite having established equal conditions before the start of the simulation cycle.

We can draw the following conclusion based on Fig. 5. Dependent researchers dominate in the scientific community (from 60% to 90%), and about 33 research projects per 1000 researchers win in the competition at each slot. In a situation where independent researchers dominate in the scientific community (from 60% to 90%), on average, about 11 research projects per 1000 researchers win in the competition at each slot. This result indicates that independent researchers carry out a more qualitative and tough examination compared with dependent researchers, and support 3 times less research projects. Thus, we can conclude that the scientific projects examination conducted by independent researchers contributes to a more rational distribution of scientific foundations.

5 Conclusion

The improvement of both competitive financing mechanisms and professional review processes is a key task for the scientific and expert community. The agent-based model of professional expertise and decision-making on the support of scientific projects described in this paper, allows for the identification of the influence of certain economic and psychological characteristics of the researchers based on the results of competitive financing from the research funds. It is shown that such economic and psychological opposition, such as “individualism – collectivism”, significantly affects the results of competitions. If the number of dependent researchers (who follow the interests of a certain group – ‘collectivism’), dominates in the scientific community, so will a greater number of research projects receive financial support. In the reverse situation, where independent researchers dominate in the scientific community, the examination processes of scientific projects become more rigid and, accordingly, scientific foundations support fewer scientific projects. These circumstances allow us to match up researchers who criticize the anonymous public review method (peer review). The lack of accounting for the economic and psychological characteristics of reviewers who assess scientific projects significantly reduces the quality of this method. As a possible solution to this problem, along with the multistage methods that are currently used to conduct expert evaluations of scientific projects, we propose to take into account such characteristics by forming psychological profiles of reviewers. In other words, each expert, on the one hand, should have sufficient qualifications to conduct examinations of scientific projects, and on the other hand, should have a suitable psychological profile.

The results obtained in this research summarise the problems of organizing the examination of not only scientific, but also other socially significant initiatives in the situations where the majority forms public opinion about one or another initiative. These include, for example, the organization of public hearings, various kinds of contests, elections, public positions, etc. At the same time, the influence of the economic and psychological characteristics of agents will probably be significant too. By slightly changing the conditions of the constructed agent-based model, it can be applied to the study of a variety of such situations.

Currently, the agent-based model of professional expertise and decision-making on the support of scientific projects has a number of limitations, which we hope will be overcome in the future. In particular, the following areas can be highlighted to improve the model:

  1. (a)

    considering the rationality (bounded rationality, up to irrationality) of the behavior of agents. Here, the mechanism of optimal decisions making by an agent in accordance with its rationality (bounded rationality, irrationality), taking into account the relation of this agent to restrictions in the area of making possible decisions may be realized;

  2. (b)

    changing the agents’ qualifications depending on the examinations to be carried out and the research projects submitted for the competition (economic and psychological opposition “reflection – antireflexia”);

  3. (c)

    taking into account the attitude of the researcher to his surroundings (economic and psychological opposition “introversion – extraversion”);

  4. (d)

    considering the possibility of examining scientific projects to the researchers who carry out and submit projects for the competition, without a formal division into the roles of producers and reviewers;

  5. (e)

    providing an opportunity to simultaneously submit projects to the competition and engage in several research projects for the researchers;

  6. (f)

    increasing the maximum number of agents in the simulation, which requires large computational power;

  7. (g)

    updating the population of agents according to their age.

Solving these problems will help to improve the predictive abilities of the model and assist it in future predictions of operating results of multi-agent autonomous socio-economic systems.