Keywords

1 Introduction

Cooperation among agents is critical for agents’ artificial intelligence (AI) and multi-agent system (MAS). Example applications include cooperative target observation [1], foraging [2], and peer-to-peer systems [3]. Coordination is important to improve the performance of the entire system; and cooperation is the first step of coordination [4]. Agent can decide whether to cooperate or defect with another agent based on their past interactions. Among previous researches, the Prisoner’s Dilemma (PD) game has been adopted widely in the field of evolution of cooperation [5]. Under the setting of prisoner’s dilemma, two players decide to choose one decisions from the two alternative decisions (cooperation, C or defection, D) at the same time. If two players take cooperation strategy, their benefits will both be R. If both players take defection strategy, they will both get P. If one player take cooperation strategy and another player take defection strategy, the cooperator will get S and the defector will obtain T. In the PD, it is assumed that T>R>P>S and T+S<2R. Obviously, if the PD game only happens for one single time, defection will be the best choice, and this cause both of players only get P. Thus, if players make decision based on their own maximization of fitness, it will result in low individual benefits.

In order to explain the phenomenon of evolution of cooperation in social and biological systems, a series of cooperation mechanisms has been proposed, including kin selection, direct reciprocity, indirect reciprocity, graph selection and group selection [6]. Graph selection (or spatial reciprocity) has become one of the most important mechanisms, and has gained wide attention in recent years [7–13]. The graph theory provides a very natural and convenient model to describe the spatial structure of cooperation evolutionary groups. Each node of the graph represents an individual participating in the game, and the link between the nodes is used to represent the adjacent links between individuals. At the same time, individuals constantly change (or maintain) their relationship with the neighbors according to the gains from the previous game. Based on such research framework, the evolution of the PD game in the past decade has been extended to various network models. It has been found that the structure of individuals’ interaction plays an important role in the evolution of cooperation.

Most of the researches on spatial PD game are based on a simplified assumption that each individual will interact with all his neighbors. That is to say, as long as two individuals are “neighbors”, they will play games definitely. However, in real life, whether to join a game with neighbors, the decision is not necessarily to be 100 %, because it is likely to be changed based on the individual’s environment or risk propensity. Traulsen et al. [14] put forward that the interaction between individuals is Stochasticity for the first time. Chen et al. proposed a model that individuals will interact with neighbors in a random rate in the spatial PD game [15]. Simulation results show that interaction intensity can promote the cooperation level maximization. Li et al. extended the research to the next step [16]. They proposed an adaptive interaction mechanism in which an individual can adjust their interaction rate with neighbors according to their incomes. If an individual’s income is higher than before, he will strengthen interaction rate with neighbors; otherwise he will reduce the interaction rate. They found that if the individual can adjust their interaction rate within a certain range, the cooperation level will be improved effectively. In addition, the individuals with low interaction rate usually occupy the cooperation cluster’s edge. Thus, the level of overall cooperation system will be improved by decreasing in the interaction rate between cooperators and defectors. It is also proved that the spatial structure can effectively reduce the invasion of defectors.

Previous researches about “interaction stochasticity” are all assume that the players’ strategies are “either all or nothing”, that is to say, the player of the game is either complete cooperator or defector. Numerous evidences in nature and human society show that strategy may be continuous, in other words, the player of the game is neither complete cooperator nor complete defector. In recent years, the scholars have carried out some researches on the strategy continuity in spatial games. Zhong et al. [17] found that in the population with a spatial structure, the equilibrium point of the continuous strategy and the discrete strategy are different. Especially the type of the game is more close to the “Chicken game”, the system of continuous strategy will show a phenomenon of “internal equilibrium”, which leads to the cooperation level in continuous-strategy system higher than in discrete-strategy system.

Based on the above consideration, the influence of strategy continuity on the evolution of cooperation in spatial PD game with interaction stochasticity is worth to research. According to our known, this problem has not been studied yet. Therefore, in this paper, we study the system in which the individuals play the PD game on a network with interaction stochasticity, and we focus on the influence of the continuity of strategies on the evolution of cooperation. We use the method of computer simulation, and compare the system with continuous strategy and the system with discrete strategy in the same initial settings. To explain conveniently, the system with players adopting continuous strategies is called continuous-strategy system (CSS) in the rest of this paper; and the system with players adopting discrete strategies is called discrete-strategy system (DSS).

The structure of this paper is as follows. In the second chapter, we give an introduction of the simulation model. The third part is the result and analysis of the imitation. Finally, the conclusion is given in the fourth part.

2 Model Description

In this paper, we adopt the matrix of PD game as Tanimoto and Sagara’s [18] work. We use Dr = P−S to represent the games close to “stag-hunt” game and use Dg = T−R to represent the games close to like “chicken” game. We also set P = 0 and R = 1, so the pay-off matrix can be shown as formula (1).

$$ M = \left( {\begin{array}{*{20}c} R & S \\ T & P \\ \end{array} } \right) = \left( {\begin{array}{*{20}c} 1 & { - Dr} \\ {1 + Dg} & 0 \\ \end{array} } \right){\text{and }}0 \le Dr,Dg \le 1 $$
(1)

We use the real number \( si(si \in \left[ {0,1} \right]) \) to represent continuous strategy. The strategy is fully cooperation strategy when si = 1 and the strategy is fully defection strategy when si = 0. When the value of si is between 0 and 1, the players adopt partial cooperation strategy. Suppose that a player has adopted strategy p in the game and his opponent uses strategy q, its income can be calculated by formula (2) according to Zhong et al. [17].

$$ \pi (p, \, q) \, = \left( {R - S - T + P} \right)pq \, + \left( {S - P} \right)p \, + \left( {T - P} \right)q \, + \, P $$
(2)

When p = 0 or 1 and q = 0 or 1, π (D, C) = T, π (D, D) = P, π (C, C) = R, π (C, D) = S. Thus, the formula (2) can also be used to calculate the income of discrete strategy. Because Dr = P−S, Dg = T−R, P = 0 and R = 1, we can get formula (3) from formula (2).

$$ \begin{aligned} \pi (p, \, q) \, & = \left( {Dr \, - \, Dg} \right)pq \, - \, Dr \, p \, + \left( { 1+ \, Dg} \right)q \, + \, P \\ & = \left( {Dr \, - \, Dg} \right)pq \, + ( - \, Dr)p \, + \left( { 1+ \, Dg} \right) \\ \end{aligned} $$
(3)

In this paper, the population is placed on a square-lattice network and the length of the side is L (L = 100). Each individual sits in a single grid with 4 neighbors and can interact with its every single neighbor. For the CSS, at the initial stage, the individual strategy is a random real number between [0, 1] which is evenly distributed on the network. For the DSS, at the initial stage, half individuals choose cooperation strategy and the others choose defection strategy. We adopt the interaction stochasticity model as Chen [15], using the variable \( \omega (\omega \in \left[ {0, 1} \right]) \) to represent the interaction rate between each individual. If ω increases, it represents the individual has more possibility to participating in games, otherwise the possibility of interaction will be lower. When ω → 1, the possibility of individual interact with its neighbors will be almost 100 %. When ω → 0, the possibility of interact will be almost 0 %, in another words, the interaction is frozen.

After the evolution starts, individuals play games neighbors with according to ω, and then update strategies by comparing the benefits with his neighbors. Based on the literature [19], individual’s fitness can be improved if individual has good performance when it collects neighbor’s strategies. We adopt the strategy updating mechanism as Chen [15]. The probability of the individual x adopting the strategy of neighbor y will be Px → y.

$$ P_{x \to y} = \frac{{P_{y} }}{{\mathop \sum \nolimits_{{z \in \varOmega_{x} }} P_{z} }} $$
(4)

In the formula (4), Py represents the neighbor y’s benefit. \( \sum\nolimits_{{z \in \varOmega_{x} }} {P_{z} } \) represents the sum of all neighbors’ benefits. After calculating each neighbor’ income respectively, individual x will learn a neighbor’s strategy by probability which is proportional to each neighbor benefit. Based on formula (4), a neighbor will have a higher probability to be learned with more income. If \( \sum\nolimits_{{z \in \varOmega_{x} }} {P_{z} } \) is zero, x will choose a neighbor randomly and adopt the neighbor’ strategy with the probability according to the Fermi rule shown in formula (5).

$$ f\left( {P_{y} - P_{x} } \right) = \frac{1}{{1 + { \exp }[ - (P_{y} - P_{x} )/K]'}} $$
(5)

Px is the total income of x with all its neighbor. K represents the impact of noise. When K → 0, x imitates y deterministically. When K → 1, x adopts y’s strategy randomly. In order to focus on the core of the research, we set K = 0.1 in this paper.

3 Result and Analysis

Based on the model introduced in part two, we use numerical simulations to test how the strategy continuity influences on the evolution of cooperation in lattice population under different rates of player’s interaction. We examine two categories of parameters:

(1) The interaction rate ω;

(2) The PD game’s type which is classified by the space of Dr and Dg.

According to Tanimoto [18], there are three sub-classes of PD which are illustrated in Fig. 1. Based on Dr ∈ [0, 1] and Dg ∈ [0, 1], Dr and Dg increase from 0.0 to 1.0 by 0.1, so the game space is represented by 11 × 11 = 121 points (the whole space of PD game, AllPD). The first sub-class of PD game is Donor & Recipient Game (DRG) which is the average of 11 points featured by Dr = Dg. The second sub-class is the boundary games between the PD and Chicken games (BCH) which is the average of 11 points featured by Dr = 0. The third sub-class is the boundary games between the PD and Stag–Hunt games (BSH) which is another average of 11 points featured by Dg = 0.

Fig. 1.
figure 1

Three sub-classes of PD, BCH, BSH and DRG.

Full size image

In the following simulation experiment, all the statistical results are reached an equilibrium in system (arrives at quasi-equilibrium). Cooperation rate (the fraction of cooperation) is an important indicator to describe system average cooperation level. It is defined as \( fc = \sum\nolimits_{i \in N} {S_{i} } /N \). All the results are average values after 50 independent runs.

Firstly we research the dependence of fc on the interaction rate ω in CSS and DSS respectively. As shown in Fig. 2, both continuous and discrete strategies on the whole value of interval have a great influence on cooperation evolution. Especially in the interval when ω < 0.4, the difference is particularly large. For both DSS and CSS, ω have a larger impact on fc. According to its general trend, fc will increase at the beginning and then drop in the DSS; and fc in the CSS is down to the bottom and increase after that in ALLPD, BCH and DRG games. From Fig. 2a we can observe that in AIIPD game, when ω > 0.3, the fcs in the DSS and CSS are no difference, but when ω < 0.3, the fc in CSS drops fast at the beginning which makes the gap of fc between DSS and CSS biggest. From Fig. 2b we can observe that in BCH game, the difference of CSS and DSS is relatively large. In the CSS, fc drops to the bottom firstly (ω = 0.1) and then rises. In the DSS, fc rise firstly (ω = 0.1) and then decreases. In all, when ω < 0.4, fc in DSS is higher than that in CSS; and when ω > 0.4, fc in CSS is higher than that in DSS. From Fig. 2c we can observe that in the BSH game, the difference of fc between CSS and DSS is very small when ω > 0.3. When ω < 0.3, fc in the DSS is higher than that in the CSS; especially when 0.1 < ω < 0.2, the gap of fc between DSS and CSS is biggest. From Fig. 2d we can observe that the trend of DRG game is like BCH game, the difference of fc in CSS and DSS reaches the maximum value at ω = 0.1.

Fig. 2.
figure 2

Correlations between ω and fc in the CSS and the DSS respectively.

Full size image

From Fig. 2, we can draw the basic conclusion that the overall performance of the CSS is worse than that of DSS based on interaction stochasticity mechanism. The difference is prominent when interaction rate is small. In order to further analyze this phenomenon, we choose four specific values ω = 0.1, 0.3, 0.5, 0.9 and compare the specific difference between the CSS and DSS. Experimental results are shown in Fig. 3.

Fig. 3.
figure 3

The fc as a function of Dr and Dg in the CSS and the DSS under different ω.

Full size image

From Fig. 3a, c, e, the region in the space Dr-Dg that is close to BCH, with the increase of Dg we observe the obvious “phase transition” from cooperation to defection. When ω = 0.1, transition point is Dg = 0.5. At this point, a rapid falling of fc to 0.0 with a slight increase of Dg. On the other hand in the CSS, we see different situations, as shown in Fig. 3b, d, f. In the CSS, in the region in the Dr < 0.4 and Dg > 0.5 (the square A), fc is always greater than 0, especially fc = 0.1 when ω = 0.1 (Fig. 3b). With the increase of ω, in square A gradually declines to 0.05 when ω reaches 1.0. In all, when Dr < 0.4 and Dg > 0.5, fc in the CSS is slightly greater than that in the DSS. The advantage of continuous strategy in this region on cooperation gradually diminishes with the increase of ω. In the CSS, in the region in the Dr < 0.4 and Dg < 0.5 (the square B), the influence of strategy continuity on cooperation shows complex changing phenomenon. In this region, fc in CSS is always lower than that in DSS, especially fc = 0.1 when ω = 0.1 (Fig. 3b). With the increase of ω, fc in square B gradually increases closely to fc in DSS when ω reaches 1.0. Besides the regions of square A and B, the difference of fc in CSS and DSS is insignificant.

From the experimental results of Figs. 2 and 3, we can see that in the full Dr-Dg parameter space, interaction stochasticity and strategy continuity influences the evolution of cooperation together. It is at a relatively low level of ω, the slightly positive influence of the strategy continuity on the fraction of cooperation is clear in the region where Dr is relatively low and Dg is relatively high. On the other hand, the massive negative influence of the strategy continuity on the fraction of cooperation is obvious in the region where Dr is relatively low and Dg is relatively low. This explains the remarkably lower fraction of cooperation in CSS than in the DSS in Fig. 2 when ω < 0.4. When ω increases, the difference between the CSS and DSS becomes smaller which implies the influence of strategy continuity reduces with greater interaction rate. This explains the results shown in Fig. 2 ω > 0.4 and also explains the significant difference of CSS and DSS in DRG.

From the above simulations, we can draw the conclusion that, strategy continuity and interaction stochasticity are two different types of dynamics in promoting cooperation. The reason of the strategy continuity can promote the cooperation level of the system is the evolution of strategies instinctively tends toward the internal equilibrium which leads higher fc in the CSS than DSS when the game is closer to BCH game. The reason that the interaction stochasticity can promote cooperation level is that it weakens the interaction strength between individuals, equivalent to reducing the individual effective number of neighbors, especially in the type of game closer to BSH game. Simulation experiment shows that strategy continuity and interaction stochasticity cannot promote cooperation in the same time, because their deep mechanisms are incompatible, especially when the type of game is closer to the DSG game.

4 Conclusions and Future Work

In this paper, we have studied the influence of strategy continuity on evolution of cooperation in lattice population with interaction stochasticity. From the simulation results, we find that the strategy continuity has a great effect on cooperation level in spatial PD game with interaction stochasticity. According to our research, when the interaction rate is low, the cooperation level is reduced if the strategy is continuous in the system. When the interaction rate is high, the cooperation levels of continuous-strategy system and the discrete-strategy system are close. The reason behind this is strategy continuity and interaction stochasticity are two different types of dynamics in promoting cooperation, but their deep mechanisms are incompatible. When interaction rate is low, the incompatible feature is prominent, especially when the type of game is closer to DSG game.

For future work: our existing approach lacks an analysis of the interaction between the microscopic individual and we need to study the mechanism behind the incompatibility phenomenon further.