1 Introduction

Classic evolutionary matrix games, introduced by Maynard Smith and Price [29], describe populations where individuals compete with each other through pairwise interactions (Maynard Smith [28], Chapters 6-7 in Hofbauer and Sigmund [20], Chapter 2.3 in MestertonGibbons [30], Chapter 6 in Broom and Rychtár̆ [6]) . After an interaction, contestants receive payoffs based on their strategies/phenotypes. The payoffs of the consecutive interactions then determine the (relative) fitness and thereby the evolutionary success of a given phenotype. However, in many situations, the outcome of an interaction involves not only a payoff, but also a time period during which contestants must wait before being ready for subsequent interactions (e.g. to recover from an injury, to handle the payoff (food), to collect energy after a long fight, etc., Holling [22], Charnov [8], Broom et al. [5], Garay et al. [15], Toupo et al. [41]). Therefore, a portion of the population is temporarily unable to interact, leading to no resource intake during the waiting period. This can significantly influence fitness and, consequently, evolutionary outcomes [16, 25]. For instance, in the classical Prisoner’s dilemma, the ESS strategy is the defector strategy. However, if the waiting time associated with interactions among defectors is long enough, the average intake of defectors may become smaller than that of cooperators, resulting in the evolutionary stability of the cooperator strategy in that model. Similarly, if the waiting time after a Hawk-Hawk interaction in the Hawk-Dove game is sufficiently long, the pure Dove strategy can be ESS even at a low cost of fighting. These examples well illustrate how the consideration of time constraints in game-theoretical models can lead to surprising phenomena, underscoring the need for further mathematical investigations.

Maynard Smith and Price [29] studied monomorphic populations, where each individual initially possesses the same (pure or mixed) phenotype. In this scenario, Maynard Smith introduced the following static definition in plain language: “An ESS is a strategy such that, if all the members of a population adopt it, then no mutant strategy could invade the population under the influence of natural selection” [28, p. 10]. This condition implies that the ESS is consistently favoured by natural selection when the frequency of mutants is low.

The concept of ESS aims to encapsulate the intuitive expectation that an evolutionarily stable strategy must be resistant to invasion. This static definition proves invaluable in applied evolutionary game theory, as it allows us to predict evolutionary outcomes without delving into complex models describing temporal population changes. However, from a theoretical perspective in evolutionary game theory, we require dynamics that elucidate the temporal evolution of the population (cf. Introduction in Cressman [10]). To address this, Taylor and Jonker [36] proposed a polymorphic approach, considering populations composed of individuals with N distinct, genetically fixed pure phenotypes. The temporal evolution of the population is then modelled using dynamics, often the replicator equation [20, 36], and a locally asymptotically stable rest point of the dynamics serves as the end point of the evolution.

It is a fundamental result in classic evolutionary game theory that, for matrix games, the corresponding state of an ESS is a locally asymptotically stable rest point of the replicator equation [21, 36, 44]. This relationship holds not only for matrix games but also for some cases where the fitness function is not bilinear in the phenotype of the focal individual and the population’s average strategy. In fact, assuming linearity only in the strategy of the focal individual is sufficient (see, for example, Thomas [37], Chapter 7.2 in Hofbauer and Sigmund [20], or Cressman et al. [11]). However, for other fitness functions, the validity of an analogous theorem remains an open question.

In the case of evolutionary matrix games under time constraints, the fitness function is neither linear in the strategy of the focal individual nor in the average strategy of the population. Similarly to the aforementioned classic result, Garay et al. [17] managed to prove the asymptotic stability of the rest point corresponding to an ESS for two-strategy games (Corollary 4.3 in Garay et al. [17]). The proof relies on the fact that the set of the convex combinations of the strategies for which the replicator equation is considered is one-dimensional. However, the higher-dimensional case remained open, except for some special cases (Theorem 4.1 in Garay et al. [17], Theorem 4.7 and Theorem 4.9 in Varga et al. [42]) where the corresponding state is either asymptotically stable or at least stable. Nevertheless, the authors’ conjecture was that the stability does not hold in general.

In this paper, we confirm that the conjecture is true by providing examples of matrix games under time constraints with an evolutionarily stable strategy such that the corresponding state of the replicator equation is unstable, even in the case of three-dimensional strategies.

One of our examples starts from the rock-paper-scissors game. This game models cyclic dominance [24, 27, 35] that can result in the cyclic coexistence of three phenotypes [20, 31, 32, 41]. In classic matrix games, the replicator dynamics cannot exhibit limit cycles ([3, 44], Chapter 7.5 in Hofbauer and Sigmund [20]). If periodic solutions exist, there are infinitely many, enveloping a (non-asymptotically) stable equilibrium point, much like in the classical rock-paper-scissors game. Moreover, if we replace the zeros along the main diagonal of the payoff matrix of this game with a parameter \(\varepsilon \), the phase portrait corresponds to a degenerate Hopf bifurcation, as the sign of \(\varepsilon \) changes [13, 20]. If \(\varepsilon <0\), the equilibrium point is an asymptotically stable focus. As \(\varepsilon \) reaches zero, periodic solutions emerge, and the equilibrium point turns into a non-asymptotically stable center. For \(\varepsilon >0\), the periodic orbits vanish again, and the equilibrium point becomes an unstable focus. However, the corresponding monomorphic strategy also loses its evolutionary stability. With increasing the mathematical complexity of the model, one can anticipate a more intricate behaviour of the replicator dynamics. Mobilia [31] and Toupo and Strogatz [40] pointed out that even allowing arbitrarily small mutations can induce a limit cycle in a Hopf bifurcation. Here, we demonstrate that arbitrary small differences in waiting times can also have a comparable effect, leading not only to the instability of the equilibrium point but also to the appearance of a limit cycle during a Hopf bifurcation. All this occurs while the strategy corresponding to the equilibrium point maintains its evolutionary stability throughout.

2 Preliminaries

We introduce some functions and notation necessary to our calculations. For detailed background (the heuristic thoughts and the mathematical derivation of the formulas), we refer the reader to Garay et al. [16] and Garay et al. [17].

We examine the relationship between the evolutionary stability of game theory and the stability of the replicator dynamics. From a population biology perspective, the dynamic description is applied to a population in which a finite but arbitrarily large number of pre-defined phenotypes can occur. This is known as the polymorphic approach. In contrast, the concept of evolutionary stability applies to a monomorphic population consisting of individuals with a distinguished resident phenotype, and we investigate what happens when a mutant phenotype appears. Only one type of mutant can appear at a time, but it can be any phenotype that differs from the resident.

2.1 Polymorphic Approach

First, we introduce the toolkit for the polymorphic approach. Consider a well-mixed large population in which individuals compete with each other through pairwise interactions that can be mathematically described with the help of a game. During the game, a player chooses from among N pure strategies with certain probabilities determined by the individual’s (heritable) phenotype, in other words, his strategy. Mathematically, we describe the set of possible strategies (phenotypes) using the N-dimensional simplex:

$$\begin{aligned} S_N=\{\textbf{q}=(q_1,q_2,\dots ,q_N)\in {\mathbb {R}}^N: q_1+q_2+\dots +q_N=1\text { and }q_1,q_2,\dots ,q_N \ge 0\} \end{aligned}$$

(where N is a positive integer and \({\mathbb {R}}\) is the set of real numbers). Then, the pure strategies correspond to the vectors \(\textbf{e}_1=(1,0,\dots ,0)\), \(\textbf{e}_2=(0,1,0,\dots ,0)\), ..., \(\textbf{e}_{N-1}=(0,0,\dots ,0,1,0)\), \(\textbf{e}_N=(0,0,\dots ,0,1)\), and the coordinate \(q_i\) of a strategy \(\textbf{q}\) is the probability that an individual with phenotype \(\textbf{q}\) employs the pure strategy \(\textbf{e}_i\) in a game.

Let \(\textbf{q}_1,\textbf{q}_2,\dots ,\textbf{q}_n\) denote the phenotypes occurring in the population, with frequencies \(x_1, x_2, \dots , x_n\) respectively. From an evolutionary perspective, a central question is how the state of the population, represented by the frequency distribution \(\textbf{x}=(x_1, x_2, \dots , x_n)\), changes over time. For a mathematical description of this, Taylor and Jonker [36] introduced the replicator equation or dynamics:

$$\begin{aligned} \dot{x}_i=x_i[W_i(\textbf{x})-{\bar{W}}(\textbf{x})]\qquad (i=1,2,\dots ,n). \end{aligned}$$
(2.1)

where \(W_i(\textbf{x})\) is the fitness of phenotype \(\textbf{q}_i\) (Taylor and Jonker [36] called it "current growth rate"), and \(\bar{W}(\textbf{x}) = x_1W_1(\textbf{x}) + \dots + x_nW_n(\textbf{x})\) denotes the average fitness of the population. The dynamics assume that the change in frequency of a phenotype is proportional to its relative fitness.

To proceed, we need to specify the mathematical form of the \(W_i\) fitness functions. Let \(a_{ij}\) denote the payoff to a player using pure strategy i while their opponent employs pure strategy j. This yields an \(N\times N\) matrix \(A=(a_{ij})\), which we refer to as the payoff matrix. With this, the expected payoff for an individual of phenotype \(\textbf{q}_i\) in the above population is \(\textbf{q}_i A (x_1 \textbf{q}_1+\dots +x_n\textbf{q}_n)\)Footnote 1 (we assume that the population is large enough to neglect the effect that an individual cannot interact with itself). In the case of evolutionary matrix games, this expected payoff also serves as the fitness function \(W_i(\textbf{x})\).

Following the model of Garay et al. [16] or Kr̆ivan and Cressman [25], we also consider that an interaction has another consequence in addition to the payoff. Specifically, if a player uses pure strategy i against an opponent using pure strategy j, then the player must wait for \(\tau _{ij}\) time units before being able to engage in another interaction. (The time of the interaction itself can be considered as 0 or a part of the \(\tau _{ij}\) time.) During this waiting time, the individual is incapable of interaction and is considered inactive from the perspective of the game. One can think of this time as necessary for the individual to process the resources acquired during the interaction or for the individual to recover from any injuries sustained (cf. Holling [22] or the handling time in optimal foraging theory, Charnov [8] and Garay et al. [14]). After the waiting period, the player looks for their next opponent. We consider the time taken for this search to be 1. During the search, the player is capable of interaction and is considered active in terms of the game. If the player encounters an individual who is inactive, no interaction occurs between them, resulting in a payoff of 0, but the search time restarts. It is worth noting that the model of Garay et al. [16] is a Markov model in which both the waiting times \(\tau _{ij}\) and the search time are independent exponentially distributed random variables. However, we will only use derived formulas in which only their expected values (\(\tau _{ij}\) and 1, respectively) appear. We assume that the sequence of interactions and searches occurs on a much faster time scale than the replicator dynamics (cf. Gunawardena [19]), allowing us to assume, from the perspective of replicator dynamics, that the population is in a stationary equilibrium. In other words, on the fast time scale, the proportion of inactive individuals is constant, and the ratio of time spent in an active state for an individual of type \(\textbf{q}_i\) matches the proportion of active individuals in the subpopulation of \(\textbf{q}_i\) individuals. Formally, if we denote this ratio by \(\varrho _i\) and \(T=(\tau _{ij})_{N\times N}\), then:

$$\begin{aligned} \varrho _i=\frac{1}{1+\textbf{q}_i T[x_1\varrho _1\textbf{q}_1+x_2\varrho _2\textbf{q}_2+ \dots +x_n\varrho _n\textbf{q}_n]} \qquad (i=1,2,\dots ,n) \end{aligned}$$
(2.2)

The denominator represents the expected value of time between two encounters, while the numerator represents the time spent searching (in the active state). Matrix T is referred to as the time constraint matrix. It can be shown that there is only one solution to the system of equations above for which \(0\le \varrho _i\le 1\) for every i (Theorem 1 and Lemma 2 in Garay et al. [16]).

Since there is no interaction in the inactive state, identifying fitness solely on the basis of expected payoffs could give a false picture. Even if the expected payoff is high, it can be misleading if such events rarely occur. To obtain an accurate representation, we must also consider the time between interactions. Therefore, we interpret fitness as the intake per unit of time:

$$\begin{aligned} W_i(\textbf{x})=\frac{\textbf{q}_iA\sum _{j=1}^nx_j\varrho _j(\textbf{x})\textbf{q}_j}{1+\textbf{q}_iT\sum _{j=1}^nx_j\varrho _j(\textbf{x})\textbf{q}_j}. \end{aligned}$$
(2.3)

Note that the term in the numerator is just the expected value of payoff in a single interaction, whereas the denominator equals the expected value of time between two consecutive encounters. Theorem 1 in Garay et al. [16] claims that if first the time of observation, then the population size tends to infinity, while the proportion of individuals of type \(\textbf{q}_i\) converges to \(x_i\), then the average intake per unit of time for an individual of type \(\textbf{q}_i\) is described by the above formula. This demonstrates that our formula used for the \(W_i\) fitness can be derived not only intuitively but also rigorously through mathematical means.Footnote 2

If we introduce the notation

$$\begin{aligned} {\bar{\varrho }}&=x_1\varrho _1+x_2\varrho _2+\dots +x_3\varrho _n \end{aligned}$$
(2.4)

and

$$\begin{aligned} \textbf{h}(x)&=x_1\frac{\varrho _1}{{\bar{\varrho }}}\textbf{q}_1+x_2\frac{\varrho _2}{{\bar{\varrho }}}\textbf{q}_2+\dots +x_n\frac{\varrho _n}{{\bar{\varrho }}}\textbf{q}_n \end{aligned}$$
(2.5)

(the former representing the proportion of active individuals in the population, the latter representing the average strategy of active individuals in the population), then \(W_i\) and \({\bar{W}}\) can be rewritten in the following forms:

$$\begin{aligned} W_i(\textbf{x})= \varrho _i(\textbf{x})\textbf{q}_iA{\bar{\varrho }}(\textbf{x})\textbf{h}(\textbf{x}), \end{aligned}$$

and

$$\begin{aligned} {\bar{W}}(\textbf{x})={\bar{\varrho }}(\textbf{x})\textbf{h}(\textbf{x})A{\bar{\varrho }}(\textbf{x})\textbf{h}(\textbf{x}), \end{aligned}$$

respectively.

This shows that our model is indeed a generalization of classical evolutionary matrix games, since if all \(\tau _{ij}\) times are zero, then \(\varrho _i(\textbf{x})\textbf{q}_iA\bar{\varrho }(\textbf{x})\textbf{h}(\textbf{x})\) simplifies to the formula \(\textbf{q}_iA\sum _i x_i\textbf{q}_i\) used to express fitness in the classical case.

2.2 Monomorphic Approach and Evolutionarily Stable Strategy

We will need a precise mathematical formulation of evolutionary stability under time constraints and further statements that connect it with the stability of replicator dynamics.

Consider a monomorphic population of individuals with \(\textbf{p}\) phenotype. Assume that some mutant individuals with \(\textbf{q}\) phenotype appear in the population. We should stress that only one type of mutant phenotype can be present in the population at a time. This type is denoted by \(\textbf{q}\). Let \(1-\varepsilon \) and \(\varepsilon \) be the frequencies of the resident and mutant phenotypes, respectively. This scenario mathematically corresponds to the above polymorphic situation when \(n=2\), \(\textbf{q}_1=\textbf{p}\), \(\textbf{q}_2=\textbf{q}\), \(x_1=1-\varepsilon \), and \(x_2=\varepsilon \). However, to emphasize that \(\textbf{p}\) is the resident and \(\textbf{q}\) is the mutant phenotype, we denote the fitness of \(\textbf{p}\) as \(\omega _{\textbf{p}}\) and the fitness of \(\textbf{q}\) as \(\omega _{\textbf{q}}\). Accordingly,

$$\begin{aligned} \omega _{\textbf{p}}(\textbf{p},\textbf{q},\varepsilon )&=\frac{\textbf{p}A[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}]}{1+\textbf{p} T[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}]}= \rho _{\textbf{p}}\textbf{p}A[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}],\\ \omega _{\textbf{q}}(\textbf{p},\textbf{q},\varepsilon )&=\frac{\textbf{q}A[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}]}{1+\textbf{q} T[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}]}= \rho _{\textbf{q}}\textbf{q}A[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}], \end{aligned}$$

where \(\rho _{\textbf{p}}(\textbf{p},\textbf{q},\varepsilon )\) and \(\rho _{\textbf{q}}(\textbf{p},\textbf{q},\varepsilon )\) are the proportions of active individuals among the \(\textbf{p}\) and \(\textbf{q}\) individuals, respectively, and they constitute the unique solution of the equation system

$$\begin{aligned} \rho _{\textbf{p}}&=\frac{1}{1+\textbf{p} T[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}]}\\ \rho _{\textbf{q}}&=\frac{1}{1+\textbf{q} T[(1-\varepsilon )\rho _{\textbf{p}}\textbf{p}+\varepsilon \rho _{\textbf{q}}\textbf{q}]} \end{aligned}$$

in the interval [0, 1] (Lemma 2 in Garay et al. [16]).

Definition 2.1

A strategy \(\textbf{p}^*\) is a uniformly evolutionary stable strategy of the matrix game under time constraints (UESS in short) if there is an \(\varepsilon _0>0\) such that

$$\begin{aligned}\omega _{\textbf{p}^*}(\textbf{p}^*,\textbf{q},\varepsilon ) >\omega _{\textbf{q}}(\textbf{p}^*,\textbf{q},\varepsilon )\end{aligned}$$

for any strategy \(\textbf{q}\not =\textbf{p}^*\) and \(0<\varepsilon <\varepsilon _0\).

The adverb “uniformly” is used here because \(\varepsilon _0\) does not depend on \(\textbf{q}\). When \(\varepsilon _0\) can depend on \(\textbf{q}\), we refer to \(\textbf{p}^*\) as an ESS without the adverb "uniformly". Clearly, if \(\textbf{p}^*\) is a UESS, it also satisfies the criteria for an ESS. The converse also holds in the case of classic matrix games (without time constraints), as shown in Theorem 6.4.1 in Hofbauer and Sigmund [20]. However, in the context of matrix games under time constraints, this equivalence remains an open question. Although we do not want to deal with this issue in this article, our aim was to explain the rationale behind employing the term "UESS". This distinction between UESS and ESS naturally arises when dealing with non-linear fitness functions [4], and UESS is considered more suitable for describing evolutionary stability in such cases [33].

Our primary objective is to present examples featuring a UESS \(\mathbf {p^*}\) where the corresponding state is an unstable rest point of the replicator equation with respect to pure strategies. Nevertheless, describing this corresponding state is not as straightforward as in the classical scenario. We must reference certain pivotal statements to grasp the connection between the monomorphic and polymorphic models.

2.3 Relationship Between the Polymorphic and the Monomorphic Approach

We introduce the shorter notations \(\omega (\textbf{q})\), \(\omega _{\textbf{p}}(\textbf{q})\), \(\rho (\textbf{q})\) and \(\rho _{\textbf{p}}(\textbf{q})\) for \(\omega _{\textbf{q}}(\textbf{p},\textbf{q},1)\), \(\omega _{\textbf{p}}(\textbf{p},\textbf{q},1)\), \(\rho _{\textbf{q}}(\textbf{p},\textbf{q},1)\) and \(\rho _{\textbf{p}}(\textbf{p},\textbf{q},1)\), respectively. These special quantities describe the initial moment when individuals of type \(\textbf{p}\) start to appear in a population composed exclusively of individuals of type \(\textbf{q}\). \(\rho (\textbf{q})\) can be explicitly expressed as the unique solution in [0, 1] to the equation

$$\begin{aligned} \rho =\frac{1}{1+\textbf{q} T \rho \textbf{q}}. \end{aligned}$$

(Lemma 2 in Garay et al. [16]). In particular,

$$\begin{aligned} \rho (\textbf{q})=\frac{-1+\sqrt{1+4\textbf{q}T\textbf{q}}}{2\textbf{q}T\textbf{q}}= \frac{2}{1+\sqrt{1+4\textbf{q}T\textbf{q}}}. \end{aligned}$$
(2.6)

This immediately implies that functions

$$\begin{aligned} \rho _{\textbf{p}}(\textbf{q})=\frac{1}{1+\textbf{p} T \rho (\textbf{q}) \textbf{q}}, \end{aligned}$$
(2.7)
$$\begin{aligned} \omega (\textbf{q})=\frac{\textbf{q} A \rho (\textbf{q})\textbf{q}}{1+\textbf{q} T \rho (\textbf{q})\textbf{q}}=\rho (\textbf{q})\textbf{q} A \rho (\textbf{q})\textbf{q} \end{aligned}$$
(2.8)

and

$$\begin{aligned} \omega _{\textbf{p}}(\textbf{q})=\frac{\textbf{p} A \rho (\textbf{q})\textbf{q}}{1+\textbf{p} T \rho (\textbf{q})\textbf{q}} =\rho _{\textbf{p}}(\textbf{q})\textbf{p} A \rho (\textbf{q})\textbf{q} \end{aligned}$$
(2.9)

all have continuous second-order partial derivatives with respect to the coordinates of \(\textbf{q}=(q_1, q_2, \dots ,q_N) \in S_N\).

It is clear for classical matrix games that if \(\textbf{p}=(p_1,p_2,\dots ,p_N)\) is a strategy then \((x_1,x_2,\dots ,x_N)\) with \(x_i=p_i\) is the corresponding state in the polymorphic system of pure phenotypes \(\textbf{e}_1,\textbf{e}_2,\dots ,\textbf{e}_N\) in the sense that \(\sum _i x_i\textbf{e}_i=\textbf{p}\). Consequently, if \(\textbf{p}\) is an ESS, then it is natural to investigate the stability property of the state \((x_1,x_2,\dots ,x_N)\) with \(x_i=p_i\) for the replicator equation with respect to \(\textbf{e}_1,\textbf{e}_2,\dots ,\textbf{e}_N\). If the replicator equation is considered for arbitrary strategies \(\textbf{q}_1,\textbf{q}_2,\dots ,\textbf{q}_n\) then it is also clear that the corresponding state(s) of \(\textbf{p}\) is (are) the state(s) \((x_1,x_2,\dots ,x_n)\) with \(\textbf{p}=x_1\textbf{q}_1+x_2\textbf{q}_2+\dots +x_n\textbf{q}_n\), that is, the state(s) at which the average strategy of the population is \(\textbf{p}\). However, for matrix games under time constraints, the situation is generally not so obvious. The answer can be found in the next lemma of Garay et al. [17], which asserts that the corresponding state is the state at which the average strategy of active individuals is \(\textbf{p}\). To recall notations \(\textbf{h}(\textbf{x})\) and \({\bar{\varrho }}(\textbf{x})\), see (2.4) and (2.5).

Lemma 2.2

(Garay et al. [17], Corollary 6.3) Consider a population of phenotypes \(\textbf{q}_1\), \(\textbf{q}_2\),...,\(\textbf{q}_n\in S_N\). Assume that \(\textbf{r}=\theta _1\textbf{q}_1+ \theta _2\textbf{q}_2+\dots +\theta _n\textbf{q}_n\) with some \(\varvec{\theta }=(\theta _1,\theta _2,\dots ,\theta _n)\in S_n\). If

$$\begin{aligned} x_i=x_i(\varvec{\theta })=\frac{\rho (\textbf{r})}{\rho _{\textbf{q}_i}(\textbf{r})}\theta _i \qquad (i=1,2,\dots ,n) \end{aligned}$$
(2.10)

then \(\textbf{h}(\textbf{x})=\textbf{r}\) where \(\textbf{x}=\textbf{x} (\varvec{\theta })=\big (x_1(\varvec{\theta }),x_2(\varvec{\theta }), \dots ,x_n(\varvec{\theta })\big )\), that is, the average strategy of active individuals in a population of \(\textbf{q}_1\), \(\textbf{q}_2\),..., \(\textbf{q}_n\) individuals with frequencies \(x_1(\varvec{\theta }),\) \(x_2(\varvec{\theta })\),..., \(x_n(\varvec{\theta })\) is \(\theta _1 \textbf{q}_1+ \theta _2\textbf{q}_2+\dots +\theta _n\textbf{q}_n=\textbf{r}\). Moreover, \(\textbf{x}(\varvec{\theta })\) is the unique state for which \(\theta _i=x_i \varrho (\textbf{x})/\bar{\varrho }(\textbf{x})\) \((i=1,2,\dots ,n)\) holds.

We consider two particular cases. One of them is when \(n=N=3\), \(\textbf{q}_1=\textbf{e}_1\), \(\textbf{q}_2=\textbf{e}_2\) and \(\textbf{q}_3=\textbf{e}_3\). Let \(\textbf{r}=(r_1,r_2,1-r_1-r_2)=r_1\textbf{e}_1+r_2\textbf{e}_2 +(1-r_1-r_2)\textbf{e}_3\) be a strategy from \(S_3\). By the previous lemma, there exists a uniqueFootnote 3 state \(\textbf{x}(\textbf{r})\in S_3\) with \(\textbf{h}\big (\textbf{x}(\textbf{r})\big ) = \textbf{r}\), namely, \(x_i(\textbf{r})=r_i\rho (\textbf{r})/\rho _{\textbf{e}_i}(\textbf{r})\). So \(x_i(\textbf{r})\) has continuous partial derivatives with respect to the variables \(r_1\) and \(r_2\). The notation \(\textbf{x}^R(\textbf{r})\) is used (“R” refers to “restricted”) if necessary to emphasize that \(\textbf{x}(\textbf{r})\) is concerned with this case.

The other case is when \(N=3\), \(n=4\), \(\textbf{q}_1=\textbf{e}_1\), \(\textbf{q}_2=\textbf{e}_2\), \(\textbf{q}_3=\textbf{e}_3\) and \(\textbf{q}_4=\textbf{p}^*:=(1/3,1/3,1/3)\). Then most of the strategies from \(S_3\), namely, the strategies in the interior of \(S_3\) can be expressed in the form \(\theta _1\textbf{e}_1+ \theta _2\textbf{e}_2+\theta _3\textbf{e}_3+\theta _4\textbf{p}^*\) in more than one way. For such a strategy \(\textbf{r}\), there is thus more than one state \(\textbf{x}=(x_1,x_2,x_3,x_4)\) with \(\textbf{h}(\textbf{x})=\textbf{r}\). So it would be ambiguous to use the notation \(\textbf{x}(\textbf{r})\) for a state with \(\textbf{h}(\textbf{x}(\textbf{r}))=\textbf{r}\). Therefore we associate the coefficient-list \(\varvec{\theta }=(\theta _1,\theta _2,\theta _3,\theta _4)\) from the convex combination \(\theta _1\textbf{e}_1+ \theta _2\textbf{e}_2+\theta _3\textbf{e}_3+\theta _4\textbf{p}^*=\textbf{r}\) to the state \(\textbf{x}=\textbf{x}(\varvec{\theta })=\textbf{x}(\theta _1,\theta _2,\theta _3,\theta _4)\) with \(x_i(\varvec{\theta })=\theta _i\rho (\textbf{r})/\rho _{\textbf{r}_i}(\textbf{r})\) rather than associating a strategy \(\textbf{r}\) to such a state. If it is necessary to emphasize that \(\textbf{x}(\varvec{\theta })\) is concerned with this case, then we use the notation \(\textbf{x}^E(\varvec{\theta })\) (“E” refers to “extended”). Observe that if the vector \((x_1,x_2,1-x_1-x_3,0)\in S_4\) is identified with the vector \((x_1,x_2,1-x_1-x_3)\in S_3\), then the map \(\textbf{x}^R\) is just the restriction of the map \(\textbf{x}^E\) to the set \(S_3\).

3 Dynamically Unstable UESS

It is well known in the case of classic evolutionary matrix gamesFootnote 4 that a UESS always implies the asymptotic stability of the corresponding equilibrium point of the associated replicator equation ([21, 36, 44] or Theorem 7.2.4 in Hofbauer and Sigmund [20]).

An analogues implication for matrix games under time constraints up to now was just proved when the the strategy space consists of 2 dimensional strategies. For higher dimensional strategies, however, the question remained open in general [17, 42].

Now we give examples exhibiting UESS with unstable corresponding equilibrium, that is, the accustomed implication of the classical model does not hold in the more general model of matrix games under time constraints. This shows that considering time effects can dramatically alter the outcome of evolution.

The strategy space of the examples is

$$\begin{aligned} S_3=\{(q_1,q_2,q_3)\in {\mathbb {R}}^3\,:\,q_i\ge 0,\,\, i=1,2,3,\text { and }q_1+q_2+q_3=1\}. \end{aligned}$$

In each example, strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) is a UESS which is verified in Sections A.1 and A.2. The corresponding state is \(\textbf{x}=\textbf{x}(\textbf{p}^*)= \textbf{x}^R(\textbf{p}^*) =\big (x_1(\textbf{p}^*),x_2(\textbf{p}^*),x_3(\textbf{p}^*)\big )\) for which \(\textbf{h}(\textbf{x})=\textbf{p}^*\) (see Lemma 2.2) (recall the notations \(\textbf{x}^R\) and \(\textbf{x}^E\) in the second paragraph after Lemma 2.2). We analyse the replicator Eq. (2.1) for the polymorphic population consisting of pure phenotypes \(\textbf{e}_1,\) \(\textbf{e}_2\), \(\textbf{e}_3\) with frequencies \(x_1,x_2\) and \(x_3=1-x_1-x_2\):

$$\begin{aligned} \dot{x}_1&=x_1[W_1(\textbf{x})-{\bar{W}}(\textbf{x})]\nonumber \\ \dot{x}_2&=x_2[W_2(\textbf{x})-{\bar{W}}(\textbf{x})]\nonumber \\ \dot{x}_3&=x_3[W_3(\textbf{x})-{\bar{W}}(\textbf{x})]. \end{aligned}$$
(3.11)

We also investigate the replicator equation with respect to the population of \(\textbf{q}_1=\textbf{e}_1,\textbf{q}_2=\textbf{e}_2,\textbf{q}_3= \textbf{e}_3\) and \(\textbf{q}_4=\textbf{p}^*=(1/3,1/3,1/3)\) individuals. This means the extension of the previous dynamics (3.11) adding a fourth equation:

$$\begin{aligned} \dot{x}_1&=x_1[W_1(\textbf{x})-{\bar{W}}(\textbf{x})]\nonumber \\ \dot{x}_2&=x_2[W_2(\textbf{x})-{\bar{W}}(\textbf{x})]\nonumber \\ \dot{x}_3&=x_3[W_3(\textbf{x})-{\bar{W}}(\textbf{x})]\nonumber \\ \dot{x}_4&=x_4[W_4(\textbf{x})-{\bar{W}}(\textbf{x})] \end{aligned}$$
(3.12)

where \(x_4\) describes the frequency of \(\textbf{p}^*\) individuals. The phase space of this dynamics is \(S_4\). The state (1, 0, 0, 0) corresponds to pure strategy \(\textbf{e}_1\) in the sense that in this state every individual follows strategy \(\textbf{e}_1\). Similarly, state (0, 1, 0, 0) corresponds to pure strategy \(\textbf{e}_2\), state (0, 0, 1, 0) corresponds to pure strategy \(\textbf{e}_3\) and state (0, 0, 0, 1) corresponds to strategy \(\textbf{p}^*\). Clearly, if the replicator Eq. (3.12) is restricted to the face with vertices (1, 0, 0, 0), (0, 1, 0, 0) and (0, 0, 1, 0) we get back the dynamics (3.11). Accordingly, this face contains the state \(\textbf{x}^E(1/3,1/3,1/3,0)\) (\(\textbf{x}_{\textbf{p}^*}\) for short) corresponding to the UESS \(\textbf{p}^*\) in which each individual follows one of the three pure strategies and the average strategy of active individuals is the UESS \(\textbf{p}^*\). Note that this state is the same as the state \(\textbf{x}^R(\textbf{p}^*)\) for the replicator equation with respect to pure strategies \(\textbf{e}_1,\textbf{e}_2,\textbf{e}_3\).

By Lemma 2.2, the segment (the green segment in Figs. 2 and 5) connecting \(\textbf{x}_{\textbf{p}^*}\) with (0, 0, 0, 1) precisely consists of the states corresponding to the UESS \(\textbf{p}^*\) and the coordinates of these states are

$$\begin{aligned} x^E_i\big (u/3,u/3,u/3,(1-u)\big ) =(u/3)\frac{\rho (\textbf{p}^*)}{\rho _{\textbf{e}_i}(\textbf{p}^*)} \qquad \qquad (i=1,2,3) \end{aligned}$$

and

$$\begin{aligned} x_4^E\big (u/3,u/3,u/3,(1-u)\big )=(1-u)\frac{\rho (\textbf{p}^*)}{\rho _{\textbf{p}^*}(\textbf{p}^*)}=1-u, \end{aligned}$$

respectively, where u runs over the interval [0, 1]. These states are equilibrium points of the replicator Eq. (3.12) (Lemma 3.2 in Garay et al. [17]). Furthermore, state (0, 0, 0, 1) at which every individual of the population follows strategy \(\textbf{p}^*\) is a stable (but not asymptotically stable) rest point of the replicator Eq. (3.12) (Theorem 4.9 in Varga et al. [42]). If the dynamics (3.12) is restricted to any of the faces with vertex (0, 0, 0, 1) then state (0, 0, 0, 1) is asymptotically stable on that face (Theorem 4.7 in Varga et al. [42]).

One of the eigenvalues at every point of the segment is always 0, the other two have zero real part at a single point of the segment which is denoted by \(\textbf{x}_0\) (the upper green point in Figs. 2 and 5). Above the point (the part of the segment falling between the states \(\textbf{x}_0\) and (0, 0, 0, 1)) both non-zero eigenvalues have negative real parts while under the point (the part of the segment falling between the states \(\textbf{x}_{\textbf{p}^*}\) and \(\textbf{x}_0\)) the real parts of the two non-zero eigenvalues are positive. To find \(\textbf{x}_0\) we have analyzed the linearisation of the right hand side of the dynamical system (3.12) along the segment between \(\textbf{x}_{\textbf{p}^*}\) and (0, 0, 0, 1).

We remark that the local property of equilibrium points and the global behaviour of the dynamics on the edges of the phase spaces \(S_3\) and \(S_4\), respectively, are mathematically exact, the eigenvalues were calculated, whereas the global behaviour in the interior of the phase spaces and in the interior of the faces of \(S_4\) are just based on numerical simulations of the phase portrait.

3.1 Example 1: Dynamically Unstable UESS in Rock-Paper-Scissors Game

The popular children’s game, the rock-paper-scissors game, describes a game with three pure strategies where each pure strategy beats precisely another pure strategy (rock beats scissor, scissors beat paper and paper beats rock). The payoff matrix has the following form

$$\begin{aligned} \left( \begin{array}{ccc} 0 &{} -a_2 &{} b_3\\ b_1 &{} 0 &{} -a_3\\ -a_1 &{} b_2 &{} 0 \end{array} \right) \qquad a_1,a_2,a_3,b_1,b_2,b_3>0. \end{aligned}$$

The corresponding replicator equation always has a rest point that is asymptotically stable (and the corresponding mixed strategy is a UESS) if and only if the determinant \(b_1b_2b_3-a_1a_2a_3\) is positive (Theorem 7.7.2 in Hofbauer and Sigmund [20]). Otherwise, the rest point is a (non-asymptotically) stable center (if the determinant is zero) or unstable (if the determinant is negative). From evolutionary aspect, it models a cyclic dominance [24, 27, 35] which can result in the coexistence of the three phenotypes [20, 31, 32, 41].

Now we show that arbitrary short waiting times can destabilize a stable coexistence. Let the payoff matrix \(A=A(s)\) and the time constraint matrix \(T=T(s)\) be

$$\begin{aligned} A=A(s)=\left( \begin{array}{ccc} 0 &{} -2+s/10 &{} 2\\ 2 &{} 0 &{} -2+s/10\\ a_{31}(s) &{} 2 &{} 0 \end{array} \right) ,\qquad T=T(s)=\left( \begin{array}{ccc} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0\\ s &{} 2s &{} 0 \end{array} \right) .\nonumber \\ \end{aligned}$$
(3.13)

where

$$\begin{aligned} a_{31}(s)=-2 + \frac{s (\sqrt{9 + 12 s}-1)}{20}. \end{aligned}$$

Then for any \(s\in (0,s_0)\) with some \(s_0\ge 3\) strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) is a UESS of the matrix game under time constraints with payoff matrix A(s) and time constraint matrix T(s) such that the corresponding sate \(\textbf{x}(\textbf{p}^*)\) is an unstable rest point of the replicator Eq. (3.11). The verification can be found in Appendix 1. Note that the determinant of A(s) is positive if \(s\in (0,3]\) so the matrix game with matrix A(s) (\(s\in (0,3]\)) has an interior UESS (which tends to \(\textbf{p}^*\) as \(s\rightarrow 0\)) (see expression (7.34) in Hofbauer and Sigmund [20]) and, consequently, this UESS is an asymptotically stable rest point of the relating replicator equation with respect to pure strategies ([21, 36, 44] or Theorem 7.2.4 in Hofbauer and Sigmund [20]).

The phase portrait of dynamics (3.11) for \(s=1\) is depicted in Fig. 1. The three vertices together with the edges between them form a heteroclinic cycle. The Jacobian matrix of the right hand side at equilibrium point

$$\begin{aligned} \textbf{x}(\textbf{p}^*)=\textbf{x}^R(\textbf{p}^*)= \left( \frac{\sqrt{21}-3}{6},\frac{\sqrt{21}-3}{6},2-\sqrt{\frac{7}{3}}\right) \end{aligned}$$

corresponding to the UESS \(\textbf{p}^*=(1/3,1/3,1/3)\) has eigenvalues with positive real part implying the instability of the rest point. The phase portrait suggests that the solutions tend to the boundary of the simplex.

Fig. 1
figure 1

The phase portrait of the replicator Eq. (3.11) with respect to the pure phenotypes \(\textbf{e}_1,\textbf{e}_2\) and \(\textbf{e}_3\). The payoff matrix A and the time constraint matrix T are given in (3.13) setting s to be 1. \(\textbf{x}(\textbf{p}^*)=\textbf{x}^R(\textbf{p}^*)\) is the state corresponding to strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) through Lemma 2.2: \(x_1(\textbf{p}^*) =x_2(\textbf{p}^*)=(\sqrt{21}-3)/6\approx 0.263\). Although \(\textbf{p}^*\) is a UESS, \(\textbf{x}(\textbf{p}^*)\) is an unstable rest point of the replicator equation. The vertices of the simplex are saddle points and the boundary of the simplex is a heteroclinic cycle which appears to be attractive. For every state \(\textbf{x}\), there is a composition \(Q^{\textbf{p}^*}\hspace{-1mm}(\textbf{x})\) exhibiting strategy \(\textbf{p}^*\) in the population in state \(\textbf{x}\) (see Sect. 3.4 for more explanation). Therefore, a subpopulation in state \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) has higher fitness than that of the whole population. The green segment is the set of states \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\)-s as \(\textbf{x}(t)\) runs over the red orbit (see the enlargement of the green segment in Fig. 6). \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) generally differs from \(\textbf{x}(\textbf{p}^*)\). One can intuitively expect that the population “try” to evolve towards \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) rather than \(\textbf{x}(\textbf{p}^*)\) but \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) varies from moment to moment which can contribute to the instability of \(\textbf{x}(\textbf{p}^*)\) in the example. In the table, we give the real parts of the eigenvalues of the linearisation of the replicator equation at the rest points indicated in the phase portrait

Full size image

The phase portrait of the extended replicator Eq. (3.12) is shown in Fig. 2. On the face determined by vertices (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), the dynamics agrees with (3.11) so the phase portrait on it is the same as in Fig. 1.

The solutions in the interior of the face determined by vertices (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 0, 1) start from (1, 0, 0, 0) and end in (0, 0, 0, 1). The situation is similar on the other two faces.

The state \(\textbf{x}_0\) on the segment matching \(\textbf{x}_{\textbf{p} ^*}\) with (0, 0, 0, 1) and corresponding to the UESS \(\textbf{p}^*\) is equal to \(\textbf{x}^E(u_0/3,u_0/3,u_0/3,1-u_0)\) where \(u_0= (71 + 9 \sqrt{21})/334\approx 0.336\). Accordingly, the coordinates of \(\textbf{x}_0\) are \((x_0)_1\approx 0.089\), \((x_0)_2\approx 0.089\), \((x_0)_3\approx 0.159\) and \((x_0)_4=1-u_0\approx 0.664\).

Fig. 2
figure 2

The phase portrait of the replicator equation with respect to phenotypes \(\textbf{e}_1,\textbf{e}_2,\textbf{e}_3\) and \(\textbf{p}^*=(1/3,1/3,1/3)\). The payoff matrix A and the time constraint matrix T are given in (3.13) setting s to 1. \(\textbf{x}_{\textbf{p}^*}=(x_{\textbf{p}^*\hspace{-1mm},1},x_{\textbf{p}^*\hspace{-1mm},2}, x_{\textbf{p}^*\hspace{-1mm},3},x_{\textbf{p}^*\hspace{-1mm},4})\) is the state corresponding to strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) through Lemma 2.2 on the face determined by the vertices (1, 0, 0, 0), (0, 1, 0, 0) and (0, 0, 1, 0): \(x_{\textbf{p}^*\hspace{-1mm},1} =x_{\textbf{p}^*\hspace{-1mm},2}=(\sqrt{21}-3)/6\approx 0.263\), \(x_{\textbf{p}^*\hspace{-1mm},3} =1-x_{\textbf{p}^*\hspace{-1mm},1}-x_{\textbf{p}^*\hspace{-1mm},2}\), \(x_{\textbf{p}^*\hspace{-1mm},4}=0\). It agrees with \(\textbf{x}(\textbf{p}^*)\) in Fig. 1. Every state on the green segment between (0, 0, 0, 1) and \(\textbf{x}_{\textbf{p}^*}\) corresponds to strategy \(\textbf{p}^*\) through Lemma 2.2. Hence, every point of the segment is a rest point of the replicator equation. One of the three eigenvalues of the linearisation of the replicator Eq. (3.12) at these states is zero. At \(\textbf{x}_0\), all of the eigenvalues have zero real part. The states on the segment under \(\textbf{x}_{0}\) have two eigenvalues with positive real part, so they are all unstable though they correspond to the UESS \(\textbf{p}^*\). The states between \(\textbf{x}_0\) and (0, 0, 0, 1) whereas have two eigenvalues with negative real part. The state (0, 0, 0, 1) is stable (but not asymptotically). In the table, we give the real parts of the eigenvalues of the linearisation of the replicator equation at the rest points indicated in the phase portrait

Full size image

3.2 Limit Cycle, Hopf Bifurcation

Previously, we have found that both the game with payoff matrix A(1) and the game under time constraints with the same payoff matrix and time constraint matrix T(1) have a UESS. Nevertheless, the stability properties of the corresponding rest points are different. While the rest point is asymptotically stable in the former case, it is unstable in the latter case, and the orbits spirally tend to the boundary of the simplex according to the numerical simulation (Fig. 1). The situation can remind us of a supercritical Hopf bifurcation. Indeed, let

$$\begin{aligned} {\tilde{A}}={\tilde{A}}(\sigma )=\left( \begin{array}{ccc} 0 &{} -19/10 &{} 2\\ 2 &{} 0 &{} -19/10\\ {\tilde{a}}_{31}(\sigma ) &{} 2 &{} 0 \end{array} \right) ,\qquad {\tilde{T}}=\tilde{T}(\sigma )=T(\sigma )=\left( \begin{array}{ccc} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0\\ \sigma &{} 2\sigma &{} 0 \end{array} \right) \nonumber \\ \end{aligned}$$
(3.14)

where

$$\begin{aligned}{\tilde{a}}_{31}(\sigma )=\frac{\sqrt{9 + 12 \sigma }-41}{20}.\end{aligned}$$

Here, \({\tilde{a}}_{31}(\sigma )\) has been chosen such that \(\textbf{p}^*=(1/3,1/3,1/3)\) is a UESS for every \(\sigma \in [0,1]\). Note that \({\tilde{A}}(1)=A(1)\). Therefore, the matrix game under time constraints with matrices \({\tilde{A}}(1)\) and \({\tilde{T}}(1)\) has a UESS such that the corresponding rest point of the replicator equation is unstable while the game with matrices \({\tilde{A}}(0)\) and \(\tilde{T}(0)\) has a UESS such that the corresponding rest point is already asymptotically stable. Further analysis reveals that a supercritical Hopf bifurcation occurs at \(\sigma =(1+\sqrt{82})/54\approx 0.186\) (see Appendix A.3). So a limit cycle around the rest point must exist when \(\sigma \) is close to \((1+\sqrt{82})/54\). For instance, for \(\sigma =19/100\), a limit cycle takes shape according to the numerical simulation. However, the orbits tending to the limit cycle wind around themselves so tightly that the limit cycle cannot be perceived easily. Therefore, we give a more spectacular example in Fig. 3.

Fig. 3
figure 3

The phase portrait of the replicator Eq. (3.11) with respect to the pure phenotypes \(\textbf{e}_1,\textbf{e}_2\) and \(\textbf{e}_3\). The payoff matrix A and the time constraint matrix T are given right from the phase portrait. \(\textbf{x}(\textbf{p}^*)=\textbf{x}^R(\textbf{p}^*)\) is the state corresponding to strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) through Lemma 2.2: \(x_1(\textbf{p}^*)=x_2(\textbf{p}^*) = 10/(15+\sqrt{1245})\approx 0.1989\). Although \(\textbf{p}^*\) is a UESS (see Appendix A.2.3 ), \(\textbf{x}(\textbf{p}^*)\) is an unstable rest point of the replicator equation. The vertices of the simplex are saddle points and the boundary of the simplex is a repelling heteroclinic cycle. It seems that there is a stable limit cycle around \(\textbf{x}(\textbf{p}^*)\). In the table, we give the real parts of the eigenvalues of the linearisation of the replicator equation at the rest points indicated in the phase portrait

Full size image

3.3 Example 2 with a Strict NE and a dynamically Unstable UESS

The payoff matrix A and the time constraint matrix T are the following in this case:

$$\begin{aligned} A=\left( \begin{array}{ccc} -5 &{} 449 - 19 \sqrt{469} &{} -1\\ -2 &{} 5 &{} 3\\ -204/5 + 2 \sqrt{469} &{} 1 &{} -2/10 \end{array} \right) ,\qquad T=\left( \begin{array}{ccc} 45 &{} 40 &{} 20\\ 1 &{} 2 &{} 7\\ 0 &{} 0 &{} 0 \end{array} \right) . \end{aligned}$$
(3.15)

Although a similar phase portrait is possible in the classical case (the last element of the first row in Figure 11 in Zeeman [44], phase portrait 12 in Fig. 6 in Bomze [3]), the unstable interior rest point corresponds to a UESS. In addition to \(\textbf{p}^*=(1/3,1/3,1/3)\), there is a further UESS, moreover, a strict Nash equilibrium which is impossible in classic matrix games (see the first paragraph after the proof of Theorem 6.4.1 on p.64 in Hofbauer and Sigmund [20]). A strategy \(\textbf{r}\) is a strict Nash equilibrium if, for every strategy \(\textbf{q}\not =\textbf{r}\), the strict inequality

$$\begin{aligned} \rho _{\textbf{q}}(\textbf{r})\textbf{q}A\textbf{r}< \rho (\textbf{r})\textbf{r}A\textbf{r} \end{aligned}$$
(3.16)

holds (Definition 2.2 in Garay et al. [17]). If \(\textbf{r}\) is a strict Nash equilibrium, then it is a pure strategy (Theorem 4.1 in Garay et al. [17]) and a UESS (Theorem 4.1 in Varga et al. [42]). In the present example, strategy \(\textbf{e}_2\) is a strict Nash equilibrium implying that the corresponding state (0, 1, 0) and (0, 1, 0, 0), respectively, is an asymptotically stable rest point of the replicator equation with respect to \(\textbf{e}_1,\textbf{e}_2, \textbf{e}_3\) and the replicator equation with respect to \(\textbf{e}_1,\textbf{e}_2, \textbf{e}_3,\textbf{p}^*\), respectively (Corollary 4.8 in Varga et al. [42]).

To see that \(\textbf{e}_2\) is a strict Nash equilibrium, one should check inequality (3.16) with \(\textbf{r}=\textbf{e}_2\). By (2.6) and (2.7) we get \(\rho (\textbf{e}_2)\) and \(\rho _{\textbf{q}}(\textbf{e}_2)\), respectively. We obtain that

$$\begin{aligned} \rho (\textbf{e}_2)\textbf{e}_2A\textbf{e}_2-\rho _{\textbf{q}}(\textbf{e}_2)\textbf{q}A\textbf{e}_2= \frac{3 - 796 \,q_1 + 38 \sqrt{469} \,q_1 - 3 \,q_2}{2 + 40\, q_1 + 2\, q_2}. \end{aligned}$$

Note that \(\textbf{q}=(q_1,q_2,1-q_1-q_2)\in S_3\) is a strategy. Thus the denominator \(2 + 40 q_1 + 2 q_2\) is positive for every \(\textbf{q}\). Consequently, in order to prove that \(\textbf{e}_2\) is a strict Nash equilibrium, it is enough to validate that the numerator \(3 - 796 q_1 + 38 \sqrt{469} \,q_1 - 3 q_2>0\) for every \(\textbf{q}\not =\textbf{e}_2\) which can easily be seen:

$$\begin{aligned} 3 - 796 q_1 + 38 \sqrt{469} \,q_1 - 3 q_2\ge 3 - 796 q_1 + 38 \sqrt{441} \,q_1 - 3 q_2&=3+2q_1-3q_2\\&=2q_1+3(1-q_2) \end{aligned}$$

which is positive for every strategy \(\textbf{q}\) different from \(\textbf{e}_2\). Hence \(\textbf{e}_2\) is a strict Nash equilibrium, as claimed.

Fig. 4
figure 4

The phase portrait of the replicator Eq. (3.11) with respect to the pure phenotypes \(\textbf{e}_1,\textbf{e}_2\) and \(\textbf{e}_3\). The payoff matrix A and the time constraint matrix T are the matrices in (3.15). \(\textbf{x}(\textbf{p}^*)=\textbf{x}^R(\textbf{p}^*)\) is the state corresponding to strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) through Lemma 2.2: \(x_1(\textbf{p}^*) = (495 - 4 \sqrt{469})/529\approx 0.772\), \(x_2(\textbf{p}^*)=(409 + 17\sqrt{469})/5290\approx 0.147\). Although \(\textbf{p}^*\) is a UESS, \(\textbf{x}(\textbf{p}^*)\) is an unstable rest point of the replicator equation. Among the vertices, only (1, 0, 0) is a saddle point; (0, 1, 0) is asymptotically stable, moreover, it corresponds to a strict Nash equilibrium while (0, 0, 1) is a source. In addition to the vertices, \(\textbf{x}^{(12)}\) and \(\textbf{x}^ {(13)}\) are two further rest points on the boundary of the simplex. Both of them are saddle points. For every state \(\textbf{x}\), there is a composition \(Q^{\textbf{p}^*}\hspace{-1mm}(\textbf{x})\) exhibiting strategy \(\textbf{p}^*\) in the population in state \(\textbf{x}\) (see Sect. 3.4 for more explanation). Therefore, a subpopulation in state \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) has higher fitness than that of the whole population. The green curve is the set of states \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\)-s as \(\textbf{x}(t)\) runs over the blue orbit (see the enlargement of the green segment in Fig. 6). \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) generally differs from \(\textbf{x}(\textbf{p}^*)\). One can intuitively expect that the population “try” to evolve towards \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) rather than \(\textbf{x}(\textbf{p}^*)\) but \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) varies from moment to moment which can contribute to the instability of \(\textbf{x}(\textbf{p}^*)\) in the example. In the table, we give the real parts of the eigenvalues of the linearisation of the replicator equation at the rest points indicated in the phase portrait

Full size image

Considering the replicator equation with respect to \(\textbf{e}_1\), \(\textbf{e}_2\) and \(\textbf{e}_3\), we find two additional rest points besides the vertices and the rest point \(\textbf{x}^R(\textbf{p}^*)\) corresponding to the UESS \(\textbf{p}^*\). There is a rest point on the edge between states (1, 0, 0) and (0, 1, 0). We denote it by \(\textbf{x}^{(12)}\approx (0.689,\,0.311,\,0)\). This is an unstable rest point of the dynamics restricted to the edge, and it is a saddle point of the dynamics with respect to the pure strategies.

The other rest point can be found on the edge between (1, 0, 0) and (0, 0, 1). It is denoted by \(\textbf{x}^{(13)}\approx (0.395,\,0,\,0.605)\). This state is asymptotically stable on the edge, but it is a saddle point of the replicator dynamics with respect to \(\textbf{e}_1\), \(\textbf{e}_2\) and \(\textbf{e}_3\). A separatrix starts from it and ends in the state (0, 1, 0).

The (interior) solutions on the (1, 0, 0) side of the separatrix start from state \(\textbf{x}(\textbf{p}^*)=\textbf{x}^R(\textbf{p}^*)\) and end in (0, 1, 0) except for one that ends in \(\textbf{x}^{(12)}\). The (interior) solutions on the (0, 0, 1) side of the separatrix start from (0, 0, 1) and end in (0, 1, 0) (see Fig. 4).

The coordinates of \(\textbf{x}(\textbf{p}^*)\) are

$$\begin{aligned}x_1(\textbf{p}^*)=\frac{495 - 4 \sqrt{469}}{529},\quad x_2(\textbf{p}^*)=\frac{409 + 17 \sqrt{469}}{5290}, \quad x_3(\textbf{p}^*)=1-x_1(\textbf{p}^*)-x_2(\textbf{p}^*).\end{aligned}$$

The phase portrait of the extended replicator Eq. (3.12) is illustrated in Fig. 5. The restriction of the dynamics onto the face determined by vertices (1, 0, 0, 0), (0, 1, 0, 0) and (0, 0, 1, 0) is just the replicator equation with respect to pure strategies, so the phase portrait on that face agrees with that in Fig. 4.

Fig. 5
figure 5

The phase portrait of the replicator equation with respect to phenotypes \(\textbf{e}_1,\textbf{e}_2,\textbf{e}_3\) and \(\textbf{p}^*=(1/3,1/3,1/3)\). The payoff matrix A and the time constraint matrix T are given in (3.15). \(\textbf{x}_{\textbf{p}^*}=(x_{\textbf{p}^*\hspace{-1mm},1},x_{\textbf{p}^*\hspace{-1mm},2}, x_{\textbf{p}^*\hspace{-1mm},3},x_{\textbf{p}^*\hspace{-1mm},4})\) is the state corresponding to strategy \(\textbf{p}^*=(1/3,1/3,1/3)\) through Lemma 2.2 on the face determined by the vertices (1, 0, 0, 0), (0, 1, 0, 0) and (0, 0, 1, 0): \(x_{\textbf{p}^*\hspace{-1mm},1}\approx 0.772\), \(x_{\textbf{p}^*\hspace{-1mm},2}\approx 0.147\), \(x_{\textbf{p}^*\hspace{-1mm},3} =1-x_{\textbf{p}^*\hspace{-1mm},1}-x_{\textbf{p}^*\hspace{-1mm},2}\), \(x_{\textbf{p}^*\hspace{-1mm},4} =0\). It agrees with \(\textbf{x}(\textbf{p}^*)\) in Fig. 4. Every state on the green segment between (0, 0, 0, 1) and \(\textbf{x}_{\textbf{p}^*}\) corresponds to strategy \(\textbf{p}^*\) through Lemma 2.2. Hence, every point of the segment is a rest point of the replicator equation. One of the three eigenvalues of the linearisation of the replicator Eq. (3.12) at these states is zero. At \(\textbf{x}_0\), all of the eigenvalues has zero real part. The states on the segment under \(\textbf{x}_{0}\) have two eigenvalues with positive real part, so they are all unstable though they correspond to the UESS \(\textbf{p}^*\). The states between \(\textbf{x}_0\) and (0, 0, 0, 1) whereas have two eigenvalues with negative real part. The state (0, 0, 0, 1) is stable (but not asymptotically). In the table, we give the real parts of the eigenvalues of the linearisation of the replicator equation at the rest points indicated in the phase portrait

Full size image

There exists a rest point on the edge between (0, 1, 0, 0) and (0, 0, 0, 1). It is denoted by \(\textbf{x}^{(24)}\). Its coordinates are \(x^{(24)}_1=x^{(24)}_3=0\), \(x^{(24)}_2\approx 0.2186\), and \(x^{(24)}_4\approx 0.7814\). The rest point is unstable with respect to the edge.

It appears that there is a separatrix connecting the rest point \(\textbf{x}^{(24)}\) with the rest point \(\textbf{x}^{(12)}\) on the face determined by the vertices (1, 0, 0, 0), (0, 1, 0, 0) and (0, 0, 0, 1). The separatrix divides the face into two parts. The (interior) orbits start from \(\textbf{x}^{(24)}\) and end in the stable (moreover asymptotically stable with respect to the face) rest point (0, 0, 0, 1) on the part falling toward the edge between (1, 0, 0, 0) and (0, 0, 0, 1) while in the asymptotically stable rest point (0, 1, 0, 0) on the other part of the face.

Also, it seems that a separatrix runs from \(\textbf{x}^{(13)}\) to (0, 0, 0, 1) on the face determined by states (1, 0, 0, 0), (0, 0, 1, 0) and (0, 0, 0, 1). The orbits in the interior of the face all end in state (0, 0, 0, 1) but the orbits on the (1, 0, 0, 0) side of the separatrix start from state (1, 0, 0, 0) whereas those falling on the (0, 0, 1, 0) side of the separatrix start from state (0, 0, 1, 0).

It seems that there is also a separatrix on the face determined vertices (0, 1, 0, 0), (0, 0, 1, 0) and (0, 0, 0, 1) connecting state (0, 0, 1, 0) to \(\textbf{x}^{(24)}\). The orbits in the interior of the face start from (0, 0, 1, 0). The orbits located on the (0, 1, 0, 0) side of the separatrix run into state (0, 1, 0, 0) while those on the (0, 0, 0, 1) side of the separatrix end in state (0, 0, 0, 1).

In this case, state \(\textbf{x}_0\) is the state \(\textbf{x}^E(u_0/3,u_0/3,u_0/3,1-u_0)\) where

$$\begin{aligned} u_0=\frac{61518486052 + 3148879999 \sqrt{469}}{131829933359}\approx 0.9839, \end{aligned}$$

so its coordinates are \((x_0)_1\approx 0.7596\), \((x_0)_2\approx 0.1446\), \((x_0)_3\approx 0.07981\) and \((x_0)_4=1-u_0\approx 0.01607\).

3.4 Composition Corresponding to a Strategy \(\textbf{q}\) in a Population of State \(\textbf{x}\)

In the context of classic matrix games, if \(\textbf{p}^*\) is a UESS then state this UESS is an asymptotically stable rest point of the replicator equation with respect to pure strategies. The intuitive explanation says that a subpopulation in state \(\textbf{p}^*\) has a higher mean fitness than the total population of state \(\textbf{x}\) and therefore the population evolves to state \(\textbf{p}^*\). In the morning of classical evolutionary matrix games [29], however, the implication was not clear; some years lasted before the discovery of the relation [21, 36, 44].

Our counterexamples show that this classical relation is not self-evident. To help the intuitive comprehension why the implication does not hold in general in the case of matrix games under time constraints, it can be worth analysing the composition corresponding to the UESS in a population of state \(\textbf{x}\).

Lemma 2.2 gives the state \(\textbf{x}\) that corresponds to a strategy \(\textbf{q}\) in the sense that the mean strategy of active individuals is just \(\textbf{q}\) and the proportion of active individuals equals the proportion of active individuals in the monomorphic population of \(\textbf{q}\) individuals, in short, \(\textbf{h}(\textbf{x})=\textbf{q}\) and \({\bar{\varrho }}(\textbf{x})=\rho (\textbf{q})\). Motivated by this lemma, take a population of pure strategists in state \(\textbf{x}\) and consider the following composition. If \(\textbf{q}=q_1\textbf{e}_1+q_2\textbf{e}_2+q_3\textbf{e}_3\) is a strategy, then let

$$\begin{aligned} \varrho _{\textbf{q}}(\textbf{x}):=\rho _{\textbf{q}}\big (\textbf{h}(\textbf{x})\big )= \frac{1}{1+\textbf{q} T \big [\sum \limits _i x_i \varrho _i \textbf{e}_i\big ]} \end{aligned}$$

and

$$\begin{aligned} Q^{\textbf{q}}(\textbf{x}):= \bigg (\underbrace{\frac{\varrho _{\textbf{q}}(\textbf{x})}{\varrho _1(\textbf{x})}q_1}_{Q_1^{\textbf{q}}(\textbf{x})}, \underbrace{\frac{\varrho _{\textbf{q}}(\textbf{x})}{\varrho _2(\textbf{x})}q_2}_{Q_2^{\textbf{q}}(\textbf{x})}, \underbrace{\frac{\varrho _{\textbf{q}}(\textbf{x})}{\varrho _3(\textbf{x})}q_3}_{Q_3^{\textbf{q}}(\textbf{x})}\bigg ). \end{aligned}$$
(3.17)

Observe that

$$\begin{aligned} Q_1^{\textbf{q}}(\textbf{x})\varrho _1(\textbf{x}) +Q_2^{\textbf{q}}(\textbf{x})\varrho _2(\textbf{x}) +Q_3^{\textbf{q}}(\textbf{x})\varrho _3(\textbf{x})=\varrho _{\textbf{q}}(\textbf{x}), \end{aligned}$$

furthermore,

$$\begin{aligned} Q_1^{\textbf{q}}(\textbf{x})\varrho _1(\textbf{x})\textbf{e}_1 A {\bar{\varrho }}(\textbf{x})\textbf{h}(\textbf{x}) +Q_2^{\textbf{q}}(\textbf{x})\varrho _2(\textbf{x})\textbf{e}_2 A \bar{\varrho }(\textbf{x})\textbf{h}(\textbf{x}) +Q_3^{\textbf{q}}(\textbf{x})\varrho _3(\textbf{x})\textbf{e}_3 A {\bar{\varrho }}(\textbf{x})\textbf{h}(\textbf{x})\\ =\varrho _{\textbf{q}}(\textbf{x})\textbf{q} A{\bar{\varrho }}(\textbf{x})\textbf{h}(\textbf{x}) =\rho _{\textbf{q}}\big (\textbf{h}(\textbf{x})\big )\textbf{q} A \rho \big (\textbf{h}(\textbf{x})\big )\textbf{h}(\textbf{x}), \end{aligned}$$

that is, a subpopulation in state \(Q^{\textbf{q}}(\textbf{x})\) corresponds to strategy \(\textbf{q}\) in the sense that the proportion of active individuals in the subpopulation is the same as the probability that a \(\textbf{q}\) individual playing against the population is active and that the mean fitness of the subpopulation is the same as the fitness of a \(\textbf{q}\) individual playing against the population.

The observation shows that \(Q^{\textbf{q}}(\textbf{x})\) is not a fixed composition in general. It varies with \(\textbf{x}\). In particular, if \(\textbf{q}=\textbf{p}^*\), that is, \(\textbf{q}\) is a UESS, then the subpopulation corresponding to the UESS varies as the state of the population varies (see green curves in Fig. 1, 4, 6 and 7). This is a distinction from the classical case in which the corresponding composition was just the same as the strategy [\(Q^{\textbf{q}}(\textbf{x})=\textbf{q}\)] independently of \(\textbf{x}\). This means that composition \(Q^{\textbf{p}^*}(\textbf{x})\) generally differs from \(\textbf{x}(\textbf{p}^*)\) [in the classical case \(\textbf{x}(\textbf{p}^*)=Q^{\textbf{p}^*}(\textbf{x})=\textbf{p}^*\)]. It seems that this slight deviation can be enough to make \(\textbf{x}(\textbf{p}^*)\) unstable, as shown by our examples.

Fig. 6
figure 6

The neighbourhood of \(\textbf{x}(\textbf{p}^*)\) in Fig. 1 and 4 are enlarged. The left figure corresponds to Example 1 while the right one to Example 2. The green curves consist of the states corresponding to \(\textbf{p}^*\) as \(\textbf{x}(t)\) runs on the orbit around \(\textbf{x}(\textbf{p}^*)\) in Fig. 1, 4 (red orbit in Fig. 1, blue orbit in Fig. 2), in other words, states \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) as defined by (3.17) at \(\textbf{q}=\textbf{p}^*\). If the composition of a subpopulation is \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) then the mean strategy of active individuals of that subpopulation is just \(\textbf{p}^*\). In classic matrix games \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\), \(\textbf{x}(\textbf{p}^*)\) and \(\textbf{p}^*\) coincide, but not in the examples. This can contribute to the instability of \(\textbf{x}(\textbf{p}^*)\)

Full size image
Fig. 7
figure 7

The initial subarc of the orbit \(\textbf{x}(t)\) around \(\textbf{x}(\textbf{p}^*)\) in Fig. 1, 4 (red orbit in Fig. 1, blue orbit in Fig. 2) and the corresponding curve drawn by composition \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) (green curves). If the composition of a subpopulation is \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) in the population of state \(\textbf{x}(t)\) then the mean strategy of active individuals in the subpopulation is \(\textbf{p}^*\). Therefore, one can intuitively expect that the population evolves towards \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) varying from moment to moment rather than \(\textbf{x}(\textbf{p}^*)\) which leads to the instability of \(\textbf{x}(\textbf{p}^*)\) in the examples. Each segment connects the composition \(Q^{\textbf{p}^*}\hspace{-1mm}\big (\textbf{x}(t)\big )\) with the corresponding state \(\textbf{x}(t)\)

Full size image

4 Discussion

Garay et al. [16] and Kr̆ivan and Cressman [25] incorporated time constraints into the model of evolutionary matrix games. As they pointed out, time constraints can essentially change evolutionary outcomes. In this article, we have continued the mathematical analyses started in Garay et al. [17], Varga et al. [42] on matrix games under time constraints, and we have demonstrated by two examples that static evolutionary stability does not imply the asymptotic stability of the corresponding state of the replicator equation in three or higher dimensions. This is an essential distinction from the case of classical matrix games, where static evolutionary stability implies dynamic stability [21, 36, 44].

Under the influence of time constraints, individuals in the population move between two states (active and inactive). Since there is no interaction in the inactive state, the duration spent in it can considerably influence how much a particular individual gains from the resources allocated through interactions. Furthermore, the time spent in the inactive state depends on the individual’s strategy. All of this can lead to that individuals of distinct strategies participate in significantly different numbers of interactions over a given period. Thus, in interpreting fitness, it is not enough to simply consider the expected payoff from an interaction, but one must also take into account the process of how interactions unfold. In this regard, matrix games under time constraints share similarities with several recent works in which various mechanisms govern the relevance of game payoffs for evolutionary success. For example, in the model of Foley et al. [12], current payoffs depend on past payoffs, leading to cyclic phenomena in the hierarchy network of the population. Constable et al. [7] introduces stochastic effects into his evolutionary model. Argasinski and Broom [1] and Argasinski and Rudnicki [2] link classical game theory with models where individuals can be in different states (e.g. age-structured population models), so success depends not only on the strategy, but also on the state. In these models, the payoff derived from the games is only a part of the process that, similarly to the time-delay in the time-constrained model, determines the evolutionary dynamics describing the population.

We provided two examples of matrix games under time constraints, wherein the equilibrium point corresponding to an evolutionarily stable strategy proved to be dynamically unstable with respect to the replicator dynamics. The first example pertains to the rock-paper-scissors game, which holds biological relevance in ecology [35], microbiology [24] or biotechnology [27]. Also, it is used for modelling interactions between cancer cells [39]. In addition to proving that arbitrary small distinctions between waiting times can destabilize the rest point corresponding to a UESS, we pointed out that a supercritical Hopf bifurcation can occur, resulting in a stable limit cycle emerging around the unstable rest point. Such a phenomenon is impossible in classic matrix games ([3, 44], Chapter 7.5 in Hofbauer and Sigmund [20]). Mobilia [31] and Toupo and Strogatz [40] obtained a similar result, although their focus was on investigating the effects of mutations rather than time constraints.

The second example exhibited a strict Nash equilibrium in addition to the dynamically unstable interior UESS. This is not possible in the classical case either since the existence of an interior ESS precludes the existence of another Nash equilibrium (see the paragraph after the proof of Theorem 6.4.1 in Hofbauer and Sigmund [20]).

In classical matrix games, if \(\textbf{p}^*\) is an ESS then state \(\textbf{x}^*\) with \(x^*_i=p_i^*\) is an evolutionarily stable state (Theorem 6.4.1 in Hofbauer and Sigmund [20]) in the population of pure phenotypes, that is, for some \(\delta >0\)

$$\begin{aligned} \sum _{i}^Nx^*_i W_i(\textbf{x})>\sum _{i}^Nx_i W_i(\textbf{x}) \qquad \text {whenever}\,\, 0<||\textbf{x}-\textbf{x}^*||<\delta , \end{aligned}$$

where \(W_i\) denotes the fitness of the i-th pure strategy. This condition means that state \(\textbf{x}^*\) exhibits higher fitness in a slightly perturbed state compared to the average fitness of that perturbed state. Therefore, one can expect that the population is steered back to state \(\textbf{x}^*\). Indeed, the above relation implies the asymptotic stability of \(\textbf{x}^*\) with respect to the replicator equation ([21, 44], Theorem 7.2.4 in Hofbauer and Sigmund [20]). Motivated by this, we investigated the composition corresponding to the UESS in a given population of pure phenotypes. Unlike the classical case, this composition depends on the current state of the population as depicted in Figs. 6 and 7. In general, it does not coincide with the state corresponding to the UESS. This discrepancy can potentially contribute to the instability of the state associated with the UESS, as demonstrated in our examples. On the other hand, this provides only a partial explanation since it is easy to give examples such that the composition corresponding to the UESS varies with the current state of the population, yet the state corresponding to the UESS is asymptotically stable. Therefore, future investigations are necessary for a deeper understanding of the stability property of the state corresponding to the UESS.

Finally, analysing the relationship between models on monomorphic and polymorphic populations can primarily appear to be a mathematical problem. However, it is related to the question of diversity in biology. In the theory of classic evolutionary matrix games, for instance, the existence of an interior monomorphic UESS implies stable diversity in polymorphic situations through strong stability [9, 10]. Therefore, making clear the difference between the monomorphic and polymorphic models is an interesting problem from the viewpoint of biology, as well.