The phenotypic gambit: selective pressures and ESS methodology in evolutionary game theory

Abstract

The ‘phenotypic gambit,’ the assumption that we can ignore genetics and look at the fitness of phenotypes to determine the expected evolutionary dynamics of a population, is often used in evolutionary game theory. However, as this paper will show, an overlooked genotype to phenotype map can qualitatively affect evolution in ways the phenotypic approach cannot predict or explain. This gives us reason to believe that, even in the long-term, correspondences between phenotypic predictions and dynamical outcomes are not robust for all plausible assumptions regarding the underlying genetics of traits. This paper shows important ways in which the phenotypic gambit can fail and how to proceed with evolutionary game theoretic modeling when it does.

Uncertainty about the underlying genetics of a trait is often used in evolutionary game theory as a justification for the ‘phenotypic gambit’—the assumption that we can ignore genetics and look at the fitness of phenotypes, the observable traits of organisms determined by some mixture of genetics and environment, to determine the expected evolutionary dynamics of a population. In phenotypic models, it is assumed that reproduction is asexual and that offspring are clones of their parent. Genetic models on the other hand take into account that in sexually reproducing organisms, offspring get their genetic material from both parents. Offspring may have a different genetic make-up from either of their parents and thus a different phenotype as well. The genes encoding for different traits become the focus of analysis rather than only the traits themselves. While the phenotypic approach both simplifies the mathematics and often generates accurate predictions of real world systems, it can also yield false predictions (Marrow et al. 1996; van Oers and Sinn 2011).

The emphasis on phenotypes rather than genotypes is so prevalent in evolutionary game theory that it is generally considered a phenotypic approach to evolutionary modeling (Hammerstein and Selten 1994, 935). However, there is no reason why evolutionary game theoretic models cannot be genetic in character. This paper will give concrete examples of genetic models in evolutionary game theory and demonstrate why the use of genetic models is important. In particular, it will show that varying which phenotype is associated with a particular genotype in genetic models can drastically affect the dynamics in a way that the phenotypic approach cannot predict or explain.

The organisms in the genetic models considered here are diploid; they have two sets of chromosomes. This means they can have two copies of a gene at a locus, or a place on a chromosome where a particular gene is found. Organisms may be homozygous, have two copies of the same allele (version of a gene), or heterozygous, have two different alleles, with respect to some locus. This paper will use evolutionary game theory to show that what assumptions are made about the heterozygote’s phenotype can change both the type of selective pressure acting on the population and the expected outcome of evolution. As we will see, this is in contrast to the relatively unimportant role often played by heterozygotes when genetics are incorporated into evolutionary game theoretic models (“Preliminaries: genotype to phenotype maps and evolutionary stable strategies, Selective Pressures and Stability and ESS Methodology” sections).

This paper will analyze several models of well-known games—namely the Nash Demand game, Hawk–Dove, and the Prisoner’s Dilemma—to show how genetics can affect the evolutionary process. This will show some ways in which the phenotypic gambit can fail to predict evolutionary outcomes. This is important because the phenotypic gambit plays an important role in evolutionary modeling, especially in what are called ‘adoptionist programs’ or optimization approaches to evolutionary theory. These approaches are built on the expectation of fitness-maximization, or the expectation that evolution will lead to some optimal set of phenotypes. Genetic constraints which would prevent the population from reaching this optimal point are “a fly in [the] ointment” for the phenotypic gambit and optimization approaches (Grafen 2014, 157). While we know the assumptions of phenotypic gambit are false, we could be comfortable relying on it if we knew that it gave us close to the right answer in most cases and if we could categorize the exceptional cases in which predictions based on the phenotypic gambit would not be approximately true (Grafen 2014).

Part of the analysis presented here will demonstrate how the well-known phenomenon of heterozygote advantage, where the heterozygote is more fit than either homozygote, can transfer over from the non-social to the social context, where evolutionary game theory is used. Heterozygote advantage has been commonly used as a counter-example to adoptionist programs (see, for example Grafen 2014). If the heterozygote is most fit, then genetic constraints prevent the population from evolving to become composed entirely of the most fit organisms, as will be further explained (in the “ESS methodology” section). This paper will show that in the social context there is an additional complication for adaptionist programs: the phenotype of the heterozygote can affect evolutionary outcomes even when its fitness is between the fitness of the two homozygotes.¹ Why this is important will be explained further (in the “Selective pressures and stability and ESS methodology” sections).

This demonstrates how the phenotype assigned to the heterozygote can importantly affect evolutionary processes and outcomes. However, it is often argued that in the long-term, the effects of these genetic constraints can be ignored and we can predict the evolutionary outcome based on phenotypic stability criteria, such as an evolutionary stable strategy (ESS) (see Marrow et al. 1996 and references therein). However, as we will see, connections between ESS methodology and long-term evolutionary outcomes rely on strong assumptions that may not be reasonable for every evolutionary process. So while the phenotypic gambit may pay off in many cases, we should be aware of its limitations and we should know how to proceed with evolutionary game theoretic modeling when the gambit fails.

The paper will proceed as follows. First, I will introduce genotype to phenotype maps and their use in evolutionary game theory. I will also review some relevant topics from game theory in a population setting and explain the concept of an ESS (“Preliminaries: genotype to phenotype maps and evolutionary stable strategies” section).  Then, I will discuss dynamics commonly used in evolutionary game theory for phenotypic and genetic models (“Dynamics for phenotypic and genetic models” section). With all this is place, I will consider models of a number of well-known games where differing assumptions about the heterozygote phenotype make significant differences for evolutionary outcomes (“Models” section). Next, I will discuss some lessons about selective pressures and ESS methodology that can be drawn from this analysis (“Selective pressures and stability and ESS methodology” sections, respectively). Finally, I will discuss some ways to think about phenotypic modeling in evolutionary game theory (“Concluding remarks” section).

Preliminaries: genotype to phenotype maps and evolutionary stable strategies

Evolutionary change depends on the fitness of organisms, which in turn depend on the organisms’ phenotypes. Thus changes in gene frequencies are crucially affected by what sort of genotype to phenotype mapping is assumed. A genotype to phenotype map simply describes what sort of phenotype is encoded for by each possible genotype, and so is responsible for connecting the genetic level with the phenotypic level. This mapping can be one-to-one, where each genotype encodes for a unique phenotype, or many-to-one, where multiple genotypes can encode for the same phenotype.² In the context of evolutionary game theory, the strategies of a game are considered to be phenotypes of the organisms, so the genotype to phenotype map in this context determines what strategy an organism of a given genotype will play. Just as in the classic examples of heterozygotism, heterozygotes game theoretic models can play a strategy that is distinct from the strategies associated with homozygotes.³

When making the phenotypic gambit, one assumes a very simple genotype to phenotype map. That is, one assumes there is simply a gene which maps one-to-one onto a strategy. Often, genetic models also assume a fairly simple genotype to phenotype map: they assume that the heterozygote strategy is a convex combination of strategies played by the homozygotes (Maynard Smith 1982; Binmore and Samuelson 2011).⁴ A convex combination in this context is a linear combination of strategies where the coefficients are positive and sum to one. So for instance, if there are two strategies, s₁ and s₂, a convex combination of the two would be αs₁ + βs₂ where α + β = 1. This means that the heterozygote could play the same strategy as one of the homozygotes (if α or β is zero), interpreted as one allele being dominant over the other. In this paper, I will provide examples where α and β aren’t zero, meaning the heterozygote plays a mixed strategy.⁵

Genotype to phenotype maps, however, are not always straightforward, especially when it comes to behavioral traits.⁶ Besides mathematical convenience, there is no reason to assume that an intermediate heterozygote phenotype must be a convex combination of the homozygote strategies in the particular way commonly assumed by these biological models, or a convex combination of homozygote strategies at all. Examples of different assumptions will be discussed (in the “Models” section).

This paper uses evolutionary game theory to analyze the effects of the genotype to phenotype map. An evolutionary game theoretic analysis begins with a game that can be used as a model of some strategic interaction an organism might participate in. The payoffs of these interactions are then taken to affect the organism’s fitness, so that a strategy which receives higher payoffs in the game can be generally expected to increase in frequency. In using evolutionary game theoretic analysis it is useful to relate the dynamical outcomes to equilibrium concepts in game theory, so I will introduce some of these concepts now.

Game theoretic equilibrium concepts are traditionally phrased in terms of a rational agent, but we can also apply these concepts in a biological setting where an organism’s phenotype is the strategy it plays in a game. It will be useful to explain at the start how we can apply these concepts in the population setting without any assumptions about the rationality of organisms. To do this, we look at the strategy a population as a whole is playing, or the effective strategy of the population, by taking the weighted average of all the strategies played in the population (Cressman 1992, 61–2). In a population setting, a Nash equilibrium corresponds to an effective strategy of the population that is a best response to itself—no other effective strategy could do better if played against it. This could be a pure Nash equilibrium, where every organism plays the same pure strategy, or a mixed Nash equilibrium, where more than one strategy is played within the population.

A pure Nash equilibrium hashed out as a population state corresponds to a phenotypic monomorphism—a state where every organism in the population has the same strategy (phenotype). A mixed Nash equilibrium could either correspond to a phenotypic monomorphism, where every organism mixes over a number of pure strategies with the same probability, or a phenotypic polymorphism in which there are multiple strategies (phenotypes) present. A phenotypic polymorphism could then have organisms playing strategies with different probabilities with the effective strategy of the population corresponding to a mixed Nash equilibrium. How the individual strategies of organisms combine to determine the effective strategy of a population can have important effects on selective pressures and evolutionary outcomes (Bergstrom and Godfrey-Smith 1998), but for the purposes here these distinctions will not be important.

One of the most commonly used concepts of evolutionary stability in evolutionary game theory, an evolutionary stable strategy (ESS), is a refinement of the Nash equilibrium concept:

A strategy s* is an ESS if and only if

• 1. u(s*, s*) ≥ u(s, s*) for any s in the strategy set, and

• 2. if u(s*, s*) = u(s, s*), then u(s*, s) > u(s, s).

where u(x, y) is the utility (or fitness benefit) an organism using strategy x gets from an interaction with an organism using strategy y. That is, (1) a strategy s* is a best response to itself—it is a Nash equilibrium—and (2) if another strategy s is also a best response to s*, then s* does better against s than s does against itself. This is a way of making precise the idea that an ESS is an ‘uninvadable’ strategy, that when a population is at an ESS other strategies cannot arise through mutation and invade the population. For population games, it is helpful to view s and s* as possible effective strategies of the population (Eshel 1996).

Dynamics for phenotypic and genetic models

In phenotypic evolutionary game theoretic models, the replicator dynamics are often used to model the evolutionary process. Under these dynamics, if the fitness of a strategy is greater than the average fitness of the population, the frequency of that strategy will increase. A strategy’s fitness is determined by its payoffs against other strategies in addition to the population composition, which determines how likely it is for organisms using that strategy to receive each of their payoffs. Let us call the allele in question A₁, its frequency p, and its fitness $w_{p}$. The frequency of the other allele, A₂, is q where $q = 1 - p$ and its fitness is denoted by $w_{q}$. The replicator dynamic is then governed by the following equation:

$$\dot{p} = p\left[ {w_{p} - \left( {pw_{p} + qw_{q} } \right)} \right]$$

where $\left( {pw_{p} + qw_{q} } \right)$ is the average fitness of the population.⁷ Although I have described this in terms of frequencies of alleles, phenotypes are determined by a single allele so tracking changes in allele frequencies is equivalent to tracking changes in phenotype frequencies.

For genetic models of randomly mating, infinite populations, the Hardy–Weinberg dynamics is oftentimes employed. This is similar to the replicator dynamics but incorporates features of sexual reproduction. In this dynamic, offspring get their genetic material from two parents, one allele from each (unlike in the replicator dynamics where offspring are assumed to be clones of their parent). Selective pressure depends on the fitness of a strategy, but ultimately the fitness of each allele determines the evolution of a population. This is because the fitness of an organism is determined by its phenotype, but the frequency of alleles passed on through random mating will determine the phenotypes present in the next generation. The evolutionary trajectory is governed by the following equation:

$$p_{next} = \frac{{w_{11} p^{2} + w_{12} pq}}{{w_{11} p^{2} + w_{12} 2pq + w_{22} q^{2} }}$$

where w_xy represents the fitness of the A_xA_y genotype. The numerator of this equation is the fitness of the A₁ allele⁸ and the denominator is the average fitness of the population.⁹ Since alleles recombine according to the laws of probability, p ² is the frequency of individuals homozygous with respect to A₁ (denoted A₁A₁), q ² is the frequency of A₂A_2- homozygotes and 2pq is the frequency of heterozygotes (A₁A₂).

In order to discuss the dynamical outcomes in the next section, it is important to first explain what I mean by the term ‘stable equilibrium’. First, a steady state is configuration of genotypes within a population where allele frequencies do not change over time in the absence of any drift or mutation. A stable equilibrium is then a steady state with the additional property of being locally asymptotically stable: when there is a small amount of mutation or drift that changes the gene frequencies, selective pressure will cause the population to return to the steady state.

The ESS concept is useful because it has been proven to match up with many common dynamics, such as the replicator dynamics and the Hardy–Weinberg dynamics, in such a way that if an effective strategy is an ESS of the game, then under certain assumptions it is also a stable equilibrium of the dynamic (Hofbauer and Sigmund 1998). The connection between the replicator dynamics and the ESS concept can be fairly straightforward: a strategy is a stable equilibrium of the replicator dynamics if it is an ESS of the game (Hammerstein and Selten 1994). However, even in phenotypic models, there are still many cases where an ESS of a game will fail to coincide with any steady states of the dynamics (Huttegger and Zollman 2013). The connection between the ESS concept and stable equilibria in models which incorporate genetics is even less straightforward. The usefulness of ESS methodology for predicting outcomes in these models will be discussed (in the “ESS methodology” section).

In what follows I will look at the effects of the heterozygote strategy on the applicability of the standard phenotypic dynamics and ESS methodology in genetic models. This is, of course, only one particular de-idealization. The models here still make many assumptions in order to simplify analysis. In particular, it is assumed that the population is infinite and that interactions between organisms are random. Removing either of these assumptions can also affect evolutionary dynamics and the usefulness of ESS methodology (see, for example, Young and Foster 1991; Nowak et al. 2010; Fogel and Fogel 2011).

Models

In this section, I will present three examples using well-known games to illustrate the effects of different phenotype to genotype maps. Each example will begin with a description of a game. This will be followed by the analysis of a version of the game with two strategies using both replicator dynamics and a genetic model where the heterozygote plays a mixed strategy (this sort of genetic model will be referred to as the ‘mixed strategy case’). I will then provide an analysis of a version of the game with three strategies using replicator dynamics, followed by a genetic model in which the heterozygote plays one of the three pure strategies (the ‘pure strategy case’). In these models, the allele associated with the first strategy listed in the payoff table will be called A₁ and its frequency p. For ease of comparison, in the examples considered here I restrict attention to cases of intermediate inheritance, where the heterozygote phenotype is ‘between’ the homozygote phenotypes.

Example 1: Nash demand

The Nash Demand game represents a situation in which two actors try to divide a resource by demanding a certain fraction of the contested resource. If the two demands together are less than or equal to the total amount of the resource, each gets the share they demanded. If the demands add up to more than the whole, neither gets anything. Similar to Skyrms (1996), I assume that there is one allele for demanding 1/3 of the resource and another for demanding 2/3. We can get the fitness of each phenotype from the payoffs in the first and third rows of Table 1 (along with the population distribution).

Table 1 Nash Demand game

Under the replicator dynamic, there is a stable polymorphic equilibrium at p = 0.5, where half the population is demanding 1/3 and the other half is demanding 2/3. In the genetic model with these same strategies, one homozygote (A₁A₁) demands 1/3 and the other (A₂A₂) demands 2/3. The heterozygote (A₁A₂) will be modeled as mixing between these two demands with equal probability. How the heterozygote mixes between the two demands is arbitrary and does not qualitatively affect the dynamic in the sense that it does not alter the effective strategy, the weighted average of the strategies played in the population, at equilibrium. Why this is true of the mixed strategy case will be explained in the next section. There is again a stable polymorphic equilibrium at p = 0.5, where 25 % of the population demands 1/3, 50 % demands 1/3 half the time and 2/3 half of the time, and 25 % demands 2/3. So, half of the demands made in the population are for 1/3 and the other half are for 2/3; the effective strategy of the population is the Nash Equilibrium.

If the homozygotes demand 1/3 and 2/3 of a resource, the heterozygote could play the commonly assumed convex combination of these and mix between demanding these amounts. However, heterozygotes may also demand some portion of the resource between 1/3 and 2/3, say 1/2, which can also be viewed as convex combination of homozygote strategies since the amount demanded is a convex combination of the amounts demanded by each homozygote. These are both possible intermediate strategies and both might be of interest from a modeling standpoint.

Under the replicator dynamic with these three strategies, there are two stable equilibria: one in which the entire population demands 1/2 and another in which half the population demands 1/3 while the other half demands 2/3 (Skyrms 1996). These effective strategies are both Nash equilibria of the game. On the other hand, in a genetic model with the heterozygote demanding 1/2, the population always evolves to a stable polymorphic equilibrium at p ≈ 0.63. This means that roughly 36.7 % demand 1/3, 46.7 % demand 1/2, and 13.7 % demand 2/3.¹⁰ The reasons for ending up at this alternative equilibrium this will be discussed in the next two sections. For now, we can note that the polymorphism in the mixed strategy case, where the population demands are split evenly between demanding 1/3 and demanding 2/3, is not possible in the pure strategy case because, even if the population were to start at this effective strategy, mating would lead to the production of heterozygotes demanding 1/2 in the next generation.

Example 2: Hawk–Dove

Maynard Smith and Price (1973) introduced the Hawk–Dove game as a description of certain animal conflicts. ‘Hawk’ refers to an aggressive animal that fights for a resource and ‘dove’ refers to an animal that is unwilling to fight. A hawk then always gets the contested resource when encountering a dove, but will split the resource and cost of entering into a conflict (injuries incurred, expended energy, etc.) with another hawk. A dove will always surrender the resource to a hawk, but will split the resource peacefully with another dove. As in Maynard Smith (1982), when the value of the resource is 2 and the cost of conflict is 4, the payoffs are as in Table 2.

Table 2 Hawk–Dove–Retaliator

Under the replicator dynamics, there is one stable equilibrium at p = 0.5 where each allele is found in equal frequency and each strategy is played by half of the population. In the mixed strategy case, there is again one stable equilibrium at p = 0.5, but this corresponds to a population where 25 % of the population always plays dove, 25 % always plays hawk and 50 % mixes between the two strategies with equal probability. The effective strategy of the population is to play each strategy with equal probability, the Nash equilibrium of the game.

In the Hawk–Dove game, the heterozygote could be a ‘retaliator,’ which will respond to a hawk’s aggressive behavior with aggressive behavior, but acts like a dove otherwise. This amounts to a novel pure strategy which is not a convex combination of the homozygotes’ strategies.¹¹ With these three strategies, under the replicator dynamics the only stable equilibrium in which half of the organisms play hawk and half play dove, like in the game with two strategies (Maynard Smith 1982).¹² When heterozygotes are retaliators, the population ends up at a stable polymorphic equilibrium at p ≈ 0.89. Roughly 69.4 % of the population plays dove, 27.8 % plays retaliator, and 2.8 % plays hawk. In the pure strategy case, there is much more dovish behavior than in the mixed strategy case. This is because the retaliator heterozygote acts as sort of a buffer for the doves against invading hawks—as hawks and thus the A₂ allele increase in frequency so do the retaliators. The hawks do much less well against retaliators than do doves, so they are prevented from comprising a large part of the population.

Example 3: (Iterated) prisoner’s dilemma

The Prisoner’s Dilemma is another game frequently encountered in the biological literature. It is often used to explain the evolution of biological altruism as an organism who ‘cooperates’ can be interpreted as sacrificing its own fitness in order to increase the fitness of another.¹³ I assume that an individual can sacrifice 1 unit of fitness to increase the fitness of another by 3 units. Further, I assume the game is iterated, or played repeatedly against the same organism, 10 times. The payoffs are shown in Table 3.

Table 3 Iterated Prisoner's Dilemma

In the game with only cooperators and defectors, there is only one stable equilibrium which corresponds to a population of all defectors (p = 0), the Nash equilibrium, for both the replicator dynamics and the mixed strategy case.

The fact that the game is iterated allows for a variety of possible additional strategies. One commonly used strategy is tit-for-tat, which cooperates on the first iteration and only defects if defected against in the previous iteration (Young and Foster 1991, Axelrod 2006, Nowak 2006). Applying the replicator dynamics to the game with these three strategies reveals that again there is only one stable equilibrium where the population is composed entirely of defectors.¹⁴ However, in the pure strategy case with the heterozygote playing tit-for-tat, there are two stable equilibria. Again, one of these is composed of all defectors, but there is another at p ≈ 0.81 where of the population roughly 65.7 % cooperate, 30.7 % play tit-for-tat, and 3.6 % defect. This is similar to the phenomenon that occurs in the Hawk–Dove game. Here, there is more altruistic behavior in the pure strategy case than in the mixed strategy case because in the pure strategy case, tit-for-tat can act as a buffer protecting the altruists from invasion by defectors (Rubin 2015).

One might be concerned that, at least with Hawk–Dove and the iterated Prisoner’s Dilemma, the heterozygote strategies chosen in the pure strategy case are more cognitively demanding than the heterozygote strategies in the mixed strategy case. These heterozygote strategies were chosen because they are salient in the literature. A reader familiar with models of these games will immediately see how the results differ from those produced in other models of the same games. However, the results are not dependent on the cognitively complexity of the heterozygote strategy; the point can be made with models using less cognitively complex heterozygote strategies. For an example of such models for Hawk–Dove, see the “Appendix”.

Selective pressures and stability

Before investigating stability criteria, there are a couple of significant observations to be made about selective pressures in the examples above. The selective pressure acting on the population at the phenotypic level is best visualized by graphing the fitness of each phenotype in the population for different proportions of strategies, as in Fig. 1A–C, The first thing to note is that in the pure strategy case there are phenotypic fitness differentials at a stable equilibrium. It might seem odd to call such a population state an equilibrium, as equilibria are generally thought of as population states in which there are no fitness differences. However, each allele has the same fitness at equilibrium, so the allele frequencies, and thus the genotype and phenotype frequencies, do not change over time. Second, this means that different types of selective pressure are at work in pure strategy cases. For example, in Fig. 1Ab, the heterozygote strategy is the most fit around the equilibrium. This points to an important aspect of the dynamics of the pure strategy case: heterozygote advantage can play a vital role in maintaining a stable equilibrium of the population.¹⁵ While not generally considered as part of evolutionary game theoretic analysis, there are noteworthy examples of heterozygote advantage. One classic example is the sickle cell mutation, which causes sickle cell anemia in people with two mutated genes, but in heterozygotes produces malaria resistance with little to no symptoms of sickle cell anemia.

Fig. 1

Graphs of the fitness of each phenotype over possible gene frequencies: w₁₁ represents the fitness of the strategy associated with of the A₁A₁ homozygote, w₁₂ the A₁A₂ heterozygote, and w₂₂ the A₂A₂ homozygote. The stable polymorphic equilibrium is indicated by a vertical line.

Neither of these features are found in the mixed strategy case. When the effective strategy is a pure Nash equilibrium, there is only one strategy present in the population, so all strategies present will have the same fitness by default (because there is only one strategy present in the population). As seen in Fig. 1Aa, Ba, when the effective strategy of the population is a mixed Nash equilibrium, all strategies present have the same fitness so the population composition is not changing. In fact, in the mixed strategy case (with intermediate inheritance), there can never be fitness differentials at equilibrium because the heterozygote’s fitness is always between the fitness of the two homozygotes’ (except at an equilibrium where all the phenotypes present have the same fitness). Since the heterozygote plays the same strategy as one homozygote with some probability and the strategy of the other homozygote with some probability, it will receive the same payoff as one or the other homozygote with some probability, meaning that its fitness is a weighted average of the homozygotes’ fitnesses. That is, the heterozygote’s fitness is a convex combination of the homozygote fitnesses because of the way the heterozygote strategy is a convex combination of homozygote strategies.

Heterozygote advantage has been traditionally presented as a problem for adaptionist thinking when there is no frequency dependent selection, so it should be unsurprising that remains a problem when there is frequency dependent selection. However, with frequency dependent selection there’s an additional complication: the phenotype of the heterozygote can affect the location of the rest point even when it is not the most fit. For example, Fig. 1Bb shows that the heterozygote retaliator strategy is never the most fit in the Hawk–Dove game. Yet, if we compare the location of the rest point in the pure strategy case to the mixed strategy case, we see that there is much more dovish behavior in the pure strategy case. The fact that heterozygotes play the intermediate retaliator strategy rather than an intermediate mixed strategy means that doves have a fitness advantage for a larger portion of the state space. Since the fitness of a strategy depends on the frequency of the other strategies in the population, the strategy assigned to the heterozygote can affect the location of the rest point by affecting the fitness of the homozygotes. In this case, the presence of the heterozygote retaliators increases the fitness of doves and decreases the fitness of hawks, because doves do better against retaliators than hawks do.

ESS methodology

These observations show that the intermediate strategy assigned to the heterozygote can qualitatively affect the evolutionary process. The effect can be highlighted by considering how ESS methodology fails to predict these evolutionary outcomes. We will first look at this stability concept for short-term evolution, evolution with a given set of genotypes. It is well-known that the heterozygote strategy, or other constraints, can make ESS predictions and dynamical outcomes fail to coincide in the short-term.

In the mixed strategy case, if the ESS can be reached by some combination of the strategies in the population, as is the case in the examples here, then the ESS is a stable equilibrium of a genetic model (Maynard Smith 1982).¹⁶ This means that the evolutionary outcome, in terms of the effective strategy of the population, can be predicted based solely on the payoff table of the game. This calls attention to the fact that the role the heterozygote usually plays in genetic models is relatively unimportant. The heterozygote may be assigned to play the same strategy as either one of the homozygotes or it may be assigned to play some mix of the two strategies, but in any of these cases the same phenotypic outcome is predicted by the model. The same phenotypic outcome will also be predicted by a model using the replicator dynamics. That is, the effective strategy of the population at equilibrium is not affected by a choice among one of these modeling assumptions.

This is in contrast to the pure strategy case, where we have seen that the choice of strategy assigned to the heterozygote can drastically affect the population composition at equilibrium and the population is prevented from reaching an ESS due to the probabilistic recombination of genes. In this case, the underlying genotype to phenotype map makes ESS methodology fail to be useful in describing the dynamic process, at least in the short term. This phenomenon is similar to that described in Hines and Bishop (1984a). They show, using the Rock-Paper-Scissors game, how an ESS can fail to be a stable equilibrium. In Hines and Bishop’s examples, both homozygotes and the heterozygote play different mixtures of rock, paper, and scissors. Notably, they consider cases where the heterozygote strategy is not a convex combination of the homozygotes’ strategies. They show how the ESS can be ‘trapped’: if we represent geometrically the set of all possible population effective strategies, the effective strategy corresponding to the ESS can fall in an empty space or ‘hole’ in this set.

Their examples illustrate how an ESS must be attainable and maintainable in order to be a stable equilibrium. In must be attainable in that some combination of the genes present in the population can yield a population effective strategy that is the ESS. For instance, if in the prisoners’ dilemma there is an allele for cooperating 50 % of the time and an allele for cooperating 100 % of the time, the ESS of playing defect 100 % of the time would not be attainable. The ESS also must be maintainable in that once the populations’ effective strategy is the ESS it will remain that way. For instance, in the pure strategy case of the Nash Demand game, the ESS of demand 1/2 is not maintainable because even if the population were to start out entirely composed of the heterozygous demand 1/2 organisms, there would be homozygotes introduced after mating. This paper makes a point similar to Hines and Bishop (1984a), but differs in that it explores how the phenotypic gambit fails in simpler cases with a more intuitive genotype-to-phenotype map. These differences motivate concerns about the use of ESS methodology for long-term evolutionary processes, which we turn to next.

If the ESS is both attainable and maintainable then, under certain assumptions, the effective strategy of a diploid population at equilibrium is an ESS (Hines and Bishop 1983) and the ESS will be a stable equilibrium of the dynamics (Hines and Bishop 1984b). However, these results depend on assumptions about the underlying genetics and genetic considerations, such as linkage¹⁷ and epistasis,¹⁸ have led some to argue that ESS methodology is best suited for analysis of long-term evolutionary processes. (Eshel 1982, 1996; Eshel and Feldman 2001). For long-term evolution, we consider that any possible allele, from some large enough set of ‘relevant’ alleles, could be introduced through mutation. The process is one of successively introducing potentially beneficial mutations into a population. Each time a mutant allele successfully invades, this starts a new process of short-term evolution which evolves to its end point before another potentially beneficial allele is introduced. This is often described as the ‘streetcar theory of evolution’ as the end points of short-term evolutionary processes can be thought of as stops along the way to the final destination reached by long-term evolution (Hammerstein 1996). While short-term and long-term evolutionary processes are connected, they are qualitatively different.

It has been suggested that in the context of long-term evolution the phenotypic gambit is a safe assumption, that predictions based on genetic models and ESS methodology will “meet at the terminus of the evolutionary streetcar” (Marrow et al. 1996). There have been many results connecting the outcomes of long-term evolutionary processes with ESS methodology, which are proven on the assumption that any possible allele can invade the population (see Hammerstein and Selten 1994; Weissing 1996; Eshel and Feldman 2001 and references therein). Note that the set of possible strategies may be still be restricted to those considered in a model, so ‘any possible allele’ invading means that the mutant alleles can code for any propensity for playing some combination of these strategies. When we generalize from simple models to more abstract scenarios, with many possible strategies and alleles and no specific game to ground our intuitions, it might be hard to tell how strong this assumption is. The models provided above can serve to make clear the sorts of intuitions we might have when thinking about the mixed strategy case or the pure strategy case and how these intuitions transfer over into thinking against a more abstract backdrop.

In the mixed strategy case, invasion by other possible mutant alleles means that the population’s genetic composition can change substantially over time. However, since the population is already at an ESS, the effective strategy of the population does not change over time. This means that any new alleles that persist in the population will have the same average fitness as the existing alleles and will only increase in frequency through drift. For instance, in example 2, the effective strategy of the population is to play hawk half of the time and dove half of the time. If a new allele were to arise within the population which caused any individual with that allele to play each strategy with equal probability, then this allele would have the same fitness as the population average fitness (since organisms with this allele play the same strategy as the population plays on average). What the underlying genetics of the end product of long-term evolution will look like cannot be predicted by ESS methodology but the effective strategy of the population can be (Eshel 1996).

When we generalize to abstract scenarios with many possible strategies and alleles, where the ESS strategy might not initially be reachable by some combination of available strategies in the population, having the mixed strategy case in mind makes considering any possible invading allele seem unproblematic: if the alleles code for different mixtures over the strategies it is reasonable to suppose that any mixture is possible in the long run. Similarly, if we think of the heterozygote as playing a different mix of the same pure strategies as the homozygotes, as in Hines and Bishop (1984a), considering mutations that allow different genotypes to play these pure strategies with different probabilities likely seems uncontroversial.

However, there is no reason to assume that a genotype-to-phenotype map needs to assign genotypes different probabilities of playing certain strategies. Genotypes might instead map to different pure strategies, as in the Nash Demand game where they mapped to different amounts an organism could demand. This leads us to consider the pure strategy case, in which the strategy with the highest fitness at the equilibrium may be played by the heterozygote and is only prevented from taking over the population by probabilistic recombination of genes. If this strategy is an ESS, as in example 1, any mutant gene that allows a homozygote to play this strategy would take over the population. Thus, if we allow that any possible mutant allele can invade, the average phenotype of the population will be different when looking at short-term versus long-term evolution.

However, in some cases there might be reason to believe that certain strategies may only be played by heterozygotes and that assigning a certain strategy to a heterozygote is the most biologically realistic modeling assumption, even in the long-term. Then, the set of relevant mutations to consider would not include those which would allow the population effective strategy to be the ESS. One might again think of sickle cell and malaria resistance as an example: in heterozygotes one allele codes for normal hemoglobin and the other for sickle cells. It is not clear that a homozygote could have both the malaria resistance associated with sickle cells, and the lack of sickle cell anemia from also having normal cells.¹⁹ While things are obviously more complex in the behavioral setting, we might want to similarly restrict the set of relevant mutations to consider. This is not to argue that the pure strategy case is more biologically plausible. It is difficult to see how we could know a priori which genotype to phenotype map is more plausible in any particular case, in which case we should not ignore the possibility that the underlying genetics of certain behaviors could be more like the pure strategy case than the mixed strategy case.

There are a couple ways to think of this in relation to the argument that we can use ESS methodology to make predictions about the outcome of long-term evolution. First, and most obviously, it shows that genetic considerations can prevent ESS methodology from being used reliably to predict long term evolutionary outcomes. Heterozygote advantage is a well-known phenomenon but examples of how it can come into play in evolutionary game theory are hard to come by and it is not often discussed as a reason why evolution might not lead to an ESS. Second, you might think of this as demonstrating how strong of an assumption it is to allow that any possible allele can invade. The assumption that any allele can invade effectively assumes away the possibility that the importance of the heterozygote strategy will be maintained in the long term by assuming it is always possible that a mutation could arise which lets the heterozygote strategy be played by a homozygote. In other words, it assumes that in the long-run the ESS will always end up being genetically attainable and maintainable.

When talking about long-term evolution, the set of possible mutations is restricted to those which encode for propensities to play strategies from some predetermined set, so it is not unreasonable to suggest that we might restrict the set of possible mutations in some other way. If there is reason for a restriction in some context, then in that context there is reason to expect that the connection between ESS methodology and long-term evolutionary stability will break down. This goes to show that one should be wary of considering ESS predictions to be robust for all reasonable genotype spaces.

Concluding remarks

This paper has shown important ways in which the phenotypic gambit can fail and, when it does fail, how one can build simple genetic models which illuminate the possibilities of evolution. The models provided here demonstrate that genetic models can lead to interesting new equilibria in several well-known games. The pure strategy case shows how the heterozygote strategy can play an important role in evolutionary game theoretic dynamics which, for obvious reasons, is not discussed in phenotypic modeling. Relatedly, the pure strategy case shows an important area where and ESS methodology can fail to apply reliably as a stability concept for both short-term and long-term evolution. The more commonly assumed mixed strategy case obscures the fact that heterozygote strategies can play an important role in evolution by assuming the heterozygote strategy is a particular kind of linear combination of the homozygotes’ strategies that makes ESS methodology easy to apply.

This is not to argue that we should be wary of models employing the phenotypic gambit. These models are a valuable way of investigating the evolutionary impacts of traits of interest. Nor is it to argue that none of the authors cited here are aware of the limitations discussed above. Maynard Smith (1982) mentioned heterozygote advantage as a problematic case for ESS methodology from the beginning. That said, models that fall under the pure strategy case are not used often because they add mathematical complexity and many times connections between ESS methodology and long-term evolutionary outcomes are taken to be robust for any type of genotype to phenotype map. The simple models here demonstrate concretely that there are plausible genotype to phenotype maps that may be of interest to modelers for which ESS methodology will not apply even in long-term evolution. Further, they show that heterozygote advantage is not the only problematic case for the phenotypic gambit and adaptionist programs; the heterozygote can affect the evolutionary outcome even when it is never the most fit.

Evolutionary game theory is useful because it provides tools for exploring the possibilities of evolutionary change, and genetic models are a necessary component of a complete exploration. Assuming we know the underlying genetics of a trait and making predictions given these assumptions provides us with important theoretical knowledge about evolutionary processes. This theoretical knowledge in turn allows us to draw conclusions about the nature of evolutionary change (Eshel 1996, 490–493). However, in drawing these conclusions it is important to consider and reconsider the assumptions made in models. When making a gambit, one should be aware of the ways in which the gambit can fail and how to proceed with modeling when it does.

Appendix 1: Hawk–Dove–Goose

One might be concerned that the analysis in this paper is somehow affected by the fact that the heterozygote strategies described in the pure strategy case for the Hawk–Dove game and the iterated Prisoner’s Dilemma are more cognitively complex than in the mixed strategy case. To ease this concern, I consider an alternative approach to modeling the heterozygote pure strategy in Hawk–Dove. A natural suggestion is that the heterozygote does not condition their behavior on the type of their counterpart, but the level of aggression displayed is somewhere between hawkish and dovish. We might call such an individual a ‘goose’.

When a goose interacts with a hawk, the hawk gets the resource because it is more aggressive. Likewise, when a dove interacts with a goose, the goose gets the resource because it is more aggressive. What happens when two geese meet? There are two plausible options. First, the two geese may get enter a conflict and end up splitting the resource and the cost of conflict. In this case we would reasonably assume that the cost of conflict for two geese is less than the cost for two hawks, as the level of aggression displayed is less. The payoffs for the interaction among these three types is shown in Table 4.

Table 4 Hawk–Dove–Goose option 1

For option 1, the replicator dynamics leads to a polymorphic equilibrium where a third of the population are geese and two-thirds of the population are hawks. By contrast, a genetic model with heterozygous geese leads to one stable polymorphic equilibrium at p ≈ 0.20, where the population is roughly 3.9 % doves, 31.7 % geese, and 64.4 % hawks.

The second option is that the two geese end up peacefully splitting the resource. The idea behind this is that geese are aggressive enough to signal aggression but not aggressive enough to actually engage in conflict. So both will signal aggression, but back down once they see the aggressive signal. The payoffs for the second option are shown in Table 5.

Table 5 Hawk–Dove–Goose option 2

In option 2, for the replicator dynamics there is one stable equilibrium, composed of 50 % geese and 50 % hawks. For the genetic model there is one polymorphic equilibrium at p ≈ 0.26, where about 6.8 % are doves, 38.6 % are geese, and about 54.6 % are hawks. In either case, we can see that the genetic models differ from the phenotypic models and, more importantly, differ from the genetic mixed strategy case discussed in the main text.