Model of Interaction Between Learning and Evolution

Abstract

The lecture characterizes the following main properties of interaction between learning and evolution: (1) the mechanism of the genetic assimilation, (2) the hiding effect, (3) the role of the learning load at investigated processes of learning and evolution. During the genetic assimilation, phenotypes of modeled organisms move towards the optimum at learning; after this, genotypes of selected organisms also move towards the optimum. The hiding effect means that strong learning can inhibit the evolutionary search for the optimal genotype. The learning load can lead to a significant acceleration of evolution.

1 Introduction

Our model continues the works by Hinton and Nowlan and Mayley [1, 2], who simulated some features of interaction between learning and evolution. We use also the quasispecies model [3, 4] and our estimations of the evolutionary rate and the efficiency of evolutionary algorithms [5].

2 Description of the Model

We consider the evolving population of modeled organisms. Each organism has the genotype and the phenotype. We assume that the genotype and the phenotype of the organism have the same form, namely, they are chains; symbols of both chains are equal to 0 or 1. The length of these chains is equal to N. For example, we can assume that the genotype encodes a modeled DNA chain, symbols of which are equal to 0 or 1, and the phenotype is determined by the neural network of the organism, the synaptic weights of the neural network are equal to 0 or 1 too. These weights are adjusted by means of learning during the organism life.

The evolving population consists of n organisms, genotypes of organisms are S_Gk, k = 1,…,n. The organism genotype S_Gk is a chain of symbols, S_Gki, i = 1,…,N. N, n >> 1, 2^N >> n. The values N and n do not change during evolution. Symbols S_Gki are equal to 0 or 1. The evolutionary process is a sequence of generations. The new generation is obtained from the old one by means of selection and mutations. Genotypes of organisms of the initial generation are random. Organisms inherit the genotypes from their parents, these genotypes do not change during the organism life and are transmitted (with small mutations) to their descendants. Mutations are random changes of symbols S_Gki. The evolutionary process is similar to that of in the quasispecies model [3, 4].

Phenotypes of organisms S_Pk are chains of symbols S_Pki, k = 1,…,n, i = 1,…,N; S_Pki = 0 or 1. The organism receives the genotype S_Gk at its birth. The initial phenotype of the organism at its birth is equal to the organism genotype: S_Pk(t = 1) = S_Gk. The lifetime of any organism is equal to T. The time is discrete: t = 1,…,T. T is the duration of the generation. The phenotype S_Pk is modified during the organism life by means of learning.

It is assumed that there is the certain optimal chain S_M, which is searched for in processes of evolution and learning. Symbols S_Mi of this chain S_M are also equal to 0 or 1; the length of the chain S_M is N. For a concrete computer simulation, the chain S_M is fixed; symbols of this chain are chosen randomly.

Learning is performed by means of the following method of trial and error. In every time moment t, each symbol of the phenotype S_Pk of any organism is randomly changed to 0 or 1, and if this new symbol S_Pki coincides with the corresponding symbol S_Mi of the optimal chain S_M, then this symbol is fixed in the phenotype S_Pk, otherwise, the old symbol of the phenotype S_Pk is restored. So, during learning, the phenotype S_Pk moves towards the optimal chain S_M.

At the end of the generation, the selection of organisms in accordance with their fitness takes place. The fitness of the k-th organism is determined by the final phenotype S_Pk in the time moment t = T. We denote this chain S_Fk, i.e. we set S_Fk = S_Pk(t = T). The fitness of the k-th organism is determined by the Hamming distance ρ = ρ(S_Fk,S_M) between the chains S_Fk and S_M:

$$ f_{k} = { \exp }\left[ {{-}\beta \rho \left( {{\mathbf{S}}_{{{\mathbf{F}}k}} ,{\mathbf{S}}_{{\mathbf{M}}} } \right)} \right] + \varepsilon , $$

(1)

where β is the positive parameter, which characterizes the intensity of selection, 0 < ε << 1. The role of the value ε in (1) can be considered as the influence of random factors of the environment on the fitness of organisms.

The selection of organisms into a new generation is made by means of the well-known method of fitness proportionate selection (or roulette wheel selection). The probability of the selection of a certain organism into the next generation is proportional to its fitness. The choice of an organism into the next generation takes place n times, so the number of organisms in the population at all generations is equal to n.

Thus, organisms are selected at the end of a generation in accordance with their final phenotypes S_Fk = S_Pk(t = T), i.e. in accordance with the final result of learning, whereas genotypes S_Gk (modified by small mutations) are transmitted from parents to descendants.

Additionally, similar to Mayley [2], we take into account the learning load, namely, we assume that the learning process has a certain burden for the organism and the fitness of the organism can be reduced under the influence of the load. For this purpose, we consider the modified fitness of organisms:

$$ f_{mk} = { \exp }\left( { - \alpha d} \right){\{ }{ \exp }\left[ {{-}\beta \rho \left( {{\mathbf{S}}_{{{\mathbf{F}}k}} ,{\mathbf{S}}_{{\mathbf{M}}} } \right)} \right] + \varepsilon \} , $$

(2)

where α is the positive parameter, which takes into account the learning load, d = ρ(S_Gk,S_Fk) is the Hamming distance between the initial S_Pk(t = 1) = S_Gk and the final S_Pk(t = T) = S_Fk phenotypes of the organism, i.e. the value d characterizes the intensity of the whole learning process of the organism during its life.

3 Results of Computer Simulation

Two modes of operation of the model are considered: (1) the regime of the evolution combined with learning, as described above, (2) the regime of “pure evolution”, that is the evolution without learning. The parameters of the model at the simulation are chosen in such manner that the evolutionary search is effective (the experience of the work [5] at this choice was used): N = 100, β = 1, P_m = N ⁻¹ = 0.01, n = N = 100, T = 2, ε = 10⁻⁶, α = 1 (P_m is the probability of change of any symbol S_Gki at mutations).

The results of the simulation are averaged over 1000 or 10000 calculations. This averaging ensures the good accuracy of the simulation; typical errors are smaller than 1–2%.

3.1 Comparison of Regimes of Pure Evolution and Evolution Combined with Learning

Figure 1 shows the dependence of the average (for the population) Hamming distance ρ = ρ(S_Gk,S_M) between the genotypes S_Gk of organisms in the population and the optimal chain S_M on the generation number G. The curve 1 characterizes the regime of evolution combined with learning; the curve 2 characterizes the regime of pure evolution. The fitness of organisms is determined by the expression (1). We can see that the pure evolution without learning (the curve 2) does not optimize organisms S_k; whereas evolution combined with learning (the curve 1) obviously ensures the movement towards the optimal chain S_M.

Fig. 1.

The dependence of the average Hamming distance <ρ> = <ρ(S_Gk,S_M)> between genotypes S_Gk and the optimal chain S_M on the generation number G. The curve 1 characterizes the regime of evolution combined with learning; the curve 2 characterizes the regime of pure evolution.

To understand, why the pure evolution does not ensure a decrease of the value ρ, let us estimate the value of the fitness (1) in the initial population. The Hamming distance ρ = ρ(S_Gk,S_M) for initial genotypes is of the order of N/2 = 50, therefore, exp(−ρ) ~ 10⁻²² and exp(−ρ) << ε. This means that all organisms of the population have approximately the same value of the fitness f_k ≈ ε. Consequently, the evolutionary optimization of genotypes does not occur in the case of the pure evolution. Thus, the movement towards S_M occurs only in the presence of learning; this movement leads to the decrease of the value ρ.

Consider the effect of the acceleration of the evolutionary process by learning (the curve 1 in Fig. 1). Analysis of the results of simulations shows that the gradual decrease of the values ρ = ρ(S_Gk,S_M) occurs as follows. First, the learning shifts the distribution of organisms n(ρ) on the value ρ towards smaller ρ, so the values ρ = ρ(S_Fk,S_M) become small enough, such that exp[−ρ(S_Fk,S_M)] is of the order of ε. Consequently, the fitnesses of organisms in the population in accordance with (1) become essentially different; so, organisms with small values ρ(S_Fk,S_M) are selected into the population of the next generation. It is intuitively clear that the genotypes of S_Gk of selected organisms should be rather close to the final phenotypes S_Fk (obtained as a result of the learning) of these organisms. Thus, the result of the selection is choosing of organisms, which genotypes are also moving to the optimal chain S_M. Therefore, values ρ = ρ(S_Gk,S_M) in the new population decrease.

The described mechanism of the genetic assimilation is characterized by Fig. 2, which shows the distributions of the number of organisms n(ρ) for given ρ in the population for different moments of the first generation. The curve 1 shows the distribution n(ρ) for ρ = ρ(S_Gk,S_M) for the initial genotypes of organisms at the beginning of the generation. The curve 2 shows the distribution n(ρ) for ρ = ρ(S_Fk,S_M) for organisms after the learning, but before the selection. The curve 3 shows the distribution n(ρ) for ρ = ρ(S_Fk,S_M) for organisms, selected in accordance with the fitness (1). The curve 4 shows the distribution n(ρ) for ρ = ρ(S_Gk,S_M) for the genotypes of selected organisms at the end of the generation. The genotypes of selected organisms S_Gk are sufficiently close to the final phenotypes of learned and selected organisms S_Fk, therefore the distribution n(ρ) for genotypes S_Gk (the curve 4) moves towards the distribution for final phenotypes S_Fk (the curve 3). A similar displacement of the distribution n(ρ) towards smaller values ρ takes place in the next generations.

Fig. 2.

The distributions n(ρ) in the first generation of evolution for different moments of the generation. The curve 1 is the distribution n(ρ) for ρ = ρ(S_Gk,S_M) for the initial genotypes before the learning. The curve 2 is the distribution n(ρ) for ρ = ρ(S_Fk,S_M) for organisms after the learning, but before the selection. The curve 3 is the distribution n(ρ) for ρ = ρ(S_Fk,S_M) for selected organisms. The curve 4 is the distribution n(ρ) for ρ = ρ(S_Gk,S_M) for the genotypes of selected organisms at the end of the generation.

It should be underlined that the decrease of values ρ at the learning should be sufficiently large in order to ensure the small role of the parameter ε and the significant difference of the fitnesses (1) of organisms after the learning, and therefore, the effective selection of organisms with small values ρ(S_Fk,S_M). In order to guarantee the effective operation this mechanism, the learning should be enough strong.

3.2 Hiding Effect

Thus, the strong learning can accelerate the evolutionary search. However, the strong learning can also prevent a finding of the optimal genotype. The curve 1 in Fig. 1 shows that at large G the decrease of values <ρ> = <ρ(S_Gk,S_M)> is limited: the final value <ρ> remains quite large, the asymptotic value <ρ> is approximately equal to 6.2. This fact is due to the following. At large G (G ~ 1000), the strong learning leads to the finding of the optimal phenotype S_Popt = S_M independently on the genotype S_Gk. Therefore, at the final stages of the evolutionary process, the genotypes S_Gk do not move towards the optimum S_M. So, the hiding effect [2] is observed.

3.3 Influence of the Learning Load on the Modeled Processes

We also analyzed the influence of the learning load on the modeled processes. For this case, the fitness of organisms is determined by the expression (2). The simulation is performed for the mentioned parameters (N = n = 100, β = 1, P_m = 0.01, T = 2, ε = 10⁻⁶, α = 1). The simulation results show that the learning load leads to the considerable acceleration of the evolutionary search for the optimal chain S_M. This acceleration is due to the fact that the learning load results in the more strong selection of organisms that have small distance ρ(S_Gk,S_Fk) between the initial S_Pk(t = 1) = S_Gk and the final S_Pk(t = T) = S_Fk phenotypes, than for the case of the fitness (1). This form of the selection in accordance with the expression (2) leads to the additional minimization of changes of phenotypes S_Pk during the learning process. The simulation demonstrates that the optimal genotype S_Gopt = S_M in the considered case is found.

The on-line paper [6] describes results of the simulation in more details.

4 Conclusion

The analysis of the simulation shows that (a) the genetic assimilation, (b) the hiding effect, and (c) the significant acceleration of the genetic assimilation and the evolutionary process under the influence of the leaning load are observed under the following assumptions:

• (1) Each organism of the evolving population has a genotype and a phenotype.

• (2) The genotype and the phenotype are chains of symbols; the both chains have the same form.

• (3). Genotypes of organisms are transmitted from parents to descendants with small mutations. The genotype of the organism is not changed during its life.

• (4) The initial phenotype of the organism at its birth is equal to the organism genotype.

• (5) There is a certain optimal chain, which is searched for by means of learning and evolution. The optimal chain has the same form as the genotype and the phenotype.

• (6) The phenotype is essentially adjusted by means of learning during the organism lifetime. During learning, the phenotype moves towards the optimal chain.

• (7) The selection of organisms into a new generation occurs in accordance with final phenotypes of organisms.