1 Motivation and history of the term

The term “emergence” has become an important topic in various branches of science, closely connected to the study of complex systems [16]. Some scientists contend that emergence is the very basis of nature [27] and the complexity we observe around us. Indeed in sciences from A—like Astrophysics to Z—like Zoology, emergence is an important concept. Typical questions asked are:

  • A: How did the structure of the Universe emerge from the Big Bang? [17]

  • Z: How do different animal species or lineages and their features emerge? (see, e.g., [9]) Some argue strongly that all physical law is emergent [24], essentially saying it is the result of collective phenomena.

Emergence has a number of connotations which need to be kept separate in order not to sink into confusion.

  1. 1.

    There is the emergence of novelty, as in the case when a new entity appears for the first timeFootnote 1;

  2. 2.

    Emergence could also mean that something becomes visible in a population or concentration sense, i.e. the entity does not appear the first time, but has now spread to the point where its impact becomes measurable;

  3. 3.

    Then there is the emergence of new levels of organization, as in the event when the first atoms formed from the primordial plasma Footnote 2;

  4. 4.

    Finally, there is ontological emergence which can be discerned from epistemological emergence. The former is used to characterize the existence of new entities whereas the latter is used to explain what has happened, e.g. in a theory. Footnote 3 Bunge [10] gives good examples for both.

Questions abound. Why is nature organized in a hierarchical way? Is this an illusion of our limits of perception which force us to accept a certain level of granularity when observing phenomena? In other words, is it a result of the scope we use for observation [31] which imposes on us the impression of a hierarchy where none is, or do levels actually exist, i.e. are they ontological categories [15]? Ultimately, this line of thoughts leads us directly to ask questions about the nature of reality [3]. One thing is clear: Not only nature, but also our systems of understanding follow a hierarchical structuring [12].

The concept of emergence has been used in various contexts and for various purposes. As such, it is in dispute within sciences and would have a lot to gain from a concise definition. Before going to a definition, however, we would like to discuss some of the history of the term and the underlying concept.

The ideas behind emergence are not new. Aristotle [5] in the fourth century B.C. has the following to say (free translation) about the unity of entities: "The whole is something over and above its parts, and not just the sum of them all.” Footnote 4 Obviously, Aristotle was referring to the interaction of parts and the resulting non-linearity in their behaviour which produces wholeness. At least that would be a characterization in modern parlance.

Another giant in philosophy, Kant [19], the first to recognize the importance of organization in living beings, the philosopher who minted the term "organism” had the following to say about it: "An organized product of nature is that in which all has purpose and is conversely a means.” Longo et al. [26] has nicely paraphrased Kant in the following sentence: “The whole exists for and by means of the parts and the parts exist for and by means of the whole.” In other words, Kant recognized the circular relationship between the cause and effect that is at the root of living matter.

Then, during the industrial revolution which is—not by coincidence—also the time of Darwin, an entire school of natural philosophers, the British Emergentists started to zoom in on emergence as a key topic, beginning with John Stuart Mill who thought about the difference between mechanical and chemical “causes”. For a thorough study of the history of the term emergence the reader is referred to Blitz’s [8] book. Suffice it to say that the term "emergent” appeared the first time in 1875 [25] and was used in contrast to the term “resultant”, characterizing a non-interacting superposition. At the time, qualitative novelty was realized to be an emergent phenomenon.

For a good definition of “emergence” we adopt Mario Bunge’s wonderful exposition in [10]: "Typically, then, wholes possess properties that their parts lack. Such global properties are said to be emergent. Every whole has at least one such property. …Emergent properties are not distributive but global. Think, for instance, of the validity of an argument, the consistency of a theory, the efficiency of an algorithm, …, the stability of an atomic nucleus, the solidity of a body”. Emergence then is the process by which these emergent properties or emergent phenomena come about.

A question immediately arises: What are the mechanisms that would allow the emergence of such properties, which in turn define the wholes and differentiate them from their parts?

2 Mechanisms for emergence

Of course, the term “mechanism” is in itself to be questioned. For our purposes here, however, we should consider it as a figure of speech. What we want to understand is how the process of emergence works. If it were a machine, how would it produce emergent phenomena? In that sense, we are looking at the inner workings of this presumed machine, its mechanism.

Downward causation, also called top-down causation has been at the centre of the discussion about mechanisms of emergence, together with its twin, upward causation, or bottom-up causation. Upward causation refers to the influence exerted on a whole by its parts. Conversely, downward causation refers to the influence exerted by wholes on their parts. However, we’d better be careful about the term "cause”, as it is frought with history. Often in this context, Aristotles’ four types of causes are mentioned, but that is not of much help because the meaning of the term cause has shifted since the times of the Athenean Academy, and “cause” is not a good translation any more for what Aristotle said. Footnote 5

Here I adopt the simple minded approach to cause, by which I mean some thing or process is a cause of another thing or process if by taking away the former, the latter disappears also. Thus, a mechanical force moving a body is the cause of its movement, since taking it away, the movement would not happen. Footnote 6 So we have a simple way of determining causes: By taking putative causes away, and observing the outcome, we see whether the hypothesis was correct or not. Genetics works like that today, by "knocking out” genes to see whether they have any influence on a certain trait. Footnote 7

Figure 1 shows the two different directions of influence. While there is no dispute that upward causation happens in nature, the claim that downward causation exists is regularly disputed, as it contradicts a reductionist philosophy still much en vogue in science. Ultimately, the reductionist claim is that without explaining phenomena through upward causation they cannot be understood and analyzed, and therefore science would be giving up its attempt to understand. This can be seen in contrast to our opinion that downward causation exists and interacts with upward causation to produce (emergent) phenomena. To clarify this: Downward causation is a prerequisite for emergence, but upward causation is necessary, too.

Fig. 1
figure 1

The two elementary directions of causation in a hierarchical system. Each level contains elements that can interact on their own level, and cause entities on other levels

Full size image

There are other ideas to reconcile the two positions, for instance by claiming weak emergence [7] which states that higher level entities are not easily described as resulting from causes of the interaction of lower level entities, yet ultimately that there is nothing else than lower level interactions causing higher level phenomena.

However, where does the sense of unity and indivisibility Footnote 8 come from at certain levels of the hierarchy? Why does the human mind perceive the person as a unity, why does an atom—at a certain level of examination—appear indivisible? Why is a multi-cellular organism not falling apart into myriads of cells? All illusions? It is this unity—some would say irreducibility—at a certain level of examination, that is the thought-provoking kernel of theories of emergence. Where does it come from when it wasn’t there before? How is downward causation able to explain the unity?

At a recent meeting of the Royal Society [14] on top-down causation pertinent questions were discussed and a theme issue was published in the Interface Focus journal of the Royal Society. If we tentatively accept the thesis that downward causation is a reality, we still need to settle the question of what mechanisms in detail it could consist of. The present state of the discussion is reflected such: There are three essential mechanisms for downward or top-down causation [13]:

  1. 1.

    Higher level entities constrain the behaviour of lower-level entities;

  2. 2.

    Feedback control is provided by the higher level;

  3. 3.

    Selection at the higher level results in certain preferred behaviour at the lower level.

All three mechanisms require the lower level entities to have a certain degree of freedom to arrange themselves into different configurations that result in a higher level entity. This principle, called causal slack at the lower level requires that there is an entire equivalence class of states at the lower level which reflects the same state at the higher level [13]. With that sort of freedom, constraints, feedback or selection can affect lower level entities. In other words, the ability is required to have multiple realizations at the lower level to lead to the same realization at the higher level. This is the very nature of micro-macro relationships and is thus perhaps not surprising, although the concept is another battlefield in philosophy [32]. The argument needs to be seen in the context of larger (multi-level) hierarchical systems. There, it requires that the higher we move in the hierarchy of levels, the larger are the equivalence classes of low level entities. Vice versa, the lower we go in the hierarchy, the more freedom entities need to have to arrange themselves. Footnote 9

The attentive reader might already presage what all this has to do with Genetic Programming. Before we go there, however, we need to emphasize one last point about the observability of entities at different levels. Each level will be characterized by its intrinsic time scale, which in turn determines what can be observed as a stable entity on that level. This brings in a dynamical perspective which is indispensable to both a consideration for stability of entities and levels as well as their formation. Indeed, it is only with the addition of time that certain paradoxes can be resolved.

The nature of the interactions (inner level, top-down and bottom-up) is now such that the entities that are observable on a certain level happen to be those whose causation is consistent between the different levels. Kim’s [20] argument that downward causation will mean causal conflict between causation originating at different levels is invalid. Footnote 10 He maintains that downward causation would result in causal conflict and because of that downward causation cannot exist. Footnote 11 However, there is nothing wrong with causal conflict if we consider its development over time. In fact, we have to look at this conflict in a dynamical system setting as it resolves itself, not as a stand-off in a static system. The dynamics of the interaction of entities at a certain level is such that only those entities stabilize whose causes converge to a consistent pattern and are not contradictory to each other. Contradictory influences (whether from different or the same level causes) will destabilize an entity and might be better called causal competition to reflect their dynamics.

3 Genetic programming and emergence

What has all this to do with Evolutionary Computation (EC) or Genetic Programming in particular? A few keywords have already been mentioned: a mechanism for downward causation - selection, stability of entities at certain levels—survival, multiple realizations—many-to-one relationships between genotype, phenotype and fitness. As we can see, these mechanisms are present in many variants of EC algorithms already.

There are, however, three aspects of Genetic Programming that make it particularly amenable to see emergence happening:

  1. 1.

    Multiple levels between genotype and fitness. This allows repeated many-to-one mappings which are a mainstay in Genetic Programming under, among others, the heading of neutral variations.

  2. 2.

    The fact that solutions are active entities in Genetic Programming. The behaviour of programs can achieve, in different ways, the same outcome. As a simple example, a calculation from input to output can be performed in different ways (i.e. using different algorithms/programs).

  3. 3.

    GP solutions are of a compositional type, i.e. behaviour is brought together from the combination of simpler entities that can be arranged more or less freely. This freedom is reflected in the fact that behavioural descriptions are variable in length. i.e. whether a program has 5 or 50 instructions, it still ends up as the behaviour of one program.

My point in this essay is summarized shortly in this statement: In Genetic Programming, emergence is a process that happens directly before our eyes. Low level entities are the genes in our artificial genomes that are subject to mutation and recombination and possibly other forces. Genomes map into phenomes and the latter into fitness values which in turn determine which phenomes (and therefore which genomes) survive. Selection is our mechanism for downward causation, fitness of the phenome the property used to determine which combination of genes ultimately survives. The entities at the highest level are the solutions to our problems which literally emerge from the Genetic Programming algorithm. Footnote 12 Table 1 summarizes the situation.

Table 1 A comparison of general terms with their special meaning in Genetic Programming
Full size table

The famous cumulative variation—selection loop of evolutionary algorithms is thus the driving force for the emergence of solutions, with variations on properties of lower level entities and selection on fitness in a bottom-up and top-down interaction (see Fig. 2). Higher level environments provide the niches which are favourable (or not) to lower level entities. As a result, variations are either preserved or removed.

Fig. 2
figure 2

The familiar variation—selection loop of a GP system as the engine for emergence

Full size image

It goes without saying that the process of Darwinian evolution in nature that is the paradigm for evolutionary computation models makes use of similar if more sophisticated mechanisms.

4 Emergent phenomena

If emergence is so tightly connected to the mechanisms of variation and selection (so as to produce top-down causation) is it a surprise that emergent phenomena abound in evolutionary algorithms? Notably in Genetic Programming emergent phenomena have been characterized since the early nineties [1, 6]. Remarkable in GP are the following phenomena:

  • Bloat [4]

  • Repetitive patterns [22]

  • Modularity [21, 33]

  • Cooperation [18]

  • Evolvability [2],

all of which have been found to be produced by evolution without being explicitly selected for by the algorithm.

Just to pick out one of these phenomena, repetitive patterns, which we have studied to some degree in the past but of which to this day only the surface is scratched: In single runs we find numerous repeated patterns [22, 23]. These often have structure: There are shorter patterns within longer patterns, but also variations and overlaps. The emergence of these patterns seems somewhat “orthogonal” to the emergence of neutral code (bloat): We find these patterns comprised of both effective and neutral code.

I believe that these patterns are emergent, a result of the presence of downward causation realized through selection. One fact supporting this argument is that, usually, patterns can be found in longer runs and for difficult problems. Obviously, the interaction of individuals via the crossover operator produces such repeated patterns, but that needs time to prove advantageous in selection. The fastest way of improvement in difficult GP problems seems to be duplication of code segments, followed by divergence. Again, selection acts to allow for emergence (and stabilization) of patterns in the population. Footnote 13 This brings about a compression of information, as the repeated segments are variants of the original segments. Figure 3 gives an example of repetitive code in a problem of speech classification by linear GP [11].

Fig. 3
figure 3

This is a program in machine language (linear) GP which happens to classify vowels of spoken language. The highlighted segments are identical copies. If you look closely, you can detect more copies, though diverged from the original in various ways (marked by vertical lines)

Full size image

5 Consequences

What is so interesting about emergence in Genetic Programming? With the still lingering dispute about whether emergence using downward causation is a valid idea, GPers can confidently declare that emergence is happening in GP for all the world to see. I believe this is due to the presence of selection as one of the driving forces of evolution, a clear case of downward causation.

No wonder that unintended emergent phenomena abound! This is a consequence of the fact that we engineered GP systems to provide emergent solutions. Side effects seem to be principally unavoidable given that we want to make use of the variation—selection loop. I am convinced that this is very much analogous to the situation in which intelligence or consciousness emerges: Intelligence can only emerge if a system is allowed to make errors, and consciousness only if mental states can degenerate into disease. In a similar vain, multi-cellular complexity growth comes with the downside of disintegration and death. Footnote 14

The concept of emergence becomes even more powerful in the context of multi-level systems. Indeed, the idea that there is no preferred direction of causation, and that all levels should be equally legitimate starting points for our investigation is at the heart of a relativity principle recently formulated for biological system [28]. What this relativity principle says is that there is no privileged level at which causation starts (say, e.g., at the lowest level, with "lowest” defined at a certain moment in the history of biology), and there is consequently no preferred direction in which causation exerts its influence. i.e., both bottom-up as well top-down causation are necessary directions of effects.

Our future work will therefore focus on multi-level systems that are equipped with a multi-level selection mechanism for the emergence of entities on different levels. We have started such an inquiry just recently by making first steps in this direction [35], but plenty of questions remain.