The Probabilistic Basis of Spatial Complexity

Abstract

This chapter sets the basis for a probabilistic approach to spatial complexity, by focusing on: (a) species richness (the number of different qualitative classes /species/ population types/ categories /covers/colours/types) occupying the spatial extent of a surface, (b) entropy (measured i.e. by Shannon’s formula, and reflecting the degree of equidistribution of these classes on the basis of their relative participation percentages) and (c) randomness of allocations of these classes. Evidently, the higher the number of different classes within a surface, space, or spatial object, the higher its spatial complexity. Further, the more disordered their allocation, the higher the spatial complexity also. These considerations however, ramify to a series of questions, i.e.: when is a string of symbols a random one? what alternative definitions of random strings are there and how do they impact spatial randomness?

From Chaos became Erebus and the black Night;

From Night were born the Aether and the Day,

“ἐκ Χάεος δ᾽ Ἔρεβός τε μέλαινά τε Νὺξ ἐγένοντο:

Νυκτὸς δ᾽ αὖτ᾽ Αἰθήρ τε καὶ Ἡμέρη ἐξεγένοντο”

(Hesiod, c.750 b.C., “Theogony” 104,123-125)

1 Spatial Entropy Versus Complexity

“A bit of entropy is a bit of ignorance”

(Seth Lloyd 2007, p.80)

While analyzing spatial complexity, one should always look for the simplest methods possible, despite the fact that spatial settings are “interesting” exactly when they are not simple (although they may be pleasant nevertheless). Understanding the relationship between entropy and complexity is a multi-faceted issue cropping up in many domains of scientific enquiry. This relationship has attracted the attention of physicists and computer scientists time and again. Generally, the higher the number of different classes (species/populations/categories/covers/colours/types…) within a surface or spatial object, the higher the spatial complexity. But this also depends on the scale under which the object is examined, as different classes or objects can be packed together so densely that they might even accept an easy synoptic description (Fig. 4.1).

Fig. 4.1

Progressively subtracting fruits from a tray gives a glimpse of spatial complexity: a high number of spatial objects (fruits in this case) is not necessarily an indication of high complexity, as they can be packed together so densely that an overly simplifying description might be attributed to them (i.e. “a fruity area”). The same applies when the spatial objects are too few against the empty backdrop (i.e. “a map with one object in it”). Yet, when both the spatial objects and their backdrop are present at comparable percentages, the increasing difficulty of description becomes obvious. This difficulty is a measure of spatial complexity of the surface which in this case is defined by the number of different covers/categories present on the surface

The number of classes present on a spatial surface is only one indication of its spatial complexity; the relative participation of each one of these classes in the object examined is another. The more different classes participate to cover the surface, the higher the spatial complexity. It is common in the scientific literature of ecology, cartography and geography to define the entropy of a map by using the Shannon formula (Shannon and Weaver 1949; Forman and Godron 1986):

$$H = - \sum\limits_{i = 1}^{V} {Q_{i} \log (Q_{i} )}$$

(4.1)

where V is the total number of “colors” present in the map depending on the map examined (i.e. if it is geographic, then V represents land cover classes, land use types, landscape types, population types etc) and Q_i is the percentage of occurrence of each color i in the map’s area, with $\sum\nolimits_{i = 1}^{V} {Q_{i} = 1}$.

If the entire map is covered by one color only, then H=0. With an increase in the number of colors V, entropy increases, but the entropy formula does not take into account the spatial allocation of these classes, as the maps (b) and (c) show (Fig. 4.2).

Fig. 4.2

Increasing spatial differentiation results in increasing Shannon entropy (calculated with a logarithm basis 2). Notice however, that spatial configurations with the same numbers of covers and the same cover percentages may yield the same entropy values for entirely different spatial configurations (i.e. cases b and c)

Thus, entropy is only a relative (although highly important) indicator of spatial complexity. Let us see two other examples (Fig. 4.3). The maps of the two upper rows have higher entropy than those of the two bottom rows: H=1 versus H=1.5 respectively. The two upper rows show maps with two colors only, while the two bottom rows show maps of equal size with three colors. The maps with three colours have a higher entropy than the maps with two colours, regardless of the spatial allocation of these colours. These also show why entropy alone is not a sufficient criterion of spatial complexity, since different spatial configurations (and therefore different spatial complexities) can correspond to exactly the same entropy values (Papadimitriou 2012).

Fig. 4.3

Two sets of maps: the maps of the upper set of two rows are binary and have H=1 while the two lower sets of rows have three colors and H=1.5 In both cases, the entropy values are the same for each set of maps. Thus, within each set of rows, entropy alone can not help us distinguish between spatial configurations with different spatial complexities each

In biogeography and ecology, the term “diversity” is widely used. In its general form, diversity means the identification of the characteristics of a map, as reflected by the number of different classes of elements in it (i.e. as reflected by the “diversity” of its elements; a term used in spatial ecology) and consists a recurrent theme in ecological research (Clarke and Warwick 1998; Anand and Orloci 2000; Petrovskaya et al. 2006; McShea 1991; Magurran 2004). The connection between diversity and complexity was discussed in ecological context by Zhang et al. (2009), who found that an increase in Shannon diversity appeared concurrently with increasing “landscape complexity”. Yet, other studies based on field observations have shown that spatial complexity and entropy (diversity) are not always positively correlated. Species richness and complexity are not always correlated either (Azovsky 2009, p.308).

Besides entropy, another index needs to be parenthetically mentioned, which is contagion. The “contagion” index was proposed by O’Neill et al. (1988), Turner (1989; 1990) and Turner and Ruscher (1988) to characterize spatial (landscape) patterns in landscape maps:

$$2n\ln (n) - \sum\limits_{i = 1}^{n} {} \sum\limits_{j = 1}^{n} {Q_{ij} \ln (Q_{ij} )}$$

(4.2)

where n is the total number of cover types in the geographical space (or landscape), Q_ij is the probability of cover type i being adjacent to cover type j, and 2nln(n) is the maximum contagion, which is attained if there is an equal probability of any two landscape types being adjacent to one another.

The problem with this index is that it may also yield the same values for entirely different spatial configurations (and hence, for configurations of entirely different topology and spatial complexity). As shown in six example 3×3 binary maps (Fig. 4.4), the 1st binary map has only one black cell, the 2nd has two, while the 3d has 3 and the 4th has four. Yet, all these maps have the same contagion index. The same applies to the binary maps 5 and 6: both have the same contagion, although their spatial configurations are completely different. But different spatial configurations most likely have different spatial complexities (this will be examined in detail in next chapters). Consequently, contagion can not be taken as a measure of spatial complexity.

Fig. 4.4

Small binary maps with the same contagion but with different entropy and different complexity (at least, as perceived visually). Binary maps 1 to 4 have the same contagion (equal to 0.918), but their entropies are different (both entropy class r and Shannon entropy H are different in each one of them). The same applies to binary maps 5 and 6

However, it is not only maps that can demonstrate the central role of entropy. As Steinhaus (1954) suggested, there is an interesting association of the concept of entropy to the complexity of a curve: the number of intersections of a plane curve and a random line intersecting that curve is equal to 2L/C, where L is the curve’s length and C is the length of the boundary of the curve’s convex hull. Plugging this into Shannon’s entropy formula and defining (arbitrarily) as the “temperature” of the curve the quantity

$$\log_{2} \left( {\frac{2L}{{2L - C}}} \right)$$

(4.3)

it is possible to derive a thermodynamic analogue of curve complexity. Supposedly, this link between geometry and thermodynamics gives a measure of the “entropy” of a curve (Mendes France 1983; Dupain et al. 1986):

$$H_{curve} = \log_{2} \left( \frac{2L}{C} \right) + \frac{{\log_{2} \left( {\frac{2L}{{2L - C}}} \right)}}{{e^{{\log_{2} \left( {\frac{2L}{{2L - C}}} \right)}} - 1}}$$

(4.4)

The entropy of a curve is also an indicator of its complexity, i.e. if the curve is described by a polynomial of degree d, then its entropy is at most 1+log₂d (Stewart 1992).

2 Spatial Randomness and Algorithmic Complexity

To a land of deep night, of disorder and utter darkness,

where even light is like darkness

“אֶ֤רֶץ עֵיפָ֨תָה ׀ כְּמֹ֥ו אֹ֗פֶל צַ֭לְמָוֶת וְלֹ֥א סְדָרִ֗ים וַתֹּ֥פַע כְּמֹו־אֹֽפֶל׃ פ”

(The Bible, Job, 10.22)

The study of spatial randomness of distributions of some population in any spatial dimension (Fig. 4.5) is a vibrant field of research, particularly in the context of spatial random processes, for which a basic introduction can be found in Adler (1981) and a more elaborate and updated in Hristopoulos (2020). Expectedly, spatial complexity depends on spatial randomness, but, as it turns out, it is difficult to assess spatial randomness in terms of algorithmic complexity because there are several alternative approaches to deciding whether a string of symbols is random or not (diverging approaches even about the one dimensional case).

Fig. 4.5

An allocation of flowers on a street created by natural forces acting both deterministically (i.e. gravity) and stochastically (i.e. affected by changing wind speed and direction), thus producing a not entirely random spatial allocation

In an early approach, von Mises (1919) defined an infinite binary string as random, so long as it has as many 0s as 1s at its “limit”. Adopting a different approach, Church (1940) defined a random string as every infinite string of which the digits can not be given by a recursive function. Later, Martin-Löf (1966a, b) suggested that random infinite strings are those that satisfy all statistical tests for randomness. Levin (1973; 1974) and Chaitin (1974; 1975) defined random strings x as those that are endowed with a maximum Chaitin-Levin complexity, meaning there exists a number c, such that for every n, this complexity is higher than the difference n-c. Following the most widely known definition by Kolmogorov (1965) however, an infinite string x is random, if its Kolmogorov complexity K(x) is maximum.

An alternative approach to randomness is Bennett’s concept of “logical depth” of an object (Bennett 1973, 1982, 1986, 1988a, b, c, 1990), measuring the time required to compute a minimal program, with its organisation of the studied object. The “logical depth” of a string is the calculation time needed (by a universal machine), in order to produce it from its minimum Kolmogorov description. Bennett’s definition has had some applications in physics and biology (Bennett 1986, 1988b) and one of the implication of Bennett’s theory is that the possibility to encounter by chance an object with large logical depth is very small. Plausibly, the possibility that Bennett’s definition might be used to assess spatial complexity should not be precluded. However, the problem with the definition of randomness becomes explicit even by considering simple cases. Consider, for instance, a binary string composed of 0s and 1s. Let the sum of the string’s elements be N. If the string is random in the Church sense, then:

$$\mathop {\lim }\nolimits_{n \to \infty } \left( \frac{N}{n} \right) = \frac{1}{2}$$

(4.5)

But there are strings satisfying this equation at the limit, which are clearly non- random (i.e. the string 0101010101010101010101…). This simple example illustrates why the riddle of defining string randomness remains unsolvable even for one-dimensional objects. Further, many strings that happen to satisfy the criteria of Von Mises and Church fail to do so for Martin-Löf’s criteria. But characterizing a string as random in the sense of Kolmogorov means that it accepts no shorter algorithmic description and therefore it has no regularity at all and passes the statistical tests required by Martin-Löf’s definition.

Yet, diverging opinions with respect to strings can be seen in the case of 2d surfaces also. For instance, complexity is often perceived to be a condition “in between order and chaos”, in such a way that (i.e. in Fig. 4.6), neither map a (ordered) nor map c are “complex”, because a is ordered and c is random. But map b is perceived as “complex”, because it is found in between order and randomness and displays distinct patterns (such as clumps of the same colour in B).

Fig. 4.6

Different perceptions of what is “complex” affects the impression of what a spatially complex surface is. According to some interpretations, map b is more complex than both maps a and c, because it displays both order (patterns of the same colour) and randomness (at the bottom right quadrant). But following the theory of algorithmic complexity, a spatial region is more complex if it is more difficult to describe by an algorithm and if it is closer to randomness: in this way, map b is more complex than a, while c (a random allocation of the three colours) is more complex than b

Spatial stochasticity can be created by a 2d Brownian stochastic motion. Here, for the purpose of illustration, it is plotted for 400,000 time steps (Fig. 4.7). As the east-west Brownian motion is tantamount to the north-south motion, the joint probability of being at a position u along the horizontal axis and v along the vertical axis is the product of the Gaussian probability densities of the respective motions:

Fig. 4.7

A stochastic Brownian motion over an empty space. When the number of time steps gradually increases (up to 400,000 here), complex patchy, rugged spatial forms emerge

$$P(u,v,t) = \frac{1}{2\pi t}e^{{ - \frac{{{(}u^{{2}} + v^{2} )}}{{{\text{2t}}}}}}$$

(4.6)

When a rugged surface (noisy, with “ups” and “downs”) is examined, its spatial complexity will be even more dependent on the level of spatial resolution at which it is examined. At this point enter issues of choice and technical capabilities: one simply has to experiment with surfaces that can be constructed on the basis of Gaussian-type functions describing a terrain (Fig. 4.8) and follow the general type (a_i,b_i,c_i are constants):

Fig. 4.8

The ruggedness of a surface is a parameter indicating its spatial complexity (a). The degree of ruggedness essentially reflects the degree of randomness of a surface. The two-dimensional cross-section of a three-dimensional landscape that is created by using Gausian-type distributions for x and y can be given by a function F(x, y) that describes the “altitude” of the landscape, such as the profile shown in b

$$F(x,y) = c_{1} e^{{ - \left( {a_{1} x^{2} + a_{{_{2} }} y^{2} + a_{3} x + a_{4} y + a_{5} } \right)}} + c_{2} e^{{ - \left( {b_{1} x^{2} + b_{{_{2} }} y^{2} + b_{3} x + b_{4} y + b_{5} } \right)}}$$

(4.7)

In a rasterized map, a string of symbols can represent a strip of squares (or parallelograms) of a spatial object. The algorithmic complexity of a string of symbols is equal to the minimum description of this string. In one dimension, if a string of symbols is random, then it has maximum algorithmic complexity; equivalently, if the allocation of colors/covers is random over a map (Fig. 4.9), then it has maximum complexity (with respect to any other such map of the same size and with the same amount of covers/colors). If it has repeating patterns, then its description can be reduced to a simpler one, by taking advantage of these repetitions, and in that case, it has a lower complexity than an incompressible string of symbols.

Fig. 4.9

A random image, generated by a computer. It has almost equiprobable allocation of cells per luminosity value (histogram on the right) and zero Moran’s autocorrelation index. Such images have maximum algorithmic complexity in the Kolmogorov sense