Cognitive Architectures: The Dialectics of Agent/Environment

Abstract

In what concerns living systems, cognition is an embodied, embedded and always situated experience. This means that it involves an entity endowed with a particular physical architecture bound in a dialectical relationship with the environment in which it is immersed, behaving according to the prompts placed by this environment, reacting, learning and adapting to it defining this way its own existential narrative and history. Highlighting the fact that human cognition stems from more simple and basic forms of cognition with which it shares essential life mechanisms, the present chapter focuses on the essential semiosic process that is inherent to the dialectics agent/environment and the role played by corporeal architectures in the construction of meaningful worlds, namely, the hybrid realities, where natural and artificial intelligence cohabit.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Subjective Worlds

Cognition is the embodied, embedded and always situated process whereby life forms bound to their respective environments in an essential dialectical relationship thrive “to persist and prevail”¹ within the existential spatio/temporal framework defined by their own corporeal dynamics.

Every species has a typical evolved intelligent architecture, the phenotypic structure, which is the joint product of its genes and the environmental variations faced during its developmental and evolutionary history. This cognitive architecture embodies vital information concerning the regulation and equilibrium of its internal live states—homeostasis—as well as the information relative to the sensorial/perceptive perceptive systems available to interact with a typical external environment defining the species specific world model.

A life form and its environment constitute a “closed purposive organization” [2] bound by a relationship of mutual influence. In regard to that which relates evolved systems, form seems to follow from function, as the existence of a particular physical structure is shaped by the specific functional needs that the organism has met along its evolutionary history and ontogeny. This functional level of explanation is essential for understanding how natural selection designs organisms and how, in the course of evolutionary time, new features were added or discarded from the species design [5].

Genetic “instructions” provide general constraints for neural development, determining the different levels of neural organisation and the specificity of the sensorial equipment that organisms belonging to different species display. These instructions define the types and forms of interaction available, and are also responsible for the entity’s capacity to identify and assign meaning to particular environmental features, responding accordingly. On this account, [20, p. 16] states:

The nature of the environment […] acquires a curious status: it is that which lends itself […] to a surplus of significance. Like jazz improvisation, environment provides the “excuse” for the neural “music” from the perspective of the cognitive system involved.

To illustrate the fundamental role played by different physical architectures in the definition of particular meaningful worlds—the Umwelten²—Uexkull [19, p. 45] takes the female tick as an existential model. Providing a glance at the way it interacts with the environment within which it is embedded across the essential timings of its life cycle, Uexkull identifies the forms of interaction with the external world that are available for the tick and how these provide the information the organism requires to exist: “Out of the egg crawls a not yet fully developed little animal, still missing one pair of legs as well as genital organs. Even in this state, it can already ambush cold-blooded animals such as lizards, for which it lies in wait on the top of a blade of grass. After many moltings, it has acquired the organs it lacked and can now go on its quest for warm-blooded creatures. Once the female has copulated, she climbs with her full count of eight legs to the tip of a protruding branch of any shrub in order either to fall onto small mammals who run by underneath or to let herself be brushed off the branch by large ones. The eyeless creature finds the way to its lookout with the help of a general sensitivity to light. The blind and deaf bandit becomes aware of the approach of its prey through the sense of smell. The odor of butyric acid, which is given off by the skin glands of all mammals, gives the tick the cue to leave its watch post and leap off. If it then falls onto something warm—which its fine sense of temperature will tell—then it has reached its prey, the warm-blooded animal, and needs only use its sense of touch to find a spot as free of hair as possible in order to bore past its own head into the skin tissue of the prey. Though it has no sense of taste, the tick pumps a stream of blood, as long as it is warm, slowly into itself […]”.

Given the needs dictated by its internal state(s) at a scheduled point in its life cycle, three features become salient in the tick’s surrounding environment:

• 1. Odor of butoric acid

• 2. Hairy surface

• 3. ±37°.

Following a sequence, each of these three cues is perceived,³ defining a pattern that is identified and assigned a value—meaning—triggering the following pre-set behaviours:

• 1. Odor of butoric acid _______________ leap off

• 2. Hairy surface ____________________ cling to it

• 3. ±37° ________________________ pump the host’s blood.

By assigning a meaning to this set of cues and acting accordingly to a final goal—laying its eggs—the tick ensures the survival of its species. Uexkull points that out (ibidem): “And now something miraculous happens. Of all the effects emanating from the mammal’s body, only three become stimuli, and then only in a certain sequence. From the enormous world surrounding the tick, three stimuli glow like signal lights […]. Through these features, the progression of the tick’s actions is so strictly prescribed that the tick can only produce very determinate effect marks. The whole rich world surrounding the tick is constricted and transformed into an impoverished structure that, most importantly of all, consists only of three features and three effect marks”.

This dialectics that binds a cognitive architecture to its environment can be seen replicated endlessly⁴ in nature, highlighting the fact that reality is perceived, “conceived”, and modelled differently depending on the “eyes of the beholder”, i.e., according to the perceptive/sensorial capacities of the cognitive agent, in other words, according to its corporeal architecture.

Cassirer [2] pointed out that whatever is alive has its own circle of action for which it is there and which is there “for” it, both as a wall that closes it off and as a viewpoint that it holds “open” for the world.

A life form and its physical world constitute a unit—a microcosm—bound by an essential dialectic relationship [5, 7]. This dialectic relationship that binds different cognitive agents⁵ to their selected environments is an ongoing dynamic process of reciprocal influence. Seeking to satisfy the existential demands of their internal states, life forms strive to cope with the environmental prompts. By identifying and adequately responding to meaningful patterns, by learning and adapting they guarantee their self-subsistence and species replication within a definite life-span and according to biologically determined timings and stages. At the core of this dialectics stands semiosis. Defined as an essential “interpretative” process present in all life processes [5,6,7, 10], semiosis is to Sebeok [17, 18] the criterial attribute of life the feature that distinguishes the animate from the inanimate. According to Ferreira [6,7,8], semiosis emerges from the structural coupling of the living entity and its environment, guaranteeing the cohesion, sustainability and prevalence of the microcosm. This interpretative capacity, this “meaning-making”, is, as Sagan [15] points out, much older than words. Damásio [4, pp. 108, 109] states that⁶ “in the beginning, there were only sensations and reactions by unicellular organisms […] sensing and responding accordingly started in this way […] messages were like irritating substances that caused the corresponding irritation. There were no “eyes” nor “ears” […] there were just the primordials of a perceiving process that, with evolution and with the development of nervous systems, would lead to world modeling, mind definition and, finally, subjectivity”.

In this sense, we can agree with Merleau-Ponty [14] that meaning exists at a pre-reflective level of existence. In fact, there seems to be a primary, pre-ontological “meaning-making capacity” present at all levels of life activity and inherent to life itself. Based on the recurring properties of previous encounters, cognitive architectures incorporate existential narratives, constituting the “know-how” that guides all present interactions. This “know-how” comprehends the capacity to identify and assign a value—meaning—to particular environmental features, simultaneously triggering the organism’s adequate response from a repertoire that is basically pre-established.

As posited by Ferreira [5, 7], independent of the type of cognition or level of semiotic complexity involved, meaning is a value—a structured entity. This value is assigned by the cognitive agent—a natural or artificial entity—to an individuated environmental feature or a cluster of features that, because of the agent’s nature and needs, emerges in the environment as a salient typical pattern.

In the diagram below, reproduced from Ferreira [7, p. 9], the oval on the left represents the set of all cognitive agents endowed with a particular physical architecture {X}, while the oval on the right represents the set of all possible environmental features {Y}. f is a function from domain X to codomain Y; the small oval stands for the image of f, i.e., the set of all possible outputs obtained when the function is evaluated at each element of the subset. In other words, the smaller oval represents the set of all possible meaningful features for X in the codomain Y, i.e., its potential self-world (Fig. 1).

Fig. 1

Meaning—a value assigned to an environmental feature

Uexkull distinguishes the Umwelt from the Innenwelt. If the Umwelt corresponds to the entity’s particular “view” of the world—its world model—the Innenwelt is defined by the internal state(s) that characterize an entity’s internal condition at a given time. Conceived as inherently systemic, the concept of Innenwelt is essential for understanding why specific environmental features emerge and take on more salience comparative to others in the organism’s lived space. In fact, salience is determined by the life form’s existential needs, as reflected by the states of its Innenwelt at a given moment of its life timeline. These states will define the priorities of the emergence of salience in what concerns the environmental features’ prominence.

The diagram reproduced in Fig. 2, [8, p. 3], aims to capture the invariants present in the dialectics essential to cognition.

Fig. 2

Modelling cognition

The diagram represents the roles and functions played by the key concepts of the model: y is a vector of dimension:⁷ (N_y × 1), which is assumed to represent all of the potential information present in the entity’s environment. Acknowledging that not all environmental features will be perceived by the agent and that other features will have different importance at different times and within different contexts, the agent’s view of its environment (Umwelt) was modeled through an (N_u × 1) vector, u. This vector is created from the environmental features vector, y, through the application of a semiotic filter, F, whose characteristics are dependent on the agent’s internal state (Innenwelt), represented through an (N_i × 1) vector, i. The agent’s particular view of the world—the Umwelt—will then influence both its actions and its consequent transition to a new internal state. This new internal state will, in turn, influence both the agent’s actions and its semiotic filter, and, through it, its environmental perception. The vectors u (Umwelt) and i(Innenwelt) are, therefore, in a dialectic relationship that determines and triggers the determine the agent’s actions. We assume that there are N_a possible actions that can be executed by the agent and collect the respective probabilities of execution in a vector, a. These actions when executed, will have an effect on the environment, allowing or not the satisfaction of the needs dictated by internal states and providing a means for learning to occur.

The process of cognition is an ongoing learning and maturation process through which lifeforms constantly rewrite narratives defining and redefining their “view” of the world and adjusting their responses accordingly. As Varela [20, p. 60] writes: “Ordinary life is necessarily one of situated agents […] situatedness means that a cognitive entity has, by definition, a perspective. This means that it isn’t related to its environment “objectively”, that is, independent of the system’s location, heading, attitudes and history. Instead, it relates to it in relation to the perspective established by the constantly emerging properties of the agent itself and in terms of the role such running redefinition plays in the system’s entire coherence”.

Situatedness is reflected in the two overlapping narratives simultaneously running in all lifeforms: one concerning their evolutionary history as a member of a species, embodying the achievements of their predecessors in their struggle for life, the other, the actualization of this evolutionary narrative by the present physical body in particular contexts and circumstances. These particular contexts and circumstances that the new lifeform will have to face and interact with, constructing a particular microcosm, may not be exactly the prototypical, i.e., the ones “expected” by the system [12]. However, in the course of the dialectics that binds the cognitive agent to its environment and in its struggle for life, the organism will always try to respond to the environmental prompts, adjusting, adapting, evolving or otherwise perishing.

2 Umwelt Overlap: The Overlap of Individual Experiences

This Concept and Corresponding Mathematical Modeling Are Defined and Developed in Ferreira and Caldas [10].

Different cognitive agents will define according to their physical bodies distinct world views. The existence of multiple subjective⁸ worlds, multiple meaningful spheres of existence apparently sharing the same spatio/temporal framework⁹ is, again, acknowledged by Uexkuhll who in the introduction to “Umwelt und Innenwelt der Tiere” invites the reader to an imaginary stroll (1934:5):

[…] a stroll into unfamiliar worlds; worlds strange to us but known to other creatures, manifold and varied as the animals themselves. The best time to set out on such an adventure is on a sunny day. The place, a flower—strewn meadow, humming with insects fluttering with butterflies. Here we may glimpse the worlds of the lowly dwellers of the meadow. To do so, we must first blow, in fancy, a soap bubble around each creature to represent its own world, filled with the perceptions which it alone knows. When we ourselves then step into one of these bubbles, the familiar meadow is transformed. Many of its colourful features disappear, others no longer belong together but appear in new relationships. A new world comes into being. Through the bubble we see the world of the burrowing worm, of the butterfly, or of the field mouse; the world as it appears to the animals themselves, not as it appears to us.

The metaphor of the soap bubble is fundamental to highlight the inherently subjective character of cognition, a subjective process that takes place in a circumscribed sphere: a virtual sphere, a figurative perimeter, traced according to the type of interactions allowed by the physical architecture of the organism and that models in the general environment the organism’s Umwelt, its meaningful world [5]. But the metaphor of the soap bubble is also fundamental to understand how these coexistent individual worlds frequently overlap at variable degrees.

Life is characterised by the crisscrossing of individual spheres of existence, of individual Umwelten. The Umwelt of the tick and that of the mammal overlap at a time t, when one becomes the host of the other. The same happening, for instance, with the wolf and the lamb in the relation predator-prey, when the prey becomes the energy supplier of the predator, or between the male eagle and the female eagle in a mating relation. Umwelten also overlap at varying degrees in the so called social species whose individual members are assigned specific roles and usually enroll in cooperative tasks guaranteeing, this way, their subsistance and the community’s existence and sustainability, as it is the case of ants or that of bees. But it is with the social species par excellence—the human being—that this overlap becomes the ground for a galaxy of existential interactions from which primarily results the notion of Oneself and that of Otherness, the interaction with the Other(s) and from this the shaping of individual and social identity Ferreira [5, 7]. It is also in the context of the overlap of multiple spheres of existence that specific relations of production emerge giving rise to particular social structurings, and “work”, understood as the creative and generative capacity to produce and change reality becomes an inherently human achievement.

3 The Observer’s Myth

Senses are an essential window to the world we live in, providing the data that build mental representations enabling the construction of particular world views. As it happens with all other lifeforms, human beings perceive and interact with the external physical world in their species-specific way. It is thanks to their cognitive architecture, the evolved physical body endowed with innate competences, namely that of symbolic encoding, that human beings are able to give shape and substance to their meaningful worlds anchored on the notion of Self and fulfilled by the net of relationships this self defines and establishes with the meaningful Other(s).

Damásio [4] points out that it was the mapping capacity provided by the emergence of nervous systems linked to a web of neural circuits that allowed for some life forms, namely human beings, the generation and definition of a cartography where patterns of activity and the spatial relations between the active elements inside a pattern are represented and ultimately minds, understood as representations of a subjective lived world can be defined.

Experience is necessarily subjective, and consequently temporal. The organisation of experience according to a temporal axis along which the multiplicity of events are placed in respect to their “before” and “after” is an essentially subjective construal. The outcome of this subjective construal is a totality in which the division of time into present, past and future is no longer a substantial division. Experience, presented in an unbroken flow, will allow the subject to flash back in time, re-experiencing facts or events, and, simultaneously, will allow them to use those past experiences as a standpoint, enhancing a better understanding of the present or anticipating/predicting the future. On this topic, Cassirer [1, p. 167] much inspired by Augustine, writes:

Strictly speaking […] we should say that the present time comprises three different relations and through them three different aspects and determinations. There is a present of past things, a present of present things and a present of future things. The present of past things is called memory; the present of present things is called intuition; that of future things is called expectation. Thus, we may not think of time as an absolute thing, divided into three absolute parts: rather, the unitary consciousness of the “now” encompasses three different basic directions and is first constituted in this triality.

Conscious of the complex way meaning is composed and conveyed among human organisms, Cassirer [2] defines the human being as “animal symbolicum”. He suggests, on the basis of Uexkull’s biology, the existence of a symbolic system, which falls between the “receptor” and “effector” systems that it shares with all of the other organisms. It is this symbolic system that allows signs to be assigned values, enhancing a three-part relationship between the “Sign-Using Self”, “Constructed Reality” and the “Other Self”.

Reality is not just the reflection that mirrors an external objective world in our eyes, a world existing independently of the subjects of experience, but rather is the result of an individual and collective symbolic construction, a construction emerging from the semiosic process that lies at the core of all forms of cognition. Cassirer says that we must break radically with the presupposition that what we call the visible reality of things is given and present at hand as a finished substratum prior to all formative activities of the mind, because it is not the reality of things that endures, but only the form that reality assumes through us.

The model that characterises the basic forms of semiosis analysed above is also found in the upper levels of semiotic structuring that characterize human cognition. Cassirer [1, p. 56] has this intuition when he writes:

If perception did not embrace an originally symbolic element, it would offer no support and no starting point for the symbolism of language […] perception, as such, signifies, intends and “says” something, and language merely takes up this first significatory function […] the word of language makes explicit the representative values and meanings that are embedded in perception itself.

In what concerns reality, we are never observers, even when we think we are, but always experiencers. In fact, though reality is perceived as external, we know that this very reality results from a semiosis grounded in a unique experiencer/experienced relationship, which the conscious mind ignores, giving the experiential subject the status of virtual observer. The subjectivity inherent to this world view was also stressed by Kant [11]:

What objects may be in themselves, and apart from all this receptivity of our sensibility, remains completely unknown to us. We know nothing but our mode of perceiving them- a mode which is peculiar to us, and not necessarily shared in by every being, though, certainly, by every human being.

Simondon [16] calls the historical and cultural context in which human cognition takes place the pre-experiential background issued from the experience of all precedent generations, a common background that only comes to life in the present individual appropriation, being in this way consequently changed by the action of those who share it. In fact, every newborn comes to life in a particular physical, economic, social, cultural and linguistic atmosphere. A physical environment where specific relations of production have not only determined the particular social structuring and social hierarchies, but have also determined the typical patterns of behaviour to be followed in all circumstances and contexts, the definition of public and domestic space [13], the creation of institutions, the architectural options, the production of artefacts and technological artefacts, and the production of art forms. It is in the restricted and very controlled life circle provided by the close family that the child seizes the concept of Otherness in the person of its caregivers, especially of its mother, learns how to designate them and how to designate itself, as it starts to shape its own identity. It is also here that it develops essential motor programs, such as that of sitting by itself, walking on two legs, or both handling a spoon in the proper conventional way and carrying the spoon with food to the mouth; it learns that this particular object is a [spoon] and not a [mug] and that its function is to handle food; it becomes aware that artefacts generally have a function associated with them, as well as the spaces defined in its home. It learns that there are behaviours and procedures to be followed in different contexts. It is here in this first restricted circle that the child is slowly introduced into a constructed reality. A world where people, with slight variations, follow the essential typical routines [9], each, eventually, subsuming sets of others that guarantee not only the biological and social existence, but also the maintenance of the necessary conditions of production on which a particular society stands at a given time of its development, e.g.

• get up at about the same time

• follow identical hygienic procedures

• have breakfast

• take the children to school

• rush to work

• get into a train, bus, etc.

• start working

• get a coffee at the local Starbucks

• stop working

• rush back home

• pick up the children (at school)

• cook dinner

• go to sleep

Though the essentials of this universe and the basic typical patterns of behaviour with their respective motor programs are incorporated into that first circle of social interaction, the learning process carries on throughout life with the broadening and diversification of social circles [7], with the consequent permanent updating of social conventions, with the introduction of new artefacts and the consequent updating of existing motor programs: how to step onto and off of an escalator, how to swipe the screen of a smartphone so that the camera is activated.

The encapsulation of meaning in symbolic forms is a cognitive demand, as human beings need to preserve and objectify experience, to reflect upon it, to create for themselves a shared model of their lived world. Symbolisation makes the translation of inherently subjective experience into an objective medium possible. By freeing meaning from the immediacy of subjective experience and turning it into a collectively sharable object, language allows it to be incorporated, redefined and reshaped in different contexts and world views.

Damásio [3] states that we will probably never know how faithful our knowledge of the world is in what concerns absolute reality. But what we need, and we have it, is a remarkable consistency in terms of the nature and content of the mental representations that our individual minds produce, and consequently are able to share collectively. This very consistency of our experience and the fact that, through language, this same consistency can be verified and confirmed by the experience of others lead us to believe that this is an experiencer-independent reality, an objective reality.

Cassirer points out that the problem refers not to the objectivity of existence, but to the objectivity of meaning. We would say that this objectivity of meaning is achieved through language, a symbolic construction in which the whole community participates and from which objectivity of being emerges.

4 Hybrid Worlds, Hybrid Agents

Digitization, the conversion of an analogue signal to binary bits, allowed information to be represented in a universal manner and be stored as data. This data can be filtered, tracked, duplicated and transmitted, infinitely, at incredible speed. Digitization has not only empowered human cognition exponentially by accelerating intrinsic semiosic processes but it has also changed the very nature of the typical environment by creating new agents, new umwelten and new overlapping of experience.

For purely analytic purposes and not taking into account other possible hybrid forms, we could consider the following main types of cognition present in the contemporary world:

• (i) The typical forms involving a natural system and its physical environment. We include in this case the forms of human interaction with the surrounding environment (analogue) and consider as physical environment the compound of physical, social, cultural and linguistic counterparts.

• (ii) Those involving natural systems—human beings—and digital interfaces existing in the analogue world, in typical human life contexts, as it is the case of all the interactions that take place on the Internet via computer or smartphone.

• (iii) The forms of cognition involving human beings interacting with virtual environments augmented reality scenarios … where displacement from the subject’s actual mental spatio/temporal framework occurs, as those induced by electronic devices operating on the external perception organs or through induction in the neural system.

• (iv) The forms of cognition involving human beings with enhanced capacities and the physical environment, as in the case of bionic components.

• (v) The forms of artificial embodied cognition involving a physical artificial system that interacts physically with its body and with the surrounding environment (physical, social, cultural, linguisticm etc.) as in the case of robotic systems.

• (vi) The embodied and/or non-embodied forms of artificial cognition interacting with a digitized world, as in the case of the Internet of Things (IoT) or in the case of Artificial Life Research.

Common to all these forms of cognition is the existence of an agent that interacts with an environment driven by certain needs and expectancies. All these instances are profoundly human in the sense that they reflect and incorporate the human view of the world and the way human beings interact with it in an essential semiosic process.