Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Solving spatial tasks is an important part of everyday activity in the lives of animals, humans, and cognitive agents, in general. Examples include chores such as setting the table and orientation while moving through the world. Humans seem to often solve these problems with more ease, if they can be visualized or are available as concrete instances in their environment, which can be manipulated. In [1] the idea of strong spatial cognition is presented to investigate and exploit this kind of spatial action- and perception-based problem solving. That is, humans solve spatial problems not only by generating symbolic representations and applying reasoning to compute solutions, but by invoking actions, perceiving their effects, and adapting the action. For example, if one has to untangle two cables, the common approach is to pick them up and start pushing and pulling on cable parts and see where a cable end can be pulled out of a loop in order to reduce the degree of entanglement.

This basic ability to utilize space and change to directly solve problems seems to be an essential part of our cognitive abilities. Therefore a cognitive agent, be it living or artificial, must have not only internal perception and reasoning functions, but also an understanding or at least an assumption of the affordances of the environment. Figure 1 illustrates this view on a full cognitive system including internal knowledge representation and reasoning, perception and action options, physical capabilities, and the inhabited environment.

Fig. 1.
figure 1

Structure of a full cognitive system [1, 2]

Full size image

Artificial cognitive agents are currently lacking a comparable ability to exploit their spatio-physical environment for efficient problem solving. One main issue with equipping artificial cognitive agents with strong spatial cognition is that the constraints and properties of this type of problem solving are still insufficiently understood. Being tightly embedded in the environment it is difficult to assess strong spatial cognition by traditional methods. This makes it hard to gain an explicit understanding of strong spatial cognition or to compare it to traditional computing approaches. In order to approach this, we propose to employ models of strong spatial cognition to gain a deeper understanding of this phenomenon, its properties, and its nature.

In the remainder of this paper, we discuss the concept of strong spatial cognition in more detail, address model design and instances, and introduce and discuss observations made during model creation and evaluation. In order to have a practical application for illustration, we utilize the shortest path search as a running example.

2 Strong Spatial Cognition

Strong spatial cognition [1, 2] aims at preserving spatial structure and directly exploits features and properties of spatial transformations and affordances. In the structure presented in Fig. 1, strong spatial cognition methods are found at the border between embodied cognition to a full cognitive system. That is, a cognitive agent utilizes its spatio-physical capabilities to invoke actions in and on spatial objects and configurations in the environment and perceives their effects. It is important to note that these actions are not necessarily performed by the agent, but can be inherent within the environment. Spatial objects and configurations are represented by themselves or by mild abstraction [2], rather than by symbolic representations. This avoids information loss due to early representational commitments, i.e., it is not necessary to decide beforehand which aspects to abstract away and which spatial reference frame to use. The advantage is that at a later point in time more information may be available to make a better-informed decision about which specific (mild) abstraction and frame of reference to apply.

The strong spatial cognition approach is to concentrate on the specific spatial problem to be solved by creating an appropriate object or spatial configuration, e.g., by removing task-irrelevant entities and features or by reconstructing the essence of the object or spatial configuration through mild abstraction, e.g., scaling, rotation, or translation. It is hard to predict the optimal object or spatial configuration to create. However, spatial problems pose a special case as natural agents have enough meta-knowledge, i.e., experiences with affordances and physical effects, about the environment to apply useful heuristics for supporting object or spatial configuration creation. The created entities are then exposed to the environment and the result either presents a direct solution or a configuration that may be more suitable for a knowledge-based approach to solve the original problem.

The insight that spatial relations and physical operations are strongly connected to cognitive processing may lead to a different division of labor between the perceptual, representational, computational, and locomotive parts of cognitive interaction than the one pursued in classical AI systems. That is, rather than putting all the configurational reasoning into the computer or agent, the strong spatial cognition approach employs (physical) spatial reconfiguration by the environment in order to simplify or nullify the problem to be solved. The approach uses structural and procedural ‘knowledge in the world’ [3], i.e., physics and affordances, to solve spatial problems by exploiting intrinsic structures of space and time. To do this successfully, the computer or agent must be equipped with meta-knowledge and respective reasoning abilities in order to identify and perform useful actions and to perceive significant effects.

Our hypothesis is that a flexible assignment of physical and computational resources for cognitive problem solving is closer to natural cognitive systems than the current purely computational approaches. For example, when cognitive agents search for a certain object in the environment, they have at least two strategies at their disposal: they can represent the object in their mind and try to imagine and mentally reconstruct where the object should or could be located, i.e., the classical AI approach, or they can visually search for the object in the physical environment. Which is better (or more promising) depends on a variety of factors including memory and physical effort and the environment. We think that most often a clever combination of both approaches will be best.

To illustrate the concept of strong spatial cognition, we use shortest path search as a running example throughout this paper. Figure 2 presents a physical network of connected nodes and the task is to identify the shortest path between two specific nodes. In order to create the network, a map is used as basis and roads are (3D) printed as strings and each intersection or location of interest is printed as a circular node. By applying a physical force/action we can pull the two specific nodes apart until the connections between them form a straight line (see Fig. 3), which explicates the shortest path. Furthermore, applying the same force/action we would also directly explicate, if there is one or more shortest paths (see Fig. 4).

Fig. 2.
figure 2

Strong spatial cognition shortest path object

Full size image
Fig. 3.
figure 3

Strong spatial cognition shortest path

Full size image
Fig. 4.
figure 4

Strong spatial cognition shortest paths

Full size image

This example illustrates several important aspects of strong spatial cognition, e.g., mild abstraction, exploiting spatial constraints and continuity, and perception and action apparatus of the involved cognitive agent and thus provides an instance of problem solving by a full cognitive system. It also highlights a number of prerequisites for successfully applying strong spatial cognition such as (I) that mild abstraction preserves required environmental properties and (II) that the perception of and actions within the environment are (mostly) cheap.

Thus, strong spatial cognition is based on the following basic assumptions: (1) mild abstraction allows to create objects and spatial configurations within the environment that preserve specific properties, (2) there exists a cognitive agent within the environment, and (3) the cognitive agent can perceive and interpret as well as act on the environment to interact with the objects or spatial configuration.

As such, strong spatial cognition has a number of relations to but also sets itself apart from contemporary approaches to analyzing and building cognitive agents. Strong spatial cognition builds on arguments of Norman [3], who argues for the importance of distinguishing “knowledge in the head” from “knowledge in the world” and Kirsh [4], who highlights the importance of physical interactions for (human) problem solving. Strong spatial cognition can also be seen in the tradition of biology-inspired approaches as pioneered by [5, 6] and Gibson’s [7] idea of affordances. However, in contrast to these biology-inspired approaches, strong spatial cognition goes beyond strictly reactive behavior, but includes physical interactions that are deliberately used to solve problems. Furthermore, strong spatial cognition interprets affordances as conditions that permit actions in spatial environments and not exclusively as conditions that can be perceptually identified [8]. Strong spatial cognition also bears resemblance to the notion of analogical representations [9, 10] and diagrammatic reasoning [11, 12] in the sense that all three stress the importance of (preserving) the spatio-physical structure of the problem (environment). While analogical representations and diagrammatic reasoning work towards representation structures that mirror important parts of the physical structure, strong spatial cognition proposes to directly exploit the spatial structure. This direct exploitation is also what sets strong spatial cognition apart from traditional knowledge-based approaches, in which facts and relations about space in general and about specific problem domains are encoded as knowledge that describes the domain (e.g., in ontologies [13] or in qualitative spatial representations [14, 15]).

A use of ambient properties comparable to strong spatial cognition is exploited in morphological computation [16, 17]. Morphological computation exploits the properties of an agent’s body (its morphology) in interactions with its environment. That is, we can perceive morphological computation as a real subset of strong spatial cognition, as it is restricted to exploiting interactions between the agents morphology and the environment while strong spatial cognition takes a more general approach in including any interaction between the agent and the environment as well as interaction occurring purely within the environment.

A more in-depth discussion of the relation of strong spatial cognition to previous approaches can be found in [2].

3 Model Design

Figures 2, 3 and 4 present how we envision the actual use of strong spatial cognition for the example of the shortest path search. By investigating this example, we identified different approaches to use forces/actions to explicate the shortest path between two nodes. That is, we apply a strong force to the start node (grasping or pinning down action) to fix it in position. Then a force (pull action) is applied to the goal node explicating the shortest path. Here, it also occurred to us that we could apply the force to all nodes, like gravity applies to all objects in our world (see Fig. 5). This simultaneously explicates all shortest paths outgoing of the start node. That is, the force works simultaneously on all nodes, i.e., the ‘computation’ and information propagation of positions happens in parallel in either case.

Fig. 5.
figure 5

Strong spatial cognition with gravity

Full size image

In essence, we identified two types of forces applicable:

  • individual force, i.e., a force that is directly applied to one specific node, and

  • ambient force, i.e., a force that is applied to all nodes (e.g., gravity).

These forces can also be present simultaneously in the environment.

However, the problem remains that this embedded approach is very hard to analyze. How do objects or spatial configurations interact with the environment? Why does this interaction seem so effective? What are the ‘computational’ properties, i.e., what properties explicate the solution? How does this ‘computation’ unfold?

To address and, ultimately, answer these questions we need a way to investigate strong spatial cognition. As argued above, a direct investigation by well known means does not seem feasible. Against this background, we argue that an appropriate way to make strong spatial cognition amenable to thorough analyses is by building and analyzing computational models of it. Such a computational modeling approach shares a number of advantages with modeling in other domains (e.g., [18]): First and foremost, a computational realization afford ways of analyses that are unfeasible for the entity that is modeled. Second, building computational models forces theoretical explicitness and precision. Third, because a computational model has to be explicit and precise, it facilitates pinpointing those aspects of the to-be-modeled entity that are still insufficiently understood, that is, modeling draws the attention to the crucial open questions.

Specifically, we propose to generate and investigate models of specific strong spatial cognition instances. These models map a spatio-physical environment to computational simulation approaches. We build on computational approaches as there exist standardized and well understood methods for the analysis of computational models. Note, however, that these models of strong spatial cognition are not actual strong spatial cognition instances or possibilities how we propose to utilize strong spatial cognition. That is, they are meant to help analyze and understand the strong spatial cognition idea and not as an implementation pattern for strong spatial cognition programs.

4 (Computational) Model Instances

In order to get a better understanding of the inner workings of strong spatial cognition, i.e., what properties or aspects are crucial, we decided to implement computational models of the shortest path search that imitate the use of gravity. For this, we assumed, that the central property of ‘environmental computation’ is parallelism.

Based on this, we decided to use the MASS (Multi-Agent Spatial Simulation) library [19, 20] for our implementation as it is specially designed to build spatial simulations with a parallel programming paradigm. The MASS library allows implementing parallel multi-agent systems by implementing the two concepts of Places and Agents. This fits very well with the internal structure of strong spatial cognition as we see it at the moment. That is, we are able to model individual agents and space they inhabit as well as other objects in space in a clear way. Places are represented as a multi-dimensional array of elements that are dynamically allocated over a cluster of multi-core computing nodes. Each Place is capable to make parallel function calls with all neighboring Places to exchange information. Agents are actors that can reside in Places and autonomously migrate to any other Place and synchronize with all other Agents according to function calls. Thus, Agents can interact with one another through setting and perceiving information within the Place they inhabit. Parallelization is realized through a set of multi-core computing nodes connected through a secure network layer. This left us with two implementation options to be considered for using MASS:

  • Place-only implementation: Each Place represents a different node. Places forward gravity as a message through a repetition of their exchange of information with their neighbors. A limitation of such a MASS implementation is to force all Places to exchange messages with their neighbors regardless of an arrival of gravity, which results in having all Places keep forwarding null messages to their neighbors.

  • Agent-based implementation: Similar to the first option, each Place represents a different node. We introduce an Agent as gravity that travels from the source node (i.e., Place) to the destination by cloning itself along all neighbors. Although an Agent is a larger object than a message, the number of Agents spawned over a simulation is much lower than the number of messages to be exchanged among Places in the first implementation.

As MASS only provides us with a fully distributed model, i.e., computations and storage are distributed, we decided to also implement a model that uses a centralized storage. The idea being that in the real world, all information is also stored in one entity, i.e., the spatio-physical environment. The computations are again set to be conducted distributively in parallel. That is, each node is realized as an individual thread and a central storage contains information regarding the position of each node as well as a set of constraints (maximal distances) between specific nodes. In an iterative approach, each node gets the instruction to drop (change its position). It checks and makes sure that the changed position does not violate any constraint and updates the position information in the central storage. If a constraint violation would happen due to a position change, the position update is restricted to the maximal change that does not violate any constraint.

All three implementations model the shortest path search using an ambient force (gravity) while applying a very strong force keeping the start node fixed in its original location. During execution, the models only simulate selected physical properties, i.e., gravitation, which moves all nodes until they are fully constrained. That is, each simulation propagates position information updates for all nodes throughout the model until no further changes are possible.

5 Model Discussion

The design and implementation of these computational models to simulate strong spatial cognition led to a couple of observations. To us the two most noteworthy are: (1) information propagation seems the most crucial aspect of the solution generation and (2) that all implementations required a representation of the object or spatial configuration as well as a definition of at least selected properties of the environment as functionality, i.e., simulations like physics.

Regarding information propagation, the strength of the strong spatial cognition approach is that the actual work of updating, synchronizing, and consistency checking of information is provided by the environment and does not require additional work from the ‘computational’ side. It is also important to note, that this data management is not only provided, but is also instantaneous (one could call it ambient), i.e., we cannot get into a world state where data is inconsistent due to some form of reader-writer problem. This also indicates a concept of parallelism in the real world and as a consequence in strong spatial cognition that is rather different from the concept of parallelism in computer science. Where parallelism in traditional computer science addresses mainly the execution, in strong spatial cognition it addresses always all aspects, i.e., execution, knowledge exchange, and knowledge storage.

Regarding the design and implementation of computational models of strong spatial cognition, it seems that strong spatial cognition approaches only explicate the knowledge representation, while in the computational approach we also have to explicate the functionality to compute the solution. Thus, the amount of representational work in strong spatial cognition seems to be more streamlined and there is a clearer distinction between information representation (i.e., the object or spatial configuration) and the possibilities for change (i.e., the environmental properties). In computational implementations both of these aspects have to be explicitly represented in a program.

Next to these two major observations, building and running the models also helped to identify the following research questions that are central for gaining a more comprehensive understanding of strong spatial cognition:

  1. (Q1)

    What is complexity in the context of strong spatial cognition?

  2. (Q2)

    How does strong spatial cognition relate to traditional computing?

  3. (Q3)

    Which axioms and properties are crucial for strong spatial cognition and its definition?

  4. (Q4)

    What are design patterns to ‘implement’ strong spatial cognition?

  5. (Q5)

    What is the relation between object or spatial configuration and environment in strong spatial cognition?

  6. (Q6)

    Which heuristics are optimal for the creation of objects or spatial configurations in given contexts?

  7. (Q7)

    To what extent and how is strong spatial cognition learned?

  8. (Q8)

    Is strong spatial cognition only available to individuals that are or were able to actively manipulate the environment or may it also derive from (passive) perception of the environment?

Question Q1 was already raised by [21]. The other questions basically address the internal workings of strong spatial cognition (Q3 and Q5), how we can utilize strong spatial cognition (Q2, Q4, and Q6), or how we develop the capability of strong spatial cognition (Q7 and Q8).

6 Conclusion

Strong spatial cognition aims at moving the computational effort from the central processor to the environment. In this paper we presented a first step towards analyzing strong spatial cognition by designing and implementing computational simulation models. The results show that especially the properties of the environmental memory in terms of instantaneous synchronization and ambient accessibility are key aspects of this approach. Furthermore, these properties seem to be strongly linked to the special form of parallelism exploited by strong spatial cognition.

As next steps we will investigate in more detail the information propagation in strong spatial cognition and computational simulation models. We intend to focus on the amount of information propagated as well as how the information flow is restricted. The idea being that this might lead to an initial comparable measure of complexity for strong spatial cognition and traditional computer science.

In addition, we will further investigate the general properties of strong spatial cognition by creating further computational simulation models of different strong spatial cognition approaches. We will also investigate the use of physics- and other simulation engines in order to generate models with environments that implicitly provide action capabilities, i.e., affordances. We believe that this is the way forward to develop a better definition of strong spatial cognition and to shed light on the question on how to connect or combine it with computational approaches.