Keywords

1 Introduction

This paper is an extension of the conference paper, Emerging Complexity: Communication between Agents in a MAS for shape-shifting TUIs , presented at CHIRA conference in 2020 [1]. The paper explored communication methods between agents, which are part of a multiagent system (MAS). The overall context of the research on which it is based, relates to the development of MAS in their use for shape-shifting technology and further implementation for tangible user interfaces (TUIs). To date a large portion of research relating to this field has been divided between agent-to-agent communication methods [2, 3] and the process of self-assembly [4,5,6,7]. The study on which the research described in this paper is based, focused on designing a physical agent: the Dod [8]. The Dod is a blueprint for an agent design, to be used as part of an artificial MAS. The design itself is highly adaptable and versatile as it reflects the changing developments in technology and material sciences. In refraining from creating a high-fidelity prototype during the original study, it was possible to explore a larger breadth of possibilities with respect to the overall function of the Dod itself and an entire MAS based on the Dod agent. The physical design of the Dod and how it can potentially communicate have been presented in the following papers [1, 9, 10]. This paper will explore a design guideline that can be used in a situation whereby significant design criteria were still unknown, as well as exploring an avenue of future work that had been detailed in the original PhD study: developing the Dod as a game-piece.

As it emerged, there is a large gap between, how shape-shifting technology should ideally work, what it should look like, and what is currently possible. Aside from the technical and design challenges of realising such technology, another obstacle is the uptake into the public realm. If there is no engagement or understanding for this kind of technology, it is questionable how successful it can be. Therefore, the Dod design as a game-piece can begin to introduce essential qualities of shape-shifting technology into social consciousness as well as foster the art of 3D thinking. As technology develops, and if the Dod design is still viable, it will be possible to integrate for use as a MAS agent design more easily. An analogy can be draw with the initial development of computer technology - since its introduction into mainstream society, even though the shape, size and construct of the computer has altered, the fundamental principle of how users interact with them are still the same: visual screen for representation of digital data and some sort of interface to enable interaction with this data (e.g. keyboard, mouse, trackpad, etc.). From an interaction perspective, shape-shifting technology represents a slightly stronger focus on the haptic modality. This modality is present in the majority of humans from a very early stage in their conceptualisation process of their environment. A function of touch is to create a sense of grounding in one’s reality and is a key element in being able to act on or in one’s environment. Unique to a person’s haptic sense is the quality of bi-directionality [11]; the ability to receive sensory input but also being able to exert a change in the environment. For example, it is possible to see a leaf on a table; its texture, colour, shape, and size, but it is not possible to affect the leaf only by looking at it (sensing). Without vision it is possible to lift the leaf, feel it to discover its size, shape, weight, temperature, and texture - it is also possible to change its size or shape by folding or tearing edges (sensing and acting). In conjunction with this sense, the act of play is not only an integral part to socialisation but also the creation of holistic experiences that aids in the process of contextualisation and the sharing of ideas and knowledge.

In turn, an important aspect of play is communication. Communication is integrated in several different levels in a project of this nature. Because the final application of shape-shifting technology is not yet clearly defined, it is necessary for MAS agent to have a degree of autonomy that enables them to interact, interpret and adapt to their environment. For example,

  • User to technology interaction,

  • Agent to environment interaction,

  • Agent to agent interaction,

  • Agent to itself interaction

Despite the physical appearance and design of the agent (the Dod) taking precedence throughout the original study, a portion of research was given to defining the type of communication method that was possible as a result of the design itself. To highlight the variety of communication techniques available to implement in MAS, section two will detail the journey of communication techniques in relation to how the original ideas for shape-shifting technology have evolved.

Section three will continue to expand on a specific design guideline that was very influential in development of the Dod: emulation rather than replication. Due to certain design parameters being unknown, it is important to explore design guidelines that do not limit the scope of creative potential and can still maintain a structured framework in order to produce viable results. Applying this kind of guideline to various stages of research ensures that there is a consistency of approach throughout the project’s development.

The existing and fictional games that influenced the development of the final game design will be presented in section four. These not only help to contextualise the Dod game-piece, but also to highlight how specific game strategies can influence players’ interactions and engagement with a game, e.g. being able to see patterns. Being able to identify patterns is fundamental to Gestalt psychology as it provides a person with grounding and the ability to organize their reality.

Lastly a description of the proposed Dod game-piece, the Dodlen, and several suggestions for game play rules will be detailed in section five. The primary concept for this avenue of research is that in order to facilitate user interaction with the concept of MAS, and in the future with shape-shifting technology, people must become familiar with it, play with it and explore it long before they actually need it. Therefore, even if the Dod design could potentially no longer be relevant, the behaviour of interacting with 3D structures may have become more familiar in general society.

2 Related Work

The Tangible Interface classification scheme provided the parameters within which the Dod was developed [12]. The categories of constructed assemblies and continuous plastic TUIs being of particular relevance. However, the initial ideas and research that inspired the Dod, were strongly influence by William Butera’s work in 2002: a paintable computer [13], Ishi’s conceptual exploration of Radical Atoms [12] and Horev’s work into the affordances presented by shape-shifting technology [14]. Going back further to research that inspired these studies, is Sutherlands vision of the Ultimate Display [15]: ‘a room within which the computer can control the existence of matter’[16]. In the domain of science fiction, the Ultimate Display concept can be said to be represented by the Holodeck in the Star Trek franchise [17]: a room that uses holographic projections and that gives a person the ability to interact with digital representations in the same manner as physical ones, i.e. a digital table would be as real and solid as its physical counterpart. The gap between reality and fiction is continuously reducing as is evidenced by the advances in Virtual Reality (VR) technology. Being able to introduce haptics into VR, via shape-shifting technology would represent a significant development in a number of different STEM disciplines.

Changing reality in real-time, and exerting control over that new reality, are interesting perspectives to consider and reference, in the development of shape-shifting technology. The question as to which forms or shapes, digital representations would take once they are made tangible, is the open-ended question with respect to this kind of technology’s design. In Butera’s work, he suggests the concept of a paintable computer; enhancing surfaces, making them interactive and intelligent. The concept is a blend of the internet of things (IoT) approach, whereby objects are connected to one another and communicate over a shared network, and multiagent computing [13]. He suggests that future computer technology could become so resilient, small and cheap, that it would be possible to deal with it in bulk and that it will be small enough to potentially blend into the background and be applicable to everything in the environment. Butera describes agents that function on a basic level, ideally suspended in a liquid, so that they can be dispersed evenly but also be used to enhance the surface onto which they are painted. An agent consists of a ‘brain’, in terms of memory, communication and energy manipulating ability [13]. The predominant method of functioning of such agents is through self-organisation. It is possible to see the correlation between digital pixels and their possible physical representation through the agents suggested by Butera. At this point it becomes clear that whilst a paintable computer cannot form 3D structures, it is beginning to address the concept of a reactive reality that a user can influence in real-time.

Expanding this concept, it is possible to envision a computer liquid that could change its physical shape, i.e. it would be able to create 3D relief structures or even be used as an independent 3D object. Such an interface would be in a quasi-liquid state when inactive and could become solid when in use, comparable to an augmented Non-Newtonian fluid (specifically rheopectic fluids [18]). These types of interfaces are described in Horev’s work, primarily highlighting the value of haptic affordances of shape-shifting structures [14]. In his work the method of communication shifts from the traditional visual interpretation of digital information to haptic representation. Communication is facilitated through textural or shape change, temperature and movement. As a result, Horev also highlights behaviours to which a shape-shifting interface may be ideally suited that are currently difficult to convey: “Hidden statuses…Process Guidance…Adaptation to Context” [14]. This kind of research is the valuable counterpart to the digital exploration of agent-to-agent communication because it considers novel ways of user interaction with shape-shifting technology but also focuses on the haptic modality. In order to accomplish 3D structures, the necessity to focus research on self-assembly techniques became evident. It could be argued that self-organisation is a product or physical manifestation of a communication protocol, whereas self-assembly used in the context of construction, indicates that an agent can use knowledge that is communicated amongst agents and apply it to form more complex structures.

As is evidenced by Butera’s work, the MAS approach appears to be the most efficient system to achieve independent, artificial, 3D structures and communication between these agents is essential. The work done by Lifton illustrates how communications between a large quantity of agents can function in a network configuration [2, 19]. He poses the questions of how to process, communicate and apply the voluminous influx of data gained through large sensor matrices contained in current machinery, particularly with respect to the robotics domain. This aspect becomes relevant when considering that the scale of these agents is envisioned to be 2–6 mm [6, 7]. Lifton presents a distributed sensory network in the form of hardware and software embodied through pushpin computing. His research supports a system whereby it is possible to incorporate the following functions:

  • add more agents to an existing system and have them integrate faster because they can benefit from the agents that came before them,

  • the ability of a system to self-repair,

  • compensate for possible failures in specific areas,

  • localise and isolate problems rather than affecting the whole system.

In research projects whereby MAS communication protocols were implemented via physical prototypes, the outcome resulted in successful agent-to-agent interaction [3, 20, 21]. However, these systems often exhibited self-organising behaviour as opposed to the required self-assembling ability. Despite this behaviour, the value of these projects resides in the behaviours that were coded into the agents. For example,

  • A MAS must be capable of dealing with agents that malfunction or, if for example a shortage of energy is detected, function with a reduced capacity of agents [21]. This requires adaptive communication protocols.

  • The ability of agents to decide which ‘role’ or function they must provide in any given situation. This is reflective of behaviours evidenced in natural systems [22] but also becomes more relevant for an agent’s spatial orientation and position in creating 3D structures.

  • Similar to natural systems being based on specific blueprints, each artificial agent can also have a range of variations based on the quality of its components (motion, learning, communicating, material makeup, etc.) [3].

  • Rare or unforeseen events that cause errors ranging in severity, either from within or due to external environmental influence (e.g. internal influence: short circuit, external influence: poke) [3].

  • Communicative errors such as multiple messages per channel, chatter between agents, random noise. This can have a domino effect on other dependant factor, e.g. failing to sense boundaries [3].

These behaviours (deciding, coping, acting, learning, sharing, etc.) should ideally be transferable to any kind of agent, requiring adaptation only to accommodate changes in physical design of the agent, i.e. analogous to the act of walking, irrespective of whether it is on two or four legs. Whilst the listed examples demonstrate the complexity and variation of scenarios, outside of the original purpose for such a system (self-assembly into user-defined 3D structures), it also indicates the layers of awareness that are fundamental to an agent’s ability to act autonomously. Self-assembly adds the conditions of structural cohesion and the physics involved in 3D construction. Similar to the necessity for agents to be able to self-assemble into 3D structures, communication protocols also have to become 3D. This not only refers to the traditional spacial axes and the natural flow of communication, but includes the levels of awareness that translate between the internal and external world of agent perception [1, 23], see Fig. 1. The model of awareness proposed by Thórisson indicates the bi-directionality of awareness but also the balance between ‘hardcoded’ behaviours found in the innermost layer and the flexible, adaptability of the remaining layers in their participation of information exchange.

Each layer is interlinked with the other, and once the process of awareness begins to include environmental influences, a range of behaviours emerges. Even though each layer or level may be clear in its function, in isolation; similar to the nature of complexity, it is the combinations of these layers that can help define the degree of intelligence for artificial agents. It is becoming clear that, in conjunction with intelligence, communication plays an integral part in defining agent autonomy. Exploring the development of agent communications can also provide valuable insight into the degree of autonomy required for shape-shifting technology. Evolving from a) functional communication: agents moving in their environment, completing a task; to b) basic autonomy: coping with an unexpected event, getting pushed and eventually c) intelligent autonomy or AI: learning from experiences, forgetting irrelevant data, making decisions, interpreting situations.

Fig. 1.
figure 1

Three levels of awareness and directions of influence.

Full size image

At this point it is important to consider again that an appealing aspect of digital, is the ability to control it; its predictability, and its order. Does intelligence enhance these qualities or does it detract from them? Is it possible to find the balance between control and artificial autonomy? What would happen if artificial agents are given a degree of autonomy and allowed to develop their own form of communication? These questions again can only truly be answered closer to a time in which MAS based shape-shifting technology becomes more clearly defined. However, as the design process develops, and in researchers’ engagement with the subject of agent autonomy, it is important to consider that certain peripheral questions as those suggested above, may become more relevant earlier than expected.

Having highlighted pivotal research that informed the technical communication methods relevant to MAS, another key influence is the discipline of biomimcry. It is important to recognise the vast diversity in how biological entities can communicate with each other and within a social setting but also the diversity that individuals themselves can present, using different combinations of their senses. For example, consider the range of influences, with respect to communication alone, in the design of the Dod:

  • with its moving arms, the Dod shares a physical likeness to an octopus or sea anemones,

  • The Dods implementation requires it to be part of a MAS, like ants or bees,

  • Being made by humans, includes the elements of creativity; drawing from a rich diversity of communicative adaptation: spoken, pictographic, sign, numeric, written languages, to name but a few variants. It not only ensures a rich variety of learning and perspectives but can accommodate adaptability and the means to convey shared experiences over multiple channels of cognition.

Considering the complexity of shape-shifting technology that is to be based on a MAS, there are still a number of undefined design parameters. Deciding which avenue of research would be most worthwhile to pursue and which could provide the greatest probability of success, it is necessary to develop versatile design guidelines. In the instance of the Dod the design guideline replication versus emulation emerged. Applying this guideline assisted in achieving rigorous, theoretical suggestions whilst still accommodating the required scope for creativity.

3 Replication Versus Emulation

An advantage of exploring the discipline of biomimicry, is the diversity of biological systems and in particular how they have developed to function successfully, being optimized for their specific purposes and environments. For example, communicating via the olfactory modality through pheromone trails [24], aural modality via echo location, or kinesthetic modality represented in dance (e.g. bees communicating through dance). To understand and research these methods, the process of replication provides a good starting point for the initial exploration. Replication entails recreating what already exists and fulfils an essential quality of deductive reasoning. Conclusions reached as a result of this approach are formed as a logical progression from an original premise(s). This also means that even if the original premises were to be incorrect, the conclusion would still be logical.

Deductive reasoning finds resonance within the STEM disciplines and in conjunction with the process of replication, allows system variables and behaviours to be determined. It is a process that can help unravel complex systems. Physicist Richard Feynman described complex systems as being constructed-of or based-on very simple rules / algorithms and that it is the interaction of these simple rules, with each other and the environment, that underlie complexity [25]. In isolating and defining the number of system variables in order to discover these ‘simple rules’, the potential to create constrained or singular conclusions is increased and each new variable may require the repetition of the entire deductive process. Following this process, if a system can be replicated near to exactness and it functions as the original, then it can be understood. It also creates a strong and secure knowledge foundation. From this point it is conceivable to gain further insight into the relationships these variables have on each other, not just as the whole system, but it is also possible to postulate further questions: ‘What if…’, ‘Consider this…’, which indicate the stage of research where the system variables are altered. Studying the effect or consequences of these alterations indicate that a system is understood to such a degree that there is potential to predict the resulting behaviour to a high probability of accuracy.

Whilst deduction is favourable to the exploration of systems that are known to exist, what aspects of reasoning change when the questions originate from a purely conceptual or idea-based perspective, where there is no previously proven data, clear evidence or a possible correlation? In this instance, the previous process of exploration, through replication, does not occur on an external system but rather on an internal one: intuition. The internal system of intuition, experience and perception often produces connections between elements that cannot occur from a grounded perspective. Lifton, Feynman and mathematician Dirac use terms such as elegance and beauty in the description of scientific processes because the emotional connection and interpretation of knowledge, has as strong an influence in the manner in which researchers interact with knowledge as would a process that is documented and which produces tangible results [19, 25, 26]. The process of analysing, connecting and living experiences in order to create a different kind of knowledge base, is a valuable asset in research that deals with significant unknown variables, e.g. how will tangible shape-shifting technology be used? what can it actually do? how sentient does it need to be? how will users react?, etc.; it incorporates a researchers phenomenology. This essentially means, that even if a core piece of research is being explored by several different researchers, each individual will be able to contribute a unique insight and perspective. As shape-shifting technology in its current vision is still hypothetical, it is a subject that benefits greatly from an inductive reasoning approach. It broadens the scope of research and allows for the variable of uncertainty. As different, individual elements of this kind of technology are being explored (e.g. communication, self-assembly, agent shape, metamaterials, etc.), it will eventually be possible to combine this research.

In contrast to deductive reasoning, the conclusions drawn from the inductive process of reasoning are based on the surrounding evidence, or comparable theories, more so than an actual, specific premise itself. They promote a probability of truth or fact (as opposed to a clear certainty), but do not need to be correct. The greater the volume of relevant data that is available in the construction of the premise(s), ensures a higher probability that the conclusions will be true. For example, consider the following conclusions from the question: ‘can a human fly as a bird does?’.

  • Deduction: A bird flies as a result of its overall design (premise). This ranges from body shape, skeletal density and muscle distribution, etc. to the shape of its wings, the function of the feathers and the mechanics and physics of air flow. Humans cannot fly exactly as birds do (conclusions).

  • Inductive: ‘if a bird can fly, and the mechanism is mimicked (premises), then can humans fly too (conclusion)?’. The variety of shapes and sizes of birds, as well as other flying creatures such as bats or insects indicates that if the proportions are correct and relative to human dimensions, it should be possible.

In the latter example the premises were true (if a bird was to be replicated as is in every detail, [27, 28]), but the conclusion that this mechanism could work for humans was false - humans cannot fly as birds do. However, it establishes the scope to consider that humans could fly like birds. In the original PhD study surrounding the Dod, it was found that inductive reasoning was a useful tool for generating questions, whereas deductive reasoning helped answer these questions. Applying this cyclical process in the same manner as the user centred design (UCD) method is applied, ensured that the Dod design could undergo testing as well as maintain an adaptive design process in order to help accommodate the unknown design variables. Furthermore, for the purposes of the study, these two types of reasoning approaches were represented in the processes of replication and emulation. As has been established deduction and replication are mutually beneficial. Inductive reasoning focuses on the essence of a system or idea, and emulation can aid in establishing this concept. Emulation entails qualities also conveyed through imitation and replication; however, an important difference is that the system that is emulating another, is not designed to be applied in the same manner as the original - it aims to surpass the original. It entails the utilisation of knowledge and adaptation to a different or new application. Using the example of flight, a key element or essence of flight is the physics of airflow over the wing structure. This knowledge, among other contributions, produced a wing shape that could enable humans to fly - not replicating the exact mechanism.

Another example of using emulation as a guideline in the design process can be seen in the development of the foot prosthetic. Whilst exact replicas of the foot / leg were initially the point of relevance for research, it is also worthwhile considering the prosthetic seen in Fig. 2.

Fig. 2.
figure 2

Purpose-orientated leg/ foot prosthetic.

Full size image

The prosthetic in Fig. 2 is dissimilar to the biological counterpart that it aims to replace, but it represents the essence of how the leg / foot structure functions and simulates that action. In the research of the Dod, the essence of what is represent by a fire ant colony and rheopectic Non-Newtonian fluids, was researched.

  • Most ant colonies represent successful and efficient, biological MAS and not only use material from their environment to build structures larger than themselves but in times of need can also use themselves as building blocks [29,30,31].

  • Non-Newtonian fluids provided valuable insight into particle characteristics that enables a fluid to be both liquid and semi-solid given specific forces it is exposed to.

These explorations lead to the following design criteria from which the Dod emerged:

  1. 1.

    A semi-spherical shape with an irregular, cratered surface.

  2. 2.

    Non-hierarchical chain of command: autonomy to function as individuals

  3. 3.

    The ability to morph: surface topology and fundamental form

  4. 4.

    One material make-up and scalability – structural affordances and inherent material qualities

  5. 5.

    Bi-directionality – the ability to assemble and dis-assemble

  6. 6.

    Behavioural simplicity

The design guideline, emulation versus replication, has been useful in this instance to facilitate a degree of adaptability in the design itself. As research continues it is important to return to the original concepts overarching the individual areas of research entailed in shape-shifting technology: Sutherland’s Ultimate Display and Ishii’s Radical Bits. What is the essence of shape-shifting technology? One interpretation could be that it is giving an ethereal substance (digital data) a physical manifestation. Consider extrapolating this concept further to encompass quantum computing; what connotations are entailed in this avenue of research?

4 Game Context

Having considered the potential of the Dod design, the next stage in the research of such a project is the application of knowledge gained throughout the research and making it accessible. A strong motivation to design the Dod primarily as an adaptable and versatile blueprint for an agent design, was to accommodate the fast-paced or short-lived lifespan of such research. A large portion of proposed shape shifting devices rarely leave a lab environment because of the possible expertise, specialist equipment or material cost involved in the project [32,33,34,35,36,37,38,39,40,41]. With the developments in material science, 3D printing, biological engineering [42,43,44] it is necessary to continuously develop research in order for it to stay relevant. In the original study regarding the Dod, four main sections of future work that explore the potential of the Dod with respect to considering further adaptations to its physical form as well as possible variations to its overall behaviour, were detailed. These are as follows:

  • developments of the Dod design (facet and arm function, methods of motion),

  • energy manipulation,

  • boundary conditions and

  • 3D thinking

The subject of 3D thinking will be addressed in this section. The importance of 3D thinking in shape-shifting technology relates to user interaction, and creative application but also eventually commercial uptake and social engagement. Thinking 3 dimensionally should be innate to people (e.g. strictly speaking even a piece of paper is 3D). However, there is a discrepancy between how people represent and convey designs, and how people actually interact with 3D space. In general, the first point of reference for the expression of ideas or designs, whether 2D or 3D, occurs predominantly through sketching. Whilst sketching is a powerful tool, it is primarily focused on visual representations on a 2D plane. The illusion of 3D can be created; however, it can be argued that an important 1st step in the act of interacting and communicating with digital data that has a tangible form (i.e. shape-shifting technology), is that of 3D visualization. An analogy can be drawn from sewing, whereby drafting a paper pattern occurs on the 2D plane, whereas draping a pattern occurs on the model; on the 3D plane. Each process, designing in 2D or 3D, provides unique insights with respect to interaction techniques and produces very different results.

The nature of digital structures has enabled technology to develop based primarily on the visual sense which is unidirectional. Large volumes of data can be processed and it is accessible by a large majority of any society. In contrast 3D exploration (e.g. sculpting, multimedia prototyping, construction) is primarily based on the haptic sense which is bidirectional. A reason as to why it is important to upgrade or adapt the manner in which people can represent 3D thinking and manipulation is because if they cannot express their ideas, and are limited by the medium they use to convey ideas, it may result in a lack of creativity and hinder the potential of 3D exploration. This has since been recognized in both academia and industry. For example, in the digital domain, 3D modelling software is available and more readily accessible via high quality open source software that can be introduced into schools more easily (e.g. Solidworks, zBrush, Maya, Blender, SketchUp, etc.). 3D printing technology has changed the prototyping and fabrication process, and is also becoming more accessible through 3D printers aimed at Hobbyists and the development of Fab Labs (e.g. Ultimaker, Form1, Makerbot, etc.). Even the process of sketching itself which was traditionally done through pen and paper has been augmented through the development of 3D pens. These tools are taking the next step into realizing 3D concepts and encouraging 3D manipulation, without first translating it into a 2D representation (e.g. 3D Magic Imagi Pen, IDO3D Vertical, AtmosFlare, 3Doodler, CreoPop, etc.). In other domains such as human computer interaction (HCI), the importance of multi-material prototyping and the process of involving users in the design process itself, including hands-on modelling, has been recognized and is being encouraged. The various forms of user research already illustrate the multimodal nature of the design discipline and how different insights are possible through the variety of engagement / exploration (e.g. multimedia brainstorming, shadowing, game-play, diary studies, simulations, verbal interviews, quantitative or qualitative questionnaires).

To date the research surrounding designing MAS for their use in shape-shifting technology for TUIs has been purely academic. To reduce the gap between theory and practice it is necessary to consider how the Dod, in its concept of being an agent, can A) become more publicly disseminated, B) how it can encourage haptic exploration through play and C) how it can potentially help train 3D thinking. A viable avenue of development that encompasses these three requirements is adapting the Dod into a game-piece or a toy, both as analog and eventually as digital format (e.g. the cuboid shape: as an analog toy is represented by Lego®, and as a digital game-piece is represented by Minecraft). This development would have an immediate benefit of public dissemination. As the technology becomes more refined, it would have the later benefits of the general public being more familiar with MAS, or programmable matter concepts. This process is comparable to how the cuboid shape has become integrated into public consciousness, in Western society.

A strong element that supports the approach of game-play, is that of learning and establishing conceptual frameworks through the act of playing. As children learn, they explore their environment through multimodal sensing thereby generating experiences that are not purely theory based. For example, Gupta and Khanna are educators that have made physics principles accessible and applicable for young children through the development of toys, e.g. a race car built on a twisted rubber band [45,46,47,48,49]. The children can create these toys themselves using readily available materials. The understanding gained through this process is invaluable because they not only learn how the toy works but also how to construct it. This type of learning encompasses a wider range of modalities than the traditional classroom setting, i.e. haptic learning as opposed to a purely cognitive uptake of knowledge. Using this core approach, other toys such as Topobo [50], Meccano, K’Nex, Baufix (Fig. 3a–3c) and Lego® all aim to foster an understanding of physics, engineering, electronics and construction concepts, among others, but also inherently encourage 3D manipulation and an understanding of a 3D environment through playing. The act of playing also supports contextualizing experiences in relation to others, encompassing a variety of behaviours and emerging social strategiesFootnote 1.

Fig. 3.
figure 3

3D thinking – (a) Meccano, (b) K’Nex, (c) Baufix.

Full size image

Some of the mainstream toys (e.g. Lego®, Meccano) are a simplified representation of the world, i.e. straight lines and 90˚ angles. There is however a much larger scope of shapes for achieving the same and sometimes even better results. For example, in Buckminister Fuller’s childhood he was very short-sighted which lead him to perceive only elements in his immediate presence. In one classroom exercise the children were asked to create a structure using peas and toothpicks. By comparison to the children who perceived their environment and mimicked the structures they saw, e.g. square, blocked houses, Fuller created what he later patented as the geodesic dome. He could only draw on his own haptic experiences, thereby creating and using an alternative building block (a triangle) and producing a unique structure [51, 52].

Ideally a multimodal learning style of learning should also be continued into adulthood as it continuously assists in developing motor skills and neuro-connectivity. Therefore, considering the benefits of haptic exploration through play and considering the construction-based toys that already exist, to base a building block on a semi-spherical shape, like the Dod, can open opportunities for a new range of creative solutions.

The following games, projects and hypothetical concepts provided inspiration regarding the potential of the Dod to be developed as a game-piece, with the primary aims of becoming more publicly accessible as well as encouraging 3D thinking and manipulation: Cellulo, Tantrix, Domino, Jenga and Kal-toh.

4.1 Cellulo

Cellulo is a project that uses swarm computing to link macro-sized robots to affect motion, haptic sensation and visual perception. The aim of the project is to make “tangible what is intangible in learning” (Özgür et al. 2017). Three primary requirements have guided the development of this project to make the interaction with data more accessible: ubiquity, practicality and flexibility. The inspiration from this project relates to the classroom environment in which it is used. Whilst the system provides valuable insights to communication techniques for MAS, the qualities necessary for encouraging interactivity are also valuable. For instance, by designing the robots to be ubiquitous follows in the recommendations of Mark Weiser’s description of an ‘excellent tool’. The tool itself should be intuitive and easy to learn and so natural to use that the focus and attention of the user is directed towards the task itself rather than be distracted by the implement used to carry out the task [53]. Practicality is an essential component of these types of robots and in this instance more accurately refers to the necessity for robustness of the materials - not only the physical outer appearance but also the interior circuit components. Cellulo is intended to be used regularly in a classroom setting, for a variety of subjects and by children of varying ages. Being able to cope with the unexpected is an advantageous design consideration.

Since Cellulo is designed for the scenario of robots in education, flexibility and versatility are key factors. Being applicable to a multitude of subjects encourages the integration of this type of technology but also ensures that the robot design is focused towards ease of use, i.e. it should not take up any of the teacher’s time to set up thereby potentially detracting from teaching the subject material itself.

4.2 Dominoes and Tantrix

Both games are based on the placement of tiles with specific values on them and can be played solo or as part of a group. A DominoeFootnote 2 tile is a cuboid with a number of dots printed on either side ranging from 1–6, like a die. A Tantrix tile is a hexagonal prism and has three lines of different colours and orientation printed on one surface going from one edge to another. In each game, a variety of other puzzles developed from the basic game play: each player chooses a number of tiles whilst the rest remain unallocated and can be used throughout the game when the player’s own tiles cannot be played.

In Dominoe tiles must be placed adjacent to each other whereby a following tile can only be placed at one of the two ends of the previously played tile, doubles act as branching points whereby tiles may be placed at one of the four sides of the tile. In Tantrix each player choses a colour and must attempt to lengthen their coloured line by the placement of other tiles. As each tile contains three colours in total it is necessary for the player to ensure that the other colours on the tile also match the previously played tiles. Whilst this can mean one player enhances another player’s position (by lengthening their line), the line orientations on the tiles (i.e. straight, curved, etc.) ensure that it is not a reliable method of gaining an advantage because the line may also be curtailed, e.g. if it does a U-turn.

In both of these games the simple aim of adjoining matching numbers or colours is enhanced and elaborated on by the physical shape of the tile. They are good examples of the principle that complexity can emerge through the interaction of a simple set of rules. These games start from a single origin point and build-up to generate a larger playing field - similar to the concept of having a seed agent that communicates throughout the entire network. Dodlen would act as a tile that allows players to build into the 3D space as opposed to remaining on a ‘2D’ surface.

4.3 Jenga

Jenga is a game based on cuboid building blocks. 54 blocks are stacked in alternating orientation of 90˚, in rows of three until a tower of 18 rows is standing. Players alternate in turn each taking one block from any layer (apart from the top two layers) and placing it on the topmost layer in alternate orientation to the existing layer, i.e. the layers of blocks must lie perpendicular to each other. A player may only use one hand but can bump or nudge a block to check its willingness to depart from its current location. A steady hand, as well as good spatial awareness and dexterity are skills practised in this game. The game ends when the tower can no longer retain a stable equilibrium and topples.

This game highlights how the visual and haptic sense combine in order to assess and anticipate the reaction of the structure via interaction with it. It is also a good example of how 3D thinking is inherently fostered.

4.4 Kal-Toh

Many sources of inspiration in the sciences could not exist without the element of creativity. The last game suggestion (Kal-toh) is first mentioned in the science fiction series Star Trek. It is a geometric, shape-shifting based game, whose aim is to train patience as well as logical and rational thought of the user in their attempt to create the final shape of two nested icosidodecahedra. The inner shape is connected to the outer shape by the centre points of the edges [54]. Similar concepts such as the 3D interlocking wooden puzzles already exist today, however the interesting difference is that the player’s moves in Kal-toh have the potential to initiate a chain reaction of shape-shifting that can work towards achieving the final structure or, contrary to the players advantage, can work towards creating greater chaos (similar to the game snakes and ladders), i.e. the game is reactive to the user’s actions. This latter quality is another important challenge for any kind of technology, current and future: coping with the unpredictability of human behaviour. Figure 4 illustrates the starting and end position of Kal-toh.

Fig. 4.
figure 4

The beginning and end stages of the fictitious game of Kal-toh [54].

Full size image

A key feature of the game is finding the “seeds of order even in the midst of profound chaos” [54]. The player adds or moves the thin rods to specific locations to set in motion a change of shape that eventually leads to the final structured geometric shape (like the reapplication of Dominoes, regarding chain-reactions). The icosidodecahedron is an Archimedean solid based on two primary polygons: pentagon and triangle, which meet in identical vertexes. Each side of a pentagon is connected to a triangle which has pentagons attached at the remaining two sides until they meet each other, i.e. similar to the Hoberman sphere. The rods used in the game enable the player to create the icosidodecahedron by outlining the edges of the required polygons.

The following points highlight the crossover between existing games and the academic findings relating to MAS agent design. They demonstrate the relevance and ability to link current and future research into agent design.

  • be robust,

  • facilitate quick learning (i.e. clear affordances),

  • be versatile, possibly have multiple puzzles in one (i.e. shape and size change),

  • pattern matching,

  • playing with others or individually,

  • haptic engagement through 3D exploration (i.e. bi-directionality)

The scope for creativity in this domain is substantial because the objectives and boundary conditions have altered. It brings a new set of challenges, but also considers the importance for momentum in research to continue developing the Dod, as well as developing the overarching concept of MAS agent design.

5 The Dodlen

The nature of the games or interactive pieces (i.e. shape, affordances, representations) described in section four have provided inspiration for the Dodlen (Dod game-piece). Elements such as complexity of moving parts (polymagnets), environmental impact of materials, power supply requirement, size or variety of game piece (Hoberman sphere) are considered in the design adaptation.

During the original study, indicators for viable designs for a game-piece started to emerge in the artistic explorations of the Dod design. The Latch Dod is an example of this concept [10]. A spring latch was contained within each arm, which in turn was encased by an origami constructed, pentagonal frustrum, see Fig. 5a. At the top of the origami spring, a pentagonal facet was attached. This feature was to maintain the concept of the original Dod and arm rotation, as can be seen in Fig. 5b. Any of the 12 facets could be depressed in order to retract or in turn release the latch spring in order to extend the arm again. An interesting behaviour emerged when all the latches are depressed (in the retracted position) and the Latch Dod was dropped from a short height (e.g. 10 cm). The spring of the latch on which the Dod landed extended providing sufficient energy to propel the whole construction up and onto another latch. This essentially triggered a chain reaction causing the Dod to jump around an area until most latches had been activated.

Whilst similar in construct to toys such as ‘Chuckle Ball’ [55, 56], the appendages of these toys are stationary and the bounce or jiggle comes from within the spherical ball provided by an internal motor. In the instance of the Latch Dod there is no power source required, i.e. the energy for motion, bounce, topological and form change is provided by the potential energy contained in each of the 12 springs.

Fig. 5a
figure 5

– c. (a) Spring latches contained in the inner core, (b) single core-arm construction, (c) Final Latch Dod.

Full size image

Another important quality in MAS to be able to build 3D structures, is the ability to self-assemble. This behaviour is simulated in the original Dod prototype through the use of permanent 3 mm ball magnets. As an alternative to using permanent magnets, the concept of polymagnets also proved to be of interest and as yet could be an avenue of future development. This relates to printing magnetic patterns thereby altering the magnetic field lines. The company Correlated Magnetics works on varying designs ranging from Latch, Align, Attach, Spring and even printing magnetic patterns onto non-linear surfaces. In their latest line of research are magnetic patterns that are encoded onto the required part so that it can be read by sensors [57]. Of interest to the Dodlen, among other future considerations, is the combination of an Attract and Spring mechanism. The pattern printed resembles a circular lock & key. Both magnets are restrained via a rod, which runs through the centre. When the latch is open the magnets repel each other and act like a spring, i.e. it cannot fully close and there is a specified distance between the magnets when at rest. When one magnet is rotated the magnetic pattern aligns and exhibits an attractive force essentially locking the magnets together and withstanding perpendicular outward forces [57]. There may be potential to print such patterns onto specific facets of the Dod and thereby automatically connecting a locking mechanism to the rotation-extension motion of the arm.

Considering these characteristics and possibilities again demonstrated the potential adaptability of the Dod design itself. For research purposes, ease of manufacturing and available resources, the Dod was developed as seen in Fig. 6 and is currently in production. It is currently still in the prototyping stage, with the aim to undergo user testing.

Fig. 6.
figure 6

Multiple Dodlens in various configurations.

Full size image

Following the design principle described in section three (emulation versus replication), it is possible to see that inspiration was taken from the original Lego block mechanism. In order to facilitate space efficiency, the round nub fits into a pentagonal shaped hole. The Dodlen’s arm are set in specific configurations ranging from 0 arms to all 12 arms being extended. The number of configurations versus the number of arms extended provide ample diversity with respect to the number of game-pieces available. For example, a Dodlen with 2 arms extended can have 4 configurations: Top and Bottom (creates straight lines), Top and near arm (creating an acute angle), Top and far arm (creating an obtuse angle) and two same hemisphere arms. There is also a degree of variance depending on whether the arms have nubs (can attach to another Dodlen) or whether they have holes (can be attached to). On each facet is either a nub or a hole and for each version of a Dodlen there is an inverted counterpart, i.e. if a Dodlen with all arms extended has nubs in each facet, then there is also a Dodlen, with all arms extended, that has a hole in each facet. It is possible to 3D print these pieces in PLA using a conventional hobbyist 3D printer, e.g. Ultimaker.

5.1 User Interaction

The Dodlens can be played with in two main ways, A) in the same manner as Lego blocks, i.e. as individual building blocks or B) as part of a game. This can initially prove challenging because it diverges from the familiar and traditional approach of building with cuboid blocks. The forms and shapes that are possible are not actively engaged with on a public level, in particular the additional consideration that spherical objects lack the ease of cohesion present in traditional cuboid building blocks. However, an interesting feature of the dodecahedron is the dual nature of its shape, i.e. it has flat faces enabling stable planes to exist, whilst at the same time embodying the essence of the curved form of a sphere. It can create straight lines by the extension of opposite arms as well as create curves along 2 axes. In this scenario it is possible for users to begin experimenting with shapes and constructs that can emerge from a semi-spherical building block. The overarching value of this interaction is fostering an engagement with unconventional results and ideally generating a new standard of acceptance. As mentioned previously, exploration and interaction through play is a useful and safe method of introducing novel ideas. The Dodlen takes inspiration from existing and strongly integrated games and toys, and has adapted 1 element: the shape. This ensures that users move from familiarity in concept (building -block) and interaction (stacking, connecting, building, etc.) and can focus on using the Dodlen rather than learning a new concept from scratch.

Mentally preparing the understanding for structures based on alternative building-blocks (or agents) gives shape-shifting technology more scope to define itself rather than be defined by existing computer concepts. The proposed Dodlen design highlights its affordances clearly and is intuitive to interact with. These qualities are important in the consideration of user interaction and engagement. It can easily be scaled to produce larger or smaller versions, and it can be constructed out of different materials, if necessary.

Implementing the Dodlen as part of game, a minimum of four playing boards that create the playing space are also envisioned. An additional 2 boards can be added to broaden the playing space. These playing boards are pentagonal in shape and connect at the same dihedral angle (116.57˚) forming a larger dodecahedron. The boards range from having a plain to varied surface topology. A plain surface topology consists of a distribution of nubs and holes to which one or multiple Dod pieces can be fixed. This distribution can be a regular or random pattern. A varied surface topology will have certain structures already in place, e.g. half or full Dodlen affixed onto the surface in specific configurations. The purpose of these is to diversify the playing field and provide varying levels of difficulty.

5.2 Game Objectives

There are two fundamental objectives represented in this game format, which are as follows:

  • Physical: Create lines or structures made up of Dodlens connecting two or more game boards.

  • Interaction: Develop game strategies that enhance engagement with 3D thinking and manipulation

For the purposes of the prototype, different colours will represent the different levels of arm extension purely for identification purposes. However, colours can then also be considered in the development of game rules to indicate specific rules or behaviours, i.e. red Dodlen will prevent any player from connecting to that Dodlen - can be used as a blocking mechanism; a yellow Dodlen means a player can only connect to a specific hemisphere, etc. The mechanism by which a player’s turn is indicated can be represented by a traditional die shape but with different symbols on its facets, see Fig. 7.

Fig. 7.
figure 7

A diagram dice from the game Coco Crazy by Ravensburger.

Full size image

Applied to the overall game, it can define which colour Dodlen a player must use, or to turn the entire game-board, change connecting direction, connect to two different game boards, etc.

The variations in the game pieces include the size of the Dodlen pieces themselves, e.g. larger Dodlen pieces can have different rules associated with them. This would reflect the concept of a seed agent discussed in research into agent-to-agent communication [21, 58]. The final game strategies are currently being refined with the aim of undergoing usability testing in order to determine elements that require further adaptations or improvement. Whilst the aim is to create an engaging game, a strong underlying motivation is to support and foster 3D spatial manipulation and interaction.

6 Conclusion

In order to sustain research, it is necessary to explore avenues which help maintain its relevance. A project such as the Dod (MAS agent design) and on a larger scale, shape-shifting technology, due to its nature is primarily hypothetical. Research into individual aspects of such technology is necessary (e.g. self-assembly, communication, interaction). However, this paper has also highlighted that adapting design guidelines to provide support and scope for creativity and logic, is essential in projects whereby significant design parameters are still unknown. In order to bridge the gap between science-fiction and reality it is important to continuously evaluate research and maintain an overview of the context into which shape-shifting technology should orientate.

The second aspect of this paper describes the Dodlen – a Dod based game-piece. It relates to returning to a fundamental technique of exploration and experiential development: playing. Through this technique, it is possible to gradually influence 3D thinking, interaction techniques and structural possibilities of MAS. When the first computer graphical user interfaces (GUIs) aimed to digitally and conceptually represent a physical desktop (i.e. trashcan icon, filing system, individual applications - notepad, paint), it followed several design heuristics later defined by Nielson, e.g. recognition rather than recall [59]. Users were already familiar with the concept of a desktop and this concept was transferred into the digital realm. Similarly, MAS based shape-shifting technology has a diverse range of elements that require familiarity on behalf of the user: structural cohesion, AI learning frameworks, hive behaviour, agent communication, haptic dexterity, etc. Encouraging a longevity of engagement with research through whichever medium, ensures a deeper understanding as users will have had time to interact, think and engage with the physical output of research as well as the theories it aims to represent.