Keywords

1 Introduction and Outlooks

Inspired by the soft material ubiquitous in all living creature’s body structures, a new family of robots has been constructed with the aim of incorporating flexible body elements. The resulting machines, called soft robots (Trivedi et al. 2008; Kim et al. 2013), have significant advantages over traditionally articulated robots in terms of morphological deformability and interactional safety. These robots are able to adapt their morphology to unstructured environments and can manipulate fragile objects without damaging them, making them especially attractive for interactions with humans. Specific possibilities include care for the elderly, prosthetics, and wearable technology (Pfeifer et al. 2012). Soft robots can generate diverse behaviours with simple types of actuation by partially outsourcing control to the morphological and material properties of their soft bodies (Shepherd et al. 2011), which is made possible by the tight coupling between control, body, and the environment (Pfeifer et al. 2007; Pfeifer and Bongard 2006; Nakajima et al. 2012a). Furthermore, production costs for these types of robots are relatively low. Thus, they can be easily incorporated into a wide range of machines and technology with various purposes (Morin et al. 2012).

Concomitant with these benefits is a great challenge in controlling their body dynamics. In general, soft body dynamics exhibit multifold properties, such as high dimensionality, nonlinearity, and a certain amount of “sluggishness” as a result of their elastic nature (Nakajima et al. 2015a, 2012b, 2011). In addition, they have higher levels of freedom of movement than traditional robots, often with greater degrees of freedom than the number of actuators (i.e., under-actuated systems), making them notoriously difficult to control.

Here, we aim to reverse this scenario and view it in a positive light. We claim that it is these same nonlinear and elastic body dynamics that make soft robots feasible and simple to control. In nature, some animals have soft bodies and control them in a sophisticated manner by capitalizing on their body dynamics. The octopus serves as a prime example (Hochner 2012). It does not have any rigid components in its body, which can move with virtually infinite degrees of freedom. One of the characteristic structures observed in the octopus arm is the muscular-hydrostat (Smith and Kier 1989; Kier and Smith 1985; Taylor and Kier 2003; Feinstein et al. 2011; Li et al. 2012a). In such structures, the volume of the organ remains constant during all movements, enabling the muscles themselves to perform all of the functions usually performed by the skeleton. By developing a series of platforms called muscular-hydrostat computers, inspired by the octopus, we have demonstrated that these body dynamics can be positively exploited as a computational resource. For example, we have shown that their muscular organization provides sufficient nonlinearity and elasticity that it can be used to embed multiple control programs. In this paper, we aim to review bio-inspirations and our series of proposed platforms. We begin by discussing the inspiration stemming from octopus movement and then introduce the physical reservoir computing approach.

2 Octopus Inspirations: Embodiment, Muscular-Hydrostats, and Embedded Motor Programs

Biological systems have certain morphologies and material characteristics that improve their adaptivity and increase their probability of survival. In nature, these body structures and organizations evolve based on the animal’s ecological niche, suggesting that the mechanism controlling animal behaviour is not only located in its brain, but that there is a tight coupling between the brain, body, and its environment, an idea that is usually termed embodiment (Pfeifer et al. 2007; Pfeifer and Bongard 2006). An octopus has hyper-redundant limbs with a virtually unlimited number of degrees of freedom; its movements are known to be highly sophisticated. Its specific body structure is specialized to permit survival in a complicated underwater environment (Hochner 2012). From a conventional control perspective, the octopus’ method of controlling its movement and harnessing its nonlinear body dynamics is outstanding and instructive. In this context, the octopus has been a good source of inspiration for designing a control strategy for soft robots (Trivedi et al. 2008; Kim et al. 2013).

One of the most characteristic body structures observed in the octopus is its muscular organization, called muscular-hydrostats (Smith and Kier 1989; Kier and Smith 1985; Taylor and Kier 2003; Feinstein et al. 2011; Li et al. 2012a). It acts to keep the volume of the organ constant during all movements, so that if the diameter of the muscular-hydrostats decreases, then its length increases, and vice versa. For example, in the octopus arm, muscles are organized into transverse, longitudinal, and obliquely oriented groups, which form the muscular-hydrostatic structure. Such flexible structures show major advantages over articulated arms that have a rigid skeleton and joints. Accordingly, in robotics, there have been several attempts to characterize the role of the muscular-hydrostat system in terms of its anatomical structure and morphology.

The nervous system of the octopus is characteristically highly distributed throughout the entire body. It has a relatively small central nervous system (CNS), which controls the large peripheral nervous system (PNS) of the arms. The CNS integrates information from the visual system, and then issues commands to the lower motor centres that control the elaborate neuromuscular system of the arms. A typical example showing the effectiveness of this nervous system distribution is the animal’s reaching behaviour. Reaching behaviour consists of a bend propagation along the arm toward the tip in a highly stereotypical and invariant way. Sumbre et al. showed that the arm extensions can be evoked in arms whose connection with the brain has been severed (Sumbre et al. 2001). Because the evoked motions in denervated octopus arms were qualitatively and kinematically identical to natural bend propagations, an underlying motor program appears to be embedded in the neuromuscular system of the arm, which does not require continuous central control.

Based on these biological findings, we have been exploring a novel view of the property of the muscular-hydrostat system combined with the PNS in terms of its functionality. In this chapter, we introduce a series of platforms based on a machine learning technique called reservoir computing.

3 Physical Reservoir Computing

Reservoir computing has been proposed as a framework to train recurrent neural networks (Jaeger and Haas 2004; Maass et al. 2002; Verstraeten et al. 2007). In this framework, low-dimensional inputs are mapped to a high-dimensional nonlinear dynamical system, typically referred to as a reservoir, and the corresponding outputs are expressed as weighted sums of the reservoir states (Fig. 18.1). To possess a computational capability, a reservoir should have both input separability and fading memory. Input separability is usually achieved through a mapping of the low-dimensional input to a high-dimensional state space. Fading memory is a property of a system that retains the influence of a recent input sequence within the system and permits the integration of stimulus information over time, enabling reproducible computation for which the recent history of the signal is significant. A remarkable property of the approach is that if the reservoir involves enough nonlinearity and memory, it can be shown that it is sufficient to add a simple linear and static readout from the high-dimensional state space to emulate nonlinear and complex computations (Fig. 18.1). The approach is extensively studied in neuroscience (Sussillo and Abbott 2009), and many applications can be found in the latest robotic applications (Li et al. 2013a, 2012b,c, 2013b; Kuwabara et al. 2012).

Fig. 18.1
figure 1

Information processing in the reservoir computing scheme. Low-dimensional inputs are injected to a high-dimensional dynamic system called a reservoir. Outputs are calculated as a weighted sum of the states of the reservoir nodes. Network training only requires the adjustment of the linear readout weights

Full size image

Because of its generic nature, reservoir computing is not limited to digital simulations of neural networks. Any high-dimensional dynamic system can serve as a reservoir if it has the appropriate properties. More specifically, if the reservoir consists of a physical system, we term the approach physical reservoir computing (e.g., see summary on robotics-related implementations in Hauser et al. 2014). A number of physical implementations for reservoirs have been proposed, such as the surface of water (Fernando and Sojakka 2003), and nonlinear mass spring systems (Hauser et al. 2011). More recently, electronic and optical implementations have also been reported (Appeltant et al. 2011; Woods and Naughton 2012).

In this chapter, we introduce a series of platforms that exploit the dynamics of soft structures as a reservoir. The first platform is a dynamic model of a muscular-hydrostat system that is inspired by the muscle organization of the octopus (Nakajima et al. 2013a). The second is a dynamic model of an entire octopus arm equipped with a muscular-hydrostat system (Nakajima et al. 2013b). It has been shown that this model can also be used as a model for certain soft robotic arms by tuning its mechanical parameters (Zheng et al. 2012). The final platform is a soft robotic arm made of silicone (Nakajima et al. 2014, 2015b, 2013c). In each platform, we demonstrate that the body dynamics of the system can be used as a successful reservoir; this means that, by simply attaching a static linear readout from the high-dimensional nonlinear dynamics, one can emulate complex nonlinear computations without altering the physical system itself. That is, we employ the physical body as part of a computational device. We call this series of platforms muscular-hydrostat computers (Nakajima et al. 2013d), and introduce them in the next section.

4 Muscular-Hydrostat Computers

In this section, we briefly introduce our physical reservoir computing platforms, called muscular-hydrostat computers. Note that the term “muscular-hydrostat” is not used in a strict sense, because the soft silicone arm that we introduce later does not have any muscular-hydrostat properties. However, we use this term because our arm shares the same operational characteristics as the octopus’ arm.

4.1 Muscular-Hydrostat System

Several models emulating the muscular-hydrostat system of the octopus have been proposed (e.g., see Yekutieli et al. 2005a,b; Kang et al. 2012). The model used in this study is originally proposed in Kang et al. (2012). The overall structure of our muscular-hydrostat system is shown in Fig. 18.2a. We use a mass-spring-damper system in the shape of a truncated cone, consisting of a base plane, a ceiling plane composed of four transverse springs, a central strut, and four longitudinal springs, which emulate the anatomical structure of the muscle alignment in an actual octopus arm. The longitudinal springs control the position and orientation of the ceiling plane, while the transverse springs control the radius of the ceiling plane. The system has an isovolumetric structure, which provides forces constantly striving to maintain the arm volume, a property of the muscular-hydrostats. Thus, all the springs are assumed to be implicitly or explicitly coupled with each other. The values for all model parameters (e.g., spring coefficients, damping) are either inspired by the octopus or directly drawn from biological data. Because of these parameters, the overall model dynamics are highly nonlinear, even though the springs used in the model are linear.

Fig. 18.2
figure 2

A series of physical reservoir computing platforms called muscular-hydrostat computers. (a ) Exploiting the muscular-hydrostat system as a computational resource, the mass-spring damper system emulates the muscle configuration of the octopus arm and the property of muscular-hydrostats, which maintain a constant system volume. Forces are provided as inputs to the springs and the corresponding dynamics of the transverse and longitudinal springs serve as a reservoir. By introducing a different time scale between I/O relations and the muscular-hydrostat system, we demonstrate that the system has sufficient computational capacity to emulate nonlinear dynamical systems. (b ) A dynamic model of the octopus arm. The arm consists of 20 compartments, which are equipped with muscular-hydrostat systems (Fig. 18.2a). The mechanical parameters of the system are drawn from bio-inspirations. By controlling the base rotation of the model as inputs, the resulting passive dynamics of the springs are exploited as a reservoir. This system can be used to emulate nonlinear dynamical systems and embed robust closed-loop control. (c ) The silicone-based soft robotic arm. The arm encloses 10 bend sensors, which monitor local bends during each time step. It is actuated at the base point and is able to rotate. It has been shown that the body dynamics (sensory time series) can be used as a successful reservoir. The behaviour of the arm successfully emulates functions that require memory and embeds a robust closed-loop control

Full size image

By providing inputs as forces on the springs and adjusting the linear readout weights using the spring length dynamics as reservoir states, we demonstrate that the system can be directly used to accomplish complex nonlinear computations (Nakajima et al. 2013a). Together with the physical reservoir computing approach, the core concept of our technique was the introduction of different time scales between I/O signals and the system movement. In real-world applications, the assumption of this time scale difference is rather natural, usually caused by the sampling rate of the hardware device compared to the continuous physical dynamics of the robots. We have shown that this system can emulate desired nonlinear dynamical systems and reported that the system demonstrates a characteristic short-term memory profile. One possible application of this technique would be to embed a local control and outsource the computational load to the body part in order to make an external controller more efficient.

4.2 Dynamic Model of the Octopus Arm

The next platform we create is a 3D dynamic model inspired by the octopus arm. The overall structure of the entire arm is shown in Fig. 18.2b (Kang et al. 2012). The arm consists of 20 compartments; each compartment implements the same muscular-hydrostats construction explained in the previous section. It is assumed that the arm is to be immersed in an underwater environment in which water friction constants are approximated by computational fluid dynamics simulations (Kang et al. 2011; Kazakidi et al. 2012). As a result, the arm has highly nonlinear body dynamics when actuated.

By using this platform, we demonstrate that the body dynamics of the arm can be exploited as a computational resource: first, to emulate complex nonlinear dynamical systems; and second, to implement closed-loop control (Nakajima et al. 2013b). Usually, closed-loop control in robots is realized through a sensory-motor loop, where controllers take sensory values and calculate the corresponding motor commands. In our experimental setting, an external controller is absent, and the body dynamics themselves are exploited to control the next motion, meaning that the controller and the body part it is controlling are the same. We used several nonlinear limit cycles to see how they can be embedded directly into the soft robotic arm without any support for nonlinearity and memory from an external controller. For roboticists, our method will offer one way to quantitatively characterize which controls are efficient for a variety of body designs. In addition, it will also help outsourcing the control load to the body parts. In addition, through the emulation of nonlinear dynamical systems, we showed that each body part has a specific role according to its task type. These obvious and specific coherencies suggest that these properties originate from the intrinsic body structure.

4.3 Soft Robotic Arm Made of Silicone Materials

It is important to demonstrate whether the techniques illustrated in these models can actually work in a real physical platform. We have developed a simple but powerful physical platform implemented with silicone material (Fig. 18.2c) (Nakajima et al. 2014, 2015b, 2013c). The platform consists of a soft silicone arm, its sensing and actuation systems, data processing via a PC, and a water tank containing fresh water as an underwater environment. Note that in a precise sense, the silicone arm does not emulate the simulated octopus arm, which was introduced in the previous section. However, they share a similar operational property in terms of morphology and density. The arm has 10 embedded bend sensors within the silicone material. The base of the arm can rotate left and right through the actuation of a servo motor. Our system has one active degree of freedom, but there is a much higher number of passive degrees of freedom in the silicone arm. By rotating the base of the arm and generating body dynamics induced by the interaction between the underwater environment and the soft silicone material, we showed that the sensory time series reflecting body dynamics can be exploited as a part of a computational device. It could be used to emulate nonlinear dynamic systems and functions that require memory by embedding robust closed-loop control into the arm. The soft silicone arm presented here only partially mimics the properties, such as morphology, and the material of a real octopus arm. We expect that the real octopus arm has a greater computational capacity than the one we present here because of its diverse and dynamic nature.

5 Discussion and Future Directions

Many biological systems have soft bodies, which are able to adapt and behave effectively in their given ecological niche. The framework presented in this study may also shed light on the role of the body in biological systems. In this context, it would be interesting to investigate whether the soft arm could be used to embed more biologically plausible behaviours in future work. For example, as we mentioned earlier, it is well known that the octopus adopts a specific strategy for reaching, called bend propagation (Fig. 18.3a). In this specific motion, it is suggested that the CNS only initiates the motion and the muscle activations themselves are handled at the PNS level (Li et al. 2012a). Several researchers have investigated this behaviour by directly extracting the muscle contraction patterns from the real octopus and externally applying these patterns to the octopus arm models. The technique presented here may yield further insights into the overall scheme by including the role of the arm’s body dynamics. Because the PNS does not have plasticity, it would be worth investigating how the arm’s body dynamics, together with the PNS, modelled as a linear and static feedback loop in the arm, embeds the motor patterns of bend propagation according to the initiation command sent by the CNS (Fig. 18.3b, c). This line of experimentation can be investigated in future work.

Fig. 18.3
figure 3

Illustration of the scheme used to embed a motor program for the bend propagation using a physical reservoir computing approach. (a ) Reaching behaviour of the octopus is based on stereotypical motion called bend propagation. The CNS only initiates the behaviour and does not handle the sequence of muscle controls, while the octopus arm together with the PNS controls the movement. (b ) Schematics illustrating how to embed the bend propagation into the modelled arm. Bend propagation is achieved by contracting the springs of the compartments in a specific order from the base to the tip. (c ) Information processing scheme used to embed the bend propagation using a physical reservoir computing approach. By exploiting the body dynamics of the modelled arm as a reservoir, it regulates the timing and strength of the local muscle contractions. The speed and degree of the bend propagation can be controlled by the initiation signal sent from the CNS as an input

Full size image

In this study, we have introduced a series of platforms that exploit body dynamics as a computational resource. The technique can be potentially applied to a wide class of soft robots because the main component required is the soft body itself, which is already present in the soft robotic platform. For example, the model can be used to produce nonlinear limit cycles, which enable locomotion in robots. These systems can be embedded into the body in a closed-loop manner without additional nonlinearity and memory support from an external controller. Recently, this line of study has been initiated with the use of physical platforms (e.g., Caluwaerts et al. 2014; Zhao et al. 2013). Consequently, different types of robot morphology, which increases the computational capacity of the body, should be explored in the future. In particular, recent advancements in biomaterial science suggest that numerous functionalities can be outsourced to the body materials. The way in which different types of materials regulate the computational capacity, from a reservoir computing point of view, is worth exploring. In addition, developments in new types of sensors, which can effectively monitor body dynamics, would make our technique applicable to further applications. Furthermore, with a novel technique that could be used to implement linear readouts as a device, it would be possible to control spatially distant points of the body through the use of body dynamics actuated on more local body parts. This approach would be especially useful when the actuation points are limited by the platform’s physical constraints. To conclude, we believe that our approach will contribute to the use of soft materials in various engineering applications in multiple fields.