1 Introduction

Animal locomotion is generated and controlled, arguably, by central pattern generators (CPGs), which are networks of neurons in the central nervous system (CNS) capable of producing rhythmic outputs, usually as a result of interactions between CNS and external stimulation via a range of bio-sensors [1–5]. Although current neurophysiological techniques have difficulties in clearly isolating such circuits from the intricate neural connections of animals, the indirect experimental evidence for their existence is strong [6–16]. The CPGs have been studied and modelled in details in terms of their biological significance [17–25], stringent mathematical forms [26–34], and different animal species [35–45]. Furthermore, CPGs mechanisms have already been implemented in mixed-signal circuit chips and applied to build autonomous robots [46–60]. In these studies, the locomotion patterns are supposed to be the outputs of musculoskeletal systems driven by CPGs, whose parameters and functions are adjusted by neuro-modulators or tonic inputs [61]. Although the CNS mechanism underlying CPGs is not entirely clear, artificial models have been widely applied to map the possible functional organisation of the CPGs network onto the motor system responsible for driving locomotion.

A motor system is usually modelled by coupled oscillators, which represent the activity of neurophysiologically simplified motor neurons. Different types of oscillators can be chosen and connected, usually with a topological shape to simulate a specific animal species [63–65]. The connection strength in the oscillator network is usually adaptive based on the external inputs or internal, CNS instructions. The nature of the parallel and distributed processing is a prominent characteristic of the circuit that can be canonically described by a group of ordinary differential equations, which usually reflect an autonomous system.

So far many coupled, nonlinear oscillator models have been suggested on CPG mechanisms for vertebrate and invertebrate animals, for instance, the biped [62, 65–67], quadruped [68–71], and hexapod models [74, 75]. Most of them use one or more units of CPGs for generating and switching among gait patterns. For instance, Schöner and colleagues [68] used a synergetic approach to study the quadrupedal locomotion. Synergy deals with cooperative phenomena. In synergy, the macroscopic behaviour of a complex system can be characterised by a small number of collective variables which in turn govern the qualitative behaviour of the system elements [72]. In [68], a network model, which consists of four coupled oscillators, is analysed. Each oscillator represents a limb of a quadruped model. The phase difference among limbs is used as collective variables to characterise the inter-limb coordinating patterns of this discrete system. Gait transitions are simply modelled as phase transitions, which can also be interpreted as bifurcations in a dynamical system. This approach is significant in the sense that it relates the system parameter changes and stability issues to gait transitions. CPG models with the similar synergetic operations have also been proposed and implemented in circuits [53, 54].

Inspired by the synergetic CPGs models, a novel, structural approach to the modelling of the complex, legged locomotion is presented based on a graph dynamics. In this approach, a concept, namely the oscillatory building blocks (OBBs) [73–75], is updated and its generalised version presented. By selecting and building with pre-configured OBBs modules, different gait patterns can be achieved for producing coordinated gait patterns, constructing locomotion prototypes and facilitating the circuit synthesis in an efficient, uniform, and systematic framework. Built upon our previous study on OBB modules, this work generalises it by introducing a mathematical CPGs framework for both gait pattern generation and transition between different patterns. The transitions can be realised by simply changing a few of system parameters. A similar prototype of the proposed OBB modules has already been implemented in an integrated circuit [57, 58], in which one control signal is used to change the gait patterns.

The work is organised in four sections. The following section presents a detailed model description, including the graph dynamics, the generalised concept of OBB modules, their architecture, and biological significance. Section 3 provides a simple computer simulation of gait pattern generation and transition as a case study using the proposed OBB modules, which is followed by a conclusion.

2 Method

This section presents the graph dynamics, namely Scheduling by Edge Reversal (SER) [76, 77], and its generalisation, namely Scheduling by Multiple Edge Reversal (SMER) [78, 79]. The dynamics of the discrete and analogue OBB modules are then introduced. It is shown that the discrete OBB module can be used to simulate the envelope shape of the relaxation oscillation of two coupled neurons in an intuitive way. A more elaborate method to simulate the oscillatory patterns and their transitions is to use analogue OBB modules and networks, as introduced in this section.

2.1 Graph dynamics

The SER graph dynamics considers a neighbourhood-constrained system represented by a set of nodes (or neurons) and by a set of directed edges with one edge for a pair of connected nodes, i.e. a directed graph \(G = (N,E)\) where N is the set of nodes, E is the set of edges defining the inter-connected topology. In SER, each pair of nodes has at most one edge connecting them.

In order to produce a periodic behaviour, the SER dynamics assume that G is acyclic, i.e. that node i cannot be reached by following any directed path starting from itself. The stage transition occurs through the reversal of the direction of all the edges of every node having all of edges directed to it. This node is thus referred to as a sink. After edge reversal every sink node becomes a source node, which has all of its edges directed away from it. The resulting directed graph is also acyclic. The edge reversal operation will continue until a repetition of an acyclic stage is reached. This periodic behaviour can be seen as a dynamic attractor. SER is simple and is fully distributed graph dynamics. A very interesting property of this algorithm is that any acyclic stage on any graph topology will have its own set of possible dynamics, like described in [76, 77]. Figure 1 illustrates a case of the SER dynamics.

Fig. 1
figure 1

Example of the SER graph dynamics. Black nodes indicate the sink (active) nodes. Notice that inside the periodic cycle (Stages 1, 2 and 3), each node becomes a sink exactly once

Full size image

The SMER, Scheduling by Multiple Edge Reversal, is a generalisation of SER where nodes can become sinks more than once inside a periodic cycle. In SMER, any node can have more than one directed edges attached to it (see Fig. 2 for a very simple example of SMER graph dynamics). Also any node is assigned a parameter referred to as its reversibility. Let’s randomly take a node, namely node i, or n i , as an example. The reversibility r i of node n i is defined as the number of the edges directed to n i that, when node n i is a sink, will be released to each of its coupled neighbours at the end of n i sinking by reversing the direction of r i edges to all of its neighbours. Immediately after this reversing operation n i becomes a source. The definition of reversibility also implies that any node, n i , in a SMER dynamics requires at least the number of r i edges directed to it in order to become a sink from a source. In the SER dynamics, all nodes are supposed to have reversibility equal to one. This is because any two neighbours only share one directed edge.

Fig. 2
figure 2

Example of SMER, the generalised graph dynamics. Nodes n i and n j have reversibility values 3 and 1, respectively. Grey colour indicates sink nodes. It is clear that the period of this graph dynamics system has four stages. Node n i becomes a sink exactly once, and node n j becomes a sink three times inside the period

Full size image

In a mathematically formal description, suppose we have two nodes n i and n j , where n i , n j  ∈ N, and they are connected with each other. The reversibilities of nodes n i and n j are r i and \(r_{j}\), respectively. We have the following rules to choose the number of edges and their directions to ensure the correct operation of SMER [78, 79].

  1. 1.

    \(\hbox{max} \{ r_{i} ,r_{j} \} \le e_{ij} \le r_{i} + r_{j} - 1\), where \(e_{ij}\) is the number of edges between node n i and n j ;

  2. 2.

    \(f_{ij} = r_{i} + r_{j} - \gcd (r_{i} ,r_{j} ).\) where gcd stands for greatest common divisor, and \(f_{ij}\) is the sum of the greatest multiples of \(\gcd (r_{i} ,r_{j} )\) that do not exceed the number of edges directed from n i to n j , and from \(n_{j}\) to \(n_{i}\), respectively, at the initial stage of the graph dynamics.

The first rule stipulates a range for the number of the edges between two coupled nodes, while the second further decides the exact number of edges in the range and their directions. Based on the two rules a dynamic attractor can be made with a flexible control of its active patterns and be immune of the system halt due to deadlock or starvation [77, 78]. Figure 2 illustrates an instance of the generalised dynamics.

In the next section, the SMER graph dynamics will be used to build the artificial CPGs by designing and implementing OBB models. It will be shown that an OBB can be designed as either a digital or an analogue circuit depending on different applications. In both types of OBBs composed of pairs of coupled nodes, a node which has possessed required resources is firing, and meanwhile inhibits its coupled counterpart from becoming a sink. In this case the sinks can be seen as firing neurons in an inhibitorily coupled neuronal network.

2.2 Discrete OBB modules and properties

Of long-standing interest are questions about rhythm generation in networks of non-oscillatory neurons, where the driving force is not provided by endogenous pacemaking cells. A simple mechanism for this is based on reciprocal inhibition between neurons, if they exhibit the property of post-inhibitory rebound [26]. The post-inhibitory rebound mechanism [80] is an intrinsic property of many neurons in the central nervous system, which refers to a period of increased neuronal excitability following the cessation of inhibition. It is often included as an element in computational models of neural networks involving mutual inhibition [81, 82]. The mutually exclusive activity between any two nodes coupled under the aforementioned graph dynamics suggests a scheduling scheme that resembles anti-correlated firing activity between inhibitory neurons exhibiting post-inhibitory rebound. The discrete and the analogue versions of the graph dynamics based OBBs can thus be customised for different rhythmic patterns. The discrete OBB modules are built by directly adopting the generalised graph dynamics, and an asymmetric Hopfield-like neuronal network [83–85] is employed for implementing the analogue OBB modules.

Instead of modelling electrophysiological activities of interconnected neurons based on membrane potential functions, an artificial CPGs network with state machine-based OBBs models the collective behaviour of a neuron set. A simple, discrete OBB module is defined to have an \(r_{i}\)-sink node, namely a node with a reversibility of \(r_{i}\), and an \(r_{j}\)-sink node (the same meaning of the \(r_{i}\)-sink), sharing the number of \(e_{ij}\) edges (edges will also be referred to as resources in the subsequent text). Two nodes work as two coupled motor neurons (the nerve cells locate in the spinal cord), respectively, with the shared resources signifying the interactive relationship between the two motor neurons that are interconnected. The exchange of the amount of \(r_{i}\) (or \(r_{j}\)) of resources between two nodes results in pre-defined firing frequencies of two motor neurons.

There is clearly a time relation between the pair of coupled nodes. The larger the reversibility value of a node is, the shorter the firing time of that node will be in a cycle (see the example in Fig. 2). A short firing time of a node means a small duty cycle (a fraction of the active time in the whole period). Therefore, the duty cycle of a node in an OBB module is easily adjustable by setting the reversibility values for the coupled nodes. Meanwhile, the firing phase between two coupled nodes is also configurable by setting an appropriate value for \(f_{ij}\) in the graph dynamics at the start stage.

The state transition of each leg, e.g. from stance to swing, and the corresponding phase relations among different legs are important to simulate the gait model [86]. The phase circulation can be represented by the circulation of the discrete OBB modules. A simple example of how a possible scheme of firing pattern circulation of OBBs modules can simulate the activity envelope of a pair of flexor and extensor motor neurons is shown in Fig. 3.

Fig. 3
figure 3

Mimicking rhythmic patterns generated by CPGs with the SMER dynamics. a Activity of simplified flexor (node n j ) and extensor (node n i ) motor neurons during walking of a cockroach and the graph dynamics simulation of its envelope. Here two directed edges are equivalently replaced by two small, black circles on an undirected link. The attaching of a small circle to a node represents its direction. b The biological signals are reproduced by courtesy of Getting [87]

Full size image

After an OBB module is constructed and configured for a coupled pair of flexor and extensor neurons, the envelope of each neuron with a pre-defined duty cycle is formed. For the binary activity in a digital circuit, only a few of sampling time instants are needed in one leg’s locomotion period representing the firing of the flexor and extensor neurons, respectively. For a more complex locomotion model, it is the individual OBB modules rather than the integrated OBB network that is governed by the generalised SMER graph dynamics. Therefore, the methodology of constructing a complex locomotion model with the discrete OBB module is fairly straightforward and useful for implementing a modular circuit for the asynchronous operations of a gait model.

2.3 Analogue OBB modules and properties

The aforementioned two sorts of graph dynamics have the potential to provide the greatest concurrency among scheduling schemes on resource-sharing systems [76, 79]. The mutually exclusive characteristic between any two coupled nodes makes the scheduling scheme suitably tailored for simulating post-inhibitory rebound, a mechanism widely employed for locomotion and other rhythmic activities [82, 88]. The generalised SMER graph dynamics depends on the initial allocation of shared resources. Different configurations of shared resources lead to different cyclic behaviours, and even to system deadlock or node starvation if shared resources are not allocated properly. This motivates investigation on how to mimic CPGs with the graph dynamics to simulate numerous rhythmic patterns while avoiding undesirable situations.

It is clear that using graph dynamics for simulating different rhythmic patterns is essentially intuitive due largely to the discrete nature of the dynamics. However, analogue behaviours are ubiquitous in the real world. They are continuous rather than discrete in the time domain, better described by analogue circuitry rather than digital one. Following the digital version, we present a novel continuous time OBB structure that is similar to asymmetric Hopfield networks [83, 84] but governed by the graph dynamics. This structure has been classified into two major categories, namely simple OBB and composite OBB, depending on the network complexity. The work also provides a general mathematical framework for describing the dynamic properties of analogue OBB modules.

Like the dynamics of cellular neural networks [89, 90], the input and output voltages of each node in an OBB module are normalised to the digital low or high level, while the internal potential is continuous within the normalised interval [0, 1]. The simple OBB modules consist of two interconnected nodes with pre-specified reversibilities. The composite OBBs can have an arbitrary number of cells interconnected in any topology. Both types of OBB modules operate under the mechanism of the generalised SMER graph dynamics in which the initial resources configuration is important to avoid possible abnormal situations like deadlock or starvation during operation.

2.4 The simple OBB module

Suppose we have a graph represented by G(N,E), where N is a set of nodes and E the set of edges connecting these nodes in the graph G, now consider a subgraph of G(N,E), namely \(G^{ij}\), having a pair of coupled nodes \(n_{i}\) and \(n_{j}\) with \(r_{i}\) and \(r_{j}\) as their reversibility, respectively. This simple network can be translated into a simple OBB module, with the nodes being mapped to the neurons (hence in this work we use nodes or neurons for the same meaning), and the resources being mapped to the synapses and associated weights. The membrane potential of neuron \(n_{i}\) at time t, \(M_{i} (t)\), is supposed to depend on three factors, i.e. the potential at last instant \(M_{i} (t - 1)\), the impact of its coupled neuron \(n_{j}\) output \(v_{j} (t - 1)\), and the negative feedback of neuron \(n_{i}\) itself \(v_{i} (t - 1)\), without considering the external impulses. Figure 4 shows the circuit representation of the module. The selection of system parameters, such as the neuron thresholds and synapse weights, are crucial for modelling the OBB module. In the model, let \(r^{\prime} = h(r)\), where h is a function of getting the highest integer scale in (\(r_{i}\),\(r_{j}\)), and then multiplying it by 10, e.g. if \(r_{i} = 77\) and \(r_{j} = 463\), the highest integer scale is 100 from \(r_{j}\), and then multiplying it by 10 we have 1000.

Fig. 4
figure 4

Diagram of a simple OBB module as the basic oscillation unit

Full size image

So \(h(r) = h(\hbox{max} (77, 463)) = h(463) = 10^{3}\). The neurons’ membrane thresholds \(\theta_{i}\) and \(\theta_{j}\) and their synaptic weights can be designed as follows,

$$\left\{ {\begin{array}{*{20}l} {\theta_{i} = \frac{{r_{i} }}{{f_{ij} }}} \\ {\theta_{j} = 1 - \theta_{i} } \\ {w_{ij} = \frac{{r_{i} }}{{r^{\prime} }}} \\ {w_{ji} = \frac{{r_{j} }}{{r^{\prime} }}} \\ \end{array} } \right.$$
(1)

The system parameters can be arranged by comparing the two nodes’ reversibilities. If \(r_{i} > r_{j}\), then we have \(\theta_{i} > \theta_{j}\) and \(w_{ij} > w_{ji}\) (i.e. asymmetric coupling), that means a node with smaller reversibility, corresponding to a neuron with lower threshold in an OBB module, will oscillate at a higher frequency than its companion does. It will also weakly stimulate its coupled neuron by contributing with a smaller weight change. \(f_{ij}\) has the same meaning as in the SMER algorithm, and we have \(\theta_{j} = \frac{{f_{ij} - r_{i} }}{{f_{ij} }} = 1 - \frac{{r_{i} }}{{f_{ij} }} = 1 - \theta_{i}\). This arrangement of model parameters ensures that the behaviour of the SMER-based, analogue OBB modules is isomorphic with the behaviour of the original SMER algorithm. The difference equation in the discrete time domain of this system can be formulated as follows: Each neuron’s self-feedback strength is \(w_{ii} = - w_{ij}\), \(w_{jj} = - w_{ji}\), respectively, and the activation function, which reflects the neuron’s output in relation of its model parameters, is a sigmoidal Heaviside type. Thus ,we have,

$$\left[ {\begin{array}{*{20}l} {M_{i} (t)} \\ {M_{j} (t)} \\ \end{array} } \right] =\left[ {\begin{array}{*{20}l} {M_{i} (t - 1)} \\ {M_{j} (t - 1)} \\ \end{array} } \right]+ \underbrace{\frac{1}{r^{\prime}} \times\left[ {\begin{array}{*{20}l} { - r_{i} } &{r_{j} } \\ {r_{i} } & { - r_{j} } \\ \end{array} } \right]_{W}} \times\left[ {\begin{array}{*{20}l} {v_{i} (t - 1)} \\ {v_{j} (t - 1)} \\ \end{array} } \right]$$
(2)

where W is the weight matrix. We have the outputs of neurons as,

$$\left\{ \begin{array}{*{20}l} v_{i} (t) =\max (0,{\text{sgn}} (M_{i} (t) - \theta_{i} )) \\v_{j} (t) = \max (0,{\text{sgn}} (M_{j} (t) - \theta_{j} )) \\\end{array}\right.$$
(3)

The designed circuit can be considered as a conservative dynamical system in an ideal case, i.e. the total energy is constant. The sum of two neurons’ membrane potentials at any given time is normalised to one. It will be shown that the system is able to develop from an arbitrary initial state into a limit cycle (for an oscillatory period), with the firing rate of each neuron adjustable based on its reversibility. However, like most dynamic systems, this model has a limitation in its dynamic range. There exists a singular point when each neuron’s membrane potential equals to its threshold. In this case, Eq. (3) becomes \(v_{i} (n) = v_{j} (n) = 0\), hence Eq. (2) accordingly becomes: \(M_{i} (n) = M_{i} (n - 1)\) and \(M_{j} (n) = M_{j} (n - 1)\). The system behaviour can be unpredictable: It may transit to a different oscillatory pattern due to small external perturbations, or halt with no perturbations. When designing an oscillatory neural network using OBB modules, it is possible to avoid the occurrence of any singular point state by presetting the initial membrane potential \(M_{i} (0)\) to an appropriate value so that \(M_{i} (0) \ne \theta_{i}\). Within the normal dynamic range except for the singular point, the circuit of the OBB module represents a starvation- and deadlock-free dynamic system, and it is a stable and periodically oscillating system no matter what initial potentials its neurons may have [75].

The combination of the duty cycles, the oscillation frequencies, and the phase difference of a coupled pair of neurons is the key set of parameters for modelling a joint of one degree of freedom (DOF). The oscillatory pattern transition, which is another important concept in addition to the pattern generation, can thus be understood as a transition from an old to a new set of the joint parameters. It is clear that the duty cycle of an extensor motor neuron plays an important role in deciding the locomotion speed of a legged animal [91–93]. In our model, the duty cycle of a neuron in a coupled, two-neuron system is dependent on the model parameters shown in Eq. (1). The choice of reversibilities of two coupled nodes thus dictates the transition between different patterns as it decides the system parameters, and hence the duty cycles. Therefore, in our study the design of transition between patterns is simplified to the selection among different reversibility values of two coupled nodes. On the other hand, the proposed pattern generation model has a linear internal dynamics, i.e. the membrane potential is a linear function of three input variables, and a nonlinear, Heaviside-type dynamics in the output of a neuron as a function of the membrane potential and the threshold of that neuron.

Suppose both coupled nodes have their reversibility changed in an amount of \(r_{i}^{d}\) and \(r_{j}^{d}\), respectively, we can re-write formula (1) to obtain the model parameters for the new pattern, as shown below.

$$\left\{ {\begin{array}{*{20}c} {\theta_{i}^{\rm new} = \frac{{r_{i} + r_{i}^{d} }}{{f_{ij}^{\rm new} }}} \\ {\theta_{j}^{\rm new} = \frac{{r_{j} + r_{j}^{d} }}{{f_{ij}^{\rm new} }}} \\ {w_{ij}^{\rm new} = \frac{{r_{i} + r_{i}^{d} }}{{r^{\prime} }}} \\ {w_{ji}^{\rm new} = \frac{{r_{j} + r_{j}^{d} }}{{r^{\prime} }}} \\ \end{array} } \right.$$
(4)

where \(f_{ij}^{\text{new}} = r_{i} + r_{i}^{d} + r_{j} + r_{j}^{d} - \gcd \left( {r_{i} + r_{i}^{d} ,r_{j} + r_{j}^{d} } \right)\). Here we make a reasonable assumption that r′ is unchanged as the amount of \(r_{i}^{d}\) or \(r_{j}^{d}\) is in the similar scale of its old value, respectively. This ensures that the originally linear membrane potential dynamics of a neuron is still linear after pattern transition. We further assume that, according to SMER, a firing neuron will releases some shared resources in the amount equal to its new reversibility to its coupled neuron, which receives them passively and will possibly become able to fire in one of the subsequent stages. The firing neuron in the dynamic model will thus decrease in its membrane potential by an amount equal to the new strength of its output synapse, while the coupled neuron will have the same amount added to its membrane potential. We can rewrite Eq. (2) for the membrane dynamics in a more general format involving pattern transition mechanism as follows.

$$\begin{aligned} &\left[ {\begin{array}{*{20}c} {M_{i} (n)} \\ {M_{j} (n)} \\ \end{array} } \right] \\ & \quad = \left[ {\begin{array}{*{20}c} {M_{i} (n - 1)} \\ {M_{j} (n - 1)} \\ \end{array} } \right] + \left( {W + \underbrace {{\frac{{s_{i} }}{r'}\left[ {\begin{array}{*{20}c} { - r_{i}^{d} } & 0 \\ {r_{i}^{d} } & 0 \\ \end{array} } \right] + \frac{{s_{j} }}{r'}\left[ {\begin{array}{*{20}c} 0 & {r_{j}^{d} } \\ 0 & { - r_{j}^{d} } \\ \end{array} } \right]}}_{\varDelta W}} \right) \\ & \quad \quad\times \left[ {\begin{array}{*{20}c} {v_{i} (n - 1)} \\ {v_{j} (n - 1)} \\ \end{array} } \right] \end{aligned}$$
(5)

where,

$$s_{k} = \left\{ \begin{array}{*{20}l} 1, &\quad {\text{if}} \,{r_{k} }\,\, {\text{changed}}\\0, &\quad {\text{if}} \,\, r_{k} \,\,{\text{not}}\,\,{\text{changed}}\\\end{array}\right.$$
(6)

is a control signal for pattern transition, here \(k \in (i,j)\). This equation indicates that, in theory, the pattern transition can be induced by the reversibility change of any one of the two coupled neurons.

2.5 The composite OBB module

A composite OBB module is a generalisation of the simple OBB module which consists of an arbitrary number of simple OBB modules. A composite OBB can have more than two neurons and therefore, a more complex topology. Suppose a graph M(N,E) containing a set of m neurons in a set \(N = \{ n_{1} ,n_{2} , \ldots ,n_{m} \}\), and a set of connections \(E = \left\langle {0} \mathrel{\left | {\vphantom {0 1}} \right. \kern-0pt} {1} \right\rangle_{ij}\), where \(\forall i,j \in N\), which defines the connection topology of this graph by using \(\left\langle 1 \right\rangle_{ij}\) and \(\left\langle 0 \right\rangle_{ij}\) to indicate the presence or absence of a connection between nodes i and j, respectively. Node i has its reversibility \(r_{i}\). There are \(e_{ij}\) shared resources on the corresponding connection \(\left\langle 1 \right\rangle_{ij}\), with their number and configuration stipulated by the rules set in the SMER section.

Different from a simple OBB module composed by exactly two coupled neurons, a composite OBB module has at least one neuron connected to at least two other neurons. Figure 5 shows that a neuron i has connections to a total number of N neurons. The composite OBB module can be dissected into various simple OBB modules. For instance, in Fig. 5, neuron i can be split into N − 1 copies, which share the same local clock of their maternal neuron i, to connect to the N input neurons. Here, a copy of neuron i, plus the corresponding input neuron and their connection, forms a simple OBB module. We terminologically regard the output neuron (like neuron i in Fig. 5) in a composite OBB module as a macroneuron and macroneuron’s copies as its clones.

Fig. 5
figure 5

Diagram of one macroneuron (represented by the right triangle) and its clones (represented by a column of the left triangles) in a composite OBB module

Full size image

According to SMER, a macroneuron of the composite OBB module operates in a “whole-or-none” mechanism in terms of the activity of its clones. The macroneuron is active if and only if all of its clones are active. Fundamentally, a composite OBB module operates based on its constituent simple OBB modules. A schematic diagram of an exemplar composite OBB module and its equivalent representation is shown in Fig. 6.

Fig. 6
figure 6

Two equivalent architectures to illustrate the SMER algorithm in an exemplar composite OBB. Here the reversibility values \(r_{i} = r_{m} = r_{n} = 1\); \(r_{j}^{{}} = 2\); \(r_{k} = 3\). a One operation stage of SMER in a composite OBB module, the node in red is active, those in black are inactive, the small circles represent resources. b An alternative, equivalent description for the stage of the composite OBB module with the macroneurons and their clones, the pink node is active with all of its clones being active; the grey nodes are inactive with at least one of its clones being inactive. It is clear that the independent operations of eight simple OBB modules (represented by eight connections and their associated clones, and resources) decide the operation of this composite OBB module (colour figure online)

Full size image

As a generalised version, a composite OBB module can represent a more complex oscillatory neuronal network and flexibly reproduce more complex cycles of rhythmic patterns than a simple OBB module. A composite OBB module is formed based on any prescribed oscillatory functionality in a specific application, so it does not have a fixed form in terms of the number of constituent clones, the connection structure, etc. Different applications will specify different macroneurons and their topology, and subsequently use them as the oscillatory building blocks. Due to the fact that a composite OBB module is dependent on specific applications and impossible to know beforehand, a rule of thumb to analyse this kind of modules is to dissect them into the subsystems of simple OBB modules where Eqs. (1)–(6) apply.

Since the output of a macroneuron i is determined by all its clones, we have,

$$V_{i} (k) = \prod\limits_{j = 1}^{n} {v_{i}^{j} } (k)$$
(7)

where \(\forall i \in N\), \(i \ne j\) and \(v_{i}^{j}\) is the output of clone j of the macroneuron i, which is linked to a corresponding clone of another macroneuron j via the connection \(\left\langle 1 \right\rangle_{ij}\). The superscript sequence \(j = 1,2, \ldots ,n\) is the clone number of a macroneuron \(n_{i}\), which also represents for the other macroneurons to which macroneuron i connects.

It is also noticeable about how to choose the initial membrane potential values for the clones which are parts of the simple OBB modules. Different choices will lead to different initial self-organisation process of a composite OBB module. To avoid system halt, no clone should be inactive if its macroneuron is designed to be active by Eq. (7). Within an appropriate parameter range (i.e. a neuron’s membrane potential is not equal to its threshold), a random selection of initial membrane potential values is allowed. After a possible initial duration whose length is determined by the choice of initial membrane potentials, the system will oscillate periodically.

3 Primer of computer simulation

In this section, the operations of the OBB modules are preliminarily demonstrated through several computer simulated works. Since a composite OBB module is fundamentally composed by some simple OBB modules (see Fig. 6), it will have the collective behaviours of the simple OBB modules. Therefore, the experiments focus on the operations of the simple OBB modules by showing some case studies of their dynamics in terms of the oscillatory patterns generation and transition.

3.1 Pattern generation

Let us suppose that a simple OBB module has two coupled neurons i and j with the reversibility values \(r_{i} = 3\) and \(r_{j} = 12\). According to the theory, we have the module parameters as follows.

$$\left\{ {\begin{array}{*{20}c} {\theta_{i} = 0.25} \\ {\theta_{j} = 0.75} \\ {w_{ij} = 0.03} \\ {w_{ji} = 0.12} \\ \end{array} } \right.$$

If we further assume that the initial membrane potentials of two neurons are \(M_{i} (0) = 0.66\) and \(M_{j} (0) = 0.34\), respectively, we can obtain the oscillatory dynamics of this simple OBB system by using MATLAB–Simulink, as shown in Fig. 7.

Fig. 7
figure 7

Simulated dynamics of a simple OBB module in time domain. Upper panel: neuron i output; Lower panel: neuron j output. When the system becomes stable, the oscillatory period is 15 s and the duty cycle of neuron i is 12 s. Note that here (and in the subsequent simulations) second is used as the nominal time unit for illustrative purpose only. In practice the time unit is closely related to the units of the system clock, and should be conditioned to any suitable unit size

Full size image

It is noticeable that the coupled neurons start with a self-organised period with the given initial membrane potentials. Then the system undergoes a stable periodic oscillation. The duty cycle of a neuron is decided by the model parameters, and thus indirectly related with the reversibilities of two coupled neurons. The state space plot of this example is shown in Fig. 8.

Fig. 8
figure 8

State space plot of the periodic oscillation of two coupled neurons under the graph dynamics. The two axes on the planar surface represent the membrane potentials of two neurons, and the vertical axis is for the firing state of neuron j. From different initial membrane potentials, the model evolves into a same sequence of periodical states like a limit cycle

Full size image

Given the initial membrane potentials of two coupled neurons in the model as \(M_{i} (0) = 0.66\), \(M_{j} (0) = 0.34\), the state of the model develops through an initial stage into a limit cycle (as shown in Fig. 8 in thick and grey line), which corresponds to a stable periodic oscillation in the time domain (as shown in Fig. 7). Another example of choice of the initial membrane potentials of the coupled neurons, e.g. \(M_{i} (0) = 0.12\), \(M_{j} (0) = 0.88\), results in a shorter journey before converging to the limit cycle (see Fig. 8).

It is clear that neuron j is active for a much shorter time duration than neuron i, which becomes active in the rest of the time when neuron j is inactive. The duty cycle of neuron i is calculated as \(\frac{4}{5}\) for a period of 15 s (Fig. 7). If we fix the reversibility value \(r_{i} = 3\), and let \(r_{j} \in\) [1, 3, 6, 9, 12, 15], we can get the duty cycle of neuron i in a corresponding array as \(\left[ {\begin{array}{*{20}l} {\frac{1}{4},} & {\frac{1}{2},} & {\frac{2}{3},} & {\frac{3}{4},} & {\frac{4}{5},} & {\frac{5}{6}} \\ \end{array} } \right],\) and the period of oscillation in an array [6, 9, 12, 15, 18]. Without loss of generality we can use frequency-conditioning techniques to round the period of oscillation of the OBB module to be as close as possible to a same value (here 12), while keeping the duty cycle of two neurons unchanged so that, in applications, we can compare different patterns generated with a same baseline. A new array of oscillation periods becomes [10, 12]. A plot of the duty cycle of neuron i and the period of oscillation of the model is shown in Fig. 9. Following Pearson’s argument [91] that a faster gait pattern corresponds to a larger duty cycle for swing phase and therefore a smaller duty cycle for stance phase, it is possible to vary the locomotion speed by adjusting the reversibilities of two coupled neurons (corresponding to the swing and stance phases, respectively) in a graph dynamics driven system. When the duty cycles of two coupled neurons are changed by using different reversibilities, the model period remains largely unchanged. This phenomenon displayed by the model is consistent with that of our previous study in implementing a nonlinear stepping pattern generation model with analogue circuitry [58], and therefore the current model provides an alternative approach to modelling the CPG mechanism amenable to circuit realisation, due largely to its characteristics of modularity and scalability. The OBB circuit is event driven and made of the discrete building blocks. These features facilitate its implementation in asynchronous sequential logic circuits.

Fig. 9
figure 9

Duty cycle and oscillation period of a simple OBB module with a fixed reversibility \(r_{i} = 3\), and variable reversibility \(r_{j}\) as shown in the text. The pattern order corresponds to the normalised oscillation periods [10, 12]

Full size image

The continuous-valued version of the SMER algorithm can be used to design a building block for generating rhythmic patterns with different duty cycle values for the activity of a coupled neurons in a flexible and systematic approach. For instance, if one more duty cycle point is needed between the values of 1/4 and 1/2, say 2/5, a composite OBB module can be designed for this additional state with the new duty cycle (see Fig. 10).

Fig. 10
figure 10

Exemplar composite OBB module containing 3 macroneurons to achieve a new duty cycle for a macroneuron k. The rest two macroneurons are used to support the design. The red colour represents for active clones while the yellow colour for active macroneurons. The shared resources are small blue circles (colour figure online)

Full size image

In this design, a composite OBB has three macroneurons labelled as k, m, and n. Each macroneuron has two clones inside. It is clear that this composite OBB module is composed of three simple OBB modules, which are clones and resources connecting macroneurons k and m, k and n, m and n, respectively. The clones inside macroneuron k are referred to as \(C_{km}^{k}\) and \(C_{kn}^{k}\), in which the subscripts represent the connections between different neuron pairs of k–m and k–n, the superscripts denote the clones are in macroneuron k. The reversibility variables of these two clones are labelled and set as \(r_{km}^{k} = 1\) and \(r_{kn}^{k} = 2\), respectively. Similarly, the clones in macroneuron m are \(C_{mk}^{m}\) and \(C_{mn}^{m}\), their reversibility variables are \(r_{mk}^{m} = 4\) and \(r_{mn}^{m} = 1\). The clones in macroneuron n are \(C_{nk}^{n}\) and \(C_{nm}^{n}\), their reversibility variables are \(r_{nk}^{n} = 3\) and \(r_{nm}^{n} = 1\). The model parameters of the simple OBB modules connecting macroneuron k, m, and n, respectively, can be derived as follows.

$$\left\{ {\begin{array}{*{20}l} {\theta_{km}^{k} = 0.25} \hfill \\ {\theta_{km}^{m} = 0.75} \hfill \\ {w_{km}^{k} = 0.1} \hfill \\ {w_{km}^{m} = 0.4} \hfill \\ \end{array} \quad } \right.\left\{ {\begin{array}{*{20}l} {\theta_{kn}^{k} = 0.5} \hfill \\ {\theta_{kn}^{n} = 0.5} \hfill \\ {w_{kn}^{k} = 0.2} \hfill \\ {w_{kn}^{n} = 0.3} \hfill \\ \end{array} } \right.\quad \left\{ {\begin{array}{*{20}l} {\theta_{nm}^{n} = 0.5} \hfill \\ {\theta_{nm}^{m} = 0.5} \hfill \\ {w_{nm}^{n} = 0.1} \hfill \\ {w_{nm}^{m} = 0.1} \hfill \\ \end{array} } \right.$$

After selecting the model parameters by computing OBB parameters based on SMER, the initial membrane potentials for the coupled clones in the simple OBB modules can be chosen randomly with the range [0, 1], as it is aforementioned that the random selection of the initial membrane potentials will possibly lead to an initial self-organisation stage whose length is dependent on the initial potential values. As long as the singular values of the membrane potentials are avoided, the OBB will reach a limit cycle. In this example, macroneuron k is expected to fire for two consecutive stages in the five-stage cycle for obtaining the pre-defined duty cycle. According to Eq. (7), the states of its two clones will be multiplied to have the target output. Here the initial membrane potentials of the coupled neurons in three simple OBB modules can be set as: \(M_{km}^{k} (0) = 0.5\), \(M_{km}^{m} (0) = 0.5\), \(M_{kn}^{k} (0) = 0.75\), \(M_{kn}^{n} (0) = 0.25\), \(M_{nm}^{n} (0) = 0.75\), \(M_{nm}^{m} (0) = 0.25\). The simulation outcome of this continuous-valued composite OBB module using MATLAB–Simulink is shown in Fig. 11.

Fig. 11
figure 11

Simulation of a composite OBB designed to show a firing duty cycle of 0.4 for a macroneuron. (A1) and (A2) show the output states of two clones of macroneuron k, respectively. (A3) is obtained through Eq. (7), the output of the macroneuron k, which meets the design requirement in duty cycle

Full size image

3.2 Pattern transition

As we have shown in Eq. (5), a change in the reversibility of any one of two coupled neurons will result in the change of model parameters, hence the change of oscillatory patterns. Therefore, the pattern transition in the OBB module-based CPG architecture is straightforward. In a Simulink simulation, one is able to use one time-related control signal, corresponding to the control variable in Eq. (6), to switch between the old and the new model parameters, derived from the old and the new reversibilities of coupled neurons. For instance, if one needs to change the reversibility of a pair of coupled neurons from \(\{ r_{i} = 3,r_{j} = 12\}\) to \(\{ r_{i} = 3,r_{j} = 3\}\), the dynamic model parameters are changed accordingly, like a switch being used to control this change. A transition between the old and the new patterns can be achieved with some possible intermediate self-organisation period (see Fig. 12).

$$\left\{ {\begin{array}{*{20}l} {\theta_{i} = 0.25} \\ {\theta_{j} = 0.75} \\ {w_{ij} = 0.03} \\ {w_{ji} = 0.12} \\ \end{array} } \right. \Rightarrow \left\{ {\begin{array}{*{20}l} {\theta_{i} = 1} \\ {\theta_{j} = 0} \\ {w_{ij} = 0.3} \\ {w_{ji} = 0.3} \\ \end{array} } \right.$$
Fig. 12
figure 12

Oscillatory pattern transition process. Transition occurs at the time instant 40; before that reversibilities are \(\{ r_{i} = 3,r_{j} = 12\}\); afterwards \(\{ r_{i} = 3,r_{j} = 3\} .\)

Full size image

It is clear that, if no transition happens, then neuron i will continue its first pattern, which becomes high at time instant 39 and lasts for 12 s till 51. The duty cycle for neuron i is 0.8 (and for neuron j is 0.2 accordingly). As a command for pattern transition occurs at time instant 40, ideally the new pattern should start immediately after this time instant. Practically a self-organisation cycle exists such that the new pattern starts at the time instant 51. This is because membrane potentials of the two coupled neurons are not ready (or, not as close as possible to their thresholds due to the operation of the old pattern) to make the transition happen immediately. After a short period, though, the model will evolve into the new designed pattern with the duty cycle of neuron i as 0.5 (neuron j as 0.5). It is suggested that this phenomenon be biologically plausible as no real creatures will act immediately, i.e. without latency, upon a command of action.

4 Discussion and conclusion

Although most CPG models proposed so far are mainly imaginary architectures with no clear neurophysiological proofs, advances on neuroscience start to reveal the existence of this widely accepted mechanism behind animal locomotion. Recently Hückesfeld et al. [94] identified the distinct clusters of motor neurons responsible for certain types of motions, and the underlying CPG mechanisms. On the other hand, the recent applications of CPG models for controlling robots are still largely following the tradition of using dynamical system methods to formulate the periodic oscillations represented by, e.g. limit cycles (for a review see [95]). Some new trends, however, are to incorporate the interactions of CPGs and the external world by using feedback signals from sensors, or to drive different joints (degrees of freedom) on animal legs rather than the legendary, more abstract phase relations between legs by ignoring the control of intra-leg joints [96–98]. Furthermore, apart from the more classic CPG models such as the van der Pol and Wilson–Cowan oscillators, new oscillators, e.g. the Morphed oscillator [99] which exhibits arbitrary limit cycle shapes and can represent first, second or nth order dynamical systems, are proposed and used to build a CPG model with the ameliorated performance like perturbation resistance [100]. Despite the mathematical soundness of these works, however, most of their effects on controlling legged robots are demonstrated by using computer simulations. Much more works remain if these models are to be implemented in hardware for autonomous control of a robot.

Recent neuromorphic hardware realisation of the CPG models has shown great potential in implementing silicon neurons and their networks to control the robot behaviours [52–60]. Nevertheless, many of the recent works are still focused on the basis of building the neural oscillators, showing their characteristics, rather than using the oscillators to construct a specific CPG architecture [101, 102]. This is possibly due to the lack of a modular and scalable approach to designing a CPG architecture by using a range of the available building blocks.

A novel oscillatory building block (OBB) model that is able to be configured to create a tailor designed architecture for both locomotion rhythmic pattern generation and transition has been proposed in this work. Since the simple OBB module is the basis from which a rhythm-producing model of high complexity can be designed, the top-level model is highly modular and scalable for design, prototype, manufacture and test. It is also an asynchronous and self-clocked system if the reversibility values and initial membrane potentials are chosen for individual simple OBBs. Because of the simplicity of the system, a hardware version of a simple OBB module can be made in such a way that, it is possible to develop a real-time hardware implementation of systems with arbitrary complexity.

The resulting continuous, linear and time-invariant system has its correctness based on the SMER graph dynamics. Therefore, a smooth operation of the designed workflow of rhythmic patterns is ensured. An intermediate self-organisation period may exist but this phenomenon is not an issue, being compatible with its biological counterparts. Pattern transition is controlled by just one model parameter, i.e. the reversibility of a neuron, in order to update the gait pattern according to the environment. Future works include the design of OBB architectures for a pre-specified, legged animal species, in order to show the model’s capability on mimicking a variety of legged locomotion patterns in a flexible and systematic approach.