A neuromechanical model exploring the role of the common inhibitor motor neuron in insect locomotion

Abstract

In this work, we analyze a simplified, dynamical, closed-loop, neuromechanical simulation of insect joint control. We are specifically interested in two elements: (1) how slow muscle fibers may serve as temporal integrators of sensory feedback and (2) the role of common inhibitory (CI) motor neurons in resetting this integration when the commanded position changes, particularly during steady-state walking. Despite the simplicity of the model, we show that slow muscle fibers increase the accuracy of limb positioning, even for motions much shorter than the relaxation time of the fiber; this increase in accuracy is due to the slow dynamics of the fibers; the CI motor neuron plays a critical role in accelerating muscle relaxation when the limb moves to a new position; as in the animal, this architecture enables the control of the stance phase speed, independent of swing phase amplitude or duration, by changing the gain of sensory feedback to the stance phase muscles. We discuss how this relates to other models, and how it could be applied to robotic control.

1 Introduction

Across animal species, the scale of bodies spans fourteen orders of magnitude, but the scale of cells spans only two (Wolf 2014). Species at the small end of this spectrum, such as arthropods, are consequently limited to neuromechanical control strategies which make use of a far smaller number of nerve and muscle cells. Arthropods such as insects have nonetheless evolved the ability to perform complex sensorimotor tasks such as walking with a robustness and agility that matches or arguably even exceeds that of their larger counterparts. As such, the mechanisms that give rise to insects’ adaptive locomotion (Dürr et al. 2018; Bidaye et al. 2017) and how these mechanisms could be applied to robotics (Buschmann et al. 2015) have been the focus of many studies.

Complementary to experimental investigation of the arthropod nervous and biomechanical systems is neuromechanical simulation (Ayali et al. 2015; Daun-Gruhn and Büschges 2011; Schilling et al. 2013; Szczecinski et al. 2014; Toth et al. 2013b; Zakotnik et al. 2006). Neuromechanical simulations of insect locomotion are helpful because they enable scientists to construct a framework within which to understand experimental data, and to test hypotheses that may be difficult to test experimentally. For practical reasons, the scope of a mathematical model of an animal is typically restricted to focus on a specific question or mechanism, whereas such restrictions can lead to simplifications that facilitate analysis, the applicability of the model to reality may be in question.

A feature of arthropod motor control that is often ignored is the passive forces from the elasticity of the legs’ joint membranes, which resist muscle contraction. Because of the small size of arthropods, particularly of insects, their locomotion is dominated by elastic forces, rather than gravitational and inertial forces, as for larger animals like humans (Hooper et al. 2009; Hooper 2012). As an animal becomes smaller, its mass decreases proportional to the length cubed (that is, a density about equal to water, times volume). However, the stiffness of elastic materials decreases proportional to their cross-sectional area, that is, length squared. Thus, an animal as small as a fruit fly, which is on the order of $\approx 0.1 \%$ the scale of a human, has elastic forces that are 1000 times as large as a human’s, relative to mass. This detail is often omitted from neuromechanical models of insect limbs (Szczecinski et al. 2014; Toth et al. 2013b), but models that do include such forces show that they play a large role in filtering motor neuron activity and its effect on limb velocity and acceleration (Zakotnik et al. 2006). For instance, even large insects such as stick insects can maintain the same posture, no matter their orientation, without actively contracting their muscles (Hooper et al. 2009). Relaxing the muscles in the leg after activity will cause the limb to passively return to some neutral position without further muscle activity (Ache and Matheson 2013). Even when the animal’s muscles are completely removed, the elastic forces from the exoskeleton passively return the limbs to their resting positions (Hooper et al. 2009). Such observations show that insects cannot rely on momentum-based control of motion, the way an animal as large as a human might, highlighting how different of a regime in which insect motor systems operate.

To increase the accuracy of modeled dynamics, our group’s recent neuromechanical model of a walking cockroach included passive viscoelastic forces in the leg joints (Rubeo et al. 2017). These passive forces resisted joint rotation away from some rest angle, ultimately impeding locomotion. To compensate for the extra forces, the motor systems for each muscle included an integration circuit (Szczecinski et al. 2017b), which enabled the neural control system to move the limbs to the intended positions (i.e., the posterior extreme position at the end of the stance phase). Using the integral of a feedback signal to counteract applied forces and drive a system to its intended state is a common approach in control theory (Khalil 2002); however, no such integration network has been identified in the insect thoracic ganglia. Therefore, we tried to find mechanisms which are known to exist in the arthropod sensorimotor system that may perform the function of an integral controller.

Our hypothesis is that this function is performed by the muscle membranes of slow muscle fibers. Like those of vertebrates, arthropod muscle fibers exhibit a range of contractile properties. Motor units consisting of predominantly slow muscle fibers are used for repeated tasks requiring low metabolic cost, whereas fast motor units are reserved for situations requiring brief bursts of power (Wolf 2014). However, arthropod slow muscle fibers may be so slow that they effectively convert motor neuron (MN) bursting into tonic forces (Morris and Hooper 1997a). This results in muscle fibers for which the tension is a function of the number of MN spikes (i.e., the integration of spikes over time), rather than the input firing frequency, at least as stimulation begins (Hooper et al. 2007; Morris and Hooper 1997b). For an animal like the cockroach, whose stepping period generally lasts between 100 and 500 ms (Pearson 1972; Watson and Ritzmann 1998) but whose slow muscle fibers take up to 1000 ms to relax (Iles and Pearson 1971), the slow muscle membrane is effectively always in this regime. However, as we will show, this mechanism alone is not sufficient to construct a functional model of arthropod joint control.

For effective joint control, this integration must be reset periodically, otherwise the antagonistic muscles will continue to accumulate tension, eventually causing motion to cease (Cruse 2002; Wolf 1990). Such resetting is a common technique in robot control, wherein the “integral” feedback must be reset when a new posture is commanded to the robot. Many studies have suggested that inhibitory MNs in the arthropod thoracic ganglia play this role. Arthropods possess various “specific” and “common” inhibitory motor neurons, but insects tend to possess only “common” inhibitory motor neurons (CIMNs) which innervate muscles throughout the entire leg (Pearson and Iles 1973; Wolf 2014). In locusts (Usherwood and Runion 1970; Hoyle and Burrows 1973; Burns and Usherwood 1979) and crabs (Ballantyne and Rathmayer 1981), the CIMN that innervates the femur–tibia extensor muscle only fires action potentials at the end of stance phase, relaxing the muscle for the transition to swing phase. These action potentials depend on feedback from sensilla in the tarsus, implying that the CIMN resets the muscles’ membrane voltages before a new posture (i.e., the anterior extreme position, AEP) is commanded to the leg (Usherwood and Runion 1970). If the CIMN is prevented from firing action potentials, tension accumulates and alternating motion stops altogether (Wolf 1990). Slow muscle fibers, plus the presence of CIMNs in slow muscle fibers (Rathmayer and Erxleben 1983), suggest a mechanism by which the cockroach can walk at frequencies higher than the dynamics of slow muscle fibers alone would allow, without recruiting faster relaxing muscle fibers (Pearson 1972; Pearson and Iles 1973; Watson and Ritzmann 1998): Over the short duration of the stance phase, the slow muscle fibers serve as an integrator of excitatory signals, driving the leg rearward to the posterior extreme position (PEP), in spite of large passive forces in the joints; then, at the end of stance phase, the CIMN actively relaxes the muscles in the leg to reset them for a rapid swing phase.

Fig. 1

Inhibitory and excitatory connections indicate the assumption of a direct synaptic connection, whereas influences represent interactions which do not. Gains highlight the conductances significant to this paper. a The system contains a CPG circuit which oscillates between stance (ST) and swing (SW) phases, governing the behavior of a pair of EMNs: the extensor (E) and the flexor (F), which innervate their respective muscles via excitatory synapses. A CIMN (CI) which innervates both muscle membranes provides an inhibitory output pulse during stance to swing transitions. Proprioceptive feedback modulates both the magnitude and timing of MN activity. Some elements of this system are modeled by abstractions which do not directly align with a single biological analog. The scope of each abstraction is outlined (magnitude control, timing control). b A muscle’s membrane potential governs the activation of its contractile element, which combined with passive muscle elements determines a muscle’s tension. The tension of the two muscles along with the joint’s passive elastic and viscous forces sum to impose a net moment on the joint

In this study, we explore the implications of controlling relatively rapid motions with exclusively slow muscle fibers in a closed-loop, neuromechanical model of an arthropod leg joint. We describe our model and demonstrate how slow muscle fibers may act as integrator units. We show how the CIMN may reset such integration by accelerating the relaxation of slow muscle fibers in a controlled manner. We show that including a CIMN reduces the work done by the muscles to execute the same motion. Then, we couple this neuromechanical model with a simple central pattern generator (CPG) and show that if the leg joint’s motion can affect the CPG phase, then the speed and duty cycle of the leg can be controlled in an animal-like, asymmetrical way by changing the gain of one sensory feedback pathway, as observed in walking stick insects (Gabriel and Büschges 2007). Finally, we show that this approach is robust to parameter values by applying the same system to multiple leg joints from our group’s previously-developed cockroach model (Rubeo et al. 2017).

2 Models and methods

2.1 System overview

This paper presents a neuromuscular model of a cockroach femoral-tibial (FTi) joint actuated by an antagonistic pair of slow muscle fibers, each of which is innervated by its own excitatory motor neuron (EMN) and a common inhibitor motorneuron (CIMN). The behavior of the motorneurons is governed by the combination of a central pattern generator (CPG), proprioceptive feedback, and a driving input from the central nervous system (CNS). An overview of the system is shown in Fig. 1, and the modeling of each element or its equivalent abstraction is discussed in greater detail in the following sections. The parameter values used are presented and discussed in “Appendix A”.

The system has 7 dynamical state variables from which all other quantities can be derived: joint angle $\theta$; angular velocity $\dot{\theta }$; commanded joint angle $\theta _\mathrm{ref}$; flexor and extensor muscle membrane potentials $U_\mathrm{fl}$ and $U_\mathrm{ex}$, respectively; and flexor and extensor muscle tensions $T_\mathrm{fl}$ and $T_\mathrm{ex}$, respectively. A step cycle was defined to consist of one swing phase ($\theta _\mathrm{ref} > 0$) and one stance phase ($\theta _\mathrm{ref} < 0$). The system was considered to have attained steady-state behavior whenever the values of all state variables as well as the cycle period matched within tolerances at the end of consecutive step cycles.

The system simulation was implemented using forward Euler integration with a step size of $10^{-2}$ ms, and simulations were run until a steady-state cycle was reached. We chose to restrict our analysis to steady-state cycles, as any transient behaviors such as acceleration from rest would likely make use of neuromuscular elements not present in our model. For example, additional fast and intermediate motor units could play a role, or the CPG could have a specialized acceleration sequence (Iles and Pearson 1971; Toth et al. 2013a).

2.2 Joint model

The FTi joint model is shown in Fig. 2a. The tibia is modeled as a thin rigid rod of mass m, length l, and uniform density. Because the independent dynamics of the tarsus are not relevant to this study, it is not modeled separately from the tibia.

Fig. 2

a The joint model, where the joint angle $\theta$ is defined to be zero at rest, and positive when the extensor contracts (counter-clockwise joint rotation). b The muscle model, where the length of the muscle $\lambda$ is the sum of $\lambda _1$, the length of the series elastic element, and $\lambda _2$, the length of the three parallel elements (damping, elastic, and contractile)

The tibia is attached to the femur via a simple hinge joint and hangs perpendicular to the femur at rest. Two muscles named the “flexor” and “extensor” apply the tensile forces $T_\mathrm{fl}$ and $T_\mathrm{ex}$ to opposite sides of the joint. Both muscles act on the tibia from constant attachment points with a lever arm of length $r_\mathrm{a}$ and are perpendicular to the tibia at rest.

We simplify the geometry of the model by neglecting the vertical component of muscle stretch. This is justified by two observations. First, during the walking cycle of a cockroach, the excursion of the FTi joint is typically small (Watson and Ritzmann 1997a, b), meaning that such vertical motion is minimal. Second, studies of the FTi joint in other insects show that this vertical displacement is negligible, even as the tibia is rotated throughout its entire range of motion (Guschlbauer et al. 2007). As such, the muscle tension is assumed to act only in the horizontal direction, resulting in the following equation for $M_\mathrm{net}$, the net moment on the joint:

$$\begin{aligned} M_\mathrm{net}={r_\mathrm{a}}\left( T_\mathrm{ex} - T_\mathrm{fl} \right) \cos \left( \theta \right) - {k_\mathrm{e}}\theta - {b_\mathrm{e}}{\dot{\theta }}. \end{aligned}$$

The magnitude of muscle forces relative to leg weight at insect scales make it possible to ignore the effects of gravity; however, the elasticity and damping of the cockroach exoskeleton, represented by the parameters $k_\mathrm{e}$ and $b_\mathrm{e}$, respectively, produce relatively large moments, which should not be neglected (Hooper et al. 2009; Hooper 2012). Assuming the tibia is a thin rod rotating about a point $r_\mathrm{a}$ from one end, its moment of inertia is given by:

$$\begin{aligned} J = \frac{ml^2}{12} + m\left( \frac{l}{2}-r_\mathrm{a}\right) ^2 . \end{aligned}$$

The inclusion of this term yields Eq. (1), the equation of motion for the joint:

$$\begin{aligned} \ddot{\theta }= \frac{1}{J}\left( {r_\mathrm{a}}\left( T_\mathrm{ex} - T_\mathrm{fl} \right) \cos \left( \theta \right) - {k_\mathrm{e}}\theta - {b_\mathrm{e}}{\dot{\theta }} \right) . \end{aligned}$$

(1)

2.3 Muscle model

The flexor and extensor are each considered to be a single slow motor unit and are modeled as Hill muscles. This model produces biologically plausible forces T while remaining relatively simple and computationally lightweight. The basic model from (Shadmehr and Arbib 1992) shown in Fig. 2b contains an active contractile element producing an activation force A, a parallel viscous dampening element with coefficient b, and parallel and series elastic elements with coefficients $k_\mathrm{pe}$ and $k_\mathrm{se}$, respectively. The Hill muscle model generates tension, T, according to the following equation:

$$\begin{aligned} \dot{T} = \frac{{{k_\mathrm{se}}}}{b}\left( {{k_\mathrm{pe}}\varDelta \lambda + b{\dot{\lambda }} - \left( {1 + \frac{{{k_\mathrm{pe}}}}{{{k_\mathrm{se}}}}} \right) T + \frac{A}{{fl\left( {\varDelta \lambda } \right) }}} \right) . \end{aligned}$$

A muscle’s deviation from its resting length $\lambda ^*$ is given by $\varDelta \lambda = \lambda - \lambda ^*$, where $\lambda$ is the length of the muscle. Following the geometry discussed in Sect. 2.2, the change in length of the flexor and extensor muscles are given by $\varDelta \lambda _\mathrm{fl} = r_\mathrm{a}\sin \left( \theta \right)$, and $\varDelta \lambda _\mathrm{ex} = - r_\mathrm{a}\sin \left( \theta \right)$, respectively. Likewise, the muscle contraction velocity is ${\dot{\lambda }}_\mathrm{fl} = r_\mathrm{a}\cos \left( \theta \right) {\dot{\theta }}$ for the flexor and ${\dot{\lambda }}_\mathrm{ex} = - r_\mathrm{a}\cos \left( \theta \right) {\dot{\theta }}$ for the extensor.

The Hill model typically accounts for the efficacy of actin and myosin filament interaction through a force-length scaling factor $fl\left( {\varDelta \lambda } \right)$. However, as our analysis is constrained to small joint angles where such a force-length scaling has a negligible effect, we simplify the model by setting $fl\left( {\varDelta \lambda } \right) = 1$ and assume the muscles operate at or near their optimal lengths (Full and Ahn 1995). This yields the following governing equations for muscle tension:

$$\begin{aligned} \begin{aligned} {{\dot{T}}_\mathrm{fl}}&= \frac{{{k_\mathrm{se}}}}{b}\left( {k_\mathrm{pe}}{r_\mathrm{a}}\sin \left( \theta \right) + b{r_\mathrm{a}}\cos \left( \theta \right) {\dot{\theta }} \right. \\&\quad \left. - \left( {1 + \frac{{{k_\mathrm{pe}}}}{{{k_\mathrm{se}}}}} \right) {T_\mathrm{fl}} + {A_\mathrm{fl}} \right) \\ {{\dot{T}}_\mathrm{ex}}&= \frac{{{k_\mathrm{se}}}}{b}\left( - {k_\mathrm{pe}}{r_\mathrm{a}}\sin \left( \theta \right) - b{r_\mathrm{a}}\cos \left( \theta \right) {\dot{\theta }} \right. \\&\quad \left. - \left( {1 + \frac{{{k_\mathrm{pe}}}}{{{k_\mathrm{se}}}}} \right) {T_\mathrm{ex}} + {A_\mathrm{ex}}\right) . \end{aligned} \end{aligned}$$

(2)

2.4 Muscle contractile element

The contractile element of each muscle generates an activation force A in response to U, its muscle membrane potential. This relationship is given by a sigmoidal activation function based on experimental data from (Blümel et al. 2012a):

$$\begin{aligned} A = \frac{{{T_\mathrm{max}}}}{{1 + \exp \left( {{S_\mathrm{m}}\left( {{x_\mathrm{off}} - U} \right) } \right) }} + {y_\mathrm{off}}. \end{aligned}$$

(3)

Here, $S_\mathrm{m}$ determines the maximum slope of the sigmoid, and $T_\mathrm{max}$ is the maximal tension the contractile element can produce, and $x_\mathrm{off}$ and $y_\mathrm{off}$ are curve fitting parameters.

In the animal, the flexor and extensor muscles are not identical to one another. Likewise, we model the flexor and extensor to have different values of $T_\mathrm{max}$ and $y_\mathrm{off}$. However, we choose to use symmetrical muscles for selected portions of this paper where the absence of confounding effects from asymmetry more clearly illustrate key results. In these instances, we set identical values for $T_\mathrm{max}$ and $y_\mathrm{off}$ for the flexor and extensor and explicitly state that we are using a symmetrical parameter set.

2.5 Muscle membrane

In arthropods, slow and intermediate motor units have muscle membranes with slow time constants relative to the motion these motor units produce. These muscle membranes effectively behave as integrators of the trains of action potentials delivered across their neuromuscular junctions (Hooper et al. 2007). We found it appropriate to model the muscle membranes as nonspiking leaky integrators, as the muscle membranes of slow motor units do not typically produce action potentials (Wolf 2014), and the insect thoracic ganglia contain many nonspiking neurons (Laurent and Burrows 1989; Büschges and Schmitz 1991). A detailed derivation of this model along with a description of the assumptions made can be found in appendix B.

The dynamics of the membrane potential U of a muscle membrane with membrane capacitance $C_\mathrm{m}$ and leak conductance $g_\mathrm{m}$ are given by:

$$\begin{aligned} C_\mathrm{m} \dot{U} = -g_\mathrm{m} U + \sum \limits _i {g_i \hat{U}_{\mathrm{pre},i} \left( \varDelta E_i - U\right) }. \end{aligned}$$

$\hat{U}_{\mathrm{pre},i}$ is defined as the activation of each of the i neurons innervating the muscle membrane. The activation represents the behavior of a MN as a fraction of its maximum activity and is therefore restricted such that $\hat{U}_{\mathrm{pre},i} \in \left[ 0,1 \right]$. Our model assumes nonspiking inputs, and the magnitude of $\hat{U}_{\mathrm{pre},i}$ acts as a compatible proxy for a biological MN’s presynaptic firing rate (Wilson and Cowan 1972; Trappenberg 2009).

The influence of the ith neuron on the muscle membrane is modulated by the synaptic properties of its neuromuscular junction. The synaptic potential relative to the muscle membrane’s resting potential, defined as $\varDelta E_{i}$ above, determines whether the synapse is excitatory or inhibitory. The synaptic conductance $g_i$ scales the magnitude of the synaptic current induced by the activation of the presynaptic MN and is analogous to the gain of the synapse.

In our model, the EMNs depolarize the muscle membranes via synapses with excitatory synaptic potentials $\varDelta E_{e}$. In contrast, the CIMN inhibits the muscle membranes via synapses with inhibitory synaptic potentials, i.e., $\varDelta E_{ci} = 0$mV. This is motivated by the fact that CIMNs typically secrete gamma-aminobutyric acid (GABA), which has a reversal potential near the resting potential of most neurons (Wolf 2014). Using an excitatory synaptic potential for EMNs and an inhibitory synaptic potential for the CIMN leads to the following governing equation:

$$\begin{aligned} \dot{U} = \frac{1}{C_\mathrm{m}}\bigg (g_{e} {{\hat{U}}}_{e} \left( \varDelta E_\mathrm{e} - U \right) - \left( g_\mathrm{m} + g_{ci} {{\hat{U}}}_{ci}\right) U \bigg ). \end{aligned}$$

(4)

Scaling $g_\mathrm{m} = 1~\mu$S effectively defines all synaptic conductances relative to the conductance of the muscle membrane and simplifies subsequent analyses. In the absence of presynaptic MN activity, the time constant of the muscle membrane is given by $\tau _\mathrm{m} = C_\mathrm{m}/g_\mathrm{m}$. However, when considering the influence of the CIMN, the membrane time constant is given by $\tau _\mathrm{m} = C_\mathrm{m}/\left( g_\mathrm{m} + g_{ci}\hat{U}_{ci}\right)$. CIMN activity effectively increases the conductance of muscle membrane, which temporarily decreases the muscle’s time constant $\tau _\mathrm{m}$. We will examine this mechanism and its effect on motor control in Sect. 3.2.

2.6 Magnitude control

It is generally accepted that the nervous systems of all walking animals contain neural circuits called central pattern generators (CPGs) that contribute to rhythmic muscle output, commonly behaving as relaxation oscillators in insects (Bässler 1986). In this functional mode, inhibitory signals from the CPG counteract a driving excitatory motor command provided by the CNS and disable (or significantly dampen) the activity of EMNs in an alternating manner (Goldammer et al. 2018). This has the effect of inhibiting sensory feedback as well. As shown in Fig. 1, the magnitude of each EMN’s activity is further modulated by information about the state of the system—either through direct signals from proprioceptive sensors, or via intermediary connections through interneurons.

Fig. 3

The input/output relationship of both EMNs during stance (a) and swing (b) phases. Our CPG and EMNs were designed to work in tandem to mimic the functional behavior of a biological CPG and strongly inhibit the antagonist EMN, effectively “disabling” it

The CPG itself is a complex neural circuit with behavior that is usually governed by some combination of mutual inhibition, neuromodulation, and proprioceptive feedback from position, force, and velocity sensors (for a review, see Bidaye et al. 2017). Recreating a truly biomimetic CPG would introduce significant additional model complexity as well as potentially confounding effects. Therefore, for the scope of this paper, we find it sufficient to combine and condense the role of the CPG and the descending motor commands into a few abstracted nodes, which approximate the idealized overall behavior of the analogous biological system and vastly simplify the analysis.

Specifically, our joint-level CPG analog has two functional objectives: (1) to determine the control direction and provide that information to the rest of the system and (2) to strongly inhibit the antagonist EMN, effectively “disabling” it.

The first objective is implemented directly through $\theta _\mathrm{ref}$, the single output of our CPG analog, which is simply a constant amplitude signal that alternates between $\theta _\mathrm{ref} = \pm \theta _\mathrm{max}$. The sign of $\theta _\mathrm{ref}$ reflects the intended direction of motion at a given time and is termed the control direction. The muscle acting in the control direction at a given moment is termed the agonist, whereas the muscle acting against the control direction is termed the antagonist.

When $\theta _\mathrm{ref} > 0$, the control direction is toward the extensor (swing phase), whereas when $\theta _\mathrm{ref} < 0$, the control direction is toward the flexor (stance phase). The sign of $\theta _\mathrm{ref}$, and thus the control direction of the entire system, is toggled by sensory feedback as described in the following Sect. 2.7.

The second objective is implemented through the combined behavior of our CPG analog and our EMNs, as shown in Fig. 3. As in biological slow motor units, each muscle membrane is innervated by an excitatory MN (EMN). In our system, each EMN receives input from three sources: the CPG, which provides the control direction; central drive (labeled as “CNS” in Fig. 1a) which encodes the intended magnitude of motion; and a joint angle proprioceptor [e.g., the chordotonal organ (Mamiya et al. 2018)], from which the input is proportional to the instantaneous joint angle $\theta$. We recognize that biological EMNs would likely be modulated by additional inputs, whether from other proprioceptive sensors (e.g., force and velocity), or from intermediary networks of interneurons. The inclusion of these sources of feedback would enable our model to mimic additional aspects of the biological system (e.g., control of the joint velocity profile). However, exploring these interactions is beyond the scope of this paper.

The magnitude of the EMN output is proportional to the difference in these two input signals, which is defined as the error $e_{i} = \pm \left( \theta _\mathrm{ref} - \theta \right)$. Note that the sign of the error is positive for the extensor and negative for the flexor to account for directionality. To fit the definition of activation from Sect. 2.5 (i.e., $\hat{U}_\mathrm{e} \in \left[ 0,1\right]$), the output of each EMN is scaled linearly from zero output signal $\hat{U}_{e} = 0$ at $\theta = \theta _\mathrm{ref}$, or $e = 0$. The maximum output signal $\hat{U}_{e} = 1$, which corresponds to the maximum firing rate of the biological EMN, is generated when $\theta = -\theta _\mathrm{ref}$.

As discussed in Sect. 2.5, the EMN neuromuscular junction has a fixed, excitatory synaptic potential $\varDelta E_\mathrm{e}$. Each EMN can only increase the membrane potential—and consequently the activation—of its respective muscle. Therefore, negative outputs are set to zero, and EMNs behave according to the following dynamics:

$$\begin{aligned} \begin{aligned} {{{\hat{U}}}_{e,i}}&= \text {min}\left( \text {max}\left( \frac{e_i}{2 \left| \theta _{com} \right| },1\right) ,0\right) \text { where: }\\ e_\mathrm{fl}&= -\left( \theta _\mathrm{ref} - \theta \right) , \text { and } e_\mathrm{ex} = \left( \theta _\mathrm{ref} - \theta \right) . \end{aligned} \end{aligned}$$

(5)

It follows that at any given time at most one EMN is active, as illustrated by Fig. 3. Moreover, the active EMN must be the agonist as long as the condition $\left| \theta \right| \le \left| \theta _\mathrm{ref} \right|$ holds. As detailed in Sect. 2.7, we define the timing control to enforce that condition, thus achieving the second functional objective of the CPG.

2.7 Timing control

In our system, a transition between stance and swing occurs once either $\theta$ reaches a threshold value of $\frac{5}{6}\theta _\mathrm{ref}$, or if the velocity $\dot{\theta }$ falls below a threshold of 0.1 rad/s. This is akin to the CPG’s phase resetting after the leg reaches the posterior extreme position (PEP) or anterior extreme position (AEP). These transition conditions were selected as an analog for tarsus touch-down, which our system does not model. The transition condition $\theta = \frac{5}{6}\theta _\mathrm{ref}$ ensures that the condition $\left| \theta \right| \le \left| \theta _\mathrm{ref} \right|$ required by Sect. 2.6 is never violated. In addition, the instantaneous transition of the CPG state, which is triggered by the much slower motion of the leg joint, gives rise to relaxation oscillator dynamics, which have been shown to drive walking in stick insects (Bässler 1986).

The purely excitatory nature of EMNs combined with the long time constants of arthropod slow muscle membranes means that arthropod muscles cannot quickly relax. However, arthropod limbs contain one to several common inhibitor (CI) MNs (Pearson and Iles 1973; Wolf 2014). A single CIMN simultaneously innervates most muscle membranes of a limb. CIMNs fire a short burst of several action potentials during the transition from stance to swing phase, which has been shown to be immediately followed by muscle relaxation throughout the limb (Usherwood and Runion 1970; Wolf 2014).

Our CIMN was modeled to produce a sustained signal of ${{\hat{U}}}_{ci} = 1$ for a duration of $\varDelta t_{ci}$ whenever the CPG transition condition was met in stance phase, corresponding to a stance to swing transition. The duration $\varDelta t_{ci}$ was selected to be a plausible value for several action potentials (10 ms). It can be noted that the exact value of $\varDelta t_{ci}$ is not critical, as the effect of the CI neuron is a function of delivered signal energy—a product of both $\varDelta t_{ci}$ and $g_{ci}$. In other words, any change in $\varDelta t_{ci}$ could be directly compensated by a change in $g_{ci}$.

As discussed in Sect. 2.4, we occasionally choose to use symmetrical muscles to better show key results. Likewise, we occasionally set the CI neuron to fire whenever a transition condition is met, not just during stance to swing transitions. In these cases, we explicitly state that the CI activity is symmetric.

3 Results

3.1 Steady-state error

We hypothesized that a slow muscle membrane acting as a leaky integrator behaves analogously to a proportional-integral (PI) controller in the slow motor units of arthropods to achieve precise joint positions in the presence of significant exoskeleton restoring forces. To examine this, the step response of the system was simulated across a range of values of the membrane time constant $\tau _\mathrm{m}$, achieved by changing the value of membrane capacitance $C_\mathrm{m}$ or membrane conductance $g_\mathrm{m}$.

Fig. 4

Minimum $e_\mathrm{ss}$ for a step response across a range of muscle membrane time constants $\tau _\mathrm{m}$ when overshoot is limited to 105%. a$\tau _\mathrm{m}$ achieved by varying $C_\mathrm{m}$ at a constant $g_\mathrm{m} = 1~\mu$S. b$\tau _\mathrm{m}$ achieved by varying $g_\mathrm{m}$ at a constant $C_\mathrm{m} = 150$ nF

To evaluate the step response of the system, the CPG output was fixed at $\theta _\mathrm{ref} = \theta _\mathrm{max}$ and did not change sign. The system’s accuracy was evaluated by calculating the percent steady-state error—a quantification of the system’s ability to eventually reach a reference state. This was defined to be:

$$\begin{aligned} e_\mathrm{ss} = \mathop {\lim }\limits _{t \rightarrow \infty } \left( 1- \frac{\theta }{\theta _\mathrm{ref}} \right) \times 100 \%, \end{aligned}$$

The percent overshoot—a measure of how far the system surpasses a constant reference state—was used to evaluate the range of motion produced by the system. For a step response, it was defined to be:

$$\begin{aligned} \text {overshoot} = \mathop {\max }\limits _{t} \left( \frac{\theta }{\theta _\mathrm{ref}}\right) \times 100 \%. \end{aligned}$$

For the cyclical stepping simulations throughout this paper, percent overshoot was defined as above with t constrained to the interval of a steady-state cycle.

Each simulation ran for a sufficiently long time for $\theta$ to converge to a steady-state value before $e_\mathrm{ss}$ was calculated. For any value of $\tau _\mathrm{m}$, it is possible to select an arbitrarily high value of $g_{e}$ to achieve an arbitrarily low $e_\mathrm{ss}$. However, this can result in physiologically unrealistic overshoot values (e.g., excessive movements that cause injury to the animal). Therefore, the minimal $e_\mathrm{ss}$ value for a given $\tau _\mathrm{m}$ was found by a binary search algorithm which only considered step responses where the overshoot did not exceed 105%. The choice of this limit was arbitrary, and changing this limit only scales the resultant relationship between $\tau _\mathrm{m}$ and $e_\mathrm{ss}$ shown in Fig. 4a.

Due to ever-present charge leakage through the muscle membrane, the EMN must continuously provide some greater than zero signal in order to maintain the degree of muscle activation required to counteract the elastic restoring force of the exoskeleton. Given that the presynaptic potential of the EMN approaches zero as $\theta$ approaches $\theta _\mathrm{ref}$, a slower (and consequently slower-leaking) membrane allows the joint to settle on an equilibrium value of $\theta$ closer to $\theta _\mathrm{ref}$. However, in this system, $e_\mathrm{ss}$ can never be zero unless physiologically implausible values of $g_\mathrm{m} = 0$ or $C_\mathrm{m} = \infty$ are used. Ultimately, slower muscle membranes attain significantly smaller steady-state errors, a property that is comparable to the behavior of a PI controller.

3.2 Muscle membrane dynamics

Although a muscle membrane can be made to depolarize arbitrarily quickly given a sufficiently large $g_{e}$, the repolarization of the membrane from a given potential is limited to the first-order decay dynamics of membrane leakage currents:

$$\begin{aligned} \dot{U} = - \frac{g_\mathrm{m}}{C_\mathrm{m}} U. \end{aligned}$$

Consequently, whereas a slower muscle membrane ($g_\mathrm{m} = 1$, yellow in Fig. 4b) can achieve significantly better positional accuracy than a faster membrane ($g_\mathrm{m} = 8$, red in Fig. 4b), this accuracy comes at the cost of a far longer repolarization time as shown in Fig. 5. The rate of repolarization limits the rate of muscle relaxation, and consequently the rate at which a joint is free to reverse its direction of motion. In extreme cases, this can cause opposing muscles to become fully activated and lock up the joints of an animal leaving it temporarily paralyzed (Wolf 1990).

Fig. 5

The exponential decay dynamics of the muscle membrane potential with no excitatory inputs

One strategy to achieve fast repolarization would be to operate at higher values of membrane potential—for example, a 10 mV drop occurs faster from 50 mV than from 20 mV. However, this strategy proves ineffective for reasons explained in the next Sect. 3.3.

The CI neuron provides a better strategy for accelerating repolarization. Input from the CI neuron acts as a temporary switch from slow to fast muscle membrane dynamics:

$$\begin{aligned} \dot{U} = - \frac{g_\mathrm{m} + g_{ci}\hat{U}_{ci}}{C_\mathrm{m}} U. \end{aligned}$$

When the CI neuron is active, the muscle membrane leak rate is effectively $g_\mathrm{m} + g_{ci}$, see ($g_\mathrm{m} = 1, g_{ci} = 7$, violet) in Fig. 5. Essentially, the activity of the CI neuron enables arthropod muscles to selectively function as fast membranes with short repolarization and muscle relaxation times when needed while otherwise preserving the aforementioned advantages of a slower muscle membrane.

3.3 Muscle activation

A critical component of the system’s behavior is the sigmoidal shape of the muscle activation curve, which maps muscle membrane potential to contractile element activation. Depending on the level of membrane potential, it can either enhance or diminish the resulting muscle activation (i.e., the active tension). The level of amplification:

$$\begin{aligned} E_\mathrm{sigmoid} = \frac{A_\mathrm{diff}}{U_\mathrm{diff}} \end{aligned}$$

is quantified as a ratio of differential membrane potential and differential muscle activation. These are defined, respectively, as:

$$\begin{aligned} U_\mathrm{diff} = \mathop {\text {mean}}\limits _{t\ \in \ \text {ss cycle}} \left( U_\mathrm{agonist} - U_\mathrm{antagonist}\right) , \end{aligned}$$

and:

$$\begin{aligned} A_\mathrm{diff} = \mathop {\text {mean}}\limits _{t\ \in \ \text {ss cycle}} \left( A_\mathrm{agonist} - A_\mathrm{antagonist}\right) . \end{aligned}$$

Fig. 6

Plots of steady-state cycle joint position (a), membrane potential (b), and muscle activation (c) from simulations of selected values of $g_{e}$ with symmetric muscles and no CI activity are shown normalized in time. Note that the plots in (b) and (c) show different ranges at equivalent scale. The plots in (d) show the range of muscle membrane potentials and subsequent contractile element activations from the respective cycle against the muscle activation sigmoid from Eq. (3) (denoted $A^*$)

Table 1 Key metrics from the simulations shown in Fig. 6

To better illustrate the significance of this activation curve, consider the simulations shown in Fig. 6, with key values from each in Table 1. When the level of excitatory input is low, as in Fig. 6i ($g_{e} = 0.5$), the muscle membrane potential fluctuates in a range that is not substantially amplified by the sigmoid, resulting in a value of $A_\mathrm{diff}$ that is insufficient to overcome the restoring forces of the exoskeleton and drive $\theta$ to the angular position threshold of the CPG, instead triggering a $\theta _\mathrm{ref}$ sign change by reaching the low velocity threshold. This is compounded by the fact that the antagonist muscle relaxes slowly due to the slow repolarization rate of the muscle membrane from low initial values, and by the slow activation of the agonist muscle directly caused by a low $g_{e}$.

Set ii ($g_{e} = 2$) represents the largest $U_\mathrm{diff}$ value that the system can achieve without a CIMN. At this level of excitatory input, $U_\mathrm{diff}$ is maximally amplified by the activation curve. As shown in Table 1, a smaller $U_\mathrm{diff}$ than in the low $g_{e}$ case is mapped to a much larger $A_\mathrm{diff}$. The range of membrane potential is still too low for membrane dynamics to facilitate faster antagonist relaxation; however, this fact is mitigated by the amplification of the sigmoid.

At higher levels of excitatory input, such as the case shown in set iii ($g_{e} = 8$), the muscle membrane quickly depolarizes, quickly activating the agonist muscle and driving the joint to $\theta = 0$. At this point, however, further motion in the joint is hindered by the co-activation of the antagonist muscle. Although at this range of membrane potential the muscle membrane repolarizes relatively quickly, resulting in a much higher $U_\mathrm{diff}$ than the other two cases, this behavior occurs in the least sensitive portion of the muscle activation curve. Consequently, the membrane potential must change significantly more before it causes a sufficient relaxation of the antagonist muscle to complete the step.

Fig. 7

Plots of steady-state cycle joint position (a), membrane potential (b), and muscle activation (c) from simulations of selected parameter sets with symmetric muscles are shown normalized in time. Note that the plots in (b) and (c) show different ranges at equivalent scale. The plots in (d) show the range of muscle membrane potentials and subsequent contractile element activations from the respective cycle against the muscle activation sigmoid from Eq. (3) (denoted $A^*$)

Table 2 Key metrics from the simulations shown in Fig. 7

3.4 Effect of CI on muscle activation

Because the membrane dynamics limit the membrane repolarization rate, the muscles in these simulations never fully relax, comparable to what has been observed in animals whose CIMNs are prevented from firing (Wolf 1990). Thus, the motion of the joint is driven primarily by the timing and degree of differential muscle activation. The performance of the system simulated without CI activity (violet in Fig. 7, $g_{e} = 7$, $g_{ci} = 0$) is hampered by the same problems as the system ($g_{e} = 8$) described in Sect. 3.3 (Table 2).

The addition of CI activity to the system ($g_{e} = 7$, $g_{ci} = 6$, red) produces drastic changes to its behavior. The high value of $g_{e}$ still allows the agonist muscle to quickly activate, albeit the rate of activation is slowed by the CI input. However, because of the CI activity, the antagonist muscle can relax much faster than before. This system operates on a far more sensitive part of the muscle activation curve, which combined with reduced muscle co-activation, enables it to operate much faster.

3.5 Control effort

Energy expenditure is a useful metric for evaluating the performance of a designed system under varying parameters. In fact, it is also a consequential quantity for insects, especially in the context of evolution. Nevertheless, precise quantification of energy expenditure is highly dependent on the physical realization of the system. For example, quantifying the metabolic cost of a muscle-driven joint would require a completely different model than quantifying the electrical power consumption of a joint driven by servomotors.

Mechanical work provides an idealized definition of energy expenditure that is not dependent on the physical realization of the model. However, calculating the work done by the joint itself over the course of a step cycle yields a trivial result because our model assumes the joint is unloaded. Instead, we extend the definition of mechanical work to include the notion of achieving our system’s intent (as defined by the control direction from Sect. 2.6) in the context of internal resistive forces.

We quantify the energy cost of our system with what we term the control effort: The work done by the active contractile elements in the muscles against internal resistive forces (i.e., the force generated by the antagonist contractile element, and the passive elasticity and damping of the exoskeleton and both muscles) to achieve the commanded range of motion over one step cycle.

Consequently, we define the instantaneous power of the system to be the instantaneous power of the agonist muscle—the muscle acting in the control direction at that moment:

$$\begin{aligned} \hat{P} = \left\{ \begin{array}{ll} \dot{T}_\mathrm{ex} \varDelta \lambda _\mathrm{ex} + T_\mathrm{ex} \dot{\lambda }_\mathrm{ex} &{}\quad \text {if}\ \theta _\mathrm{ref} > 0\\ \dot{T}_\mathrm{fl} \varDelta \lambda _\mathrm{fl} + T_\mathrm{fl} \dot{\lambda }_\mathrm{fl} &{}\quad \text {if}\ \theta _\mathrm{ref} < 0\\ 0 &{}\quad \text {otherwise}. \end{array}\right. \end{aligned}$$

Integrating the instantaneous power over a full steady-state cycle yields the control effort:

$$\begin{aligned} E = \int _\mathrm{cycle} \hat{P}. \end{aligned}$$

Fig. 8

Control effort required to achieve a given step frequency at various values of $g_{ci}$. For reference, approximately 10 Hz is a typical step frequency for cockroaches (Pearson and Iles 1973; Bender et al. 2010)

A binary search algorithm determined the lowest control effort required to achieve a given step frequency by varying $g_{e,fl}$ and $g_{e,ex}$, considering only simulations that resulted in an overshoot between 95 and 105%. The results shown in Fig. 8 demonstrate not only that greater values of $g_{ci}$ reduce the energy expenditure across all step frequencies, but also that they increase the maximum frequency at which the joint can move. These effects likely stem from the reduced co-activation of the antagonist muscle, which is one of the primary sources of resistance for the agonist.

3.6 Reduced control parameters

Fig. 9

Simulations run with asymmetric muscles and asymmetric CI activity at $g_{ci} = 6$. The dotted teal lines overlaid on each map denote the range of overshoot (95–105%). The solid teal lines represent a 2D-slice through the data, a plot of which is shown by each map

Arthropods can control step frequency by changing the gain of feedback pathways only in stance phase muscles (Gabriel and Büschges 2007). We wished to test if this simple model could recreate that result. To test this, we simulated the system for a broad sweep of $g_{e,fl}$ and $g_{e,ex}$ values. We sought parameter combinations for which increasing $g_{e,fl}$ would:

• Increase the speed of stance phase motion (i.e., flexion);

• Not significantly affect the speed of swing phase motion (i.e., extension); and

• Not significantly affect the range of motion of the joint.

Figure 9 shows the results. Increasing the flexor gain $g_{e,fl}$ shortened the stance phase duration and consequently increased the step frequency, with a negligible effect on the swing phase duration and the overshoot of the joint rotation. In contrast, increasing the extensor gain $g_{e,ex}$ increased overshoot with a negligible effect on stance phase duration or step frequency. This result suggests that key properties of an arthropod walking cycle can be independently changed through a reduced parameter set in the form of $g_{e,fl}$ and $g_{e,ex}$.

Figure 10 shows that the CIMN further isolates the effects of each parameter $g_{e,fl}$ and $g_{e,ex}$. When $g_{ci} = 0$, the value of both $g_{e,fl}$ and $g_{e,ex}$ must be modified to keep the overshoot between $95 \%$ and $105 \%$ (the region between the teal lines). Simultaneously, the stepping frequency does not change monotonically in this region, leading to further complication in controlling the limb’s motion. When $g_{ci} = 6$, however, the range of motion hardly changes as $g_{e,fl}$ is increased and $g_{e,ex}$ is constant (e.g., 4). Increasing $g_{e,fl}$ alone also clearly changes the stepping frequency. In fact, changing $g_{e,ex}$ can hardly modify the stepping frequency at all. These results show that the CIMN may be critical for simplifying the control of periodic motion seen in walking.

Fig. 10

Simulations run with asymmetric muscles and asymmetric CI activity at varying values of $g_{ci}$. The dotted teal lines overlaid on each map denote the range of overshoot (95% to 105%). The solid teal lines represent a 2D-slice through the data, a plot of which is shown below each map

3.7 Robustness across model parameter values

We sought to test if the results from Sect. 3.6 would apply to a different set of model parameters. All of the analyses up to this point used the parameters from the hind leg of our group’s previous cockroach model (Rubeo et al. 2017). To test the robustness of this approach, we repeated our analysis using vastly different parameter sets taken from the front and middle legs. Figure 11 shows that each leg’s parameter set displayed the same overall pattern, centered and scaled around a different range of $g_{e,fl}$ and $g_{e,ex}$ values. This suggests that arthropods can adapt an identical neural architecture across vastly different limbs simply by tuning the gain of a few motor neurons.

Fig. 11

Simulations run with asymmetric muscles and asymmetric CI activity at $g_{ci} = 6$ using varying parameter sets. The dotted teal lines overlaid on each map denote the range of overshoot (95–105%). The solid teal lines represent a 2D-slice through the data, a plot of which is shown below each map

4 Discussion

In previous work, our group assembled a dynamical model of cockroach walking (Szczecinski et al. 2014; Rubeo et al. 2017). Cockroaches walk with stepping frequencies up to 10 Hz or higher (Bender et al. 2010). They generate long, rapid leg strides despite having many muscle fibers that take between 100 and 1000 ms to relax (Iles and Pearson 1971). In addition, they do not use their fast MNs, except when running at their highest speeds (Pearson 1972; Watson and Ritzmann 1998). When our group initially constructed the cockroach model with slow MNs alone, it could hardly move. Therefore, the muscle fiber dynamics in our group’s previous studies were made to be very fast (time constant of 5 ms, $\approx$15 ms to relax). However, in this case, proportional afferent feedback from joint proprioceptors could not move the joints to the intended positions, requiring the addition of a neural integrator to accumulate error over time, and ensure the muscles pulled the limb to the intended position in each step (Khalil 2002; Szczecinski et al. 2017b). This solution enabled the model to run at cockroach-like speeds, but our group could not directly justify the inclusion of a neural integrator circuit stimulating the muscles. In addition, changing the walking speed was effectively impossible, requiring the rate of integration to be reconfigured at every walking speed.

The present study addresses the shortcomings of our group’s previous work by introducing a neuromuscular model of joint control that is simultaneously more biologically accurate and more flexible in its function. We hypothesized that we could produce an accurate neuromuscular controller by modeling slow muscle fibers as muscles whose membrane have long (i.e., 150 ms) time constants, rather than short time constants with neural integrator circuits. The slow dynamics of the muscles function as integrators (Hooper et al. 2007; Morris and Hooper 1997b), effectively accumulating input to drive the limb to the intended position. However, as we showed, such integration causes the muscle force to saturate after repeated stimulation, a phenomenon that can be observed in insects when their slow muscle fibers are stimulated at high frequencies (Bässler and Stein 1996; Morris and Hooper 1997a) and can be resolved by activating the CI directly (Bässler and Stein 1996). Therefore, we added a CI MN to our model, which actively relaxes the muscles in the leg at the end of each step. This resolves the saturation problem, resulting in accurate limb positioning and no force buildup in the leg muscles from step cycle to step cycle. The model presented in this work further reinforces the importance of including passive forces and inhibitory motor neurons in neuromechanical models.

Agreement with biological data Despite the simplicity of this model, it captures several key biological results regarding insect joint control and will inform the development of more sophisticated models in the future. Our model verifies that insects can indeed control motion with muscle fibers whose time constants are much longer than the motion in question (Iles and Pearson 1971; Pearson 1972). Critical to this capability, however, is the common inhibitory (CI) motor neuron, which drives slow muscle fibers’ potentials toward rest, actively relaxing muscles (Iles and Pearson 1971; Rathmayer and Erxleben 1983; Usherwood and Runion 1970; Wolf 1990, 2014), particularly while the animal walks (Wolf 1990). This structure enables the slow muscle fibers’ membranes to act like integrators, accumulating incoming excitatory spikes from the slow motor neurons (Morris and Hooper 1997b). The slow motor neurons receive information from proprioceptors (Gabriel and Büschges 2007), which means that the muscle membrane voltage, and thus the contractile force of the muscle, is related to the integral of the leg state over time. Such a structure is an implementation of a feedback integral controller, which is known to increase the accuracy of control systems (Khalil 2002). In addition, modulating the gain of sensory feedback, and thus this rate of integration, controls the speed of stick insect stepping on a treadmill (Gabriel and Büschges 2007). Simplifications and assumptions are unavoidable in the design of models which approximate physiological systems, and consequently, a variety of distinct approaches can be used to effectively model a given system. Likewise, our approach is by no means definitive, but we believe that the presented model provides a simplified, yet valid abstraction of what is known about insect joint control. It is unknown how well this approach may apply to highly-specialized joints, such as those used for jumping in some insects (Burrows and Morris 2001). However, we believe this model to be a valuable starting point for future studies, and will be extended to capture more aspects of insect neurophysiology.

Future extensions to the model We plan to expand this model to assemble a more complete, entirely neuromorphic closed-loop joint controller. In this study, we made several simplifications to aid in tractability. For instance, we only included one sensory afferent in our study, but previous studies have outlined how multiple proprioceptive pathways converge on the motor neurons (Sauer et al. 1996). Including such detailed feedback pathways will enable us to study how modulating each sensory type separately may impact the control of periodic motion.

Despite the abundance of nonspiking neurons in the insect thoracic ganglia, some of these sensory pathways are mediated via spiking interneurons (Sauer et al. 1996). In addition, motor neurons themselves generally fire action potentials. However, none of the units in the present study fired action potentials. Therefore, we will need to expand our model to include spiking neuron and synapse models. We have developed methods to tune spiking models using our functional subnetwork approach to network tuning (Szczecinski et al. 2017b). We believe that spiking neurons will produce more varied dynamic responses than nonspiking neurons, which may have additional relevant computation abilities (e.g., sensory adaptation via spiking threshold accommodation).

We also plan to expand this model to include fast muscle fibers and motor neurons, which are necessary for insects to move their limbs at the highest speeds during running (Iles and Pearson 1971; Pearson 1972; Schmitz 1986; Watson and Ritzmann 1998) or jumping (Burrows and Morris 2001). In cockroaches, fast MNs fire out of phase with the slow MNs when the animal runs (Watson and Ritzmann 1998). However, in the locust, fictive locomotion patterns from deafferented thoracic ganglia cause the fast MNs to exhibit subthreshold depolarizations in phase with the slow MNs’ action potentials (Ryckebusch and Laurent 1993). This suggests that both the fast and slow MNs receive central drive from the same source, but that fast MNs have higher apparent spiking thresholds, preventing them from firing action potentials when the animal walks slowly. This also suggests that sensory feedback may innervate fast and slow MNs differently, producing the observed phase difference in cockroaches. Incorporating spiking MNs and fast and slow fibers into our model will let us investigate possible networks that can reproduce these data.

One final simplification in the present model is our reduction of the central pattern generator (CPG) to a square-wave generator whose phase could be reset by intra-joint feedback. CPGs are important to form a complete motor program. When an animal is running fast, it apparently depends on central patterns for proper coordination. However, when insects walk more slowly, sensory feedback can very strongly affect a CPG’s phase [(Akay et al. 2001, 2004, 2007; Hess and Büschges 1999), for a review see (Bidaye et al. 2017)]. This is likely the case because when a CPG’s central drive is low, its phase is more easily adjusted by sensory inputs (Shaw et al. 2015; Szczecinski et al. 2017a). This effect is even evident in fast running insects like cockroaches when they walk at lower speeds, during which their coordination patterns are less coordinated, suggesting less reliance on central patterns and more reliance on sensory feedback (Bender et al. 2011). Therefore, in this study, the CPG is formulated simply as a switch whose state is flipped by sensory feedback.

The behavior of CPGs and how to model it is a rich field, but outside the scope of this work (for a review, see (Bidaye et al. 2017)). However, our group has extensive experience in designing and tuning biologically based CPG architectures, and their responses to sensory entrainment (Deng et al. 2019; Szczecinski and Quinn 2017a; Szczecinski et al. 2017a). We believe that we can map the behavior shown in this paper onto a more detailed neural architecture that more accurately reflects the structure of the nervous system.

Application to robotics Insects are capable walkers. As such, they have served as the models for many walking robots (for reviews, see (Buschmann et al. 2015; Ritzmann et al. 2000). However, the scaling of robots and the insects they mimic often differ, leading to a mismatch in the control strategy (Hooper 2012). This mismatch could be overcome by building a robot with viscoelastic forces scaled to produce the same passive mechanical dynamics, even with its much larger inertia, such as our group’s robot DrosophiBot (Goldsmith et al. 2019). Such a robot could then apply the control strategy presented here, with simulated muscle fibers whose time constants are long for timescale of the robot. This would represent a more thorough test of this system than described in this paper, requiring that this strategy be generalized to all of the joints in all of the legs. In our group’s previous work, we performed a similar generalization, demonstrating how all of the leg joints in the body might undergo “reflex reversals” when the animal walks in paths of varying curvature (Szczecinski and Quinn 2017b). We believe we could follow a similar process with this control structure, wherein we take a structure informed by biology, apply it to all of the leg joints in a robot, and then tune the structure differently for each joint, depending on the role it plays in locomotion or posture. Such an approach seems likely to work, given the success we had applying the control structure in this paper to vastly different leg segments.

Appendices

Appendix A: Parameter values

Variable parameters Within the parameter sets shown in Table 3, the values for m and l reflect those from the legs of a cockroach as collected for a previous study. The values of $k_\mathrm{e}$ and $b_\mathrm{e}$ as well as $T_\mathrm{max}$ were taken from our group’s previous model (Rubeo et al. 2017). Values for $y_\mathrm{off}$ were calculated as described below.

Table 3 Parameter sets used throughout the paper. Unless specified otherwise, the hind leg parameter set was used

Joint model As the attachment points can vary greatly between the individual muscles of a joint, and because those values are difficult to accurately quantify, we used the same value of $r_\mathrm{a} = 1$ mm from our group’s previous model for all muscles (Rubeo et al. 2017).

Muscle model The coefficients of the muscle model, $b = 0.1~\frac{{\text {N}\cdot \text {s}}}{{\text {m}}}$, $k_\mathrm{pe} = 11.24~\frac{{\text {mN}}}{{\text {mm}}}$, and $k_\mathrm{se} = 45~\frac{{\text {mN}}}{{\text {mm}}}$ were taken from our previous model (Rubeo et al. 2017) and match the values for $k_\mathrm{pe}$ and $k_\mathrm{se}$ reported in (Blümel et al. 2012b).

Muscle contractile element The maximum slope of the muscle activation sigmoid $S_\mathrm{m} = 0.3$ and the curve fitting parameter $x_\mathrm{off} = 10$ mV were selected to match our curve to the data presented in (Blümel et al. 2012a). The value of the second curve fitting parameter $y_\mathrm{off}$ was calculated for a given value of $T_\mathrm{max}$ to ensure that the contractile element produced zero force when the muscle membrane was at its resting potential $U = 0$ mV.

Muscle membrane L-glutamate is the excitatory neurotransmitter at the neuromuscular junctions of most arthropods (Shinozaki 1988), consequently our excitatory synapses were implemented with $\varDelta E_{e} = 40$ mV, based on measurements of the excitatory post-synaptic potential induced by the application of L-glutamate to insect muscle membranes presented in (Jan and Jan 1976a, b). A physiologically plausible value of $g_\mathrm{m} = 1~\mu$S was selected to allow us to define all synaptic conductances relative to the conductance of the muscle membrane, as discussed in Sect. 2.5. A value of $C_\mathrm{m} = 150$ nF was selected to achieve a membrane time constant $\tau _\mathrm{m} = C_\mathrm{m}/g_\mathrm{m} = 150$ ms that is similar to the value for arthropod slow muscles reported in literature (relaxation time $\approx$ 450 ms) (Iles and Pearson 1971).

CPG The commanded amplitude $2\theta _\mathrm{max} = 0.5$ rad was selected to match the realistic range of motion of a cockroach FTi joint (Watson and Ritzmann 1997a, b).

Appendix B: Derivation of muscle membrane model

The Hodgkin–Huxley model defines the dynamics of a neural or (in our case) muscle membrane potential V with respect to a membrane capacitance $C_\mathrm{m}$, and trans-membrane leakage, synaptic, and applied currents:

$$\begin{aligned} C_\mathrm{m} \dot{V} = I_{leak} + I_{syn} + I_{app}. \end{aligned}$$

As there is no external current injected into the muscle membrane, we set $I_{app} = 0$. The leakage of ions through the membrane is governed by Ohm’s Law, where $g_\mathrm{m}$ is the membrane’s constant leak conductance (i.e., inverse of resistance) and $E_r$ is the resting potential of the membrane:

$$\begin{aligned} I_{leak} = g_\mathrm{m} \left( E_r - V\right) . \end{aligned}$$

The sum of ion flow across each neuromuscular junction defines the synaptic current:

$$\begin{aligned} I_{syn} = \sum \limits _i {G_i \left( E_i - V\right) }, \end{aligned}$$

where $E_i$ is the synaptic potential for the ith synapse, and $G_i$, the instantaneous conductance of the ith synapse, is defined as a function of the maximum synaptic conductance $g_i$ and $V_{pre,i}$, the instantaneous potential of the presynaptic neuron:

$$\begin{aligned} G_i = g_i \cdot \left\{ \begin{array}{ll} 0 &{}\quad V_{pre,i} < E_{lo} \\ \frac{V_{pre,i} - E_{lo}}{E_{hi} - E_{lo}} &{}\quad E_{lo} \le V_{pre,i} \le E_{hi} \\ 1 &{}\quad V_{pre,i} > E_{hi} \end{array} \right. . \end{aligned}$$

Here, $E_{hi}$ and $E_{lo}$ are the upper and lower thresholds of the synapse, respectively. When we require that $E_{lo} = E_r$, and define $R = E_{hi} - E_{lo}$ to be the operating range of the presynaptic neuron, the instantaneous conductance of the ith synapse can be written as:

$$\begin{aligned} G_i = g_i \cdot \left\{ \begin{array}{ll} 0 &{}\quad V_{pre,i} < E_{r} \\ \frac{V_{pre,i} - E_{r}}{R} &{}\quad E_{r} \le V_{pre,i} \le \left( R-E_{r}\right) \\ 1 &{}\quad V_{pre,i} > \left( R-E_{r}\right) \end{array} \right. . \end{aligned}$$

This expression reduces to:

$$\begin{aligned} G_i = g_i \frac{V_{pre,i} - E_r}{R} \end{aligned}$$

if we restrict the presynaptic voltage $V_{pre,i} \in \left[ E_{lo},E_{hi}\right]$ or alternatively $V_{pre,i} \in \left[ E_r,R-E_r\right]$.

To further simplify analysis, we define a new variable $U = V - E_r$, which quantifies the voltage of the neural or muscle membrane relative to its resting potential. Note that because V does not appear outside of this derivation, we refer to U throughout the paper simply as the muscle membrane potential. In a similar manner, we define the presynaptic voltage relative to the resting potential $U_{pre,i} = V_{pre,i} - E_r$. In terms of this variable, the instantaneous conductance of the ith synapse becomes:

$$\begin{aligned} G_i = g_i \frac{U_{pre,i}}{R}, \end{aligned}$$

where $U_{pre,i}$ is restricted such that $U_{pre,i} \in \left[ 0,R\right]$.

Furthermore, it is convenient to select $R = 1$, effectively representing the behavior of all presynaptic neurons as a fraction of their maximum activity. We call $\hat{U}_{pre} \in \left[ 0,1 \right]$ the activation of the ith presynaptic neuron. When we also define the synaptic potential relative to the resting potential $\varDelta E_i = E_i - E_r$ the sum of synaptic currents simplifies to the form:

$$\begin{aligned} I_{syn} = \sum \limits _i {g_i \hat{U}_{pre} \left( \varDelta E_i - U\right) }, \end{aligned}$$

and using the same variables, the leak current simplifies to:

$$\begin{aligned} I_{leak} = -g_\mathrm{m} U. \end{aligned}$$

Given that $\dot{U} =\dot{V}$, combining the modified current terms together into the original equation yields:

$$\begin{aligned} C_\mathrm{m} \dot{U} = -g_\mathrm{m} U + \sum \limits _i {g_i \hat{U}_{pre,i} \left( \varDelta E_i - U\right) }. \end{aligned}$$

For more details regarding these manipulations of the neural and synaptic models, see (Szczecinski et al. 2017b).