Introduction

Legs are used by robots to push on their environment with end effectors and provide an active suspension [1], allowing the motion of the main body of the robot to be smoother than the underlying terrain profile. We will colloquially refer to all end effectors used to push on the environment as feet, whatever their form and function. A leg can be temporarily lifted off the ground to make a step, so that discontinuous terrains can be overcome, allowing locomotion in places out of reach otherwise. The number of legs, their kinematic structure and degrees of freedom, and the type of feet (flat, deformable, pointy, wheeled) can vary but the underlying principles are the same. As a matter of fact, the last few years have seen approaches originally developed for biped humanoid robots with flat feet quickly adapted to quadruped robots and all forms of feet.

The control objectives are to find adequate positions for establishing contact between the feet and the environment, and execute a corresponding dynamically feasible motion. The general approach which permeates almost all forms of legged robots locomotion now makes extensive use of numerical trajectory optimization and its online implementation, model predictive control (MPC), in order to handle alternating contacts and the corresponding constraints on contact forces, which are the key difficulty in legged locomotion control.

Aside from adapting this general approach to different numbers of legs and different types of feet, the main progress over the last few years in the field of legged robots locomotion control has been how this trajectory optimization is handled, with refined models and refined numerical methods. This has enabled transitioning from biped walking on mostly flat ground to more general multi-contact locomotion on uneven terrain with various legged robot morphologies.

Typical Approach to Legged Locomotion Control

The Dynamics of Legged Locomotion

The Newton equation of motion of a robot makes it clear that external forces fi generated through contact with the environment are needed to move its center of mass (CoM) c in a direction other than that of gravity g:

$$ m (\ddot{c}-g)=\sum\limits_{i}f_{i}, $$
(1)

where m is the total mass of the robot. The related Euler equation of motion makes it clear that the positions of the contact points si with respect to the CoM c are critical to controlling the angular momentum L of the robot body around its CoM at the same time:

$$ \dot{L}=\sum\limits_{i}(s_{i}-c)\times f_{i}. $$
(2)

The artificial synergy synthesis approach proposes to focus on this centroidal dynamics Eqs. 1–2 which is directly bound to contact forces, separately from the precise joint posture of the robot which is more directly bound to joint torques [3], and illustrated in Fig. 1. This approach is central today to legged locomotion control, but in situations where balance and posture must be tightly coordinated, the full Lagrangian dynamics of the complete articulated body is preferred as it naturally embeds the centroidal dynamics [4, 5].

Fig. 1
figure 1

Illustration of the centroidal dynamics and its connection to the whole-body dynamics (adapted from [2])

Full size image

One challenge of legged locomotion is that contact forces are usually unilateral: the robot can push on contact surfaces but not pull. Consequently, the forces fi can be oriented only in specific directions, constrained by the limits of friction. Additionally, each contact is binary: either there is contact and a contact force, or there is no contact and no contact force. Impacts can also occur when a leg collides with a surface. Contact switching and impacts are discrete events affecting the continuous time dynamics Eqs. 1–2: the dynamics of legged robots is hybrid in this respect, which constitutes a second challenge.

It is possible in theory to define the set of viable states, from which the robot is able to summon appropriate contact forces to avoid falling. Cyclic motions and equilibrium points are easy to identify as viable, and if the robot is able to reach such a cycle or equilibrium in a few steps from a given state, then it is viable as well [6]. This is the essence of the capturability analysis [7], which is key to many of the existing approaches to legged locomotion control [8].

Control Architecture

A sequence of contact points si is usually planned beforehand in both space and time using random sampling methods, considering the environment of the robot and its goal, accounting for kinematic and static balance constraints [9,10,11]. This is called contact planning. Pre-planned steps can be adjusted later during execution, to adapt to unstable terrain and perturbations [12]. A corresponding CoM motion and angular momentum can be obtained online with a model predictive control scheme [13, 14], accounting for the centroidal dynamics Eqs. 1–2 and making sure that the state of the robot is always kept capturable [8].

When considering locomotion on a flat ground, with co-planar contact points si, the forces fi are typically reduced to their Zero-tilting Moment Point (ZMP), which is bound to the convex hull of contact points [6, 16] (non co-planar contact points can be considered as well with a polyhedral projection of unilateral contact constraints [17]). In this case, efficient linear formulations are possible by pre-defining the vertical motion of the CoM. The most common is the linear inverted pendulum (LIP) model, which assumes a planar CoM motion and zero angular momentum [18]. Accounting for a non-zero angular momentum does not affect the linearity of this model but has rarely been considered [19], probably due to limited benefits when walking on flat ground [20].

The execution of the computed CoM motion, angular momentum and contact positions with the whole body of the robot is usually handled with inverse kinematics [21] or feedback linearization of the whole-body dynamics in the Cartesian space [22], using Quadratic Programs (QPs) to account for instantaneous kinematic, dynamic, force and torque constraints [22,23,24]. Hierarchical QPs are sometimes considered to account for different priorities in the control objectives [25], but singularities can be an issue [26]. Inverse kinematics and feedback linearization can be very sensitive to model inaccuracies, especially on the position, steadiness and stiffness of the ground contact, which are crucial elements for the balance of the robot. Passivity-based approaches are a promising alternative, much less dependent on accurate contact models [27,28,29].

This general approach to legged locomotion control, illustrated in Fig. 2, has been developed originally for biped walking and demonstrated successfully in a wide array of humanoid robots including Kawada’s HRP-2 [13], Honda’s Asimo [30], Aldebaran’s Nao [31] and Boston Dynamics’ Atlas [32••]. More recently, it has been adapted to quadruped robots such as ANYbotics’ ANYmal [33], IIT’s HyQ [34] and MIT’s Cheetah [35], and robots with feet rolling on the ground such as Aldebaran’s Pepper [36] or ETHZ’s wheeled ANYmal [37] and Ascento [38].

Fig. 2
figure 2

Typical control architecture, composed of three main stages: contact planning, centroidal dynamics resolution, and whole-body control (adapted from [15])

Full size image

Recent Progress in Contact Planning

Contact planning is a particular instance of motion planning, which is known to be PSPACE-hard in general, and therefore NP-hard [39]. In the case of legged locomotion, the discrete nature of contact states is a particular source of combinatorial explosion, adding to the overall non-convexity of the motion planning problem. In order to tackle this unfavorable computational complexity, mixed-integer formulations have been proposed to exploit state-of-the-art off-the-shelf solvers and find globally optimal sequences of contacts in a matter of seconds or even less, accounting for kinematic and even dynamic feasibility constraints on uneven terrain [40,41,42].

An alternative approach builds on the volume reachable by the feet of the robot to provide a necessary and sufficient condition for the existence of a contact sequence. This way, the motion of the robot can be approached without having to specify the exact contact sequence in the first hand, which reduces dramatically the complexity of the motion planning problem, enabling fast online replanning to adapt to changing environments in a fraction of a second. The exact contact sequence is recovered in a second stage using a variety of heuristics such as maximizing robust quasi-static balance, accounting for kinematic and even dynamic feasibility constraints on uneven terrain [15, 43,44,45]. This multistage approach can be very efficient but the reliance on heuristics can be a source of failure.

Recent Progress with Reduced Models

Divergent Component of Motion

One strength of the LIP model is that it is simple enough to lend itself to thorough mathematical analysis. Notably, the divergent part of its dynamics can be represented by a single component, the divergent component of motion (DCM), rather than the full CoM state (position and velocity). Linear feedback control of this DCM emerged as an efficient balance control strategy for biped locomotion over flat terrain [20], simple enough to be demonstrated on a variety of platforms, including position-controlled [46] and torque-controlled robots [47].

The DCM follows a first-order dynamics with respect to contact forces, simpler than the CoM’s second-order dynamics Eq. 1, which helps to simplify other aspects of the control of legged robots. Three recent progresses come to mind. First, the DCM allows removal of the planar CoM constraint to generate 3D walking trajectories with no change in model dynamics [48]. Second, online step-timing adaption can be included in the trajectory optimization while keeping a small convex problem to solve thanks to a suitable change of variable in the analytical DCM trajectory [49]. Third, the DCM can be used as a viability condition in the MPC problem for advanced stability and feasibility proofs [50].

Centroidal Dynamics

In multi-contact scenarios such as locomotion over uneven terrain or in confined spaces, reducing the model of contact forces to their ZMP is limiting. Approaches based on the full centroidal dynamics Eqs. 1–2 have emerged as a solution to compute a feasible CoM trajectory, angular momentum and corresponding contact forces in generic contact configurations. Contact sequences can even be optimized simultaneously [51].

Formulations exploiting the structure of the trajectory optimization problem such as sparsity and convexity have been developed [41, 44, 52]. Additionally, the inertia of quadruped robots is often approximated with that of a single rigid body [34, 35, 51], and locomotion-specific heuristics can be designed to regularize the underlying nonlinear optimization problem so that it can be solved faster around nominal behaviors [53]. With these improvements, approaches based on the full centroidal dynamics can run online with computation times below 10 ms [54].

The centroidal dynamics does not account for kinematic or actuator torque limits. To partly remedy these limitations, extensions of the contact wrench cone to include torque limits have been proposed [55] or learning-based approaches have been introduced to represent more complex whole-body constraints into the centroidal model itself [56].

Mixed Models

An alternative, intermediate solution between reduced and full dynamic models is to combine the centroidal dynamics Eqs. 1–2 with a kinematic model of the complete articulated body as a way to elaborate dynamically feasible whole-body motions while maintaining moderate computational complexity [57, 58]. With this approach, a locally optimal feedback policy can be computed and updated online in less than 70 ms [59]. The centroidal dynamics can also be combined formally with a whole-body dynamic model using an alternating descent method to account for additional aspects of the full dynamics of the robot [60]. An earlier approach combined in a unified, single QP the instantaneous full dynamics of the robot with a LIP model over a receding horizon to capture the long term locomotion dynamics [61].

Recent Progress with Whole-Body Models

Trajectory Optimization

The complete nonlinear dynamics of the robot can be used to obtain efficient coordinations of limb motions, taking into account all kinematic and dynamic feasibility constraints and objectives such as minimal energy consumption through trajectory optimization strategies such as collocation [62, 63] or multiple-shooting [64] (which bring complementary advantages in terms of speed, accuracy, stability and precision [65]). Contact sequences can even be optimized simultaneously, using relaxations of the contact model [66, 67]. Thanks to efficient open-source numerical solvers [68,69,70] exploiting the sparsity of the trajectory optimization problem and exact derivatives of the dynamic models [71,72,73], whole-body motions can be computed online in less than 7 ms [70, 74, 75].

Reinforcement Learning

The full dynamic model of the robot can also be used offline to compute directly an optimal feedback control policy operating from raw sensor measurements and an estimate of the robot attitude, outperforming more traditional approaches in challenging and difficult to model indoor and outdoor environments thanks to mature reinforcement learning techniques [76, 77••]. Key ingredients for this approach to be effective are as follows: (i) an efficient simulator to generate a large amount of sample behaviors [78], (ii) a curriculum strategy to guide the optimization of the feedback control policy from simple scenarios to more complex ones and (iii) accounting for difficult to model uncertainties in the actuator models using a neural network trained on real hardware measurements.

Conclusions

Legged locomotion means establishing a sequence of contacts between the feet and the environment of the robot, and executing a corresponding kinematically and dynamically feasible motion. Numerical trajectory optimization plays a central role, to handle alternating contacts and the corresponding constraints on contact forces. Models of varying complexity, from the simplest LIP model, to full centroidal dynamics Eq. 1–Eq. 2, to models of the complete articulated body can be used either alone or in combination to handle different aspects of locomotion. Refinements in these models and advanced numerical methods, either off-the-shelf or specifically tailored to legged locomotion, enabled a transition in recent years from flat terrain to generic indoor and outdoor environments. The first successful demonstration of reinforcement learning for quadruped locomotion in real, complex situations was also achieved in the same way.

The central role of numerical optimization, both online and offline, led the legged locomotion research community to heavily invest in and contribute to the development of new optimization methods and efficient numerical software. Notable contributions were made to Lexicographic Programming to safely handle multiple objectives with varying priorities, differential dynamic programming (DDP) to leverage the sparsity of nonlinear trajectory optimization problems, and even combinations of these two approaches [79]. This involved the development of efficient and versatile open-source software for dynamic models [80,81,82], simulation [83], state estimation [84] and control design [85].

Open-source software, and open-source hardware with printable parts, simplified electronics and actuation [86, 87•] could facilitate future reproducibility, dissemination and democratization of well-established, robust and extensively tested solutions, fostering faster iterations and innovations with the possibility to easily share, extend and improve existing implementations (a list is provided in Table 1).

Table 1 Open-source software for legged robots
Full size table

Interestingly, the existing literature on legged locomotion control relies almost exclusively on state feedback, but the position and velocity of the CoM cannot be measured directly as it is a virtual point and contacts with the environment can be difficult to detect and measure. State estimation is therefore a crucial component, which has been surprisingly little explored in this respect [2, 84, 89,90,91,92,93]. The reliance on state estimation and feedback makes existing legged robots extremely dependent on accurate hardware, expensive and brittle. Sensor-based control could be an alternative, largely unexplored so far [94]. Robust control could be another alternative, not much explored either [95,96,97,98]. A recent conclusion from robust control analysis of legged balance is that perfectly stable feedback can be obtained at surprisingly low frequency, as demonstrated with both humanoid [97] and quadruped robots [59].

Building on the open-source software and hardware mentioned earlier, such control approaches could contribute to the development of cheaper, more robust and versatile robots, which looks like the next logical step now that legged locomotion is being demonstrated robustly in generic indoor and outdoor environments and approaching real industrial scenarios [99, 100]. Deformable elements could also contribute to this objective [101].