The Flocking Problem

Abstract

The flocking and the rendezvous problems are the two most basic distributed robotics problems. This chapter gives a formal definition of the flocking problem for unicycles and discusses the extent to which it has been solved.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

3.1 Introduction

In his influential paper [36], Reynolds describes his algorithm to simulate flocks of birds by saying that each simulated bird is implemented as an independent actor that navigates according to its local perception of the dynamic environment, the laws of simulated physics that rule its motion, and a set of behaviors programmed into it by the “animator.” The aggregate motion of the simulated flock is the result of the dense interaction of the relatively simple behaviors of the individual simulated birds.

In this chapter we formulate a version of the flocking problem in which n robots are required to move in the same direction with the same speed. Like Reynolds, we require each robot to be an independent actor that takes decisions based on limited information it can sense about its neighbours. To simplify, for now we think of just robots in the plane modelled as unicycles. Later, in Chap. 6, we consider flying robots.

Under appropriate assumptions, the flocking problem for unicycles has a fascinating connection with the problem of synchronizing coupled oscillators. In this context, the Kuramoto model inspires a control law for flocking. We will establish a connection between this control law and the flocking algorithm investigated by Jadbabaie et al. [16].

3.2 Problem Formulation

We begin with n unicycles moving at unit speed:

$$\begin{aligned} \dot{z}_i&= \mathrm {e}^{j\theta _i}\\ \dot{\theta }_i&=\omega _i. \end{aligned}$$

Recall that $z_i$ is the ith robot’s position, $\theta _i$ its heading angle, and $\omega _i$ the steering control. The steering controls are restricted to depend on certain sensed variables as required by the nature of how the cameras function. We denote by $r_i=\mathrm {e}^{j\theta _i}$ the unit vector tangent to the trajectory. This is the velocity vector of unicycle i. Synchronizing the velocity vectors of the unicycles corresponds to synchronizing the heading angles $\theta _i$.

Let z denote the vector whose components are the unicycle positions:

$$ {z}=(z_1, \dots , z_n). $$

The neighbours of robot i are those robots that are visible by robot i. For example, if robot i carries an omnidirectional camera then its neighbours are those robots whose distances from robot i are not greater than the camera range. The set of neighbours of robot i is denoted by $\mathcal {N}_i(z)$ and is a set of indices. We repeat: $z_i$ is the location of robot i (in the global coordinate frame), z is the vector of positions, and $\mathcal {N}_i(z)$ is the set of indices k such that robot k is visible by robot i.¹ Notice that the neighbourhood class $\mathcal {N}_i(z)$ depends on z. The visibility graph $\mathcal {G}(z)$ is defined to have n nodes and an edge between two nodes if they are neighbours. If the robot carries an omnidirectional camera, $\mathcal{G}(z)$ is an undirected graph: if robot i can see robot k, then robot k can see robot i. If the camera is not omnidirectional, $\mathcal{G}(z)$ is a directed graph.

Next, we need to define what steering control laws $\omega _i$ are admissible. This is the case if they are locally Lipschitz and they depend only on sensed data from the onboard cameras. Let robot k be a neighbour of robot i. We assume robot i can see the position and heading of robot k in its local frame; that is, vectors $z_k-z_i$ and $r_k$ in the local frame of robot i. To repeat, if robot k is a neighbour of robot i, then robot i has the following sensed data:

$$ \langle z_k-z_i, r_i \rangle , \quad \langle z_k-z_i, s_i\rangle , \quad \langle r_k, r_i \rangle , \quad \langle r_k, s_i\rangle , $$

or equivalently,

$$ \langle z_k-z_i, r_i \rangle , \quad \langle z_k-z_i, s_i\rangle , \quad \theta _k - \theta _i. $$

Recall that s is the counterclockwise rotation of r by $\pi /2$: $s=j r$. Figure 3.1 shows the definition of the sensed data for two robots.

Fig. 3.1

Sensed data for two unicycles

The flocking problem is to find, if they exist, admissible steering controls $\omega _i$ so that there exists $\varepsilon >0$ such that for all initial conditions satisfying

• (i) $(\forall i,k=1,\ldots ,n) \ |\theta _i(0) - \theta _k(0)| < \varepsilon$,

• (ii) $\mathcal{G}(z(0))$ is connected,

then

$$ (\exists \theta _{ss}) \lim _{t \rightarrow \infty } \theta (t) = \theta _{ss} \mathbf {1}. $$

Thus we require that if initially the robots’ heading angles are sufficiently close to each other and the visibility graph is connected, then the heading angles will synchronize. Notice that this is a mathematical problem and that the following real issues are not addressed: robots are not of zero dimensions and therefore collisions are a real issue; real robots cannot be programmed all to go at the same speed and therefore a control system would be required to keep the robots in a pack.

Despite the apparent simplicity of the formulation above, the flocking problem remains unsolved. The challenge is to design admissible control laws that preserve the connectivity of the visibility graph while achieving synchronization of the heading angles. If we make the unrealistic assumption that the visibility graph is fixed, i.e., the neighbour sets $\mathcal{N}_i$ are constant, then the flocking problem admits a simple solution, which we discuss next.

3.3 Flocking with Fixed Neighbours

If the visibility graph $\mathcal{G}$ is fixed (i.e., the neighbour sets $\mathcal{N}_i$ are constant) and connected, then we no longer have to worry that the robots might come in and out of each other’s field of view. In this case we may ignore the robots’ positions and consider just their heading angle dynamics

$$\begin{aligned} \dot{\theta }_i = \omega _i, \quad i =1,\ldots ,n. \end{aligned}$$

(3.1)

In this context, we seek controls $\omega _i$ that rely only on the sensed data $\{\theta _i - \theta _k: k \in \mathcal{N}_i\}$, and asymptotically stabilize the flocking manifold

$$\begin{aligned} \mathcal{F} = \{ (\theta _1,\ldots ,\theta _n): \theta _1 = \cdots = \theta _n \}. \end{aligned}$$

(3.2)

We will also want the angles to converge to a constant, not just to each other.

Before proceeding with the solution of this problem, we need to clarify what the state space of (3.1) is. Each $\theta _i$ is an angle measured modulo $2\pi$. When we write $\theta _i = \theta _k$, we mean that $\theta _i$ is equal to $\theta _k$ plus some integer multiple of $2\pi$. An angle, therefore, is not a real number, but rather an equivalence class of real numbers. To make things precise, consider the equivalence relation on the real line

$$ \theta _1 \sim \theta _2 \iff (\exists l \in \mathbb {Z}) \ \theta _1 - \theta _2 = 2 \pi l. $$

For $\theta \in {\mathbb R}$, denote by $[\theta ]$ the equivalence class of $\theta$, defined as

$$ [\theta ] :=\{ \theta + 2 \pi l: l \in \mathbb {Z}\}. $$

Finally, let $({\mathbb R}/{\sim })$ denote the set of all equivalence classes just defined. Then each angle lives in $({\mathbb R}/{\sim })$, and the state space of (3.1) is the n-fold product $({\mathbb R}/{\sim }) \times \cdots \times ({\mathbb R}/{\sim })$.

How can we visualize the set $({\mathbb R}/{\sim }) \times \cdots \times ({\mathbb R}/{\sim })$? First, it is not a vector space, for the sum of two angles is well-defined but scalar multiplication is not (try to multiply $[\pi ]$ by 1 / 2; do you get a unique angle in $({\mathbb R}/{\sim })$?). A useful way of visualizing $({\mathbb R}/{\sim })$ is to identify each element $[\theta ] \in ({\mathbb R}/{\sim })$ with the point $\mathrm{e}^{j\theta }$ on the unit circle in the complex plane, which we denote by $\mathbb {S}^1$. For an exercise, convince yourself that this identification works. Prove that the map $({\mathbb R}/{\sim }) \rightarrow \mathbb {S}^1$, $[\theta ] \mapsto \mathrm{e}^{j\theta }$ is a bijection.² From now on we will drop the notation $[\theta ]$ and use $\theta$ to denote an angle in $({\mathbb R}/{\sim })$. We will also identify angles with points on the unit circle.

To summarize, the state space of (3.1) is the n-fold product $\mathbb {S}^1 \times \cdots \times \mathbb {S}^1$, called the n-torus, and the flocking problem with fixed visibility graph corresponds to the synchronization of points on the unit circle. Since the n-torus is not a vector space, it is not possible to solve the flocking problem with a linear control law. More concretely, for a control law to be well-defined on the n-torus, it must be a $2\pi$-periodic function of the angles $(\theta _1,\ldots ,\theta _n)$, and therefore nonlinear.

In 1975 the Japanese researcher Yoshiki Kuramoto proposed a simple model of synchronization of coupled oscillators. Kuramoto represented the limit cycle behaviour of each oscillator by the motion of a point $\theta _i$ on the unit circle $\mathbb {S}^1$, and formulated the following coupled differential equation representing the interconnection of the oscillators:

$$\begin{aligned} \dot{\theta }_i = \mu _i -\frac{K}{n} \sum _{k=1}^n \sin (\theta _i-\theta _k), i=1,\ldots ,n. \end{aligned}$$

(3.3)

The scalar $\mu _i\ge 0$ represents the ith oscillator’s natural frequency and $K>0$ determines the strength of the coupling among oscillators. This equation is known as the Kuramoto model of coupled oscillators. In a moment we will derive the Kuramoto model, but first we remark on its qualitative properties.

When the natural frequencies $\mu _i$ are positive, Kuramoto observed that as K is increased beyond a minimum threshold $K_{\min }$ (the so-called critical coupling strength), some of the points $\theta _i$ begin to rotate around the unit circle in a cohesive group, although not synchronized. The larger the ratio $K/K_{\min }$, the greater the portion of oscillators moving cohesively and the smaller the distance between the $\theta _i$’s. In the limit as $K \rightarrow \infty$, all $\theta _i$’s become synchronized. Kuramoto’s results stimulated a vigorous research activity in the physics and dynamical systems communities. Some of that research is summarized in the paper by Strogatz [41]. Recently, Dörfler and Bullo [10] made a breakthrough by precisely characterizing, among other things, the critical coupling strength $K_{\min }$, the presence of a locally exponentially stable set in the state space of (3.3), the size of the domain of attraction of this set, and the bound on the distances between the $\theta _i$’s on this set.

When the natural frequencies $\mu _i$ are all zero, the qualitative properties of model (3.3) are much simpler to analyze. As a matter of fact, we will see that for any $K>0$ the manifold³ $\{(\theta _1,\ldots ,\theta _n): \theta _1 = \cdots = \theta _n\}$ is locally asymptotically stable. In particular, when the initial conditions $\theta _i(0)$ are sufficiently close to each other, all phases $\theta _i$ converge to a common constant $\theta _{ss}$. This result is precisely what we need to solve the flocking problem. Before stating a formal result, we derive the Kuramoto model.

3.3.1 Derivation of the Kuramoto Model

Consider a collection of points $p_1(t),\ldots ,p_n(t)$ moving on the unit circle, with $p_i(t)=\mathrm {e}^{j \theta _i(t)}$. Differentiate with respect to time: $\dot{p}_i = \mathrm {e}^{j \theta _i}j \dot{\theta }_i.$ Now let $v_i=\dot{\theta }_i$ and substitute into the preceding equation:

$$\begin{aligned} {\dot{ p}_i} = { v_i} { jp_i.} \end{aligned}$$

(3.4)

Since the circle has unit radius, the scalar $v_i$ represents the linear speed of the point $p_i$. This speed could be positive or negative. Notice that if we view $p_i$ as a vector from the origin and view multiplication by j as rotation by $\pi /2$, then $jp_i$ can be viewed as tangent to the circle at the point $p_i$—see the picture on the left in Fig. 3.2.

Fig. 3.2

Left The vectors $p_i$ and $jp_i$. Right The contribution to the linear speed $v_i$ of the interaction between points i and k

Now we propose a feedback law for $v_i$ in Eq. (3.4); see the picture on the right in Fig. 3.2. First we suppose that when there are no interactions, point $p_i$ has a nominal speed $\mu _i$. To model the interaction between $p_i$ and $p_k$, we add a term proportional to the projection of $p_k$ onto the line tangent to the circle at $p_i$, that is, $\langle p_k, jp_i \rangle$. We then sum all these contributions to arrive at the control law

$$ v_i = \mu _i+\frac{K}{n} \sum _{k=1}^n \langle p_k, jp_i \rangle . $$

The coefficient K / n determines the interaction strength between points.

Now we need to express $v_i$ in terms of the angles $\theta _k$ rather than the points $p_k$. We observe that

$$\begin{aligned} \langle p_k,jp_i \rangle&=\mathrm {Re}\ ( \bar{p}_k jp_i) \\&= \mathrm {Re} \left( \mathrm {e}^{-j \theta _k}j \mathrm {e}^{j \theta _i} \right) \\&=-\sin (\theta _k-\theta _i). \end{aligned}$$

Therefore,

$$ \dot{\theta }_i = v_i = \mu _i - \frac{K}{n} \sum _{k=1}^n \sin (\theta _i-\theta _k). $$

This is the Kuramoto model in (3.3).

3.3.2 Solution of the Flocking Problem

We now return to the flocking problem for system (3.1). Kuramoto’s model (3.3) with natural frequencies $\mu _i=0$ suggests the following choice⁴ of control laws:

$$ \omega _i = - \sum _{k=1}^n a_{ik} \sin (\theta _i-\theta _k), \quad a_{ik}>0. $$

These controls require each robot to see all other robots (i.e., the visibility graph $\mathcal{G}$ is complete). It turns out that we may simply restrict the sum to the index set of the neighbours of robot i:

$$\begin{aligned} \omega _i = - \sum _{k \in \mathcal{N}_i} a_{ik} \sin (\theta _i-\theta _k), \quad a_{ik}=a_{ki}>0. \end{aligned}$$

(3.5)

Theorem 3.1

If the visibility graph is fixed, undirected, and connected, then for any $a_{ik} = a_{ki}>0$, $i,k=1,\ldots , n$, the control law (3.5) makes the flocking manifold $\mathcal{F}$ in (3.2) locally asymptotically stable and solves the flocking problem.

Proof Sketch Suppose for simplicity that the visibility graph is complete and $a_{ik}=1$, so that the closed-loop system reads as

$$\begin{aligned} \dot{\theta }_i = - \sum _{k=1}^n \sin (\theta _i-\theta _k), \quad i=1,\ldots , n. \end{aligned}$$

(3.6)

This equation has the form $\dot{\theta }= g(\theta )$, where the $g(\theta )$ is the gradient of a positive definite function. Indeed, let $r\mathrm {e}^{j \psi }$ denote the average of the points $\mathrm {e}^{j \theta _1}, \dots , \mathrm {e}^{j \theta _n}$. Of course, r and $\psi$ are functions of $\theta$, and so we have

$$ r(\theta )\mathrm {e}^{j \psi (\theta )}=\frac{1}{n} \left( \mathrm {e}^{j \theta _1}+ \cdots + \mathrm {e}^{j \theta _n} \right) $$

and therefore

$$ r(\theta )=\frac{1}{n} \left| \mathrm {e}^{j \theta _1}+ \cdots + \mathrm {e}^{j \theta _n} \right| . $$

The average of n points on the unit circle lives inside the unit disc, and therefore $r(\theta )$ is a real number between 0 and 1. It equals 1 if and only if the n points are equal, that is, the n angles are equal, and this is the case when the angles are on the flocking manifold $\mathcal{F}$.

Define the function

$$\begin{aligned} V(\theta )&= \frac{n^2}{2} r(\theta )^2 \\&= \frac{1}{2} \left| \mathrm {e}^{j \theta _1}+ \cdots + \mathrm {e}^{j \theta _n} \right| ^2 \\&= \frac{1}{2} \left( \mathrm {e}^{j \theta _1}+ \cdots + \mathrm {e}^{j \theta _n} \right) \left( \mathrm {e}^{-j \theta _1}+ \cdots + \mathrm {e}^{-j \theta _n} \right) . \end{aligned}$$

Thus

$$ \frac{\partial V(\theta )}{\partial \theta _i} = \sin (\theta _1-\theta _i) + \cdots + \sin (\theta _n-\theta _i) $$

and therefore (3.6) can be written $\dot{\theta } = \partial V(\theta )/\partial \theta .$ This is a gradient equation. If $\theta (0)$ is chosen so that all the phases are close enough together, then $r(\theta (0))$ will be close to 1, and therefore $\theta$ will move in a direction to increase $V(\theta )$, that is, increase $r(\theta )$, until in the limit $r(\theta )=1$ and the phases are synchronized.

In the general case of $a_{ik} \ne 1$ and connected but not complete graph, the function V should be replaced by

$$ V(\theta )= - \sum _{i=1}^n \sum _{k \in \mathcal{N}_i, k<i} a_{ik} (1 - \cos (\theta _i - \theta _k)). $$

We will see in Chap. 6 that $-V(\theta )$ can be regarded as the potential of a collection of springs with constants $a_{ik}$ that connect point-masses on the unit circle.

With the above definition of V the closed-loop system with controls (3.5) can still be written as $\dot{\theta }= \partial V(\theta )/\partial \theta$, and it is still true that solutions in a neighbourhood of the global maximum of V converge to the global maximum. It can be shown that if the graph is connected, the global maximum of V is attained on the flocking manifold $\mathcal{F}$. The details of this argument are presented later in the proof of a flocking theorem for flying robots (Theorem 6.1).   $\square$

3.4 The Control Law of Jadbabaie, Lin, and Morse

The paper by Jadbabaie, Lin, and Morse [16] is on the flocking problem. Since it has been referenced by many other papers, it is instructive to review it from the viewpoint of this chapter.

Although it is not stated in these terms, the paper studies a system of unicycles moving at constant speed in the plane. It is assumed that if unicycle i can see unicycle j, then j can see i. That is, the visibility graph is undirected at all times. The paper studies a single control strategy in discrete time, namely, at time $t+1$ (t an integer) unicycle i changes its heading to the average heading at time t of itself and its neighbours:

$$\begin{aligned} \theta _i(t+1) = \frac{1}{1+n_i(t)} \left( \theta _i(t) + \sum _{k \in \mathcal{N}_i(t)} \theta _k(t) \right) . \end{aligned}$$

(3.7)

Here $n_i(t)$ is the number of neighbours of robot i at time t. It is proved that all the heading angles converge to a common value provided $\mathcal {G}(t)$ has a connectedness property over time, namely, there exists $T>0$ such that the union graph $\bigcup _{t_0\le t \le t_0+T}\mathcal {G}(t)$ is connected for all $t_0$. Here, the union of graphs with the same node set is obtained by taking the union of the edges. Unfortunately, the condition on $\mathcal {G}(t)$ is not checkable—it would require an infinite time simulation. The proof uses a theorem of Wolfowitz on ergodicity (1963).

In the theorems in [16] the vector $\theta (0)$ of initial heading angles is fixed in time. This allows the visibility graph to be a function of t alone, and not both t and $\theta (0)$. In actuality, as we remarked at the beginning of this chapter, for a sensible model of limited visibility, the graph is state dependent, $\mathcal{G} = \mathcal{G}(z)$. If the control strategy is given and if the state z(t) evolves uniquely from z(0), the visibility graph is a function of time, $\mathcal {G}(z(t))$.

Is the control law of [16] admissible? If we ignore the problem with the time-varying visibility graph, it is admissible in the following sense. In continuous time, consider the controls

$$\begin{aligned} \omega _i= - \frac{1}{n_i(t)+1} \sum _{k \in \mathcal{N}_i(t)} (\theta _i - \theta _k). \end{aligned}$$

(3.8)

These are admissible because they rely on relative angles of neighbouring robots. The forward-Euler discretization with unit sampling interval of the closed-loop system is

$$ \theta _i(t+1) = \theta _i(t) - \frac{1}{n_i(t)+1} \sum _{k \in \mathcal{N}_i(t)} (\theta _i(t) - \theta _k(t)), $$

which coincides with (3.7). Thus the update (3.7) results from a forward-Euler discretization of an admissible control law.

The update law (3.7) is not $2\pi$-periodic. To illustrate the relevance of this property, consider three robots ($n=3$) and assume that 1 can see 2, 2 can see 1 and 3, and 3 can see 2. Initialize the heading angles as follows: $\theta _1(0) =0$, $\theta _2(0)=2 \pi$, $\theta _3(0) =4\pi$. Thus the initial headings are identical modulo $2 \pi$. At the next time step we have

$$ \begin{aligned} \theta _1(1)&=\frac{1}{2} (0 + 2 \pi ) = \pi \\ \theta _2(1)&=\frac{1}{3} (2 \pi + 0 + 4 \pi ) = 2 \pi \\ \theta _3(1)&=\frac{1}{2} (4 \pi + 2 \pi ) = 3 \pi . \end{aligned} $$

Thus the heading angles move away from the flocking manifold. The flocking manifold is unstable. More precisely, the problem with the control law (3.7) is that it treats the angles $\theta _i$ as real numbers, rather than as elements of $\mathbb {S}^1$. The control law (3.5), on the other hand, is $2\pi$-periodic and it respects the geometry of the state space.

As a final remark, notice the relationship between the Kuramoto-inspired control law (3.5) and the control law (3.8). If the visibility graph is constant and undirected, and we set $a_{ik} = 1 / (n_i+1)$, then (3.8) is the linearization of (3.5) at any point of the flocking manifold.