Robust Coordination with Transmission Delay and Measurement Noises

Abstract

This paper is concerned with the problem of distributed stochastic approximation in single-integer multi-agent systems on general directed unbalanced networks with measurement noises and transmission delay. The time-varying control gains satisfying the stochastic approximation conditions are introduced to attenuate noises. Then based on Lyapunov technique, the convergence result of mean square consensus is established provided that the transmission delay is bounded.

1 Introduction

Consensus seeking is of fundamental importance in distributed coordination of multi-agent systems [1, 2]. Therefore, consensus problem has gained a significant interest in the last decade and has been studied under a wide variety of conditions (such as networks with undirected or directed links, switching or fixed topology). Consensus problem involves designing networked interaction protocols that enable agents reaching a global agreement. Being a special case, average consensus is important in many applications and requires agents agreeing on the exact average of their initial states.

In practical applications, information communication between interacting agents of networks may suffer from transmission delays and measurement noises. Thus, robust consensus protocols that can cope with the joint effects of delays and noises are of more practical interest. However, up to now, most of contributions have just separately dealed with the effect of transmission delays and measurement noises on consensus. It is now known that stochastic approximation with decreasing control gains is a powerful tool to deal with noisy measurements. With which, each agent can gradually reduce the weights assigned to its neighbors and thus attenuate the measurement noises. Based on this idea and by converting the delayed system into a delay less one via the augment method, [3] recently proposed stochastic approximation protocols for randomly switching directed networks in the presence of both communication delay and noisy measurements, whereby convergence of mean square consensus is conducted via ergodic backward products of compatible nonnegative matrices. Stochastic approximation protocols are also adopted for non-leader–follower and leader–follower multi-agent systems on undirected networks [4]. In order to overcome the difficulties induced by the delays and noises, an auxiliary system is introduced via the augment method. Then based on the auxiliary system and by the algebraic theory, the robust consensus problem is transferred into that of analyzing the consensus for a delay-free system with input of color noises. Similarly, via the augment method and using results from non-reversible Markov chains [5], the impact of delays on consensus for directed networks is recently characterized by [6]. Without using the augment method, [7] recently proposed an effect stochastic approximation protocol to deal with transmission noises and bounded delay of multi-agent system on directed balanced networks, where the convergence analysis is mainly based on matrix decomposition. It is noted that, except that [3], many existing distributed stochastic approximation protocols can only be applicable to directed balanced networks, where the corresponding adjacent matrices must be doubly stochastic. However, using doubly-stochastic matrices implies a feedback communication between a pair of agents which is not always possible and may bring about implementation issue in practice. In this paper, motivated by [7] and [8], we develop a stochastic Lyapunov approach to analyze robust consensus over directed unbalanced networks for the case of coexistence of transmission noises and bounded delay, and our aim is also to make a contribution within the stochastic approximation context.

The rest of this paper is structured as follows: In Sect. 2, we formally state the problem of interest and briefly reviews some preliminaries on graph theory. In Sect. 3, we deduce the mean square consensus. Then finally is the concluding remark.

2 Problem Formulation

We consider a multi-agent network with agent (or node) set $\varvec{\mho} \,{ = }\,\left\{ { 1 , 2\ldots ,N} \right\}$, for which each agent has the following discrete-time single-integrator dynamics

$$ x_{i} (k + 1)\, = \,x_{i} \left( k \right) + u_{i} \left( k \right),\quad k = 0,1,2, \cdots ,\;\;i = 1,2, \cdots ,N $$

(1)

where $x_{i} \left( k \right)$ is the state of agent $i ,$ $u_{i} \left( k \right)$ is the control input or protocol of agent $i.$

The topology of the communication network can be modeled by a directed graph (or digraph) $\varvec{G} = \left( {\varvec{\mho} ,\varvec{\mathcal{E}},W} \right)$, where $\varvec{\mathcal{E}} = \left\{ {e_{ij} = \left( {i,j} \right)\left| {i,j \in \varvec{\mho}} \right.} \right\}$ denotes the edge set and the corresponding weight adjacent matrix is $W = \left( {w_{ij} } \right) \in R^{N \times N}$. The edge $e_{ij} \in \varvec{\mathcal{E}}$ represents that agent $i$ can receive information from agent $j$, then agent j is called an in-neighbor of agent i and $w_{ji} > 0$, otherwise $w_{ji} = 0$. A directed graph is undirected if $e_{ij} \in \varvec{\mathcal{E}} \Leftrightarrow e_{ji} \in \varvec{\mathcal{E}}$ for $\forall i \ne j$. A digraph $\varvec{G}$ is strongly connected if for any distinct nodes $i$ and $j$, there exists a path that connects i and $j$. A digraph $\varvec{G}$ is balanced if the in-degree and out-degree of each $i \in \mho$ are same. $W$ is said to be nonnegative if its entry $w_{ij} \ge 0$ for all $i$ and $j$. $W$ is said to be stochastic if it is nonnegative and satisfies $\it{W1} = \it{1}$, where $\it{1} = \left( {1,1, \cdots ,1} \right)^{T} \in R^{N}$. Moreover, $W$ is called doubly stochastic if it is stochastic and satisfies $\it{1}^{T} W = \it{1}^{T}$.

Due to the random communication environment and time delay, at each time, agent i receives the delayed noisy measurements of its neighbor $j$’s state given by

$$ y_{ji} \left( {k{ - }\tau } \right) = x_{j} \left( {k{ - }\tau } \right) + \xi_{ji} \left( {k{ - }\tau } \right) $$

(2)

where $\xi_{ji} \left( {k{ - }\tau } \right)$ describes the measurement noise affecting the information transmission along the directed link $e_{ji}$ at time $k$. $x_{j} \left( k \right)$ is the state of agent j; $\tau$ is the time delay.

Then the control input for agent i at time $k$ is proposed as

$$ u_{i} \left( k \right) = \alpha \left( k \right)\sum\limits_{{j{ = 1},\,j \ne i}}^{N} {w_{ij} \left( {y_{ji} \left( {k{ - }\tau } \right) - x_{i} \left( {k{ - }\tau } \right)} \right)} $$

(3)

which means that agent i updates its current state by taking a weighted average of its own delayed state and the delayed noisy states of its neighbors. Where $w_{ij}$ are the entries of the stochastic adjacent matrix $W$ associated with the digraph $\varvec{G}$. The scalars $a\left( k \right) \in \left( {0,1} \right]\left( {k = 0,1, \cdots } \right)$ are the time-varying control gains, which are used to attenuate measurement noises.

Before further proceeding, we need the following assumptions.

Assumption 1

In the stochastic approximation approach, the control gain sequence $\left\{ {\alpha \left( k \right),k \ge 0} \right\}$ satisfies the following:

$$ \sum\limits_{k = 0}^{\infty } {\alpha \left( k \right)} = \infty \,\,{\text{and}}\,\,\sum\limits_{k = 0}^{\infty } {\alpha^{2} \left( k \right)} < \infty $$

Assumption 2

The digraph $\varvec{G}$ is strongly connected. $W$ is stochastic and has positive diagonal entries, that is, there exists a positive constant $\rho$ such that $w_{ii} = 1 - \sum\limits_{{j{ = 1,}\, \, j \ne i}}^{N} {w_{ij} } \ge \rho > 0$ holds for each $i \in \varvec{\mho}$. Meanwhile, $w_{ij} \in \left\{ 0 \right\} \cup \left[ {\rho ,1} \right]$ for $i,j \in \varvec{\mho}$ with $i \ne j$.

Assumption 3

There exists a positive integer $d^{ * }$ such that $0 \le \tau \le d^{ * }$.

Assumption 4

All the (delayed) measurement noises are zero mean and have uniformly bounded variance, that is, $E\left[ {\xi_{ji} \left( {k - \tau } \right)} \right] = 0$ and $E\left[ {\xi_{ji}^{2} \left( {k - \tau } \right)} \right] \le \sigma^{2}$ for all $i,j \in \varvec{\mho}$, $k \ge 0$ and $0 \le \tau \le d^{ * }$, where $E\left[ \cdot \right]$ denotes the expectation operator. Moreover, $E\left[ {\xi_{ji} \left( {k{ - }\tau } \right)\xi_{l\kappa } \left( {t{ - }\tau } \right)} \right] = 0$ for all $i,j,l,\kappa \in \varvec{\mho}$ and $k \ne t.$

Applying the protocol (2) to (1) and by Assumption 2, we obtain the closed-loop system for agent i:

$$ \begin{aligned} x_{i} (k + 1) & = x_{i} \left( k \right) + \alpha_{k} \sum\limits_{{j{ = 1, }\,j \ne i}}^{n} {w_{ij} x_{j} \left( {k - \tau } \right)} - \alpha_{k} \sum\limits_{{j{ = 1, }\,j \ne i}}^{n} {w_{ij} x_{i} \left( {k - \tau } \right)} + \alpha \left( k \right)\xi_{i} \left( {k{ - }\tau } \right) \\ & = x_{i} \left( k \right) - \alpha_{k} x_{i} \left( {k - \tau } \right) + \alpha_{k} \sum\limits_{{j{ = 1}}}^{n} {w_{ij} x_{j} \left( {k - \tau } \right)} + \alpha \left( k \right)\xi_{i} \left( {k{ - }\tau } \right) \\ \end{aligned} $$

(4)

where $\xi_{i} \left( {k{ - }\tau } \right) = \sum\nolimits_{{j{ = 1, }\,j \ne i}}^{N} {w_{ij} \xi_{ji} \left( {k{ - }\tau } \right)}$ denotes the aggregated delayed measurement noises that agent i receives from all of its neighbors at time $k$.

Denote $x\left( k \right) = \left( {x_{1} \left( k \right), \cdots ,x_{N} \left( k \right)} \right)^{T}$, $\xi \left( k \right) = \left( {\xi_{1} \left( k \right), \cdots ,\xi_{N} \left( k \right)} \right)^{T}$. Then (4) can be further written in a compact form

$$ x(k + 1) = x\left( k \right) + \alpha \left( k \right)\left( {W - I} \right)x\left( {k{ - }\tau } \right) + \alpha \left( k \right)\xi \left( {k - \tau } \right) $$

(5)

the initial states are $x\left( 0 \right),x\left( 1 \right), \cdots ,x\left( \tau \right)$ and $\xi \left( l \right) = 0$ for $l < 0$. Furthermore, denote the regression function $H\left( {x\left( k \right)} \right) = \left( {W - I} \right)x\left( k \right)$, then it is yielded from (5) that

$$ x(k + 1) = x\left( k \right) + \alpha \left( k \right)\left[ {H\left( {x\left( {k - \tau } \right)} \right) + \xi \left( {k - \tau } \right)} \right] $$

(6)

Thus, by Assumption 4, we can easily obtain the following results for the aggregated delayed noise $\xi_{i} \left( {k - \tau } \right)$, that is, $E\left[ {\xi_{i} \left( {k - \tau } \right)} \right] = 0$ and $E\left[ {\xi^{T} \left( {k - \tau } \right)\xi \left( {t - \tau } \right)} \right] = 0$ for $k \ne t$. Meanwhile, by the definition of aggregated delayed noise $\xi_{i} \left( {k - \tau } \right)$ and Assumptions 2 and 4, there holds that

$$ \left| {E\left[ {\xi_{i} \left( {k - \tau } \right)\xi_{j} \left( {k - \tau } \right)} \right]} \right| \le \sigma^{2} \left( {1 - w_{ii} } \right)\left( {1 - w_{jj} } \right) \le \sigma^{2} \left( {1 - \rho } \right)^{2} $$

(7)

Remark 1

Note that $W$ is stochastic and has positive diagonal elements such that $\varvec{G}$ is strongly connected. Then by [9], W is primitive and 1 is a simple maximal eigenvalue of W with algebraic multiplicity one. Furthermore, there exists unique normalized positive left eigenvector $\pi = \left( {\pi_{1} , \cdots ,\pi_{N} } \right)^{T}$ corresponding to eigenvalue 1 and satisfies

$$ \pi^{T} W = \pi^{T} ,\pi^{T} 1 = \sum\limits_{i = 1}^{N} {\pi_{i} } = 1\,\,{\text{and}}\,\,\mathop {\lim }\limits_{k \to \infty } W^{k} = 1\pi^{T} $$

(8)

If W is doubly stochastic, then $\pi^{T} = \frac{1}{N}\left( {1,1, \cdots ,1} \right)^{T}$.

For analyzing the convergence of (5), we introduce the following definition.

Definition 1

(Mean Square Consensus [3, 4]). All agents are said to reach mean square consensus if there exists a common random variable $x^{ * }$ such that $\mathop {\lim }\limits_{k \to \infty } E\left| {x_{i} \left( k \right) - x^{ * } } \right|^{2} = 0$ and $E\left\| {x\left( k \right)} \right\|^{2} < \infty$ for all $i \in \mho$ and $k \ge 0.$

3 Convergence Analysis

Before going further, we need some useful lemmas that will be used in the proof of our result.

Lemma 1 [7]

Suppose that Assumptions 1, 2 and 3 hold, then $\sum\nolimits_{k = \tau }^{\infty } {\alpha \left( k \right)} \xi \left( {k - \tau } \right) < \infty$ and $H\left( {x\left( {k - \tau } \right)} \right) - H\left( {x\left( k \right)} \right) \to 0$ as $k \to 0.$

Lemma 2 [10]

Suppose that Assumption 1 holds, meanwhile

• (a) $\chi \left( k \right) = \varpi \left( k \right) + \sigma \left( k \right),$ where $\varpi \left( k \right) \to 0$ as k→0 and $\sum\limits_{k = 0}^{\infty } {\alpha \left( k \right)} \sigma \left( k \right) < \infty$, $\alpha \left( k \right)$ is defined in Assumption 1;

• (b) The matrix $B$ only has negative and zero eigenvalues with the zero eigenvalue having the same algebraic multiplicity and geometric multiplicity.

Then the sequence $y(k + 1) = y\left( k \right) + \alpha \left( k \right)\left( {By\left( k \right) + b + \varpi \left( k \right) + \sigma \left( k \right)} \right)$ satisfies that $\mathop {\lim }\limits_{k \to \infty } \text{dist} (y\left( {\text{k}} \right),\Upxi ) = 0,$ where $\Upxi = \left\{ {y\left| {By + b = 0} \right.} \right\}$ and $\text{dist} (y\left( k \right),X) = \mathop {\inf }\limits_{x \in X} ||y\left( k \right) - x||$.

Remark 2

It yields from Remark 1 that the matrix $W - I$ only has negative and zero eigenvalues with the zero eigenvalue having the same algebraic multiplicity and geometric multiplicity.

Note that (5) can be further rewritten as

$$ \begin{aligned} x(k + 1) & = x\left( k \right) + \alpha \left( k \right)\left( {W - I} \right)x\left( k \right) + \alpha \left( k \right)\left[ {H\left( {x\left( {k - \tau } \right)} \right) - H\left( {x\left( k \right)} \right)} \right] + \alpha \left( k \right)\xi \left( {k - \tau } \right) \\ & = P\left( k \right)x\left( k \right) + \alpha \left( k \right)\left[ {H\left( {x\left( {k - \tau } \right)} \right) - H\left( {x\left( k \right)} \right)} \right] + \alpha \left( k \right)\xi \left( {k - \tau } \right) \\ \end{aligned} $$

(9)

where $P\left( k \right) = \left( {1 - \alpha \left( k \right)} \right)I + \alpha \left( k \right)W.$ Note the fact that W is stochastic and $a\left( k \right) \in \left( {0,1} \right]\left( {k = 0,1, \cdots } \right)$ are time-varying, and thus $P\left( k \right)$ is also stochastic and time-varying; meanwhile, $P\left( k \right)$ also satisfies (8).

Denote the following Lyapunov function

$$ \begin{aligned} V\left( {x\left( k \right)} \right) & = x^{T} \left( k \right)\left( {I - \pi 1^{T} } \right)D\left( {I - 1\pi^{T} } \right)x\left( k \right) = x^{T} \left( k \right)\left( {D - \pi \pi^{T} } \right)x\left( k \right) \\ & = \sum\limits_{i = 1}^{N} {\pi_{i} \left( {x_{i} \left( k \right) - \pi^{T} x\left( k \right)} \right)^{2} } \\ \end{aligned} $$

(10)

where the diagonal matrix $D = diag\left( {\begin{array}{*{20}c} {\pi_{1} } & \cdots & {\pi_{N} } \\ \end{array} } \right)$. The second equality in (9) is yielded from $1^{T} D = \pi^{T}$, $D1 = \pi$ and $\pi^{T} 1 = 1$. The function $V\left( {x\left( k \right)} \right)$ is actually a weighted distance Lyapunov function that measuring the weighted spread of the vector $x\left( k \right)$ components with respect to the weighted average value $\pi^{T} x\left( k \right)$.

Lemma 2 [8]

If Assumption 2 holds, then

$$ \begin{aligned} V\left( {x\left( {k + 1} \right)} \right) & = x^{T} \left( k \right)P^{T} \left( k \right)\left( {D - \pi \pi^{T} } \right)P\left( k \right)x\left( k \right) \\ & \le \left( {1 - \frac{\eta \left( k \right)}{{2\left( {N - 1} \right)}}} \right)V\left( {x\left( k \right)} \right) \\ \end{aligned} $$

(11)

where $1 > \eta \left( k \right) = \pi_{\hbox{min} } \rho \alpha \left( k \right) > 0$ and $\pi_{\hbox{min} } = \mathop {\hbox{min} }\limits_{i \in \mho } \pi_{i} .$

With these lemmas in place, then under the proposed protocol (2), all agents of the directed network will reach mean square consensus.

Theorem 1

If Assumptions 1-4 hold, then for each agent $i$ $\left( {i \in \mho } \right)$, there holds

$$ \mathop {\lim }\limits_{k \to \infty } E\left| {x_{i} \left( k \right) - x^{ * } } \right|^{2} = 0 $$

with $E\left( {x^{ * } } \right) = \hat{\pi }^{T} \hat{x}\left( \tau \right),$ where $\hat{\pi }$ satisfies $\hat{\pi }^{T} P = \hat{\pi }^{T}$. That is, all the agents converge in mean square to a common random variable $x^{ * }$, and its mathematical expectation is the weighted average of agents’ initial states; furthermore, the variance of $x^{ * }$ is bounded.

4 Conclusion

The paper considers the robust consensus algorithm that performs correctly despite coexistence of transmission noises and bounded delay. We do not model the behavior of the bounded delay using any virtual node. For future work, we will characterize the relationship between the convergence rate and the time-varying control gains.