Multiple UAV/UGV Heterogeneous Control

Abstract

Multiple unmanned aerial vehicles (UAVs) can be used to cover large areas searching for targets. However, sensors on UAVs are typically limited in operating airspeed and altitude, combined with attitude uncertainty, placing a lower limit on their ability to resolve and localize ground features. Unmanned ground vehicles (UGVs) can be deployed to accurately locate ground targets, but they have the disadvantage of not being able to move rapidly or see through such obstacles as buildings or fences. This chapter mainly focuses on heterogeneous coordinated control for multiple UAVs/UGVs and cooperative search problem for multiple UAVs. On the basis of introduction of UAV/UGV mathematical model, the characteristics of heterogeneous flocking is analyzed in detail. Two key issues are considered in multiple UGV subgroups, which are Reynolds rule and Virtual Leader (VL). Receding horizon control (RHC) with particle swarm optimization (PSO) is proposed for multiple UGV flocking, and velocity vector control approach is adopted for multiple UAV flocking. Thus, multiple UAV and UGV heterogeneous tracking can be achieved by these two approaches. Then a time-delay compensation approach of heterogeneous network control for multiple UAVs and UGVs is described to handle the time delay in network control system. What’s more, a differential evolution (DE)-based RHC design for cooperative area search using multiple UAVs is presented. In this approach, an extended search map is used to represent the environment information on the search region.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

6.1 Introduction

Unmanned aerial vehicle (UAV) has the advantages of zero casualties, high-speed overload, good stealth performance, short operational preparation time, and relatively low life-cycle cost. These advantages increase the capability of high-risk targets penetration, suppressing enemy air defense, deep target attacking, and dominating the battlespace (Duan et al. 2013). Unmanned ground vehicle (UGV) is generally capable of operating outdoors and over a wide variety of terrain, functioning in place of humans. Multiple UAVs can be used to cover large areas searching for targets. However, sensors on UAVs are typically limited in operating airspeed and altitude, combined with attitude uncertainty, placing a lower limit on their ability to resolve and localize ground features. UGVs on the other hand can be deployed to accurately locate ground targets, but they have the disadvantage of not being able to move rapidly or see through such obstacles as buildings or fences. Therefore, multiple UAV/UGV heterogeneous cooperation provides a new breakthrough for the effective application of UAVs and UGVs (Tanner and Christodoulakis 2006; Hsieh et al. 2007; Duan and Liu 2010; Duan et al. 2010a, b). Multiple UAV and UGV heterogeneous cooperation scenario can be shown with Fig. 6.1.

Fig. 6.1

Multiple UAV and UGV heterogeneous cooperation scenario (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

For multiple UAV/UGV heterogeneous system, dynamic tracking is a typical problem. The ground moving target should be positioned dynamically and automatically, which make the target within the perception of the airborne camera (Ariyur and Fregene 2008). Trentini and Beckman (2010) summarize the two advanced research projects of about UAV rotorcraft and UGV cooperating in the battlespace currently approved for funding by Defence R&D Canada. Phan and Liu (2008) proposed a cooperative control framework for a hierarchical UAV/UGV platform for wildfire detection and fighting.

In order to realize the multiple UAV and UGV heterogeneous coordinated movement, the following control objectives should be satisfied:

• 1. The UGV subgroups should be in flocking motion.

• 2. The control center receives information from the UGV subgroups and sends the central position information to the UAV subgroups at the same time.

• 3. The subgroups of multiple UAVs should follow and hover over the subgroups of multiple UGVs stably.

The constraints of the heterogeneous movements are as follows:

• 1. The subgroups of multiple UGVs could satisfy the Reynolds rule: cohesion, separation, and alignment (Duan et al. 2010a). In this way, the flocking movement can be realized.

• 2. The subgroups of multiple UAVs could receive the changing position center information of the UGV subgroups and follow the movements of multiple UGVs.

• 3. The subgroups of multiple UAVs should avoid collisions during flight.

6.2 Multiple UAV/UGV Heterogeneous Coordinated Control

6.2.1 Mathematical Model for UAVs and UGVs

In multiple UAV/UGV heterogeneous coordinated motion, UAVs and UGVs are all considered as particles. UGVs are moving on a plane, and the status variable of UGV_i is ${x}_{\mathrm{ugv}i}={\left({x}_i,{y}_i,{\dot{x}}_i,\dot{y}\right)}^T$, i = 1, 2, …, Numugv. The emotion equation can be expressed according to the following dynamics (Duan et al. 2011):

$$ \left\{\begin{array}{c}\hfill {\dot{r}}_i={v}_i\hfill \\ {}\hfill {\dot{v}}_i={u}_i\hfill \end{array}\right. $$

(6.1)

For i = 1, 2, …, Numugv, r_i = (x_i,v_i) is the position vector of the ith UGV, ${v}_i=\left({\dot{x}}_i,{\dot{y}}_i\right)$ is the velocity vector, and ug_i = (u_xi,u_yi) is the control input. Thus, the state vector of the ith UGV can be defined as ${x}_{\mathrm{ugv}i}=\left({x}_i,{y}_i,{\dot{x}}_i,{\dot{y}}_i\right)$.

In the multiple UAV flocking motion without a leader, all the UGVs have the same status; thus, the velocity of the whole group is random when a stable and coordinated motion is achieved (Jadbabaie et al. 2003; Palejiya and Tanner 2006). In this section, a virtual UGV acting as a Virtual Leader (VL) is adopted to lead the whole UGV group to move in the right direction. The VL is a simulation of the instructions sent by the control center, and then received by each UGV. Since the VL will not be broken down, the instructions of the control center can be ensured to be executed by the UGV group. The motion equations of the VL are the same as those of the other UGVs, as shown in (6.1), and its state vector is ${x}_{vl}={\left({x}_{vl},{y}_{vl},{\dot{x}}_{vl},{\dot{y}}_{vl}\right)}^T$, where r_vl = (x_vl,y_vl) and ${v}_{vl}=\left({\dot{x}}_{vl},{\dot{y}}_{vl}\right)$.

In the heterogeneous coordinated motion, the UAV group will follow the UGV group. The state vector of the jth UAV is x_uavj = (v_j,γ_j,χ_j,x_j,y_j,z_j) for j = 1, 2, …, Numuav, and its control inputs are thrust T_j, load factor n_j, and bank angle μ_j. The following equations are adopted as the dynamic model of a UAV:

$$ \begin{array}{l}\dot{v}=g\Big[\left(T-D\right)/W- \sin \gamma \Big]\\ {}\dot{\gamma}=\left(g/v\right)\left(n \cos \mu - \cos \gamma \right)\\ {}\dot{\chi}=\left( gn \sin \mu \right)/\left(v \cos \gamma \right)\\ {}\dot{x}=v \cos \gamma \cos \chi \\ {}\dot{y}=v \cos \gamma \sin \chi \\ {}\dot{z}=-v \sin \gamma \end{array} $$

(6.2)

where v is the airspeed of the UAV; γ is the flight path angle; χ is the flight path heading; x, y, and z denote the position; g denotes the acceleration of gravity; D denotes the drag; and W denotes the weight.

6.2.2 Multiple UGV Coordinated Control Based on RHC

The initial speed and direction of each UGV in the group are different from each other, the designed controller should make the UGVs gradually meet Reynolds rule in the motion of following the VL, and the effect of multiple UGV cooperative movement must be achieved. For the ith UGV, the control input is ug_i = (u_xi,u_yi), i = 1, 2, …, Numugv. The control input of the whole UGV group can be defined as U_ugv = (ug₁, ug₂, …, ug_Numugv) = {U_ugv(t)| ∀ t ∈ [0,T]}, and the state of the whole UGV group is ${X}_{ugv}=\left({x}_{ugv1},{x}_{ugv2},\dots, {x}_{ugv Numugv}\right)\in {\mathfrak{R}}^{4\times Numugv}$. Thus, the multiple UGV motion equations can be described as

$$ {\dot{X}}_{ugv}(t)=f\left(t,{X}_{ugv}(t),{U}_{ugv}(t)\right) $$

(6.3)

Let X_ugv(0) = X_ougv represent the initial state of the UGV group, then the state at any time t ∈ (0, T] can be determined by the following equation:

$$ {X}_{ugv}(t)={X}_{ugv}(0)+{\displaystyle {\int}_{\!\!\!0}^{t}f\left(\tau, {X}_{ugv}\left(\tau \right),{U}_{ugv}\left(\tau \right)\right) d\tau} $$

(6.4)

If the initial states are definite, X_ugv(t) can only be obtained by U_ugv, which can be also expressed by X_ugv(t|U_ugv).

In the research on flocking conducted by Tanner and Olfati-Saber (Tanner et al. 2003; Olfati-Saber 2006), the cohesion and separation rules of Reynolds are satisfied by designing a proper artificial potential field. Each UGV matches its velocity to its neighbors, and the alignment rule can be fully satisfied. For the flocking motion following a VL, the control input of each UGV (u_i, i = 1, 2, …, Numugv) will include an additional part to coordinate its position and velocity with the VL.

There are complexities and diversities of control objectives in the movement of multiple UGV heterogeneous movement, which orient the optimal control strategy from the unconstrained quadratic optimization problem to multi-objective optimization problem. RHC has been proved to be more successfully optimized online in a dynamic environment, which is based on the simple idea of repetitive solution of an optimal control problem and state updating after the first input of the optimal command sequence (Zhang et al. 2010; Zhang and Duan 2012). The main idea of RHC is the online receding/moving optimization. It breaks the global control problem into several local optimization problems of smaller sizes, which can significantly decrease the computing complexity and computational expense. Particle swarm optimization (PSO) is a population-based stochastic optimization technique, which is inspired by the social behavior of bird flocking or fish schooling. It is demonstrated that PSO can find better results in a faster, cheaper way compared with other methods. Hybrid RHC and PSO approach is developed for multiple UGV movement in this section.

Consider searching space with n dimensions, the particle population is m, and the position of the ith particle can be expressed with X_i = (x_i1, x_i2, ⋯, x_in), and the velocity is V_i = (v_i1,v_i2, ⋯,v_in). The current best solution is P_i = (p_i1, p_i2, ⋯, p_in), and the global-best solution is P_g = (p_g1, p_g2, ⋯, p_gn). For the tth generation, the position and velocity updating rule can be expressed as follows:

$$ \begin{array}{l}{V}_i\left(t+1\right)=\chi \cdotp \left({V}_i\right(t\left)+{c}_1\cdotp {r}_1\cdotp \right({P}_i(t)-{X}_i(t)\left)+{c}_2\cdotp {r}_2\cdotp \right({P}_g(t)-{X}_i(t)\left)\right)\kern2em \\ {}{X}_i\left(t+1\right)={X}_i(t)+{V}_i\left(t+1\right)\end{array} $$

(6.5)

where r₁ and r₂ are two independent random numbers between (0,1), c₁ and c₂ are two learning factors, and χ is a constant number between (0,1). The object function for RHC can be expressed as follows:

$$ \begin{array}{l}\begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill & \hfill \begin{array}{cc}\hfill \underset{u}{ \min }J=f\left({X}_{ugv},{U}_{ugv};{t}_c,{T}_p\right)\hfill & \hfill \hfill \end{array}\hfill \end{array}\\ {}\begin{array}{cc}\hfill \hfill & \hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill \end{array}J={\displaystyle {\int}_{t_c}^{t_c+{T}_p}F\left({X}_{ugv},{U}_{ugv}\right) dt}\\ {}\begin{array}{ccc}\hfill subject\hfill & \hfill to\hfill & \hfill {\dot{X}}_{ugv}=f\left(t,{X}_{ugv},{U}_{ugv}\right)\hfill \end{array}\\ {}\begin{array}{ccc}\hfill \hfill & \hfill \hfill & \hfill \begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill L{L}_{ugv}\le \left[\begin{array}{c}\hfill {X}_{ugv}\hfill \\ {}\hfill {U}_{ugv}\hfill \end{array}\right]\le U{L}_{ugv}\hfill \end{array}\hfill & \hfill \hfill \end{array}\hfill \end{array}\end{array} $$

(6.6)

where t_c is control horizon, T_p is predictive horizon, t_c ≤ T_p, and LL_ugv and UL_ugv denote the upper and lower bound, respectively.

The topology of the wireless network connecting the UGVs is an adjacency graph G = {V,E}. The set of vertices V = {n₁,n₂, …,n_Numugv} represent the UGVs, and the set of edges E = {(n_i,n_j) ∈ V × V|n_i ∼ n_j} represent the adjacency relation between the UGVs. Let A = (a_ij) denote the adjacency matrix of G, then a_ij ≠ 0 ⇔ (i,j) ∈ E, and A is symmetric, A^T = A. Let N_i denote the set of the UGVs that are adjacent to the ith UGV:

$$ {N}_i=\left\{j\in V:{a}_{ij}\ne 0\right\}=\left\{j\in V:\left(i,j\right)\in E\right\} $$

(6.7)

Let R_ugv represent the maximum detecting range of a UGV, and R_ugv > 0; thus N_i can be described as

$$ {N}_i=\left\{j\in V:\Vert {r}_i-{r}_j\Vert \le {R}_{ugv}\right\} $$

(6.8)

where || · || is the Euclidean norm. Then, for R_ugv > 0, the set of edges E can be described as

$$ E=\left\{\left(i,j\right)\in V\times V:\Vert {r}_i-{r}_j\Vert <{R}_{ugv},i\ne j\right\} $$

(6.9)

• (1) Potential Cost

A potential field is established between UGVs in order that the cohesion and separation rules can be satisfied. The potential function between UGV i and UGV j is V_ij (see Fig. 6.2). V_ij is a nonnegative, differentiable, and unbounded function of the distance ${\overline{\nu}}_{ci}^{t_k}$. V_ij obtains its unique minimum when the distance ‖r_i − r_j‖ is the desired distance d_ugv.

Fig. 6.2

The curve of the potential function V_ij (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

$$ {V}_{ij}=\frac{1}{2}{\displaystyle \sum_i{\displaystyle \sum_{j\ne i}\psi \left(\Vert {r}_i-{r}_j\Vert \right)}} $$

(6.10)

In this section, we adopt the potential function introduced by Reza and define the σ-norm as follows:

$$ {\Vert x\Vert}_{\sigma }=\frac{1}{\xi}\left[\sqrt{1+\xi {\Vert x\Vert}^2}-1\right] $$

(6.11)

where the constant ξ > 0. The gradient of σ-norm can be expressed by

$$ {\sigma}_{\xi }(x)=\frac{x}{\sqrt{1+\xi {\Vert x\Vert}^2}}=\frac{x}{1+\xi {\Vert x\Vert}_{\sigma }} $$

(6.12)

According to the definition of σ-norm, V_ij can also be rewritten as V_σij:

$$ {V}_{\sigma ij}=\frac{1}{2}{\displaystyle \sum_i{\displaystyle \sum_{j\ne i}{\psi}_{\alpha}\left({\Vert {r}_i-{r}_j\Vert}_{\sigma}\right)}} $$

(6.13)

When ‖r_i − r_j‖ ≥ R_ugv, the following function of φ_a(‖r_i − r_j‖) is introduced:

$$ \begin{array}{l}{\varphi}_{\alpha}\left(\Vert {r}_i-{r}_j\Vert \right)={\rho}_h\Big(\Vert {r}_i-{r}_j\Vert /{\Vert {R}_{ugv}\Vert}_{\sigma}\Big)\varphi \left(\Vert {r}_i-{r}_j\Vert -\Vert {d}_{ugv}\Vert \right)\\ {}\begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\varphi (x)=\frac{1}{2}\left[\right(a+b\left){\sigma}_1\right(x+c\left)+\right(a-b\left)\right]\end{array} $$

(6.14)

where ${\sigma}_1(x)=x/\sqrt{1+{x}^2}$. a, b, c satisfy b ≥ a > 0, $c=\left|a-b\right|/\sqrt{4 ab}$, and φ(0) = 0.

$$ {\rho}_h(x)=\left\{\begin{array}{l} \begin{array}{cc} 1 & \end{array}\hfill x\in \left[0,h\right) \\ {} \begin{array}{cc} \frac{1}{2}\left[1+ \cos \left(\pi \frac{\left(x-h\right)}{\left(1-h\right)}\right)\right] &x\in \left[h,1\right] \end{array} \\ {} \begin{array}{cc} 0 &\end{array} \hfill {\it otherwise} \end{array}\right. $$

(6.15)

where h ∈ (0,1).

ψ _a(‖r_i − r_j‖) and φ_a(‖r_i − r_j‖) satisfy the following equation:

$$ {\psi}_{\alpha}\left(\Vert {r}_i-{r}_j\Vert \right)={\displaystyle {\int}_{\Vert {d}_{ugv}\Vert}^{\Vert {r}_i-{r}_j\Vert }{\varphi}_{\alpha }(s) ds} $$

(6.16)

Then, the collective potential cost of the whole UGV group can be described as

$$ {F}_{\it potential}={\displaystyle \sum_{i=1}^{\it Numugv}{\displaystyle \sum_{j\in {N}_i}{\varphi}_{\alpha}\left({\Vert {r}_i-{r}_j\Vert}_{\sigma}\right){n}_{ij}}} $$

(6.17)

where n_ij = σ_ξ(r_j − r_i).

The UGVs will maneuver to lower the collective potential, until the group converges to a stable and coordinated flocking motion, which has the lowest collective potential.

• (2) Consensus Cost

Each UGV will match its velocity with its neighboring flockmate to satisfy the alignment rule. The consensus cost is defined as

$$ {F}_{\it consensus}={\displaystyle \sum_{i=1}^{\it Numugv}{\displaystyle \sum_{j\in {N}_i}\left|{a}_{ij}(r)\left({v}_j-{v}_i\right)\right|}} $$

(6.18)

where a_ij(r) in the adjacent matrix A can be obtained by

$$ {a}_{ij}(r)={\rho}_h\left({{\Vert {r}_j-{r}_i\Vert}_{\sigma }/\Vert {R}_{ugv}\Vert}_{\sigma}\right)\in \left[0,1\right]\begin{array}{cc}\hfill \hfill & \hfill j\ne i\hfill \end{array} $$

(6.19)

When the multiple UGVs match their velocities with neighbors, the consensus cost of the whole group will be lowered. When a stable flocking motion is achieved, the consensus cost F_consensus is close to zero.

• (3) Following Cost

All the UGVs should regulate their motions to follow the VL; thus, the following cost is defined as

$$ {F}_{\it follow}={\displaystyle \sum_{i=1}^{\it Numugv}\left|{c}_1\left({r}_i-{r}_{vl}\right)\right|+\left|{c}_2\left({v}_i-{v}_{vl}\right)\right|}\begin{array}{cc}\hfill \hfill & \hfill {c}_1,{c}_2>0\hfill \end{array} $$

(6.20)

In the multiple UGV flocking motion, each UGV follows a VL, and the UGVs will regulate their velocity according to the position and velocity of the VL to lower the following cost.

Finally, the cost function of RHC can be described as

$$ F\left({X}_{ugv},{U}_{ugv}\right)={F}_{potential}+{F}_{consensus}+{F}_{follow} $$

(6.21)

This cost function will be used as the total objective function, which can be optimized by PSO algorithm. The solution is the optimal control input of each UGV, which will lower the cost value of the whole group gradually and lead to a coordinated flocking motion.

6.2.3 Multiple UAV Coordinated Control Based on Velocity Vector Control

According to the heterogeneous mission requirements, the multiple UAV subgroups need stability in the movement to follow and hover over the multiple UGVs, and each UAV should avoid collision during flight. In this way, the multiple UAV and UGV heterogeneous coordinated movement is formed.

According to the control input ua_i = (T_i,n_i,μ_i), where i = 1, 2, …, Numuav, T_i denotes the thrust of UAV_i, n_i denotes the overload, and μ_i denotes the banking angle. The input control vector can be expressed with U_uav = (ua₁, ua₂, …, ua_Numuav) = {U_uav(t)| ∀ t ∈ [0,T]}, and the state vector can be defined as X_uav = (x_uav1,  x_uav2, … x_uavNumvua) ∈ R^{6 × Numuav}. Then, the dynamics for UAVs can be written as follows:

$$ {\dot{X}}_{uav}(t)=f\left(t,{X}_{uav}(t),{U}_{uav}(t)\right) $$

(6.22)

The control policy of changing the control input of U_uav into velocity vector U_uav can ensure that all agents eventually align with each other and have a common heading direction while at the same time avoid collisions and group into a tight formation (Gowtham and Kumar 2005), and ${\overline{\boldsymbol{v}}}_c=\left({\overline{\boldsymbol{v}}}_{c1},{\overline{\boldsymbol{v}}}_{c2},\dots, {\overline{\boldsymbol{v}}}_{cNumuav}\right)$. The velocity vector ${\overline{\boldsymbol{v}}}_{ci}$ of UAV_i includes velocity v_ci, banking angle γ_ci, and yaw angle χ_ci, i.e., ${\overline{\boldsymbol{v}}}_{ci}=\left({v}_{ci},{\gamma}_{ci},{\chi}_{ci}\right)$. Suppose the velocity vector satisfies the following equations:

$$ \begin{array}[b]{l}{\dot{v}}_{ci}={\omega}_v\left({v}_{ci}-{v}_i\right)\\ {}{\dot{\gamma}}_{ci}={\omega}_r\left({\gamma}_{ci}-{\gamma}_i\right)\\ {}{\dot{\chi}}_{ci}={\omega}_{\chi}\left({\chi}_{ci}-{\chi}_i\right)\end{array} \vspace*{-2pt}$$

(6.23)

where ω_v, ω_γ, and ω_χ are gain constants corresponding to velocity, banking angle, and yaw angle, respectively.

According to (6.2) and (6.23), the thrust T_ci can be obtained as the following:

$$ {T}_{ci}={D}_i+{\omega}_v{W}_i\left({v}_{ci}-{v}_i\right)/g+{W}_i \sin {\gamma}_i $$

(6.24)

The overload n_ci can be expressed with

$$ {n}_{ci}=\sqrt{{\left({\omega}_{\gamma }{v}_i\left({\gamma}_{ci}-{\gamma}_i\right)+ \cos {\gamma}_i\right)}^2+{\left({\omega}_{\chi }{v}_i\left({\chi}_{ci}-{\chi}_i\right) \cos {\gamma}_i/g\right)}^2} $$

(6.25)

The pitch angle μ_ci can be expressed with

$$ {\mu}_{ci}= \arctan \left(\frac{\omega_{\chi }{v}_i\left({\chi}_{ci}-{\chi}_i\right) \cos {\gamma}_i/g}{\omega_{\gamma }{v}_i\left({\gamma}_{ci}-{\gamma}_i\right)+ \cos {\gamma}_i}\right) $$

(6.26)

The resistance D_i can be obtained according to the following equation:

$$ {D}_i=0.5{v}_i{}^2S{C}_{D0}+2k{n}^2{W}_i{}^2/\left(\rho {v}_i{}^2S\right) $$

(6.27)

where S denotes the reference square of the UAV, C_D0 denotes zero lift drag coefficient, k denotes induced drag coefficient, and ρ denotes the density of atmosphere.

The corresponding velocity vector can be expressed as the following:

$$ {\overline{\boldsymbol{v}}}_{ci}={c}_a{\overline{\nu}}_{ai}+{c}_{tc}{\overline{\nu}}_{tc i} $$

(6.28)

where ${\overline{\nu}}_{ai}$ and ${\overline{\nu}}_{tci}$ denote collision avoidance vector and hovering velocity vector, respectively, i = 1, 2 …, Numuav, c_a and c_tc are the corresponding weight coefficients, and 0 < c_a < 1, 0 < c_tc < 1, c_a + c_tc = 1.

• (1) Collision Avoidance Velocity Vector

Multiple UAVs hover over multiple UGVs, and collision should be avoided to ensure safe flight of UAVs. The collision avoidance strategy of priority mechanism is adopted in this section. The priority number level is assigned to each UAV in the multiple UAV groups, and the small number with high priority and the UAVi with low priority can avoid the UAV_j(j < i) with high priority. The collision avoidance velocity vector ${\overline{\nu}}_{ai}$ of the UAV_i with low priority can be calculated by the average velocity of UAVi and UAVj, and the direction of ${\overline{\nu}}_{ai}$ is pointing to UAV_i along the UAV_j (see Fig. 6.3).

Fig. 6.3

The collision avoidance velocity vector ${\overline{\nu}}_{ai}$ of UAV_i (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

The weight coefficient c_a of collision avoidance is decided by the distance da_ij between two UAVs and the security collision distance d_avoid, which can be expressed by

$$ {c}_a= \exp \left(\left|d{a}_{ij}-{d}_{avoid}\right|/{\sigma}_a\right) $$

(6.29)

where σ_a > 0 and 0 < c_a < 1.

• (2) Track Hovering Velocity Vector

Multiple UAV subgroup can receive messages from the control center and position the center of UGV subgroups. Suppose the minimum hovering velocity of the UAV is v_min, the hovering velocity is ω_circle, and the minimum hovering radius can be expressed by r_circle = v_min/ω_circle. The track hovering velocity vector depends on the distance dtc_i between UAV_i and the center C in the horizontal direction.

• Case 1: When dtc_i > 3r_circle, UAV_i is far away from multiple UGV subgroups, the vector ${\overline{\nu}}_{tci}$ may maintain the current velocity value or increase a little. The direction points to the center C of multiple UGV subgroups (see Fig. 6.4, marked with ★).

  Fig. 6.4

  

  Multiple UAV subgroups hovering over the center C of multiple UGV subgroups (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

• Case 2: When dtc_i ≤ 3r_circle, UAV_i hovers in the vicinity of multiple UAVs with ${\overline{\nu}}_{tci}$. When the direction and speed of UAV_i are the same with the multiple UGV subgroups, we can determine whether UAV_i follows the center C. If yes, then UAV_i continues to hover in the vicinity of multiple UAVs. Otherwise, UAV_i will follow the center C. In the following process, UAV_i maintains the same value or increases a little, while the direction keeps unanimous with the multiple UGVs (see Fig. 6.4). With ${\overline{\nu}}_{ai}$ and ${\overline{\nu}}_{tci}$, the velocity vector ${\overline{\boldsymbol{v}}}_{ci}$ of UAVi can be obtained by (6.28), and the vector command group ${\overline{\nu}}_c$ of multiple UAV subgroup can also be obtained.

Multiple UAV and UGV heterogeneous cooperation process can be illustrated by Fig. 6.5.

Fig. 6.5

Flowchart of multiple UAV and UGV heterogeneous cooperation (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

6.2.4 Multiple UAV/UGV Heterogeneous Cooperation

The feasibility and effectiveness of our proposed method are verified by series of comparative experiments with artificial potential field method. In the experiments, there are 6 UAVs, 10 UGVs, and 1 control center. The initialized parameters of multiple UGV subgroup are set as follows: d_ugv = 15, R_ugv = 1.2d_ugv, ξ = 0, a = 5, b = 5, h = 0.9, c₁ = 1, c₂ = 1, T_p = 3s, δ = 1s, t_c = 1s, ps = 20, w_{max = 1.2}, w_min = 0.1, vp_max = 4, pc₁ = 0.5, pc₂ = 0.5, Nc_max = 80.

The initialized parameters of multiple UAV subgroup are set as follows: ρ = 1.25kg/m³, W = 14,400 kg, the reference square = 30 m², T_max = 15,000 kg, n_max = 7, k = 0.1, C_D0 = 0.02, ω_v = 1, ω_γ = 0.2, ω_χ = 1, g = 9.8 m/s². v_min = 100 m/s, v_max = 200 m/s, ω_circle =(π/12)rad/s, ω_χ max =(π/9)rad/s, ω_γ max =(π/9)rad/s. d_avoid = 30 m, σ_a = 10.

The initialized position of VL in multiple UGV subgroup is (1,020,1,020)m, and the initialized velocity is 25 m/s. The initialized status of multiple UGV subgroup and VL are shown with Fig. 6.6 (“■” denotes VL).

Fig. 6.6

The initialized status of multiple UGV subgroup and VL (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

Table 6.1 The initialized status of multiple UAV subgroup

The initialized status of multiple UAV subgroup is listed by Table 6.1, and the initialized status of multiple UAV subgroup is shown with Fig. 6.7 (“•” denotes UAV).

Fig. 6.7

The initialized status of multiple UAV subgroup (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

Figure 6.8 gives the multiple UAV and UGV heterogeneous cooperation results by using artificial potential field method.

Fig. 6.8

Multiple UAV and UGV heterogeneous cooperation results by using artificial potential field method. (a) Traces of multiple UAVs and UGVs heterogeneous cooperation. (b) Traces of multiple UGV subgroup before 150 s. (c) Traces of multiple UAV subgroup before 150 s (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

Figure 6.9 gives the multiple UAV and UGV heterogeneous cooperation results by the hybrid method proposed in this paper.

Fig. 6.9

Multiple UAV and UGV heterogeneous cooperation results by the hybrid method proposed in this paper. (a) Traces of multiple UAVs and UGVs heterogeneous cooperation. (b) Traces of multiple UGV subgroup following VL before 150 s. (c) Traces of multiple UAV subgroup before 150 s (Reprinted from Duan et al. (2011), with kind permission from Springer Science + Business Media)

The results in Figs. 6.8 and 6.9 demonstrate that the proposed approach in this paper can guarantee stable convergence, robust tracking, and high efficiency. It clearly shows the superiority of the proposed algorithm over the traditional artificial potential field method. Simulations with different conditions are also conducted to verify the feasibility and effectiveness of the proposed controller.

Besides, experiments about heterogeneous coordinated control for multiple UAVs/UGVs have been conducted by applying a low-cost quadrotor and three ground vehicles. The red vehicle acts as the target. The quadrotor and the other two vehicles aim at pursuing the red vehicle by complementing each other’s advantages. As we have explained, UAV can be used to cover large areas searching for target while sensors on UAV are typically limited in operating airspeed and altitude. UGV can be deployed to accurately locate ground targets. Screenshots of the experiment video are illustrated in Fig. 6.10.

Fig. 6.10

Screenshots of the experiment about heterogeneous coordinated control for multiple UAVs/UGVs

6.2.5 Time-Delay Compensation of Heterogeneous Network Control

In recent years, network-based control has emerged as a topic of significant interest in the control community. It is well known that in many practical systems, the physical plant, controller, sensor, and actuator are difficult to be located at the same place, and thus signals are required to be transmitted from one place to another. The network- induced time delay in network control system (NCS) occurs when sensors, actuators, and controllers exchange data across the networks. This delay can degrade the performance of control systems designed without considering it and even destabilize the system.

The use of multiple UAVs in concert with UGVs affords a number of synergies. First, UAVs with cameras and other sensors can obtain views of the environment that are complementary to views that can be obtained by cameras on UGVs. Second, UAVs carry over obstacles while keeping UGVs in their field of view, providing a global perspective, and monitoring the positions of UGVs while keeping track of the goal target. This is especially advantageous in three dimensions where UAVs can obtain global maps and the coordination of UAVs and UGVs can enable efficient solutions to the mapping problem. Third, if UAVs can see the UGVs and the UGVs can see UAVs, the resulting three-dimensional sensor network can be used to solve the simultaneous localization and mapping problem, while being robust to failures in sensors like GPS and to errors in dead reckoning. We describe our work in time-delay compensation of heterogeneous network control for multiple UAVs and UGVs.

Suppose the sampling period for the multiple UAVs and UGVs is T, and the maximum time delay is n_τT, where n_τ is an integer and n_τ > 1. The control sequences for UGV_i can be denoted with

$$ U{g}_i^{t_k}=\left(u{g}_i^{t_k},u{g}_i^{t_{k+1}},\dots, u{g}_i^{t_{k+ Tp-1}}\right) $$

(6.30)

where $u{g}_i^{t_k},u{g}_i^{t_{k+1}},\dots, u{g}_i^{t_{k+ Tp-1}}$ is the input sequence of UGV_i. The predictive control sequences for all the UGVs at time t_k is

$$ {U}_{ugv}^{t_k}=\left(U{g}_1^{t_k},U{g}_2^{t_k},\dots, U{g}_{Numugv}^{t_k}\right) $$

(6.31)

Due to the time delay, we can obtain the motion equation for UGV:

$$ {\dot{X}}_{ugv}(t)=f\left(t,{X}_{ugv}(t),{U}_{ugv}^{t_{k-n}}(t)\right) $$

(6.32)

where 0 ≤ n ≤ n_τ and $n\in \mathfrak{R}$.

In the absence of the presence of network delay, UGV_i only uses $u{g}_i^{t_k}$, and $\left(u{g}_i^{t_{k+1}},\dots, u{g}_i^{t_{k+ Tp-1}}\right)$ is abandoned. The new control input is obtained in the next iteration. While in the case of random long-period time delay, UGV_i may not receive the control input at the moment t_cur, and multiple UGVs can hardly meet the requirements of cooperative motion. In this case, the predictive control sequences can be all sent to the UGVs and are stored in various UGVs.

Receding horizon control (RHC) and PSO are adopted in this approach. The objective function can be defined as

$$ \begin{array}{l}\begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill & \hfill \begin{array}{cc}\hfill \underset{u}{ \min }J=f\left({X}_{ugv}^{t_{k-{n}_1}},{U}_{ugv};{t}_c,{T}_p\right)\hfill & \hfill \hfill \end{array}\hfill \end{array}\\ {}\begin{array}{cc}\hfill \hfill & \hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill \end{array}J={\displaystyle {\int}_{t_c}^{t_c+{T}_p}F\left({X}_{ugv}^{t_{k-{n}_1}},{U}_{ugv}\right) dt}\\ {}\begin{array}{ccc}\hfill subject\hfill & \hfill to\hfill & \hfill {\dot{X}}_{ugv}^{t_{k-{n}_1}}=f\left(t,{X}_{ugv}^{t_{k-{n}_1}},{U}_{ugv}\right)\hfill \end{array}\\ {}\begin{array}{ccc}\hfill \hfill & \hfill \hfill & \hfill \begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill L{L}_{ugv}\le \left[\begin{array}{c}\hfill {X}_{ugv}^{t_{k-{n}_1}}\hfill \\ {}\hfill {U}_{ugv}\hfill \end{array}\right]\le U{L}_{ugv}\hfill \end{array}\hfill & \hfill \hfill \end{array}\hfill \end{array}\end{array} $$

(6.33)

where 0 ≤ n₁ ≤ n_τ and ${n}_1\in \mathfrak{R}$. For multiple UAV subgroup, whose speed is much larger than UGV, there is a tracking delay between multiple UAV subgroups of multiple tracking UGV subgroups. However, UAV can be quickly followed up by tracking the spiral vector. Therefore, UGV_i can obtain the tracking hovering velocity vector ${\overline{\nu}}_{{}_{tci}}^{t_{k-n2}}$ according to the latest multiple UGV center location information, where 0 ≤ n₂ ≤ n_τ, and ${n}_2\in \mathfrak{R}$. The vector group of multiple UAV subgroups can be defined as

$$ {\overline{\boldsymbol{\nu}}}_c^{t_k}=\left({\overline{\boldsymbol{\nu}}}_{c1}^{t_{k-n1}},{\overline{\boldsymbol{\nu}}}_{c2}^{t_{k-n2}},\dots, {\overline{\boldsymbol{\nu}}}_{cNumuav}^{t_{k-{n}_{Numuav}}}\right) $$

(6.34)

The time delay of multiple UGV subgroup sending the status information to the control center can be defined with τ_gc, and the time–event-driven approach is adopted in the control center. The time delay of the control center sending the status information to each UGV can be defined with τ_gc, and the time delay of the center of multiple UGV subgroup sending the status information to UAV can be defined with τ_ca.

6.2.5.1 Status Buffer of Control Center

UGV _i(i = 1, 2, … Numugv) sends the status information to the control center respectively, and the time-driven approach is adopted in this process. Due to the existence of time delay τ_gci, the arrival time is random. So time–event-driven mode is adopted in the control center. When τ_gci > T, the control center automatically starts the control algorithm by using the status information of UGV_i.

In the status buffer of the control center, the older state information will be automatically deleted with the advance of the new status information. The updating process for status buffer of control center can be shown with Fig. 6.11. In which, t = t_k + △ t, t_k = kT, △ t ≤ T, the simulation time is denoted with Time _ length, and the output of control center is U_ugv, i = 1, 2, …, Numugv.

Fig. 6.11

The updating process for status buffer of control center

6.2.5.2 UGV Control Input Buffer

Because there is a time delay τ_cgi between control center and UGV_i, it is necessary to set UGV_i control input buffer. In this way, the control sequences can be saved. Based on the maximum network delay n_τT, the buffer length is set to n_τ. The update of UGV_i control input buffer can be divided into two parts: the time-driven update and the event-driven update. The updating process for UGV_i control input buffer at moment tk can be shown with Fig. 6.12.

Fig. 6.12

The updating process for UGV_i control input buffer at moment t_k

6.2.5.3 UAV Center Location Information Buffer

The control center sends the center location information ${C}_{{}^i}^{t_k}$ of multiple UGVs to a multiple UAV subgroup by using the time-driven approach. Due to the existence of time delay τ_cai, the time of UAV_i(i = 1, 2, …, Numuav) receiving ${C}_{{}^i}^{t_k}$ is random. However, the time-driven approach is adopted in UAV_i. When τ_cai > T, UAV_i uses the velocity vector instruction ${\overline{\boldsymbol{v}}}_{ci}$ with the center of the historical status information. The center of the updating buffer can also be divided into two parts: the time-driven update and the event-driven update. In time t_k, the older state information will be automatically deleted with the advance of the new center location information ${C}_{{}^i}^{t_k}$. The updating process for UAV_i center location information buffer at moment t_k can be shown with Fig. 6.13.

Fig. 6.13

The updating process for UAV_i center location information buffer at moment t_k

The transfer timing for multiple UAV subgroup center location information can be shown with Fig. 6.14.

Fig. 6.14

The transfer timing for multiple UAV subgroup center location information

6.3 DE-Based RHC for Multiple UAV Cooperative Search

The search problem has been extensively studied in the literature, starting off with a single-agent problem and further extended to multi-agent search. In military applications, multiple UAV coordinated search is an important means of getting battlefield information in the future war. Compared to the problem of a single searcher, the problem becomes more complex when we consider a team of agents that are cooperatively searching the targets in an area.

For the flight path-planning problem in UAV targets searching, the traditional method is based on search theory, designing search routes covering task areas from the perspective of maximizing the probability of target detection. Such routes are usually fixed pattern, such as scanning-line mode to achieve a complete coverage of the target area. This method is of simple route calculation, fast, and able to guarantee a certain probability of target detection, but the flight route is fixed and the search efficiency is low. Another important method is a dynamic search method based on the search map. The method is based on two-dimensional discrete map to store targets and environmental information. Based on search map information, different strategies for online calculation of the next time search path can be used, such as the random strategy, the local optimal strategy, and the global maximum strategies. These methods can be used in target searching effectively based on real-time detection of information. The difficulty lies in how to quickly calculate the safe search route to the next point. In this study, search map is used for the cooperative area searching of multiple UAVs.

The main concern of this study for multiple UAV search is about how to control multiple UAVs for cooperative search for ground targets. In other words, the problem of cooperation between multiple UAVs is the key for the multiple UAV search problem. Multiple UAV search problem is a complex optimization and control problem with a large amount of information in process of solution and a high dimension. Recent years, biological swarm intelligence provides a good idea for solving multi-objective UAV distributed coordinate search problem. In view of the flexibility of the intelligent optimization methods based on biological evolution and its advantages in solving high-dimensional problems, in this study, an intelligent optimization method of DE is used for the solution of the multiple UAV cooperative search problem.

Another important issue for multiple UAV cooperative search problem is the requirement of real time and security. Some researchers apply the thought of RHC into the cooperative search problem. Using the online task optimization method based on rolling window, the optimization search strategy can respond quickly for environment changes by optimizing and rolling online. In this study, RHC is used to realize the real time and security during searching process of multiple UAVs.

6.3.1 Model Description for Cooperative Search

6.3.1.1 Some Hypotheses Involving UAV Platform

UAV platform is a direct implementation of the search task and also the controlled object engaged in our study. As the study focuses on the search method in UAV area searching, but not the low-level control of UAV platform, some hypotheses are set in the study:

• The UAV platform is small tactical UAV.

• There is an automatic flight control system for each UAV platform.

• The UAV high-level mission control and the low-level flight control can be considered decoupling.

Besides, in order to reflect the physical characteristics of the UAV, a set of parameters related to the flight performance is constrained:

• Maximum cruise velocity v_max: It’s a basic performance parameter for the UAV platform, which decides the movement pattern of the UAV in the task area.

• Maximum flight height h_max: To image sensor, the flight height decides the detection range of airborne sensors directly. It affects the effects of UAVs to the target search, detection and identification.

• Maximum duration time t_max: Decided by the amount of fuel on UAV, it limits the longest time that UAV can perform the task in search area.

• Minimum turning radius R_min: The minimum turning radius describes the UAV’s mobility. Together with the velocity parameter of UAV platform, it decides the flight path in a certain input.

In control of UAV flight track point, a particle model of three degrees of freedom is considered. To facilitate follow-up studies, a discrete form of expression is established to describe the flight control model. When the total number of UAVs is Nv, for the platform of UAV_i, (k = 1, 2, …, Nv), at time k, the dynamic characteristic can be described by the motion model as follows:

$$ \left\{\begin{array}{l}{x}_i\left(k+1\right)={x}_i(k)+{v}_i(k) \cos \left({\sigma}_i\right(k\left)\right) \cos {\varphi}_i\left(\right(k\left)\right)\cdot {t}_s\\ {}{y}_i\left(k+1\right)={y}_i(k)+{v}_i(k) \cos \left({\sigma}_i\right(k\left)\right) \sin {\varphi}_i\left(\right(k\left)\right)\cdot {t}_s\\ {}{z}_i\left(k+1\right)={z}_i(k)+{v}_i(k) \sin \left({\sigma}_i\right(k\left)\right)\end{array}\right. $$

(6.35)

where t_s is the decision interval, (x_i(k), y_i(k), z_i(k)) ∈ R³ is the position of UAV_i at time k in the three-dimensional search space, ν_i(k) ∈ R is the velocity of UAV_i at time k, φ_i(k) ∈ [0, 360) is the yaw angle of UAV_i at time k, and ${\sigma}_i(k)\in \left[0,{\sigma}_{\max_i}\right)$ is the climb angle of UAV_i at time k.

6.3.1.2 Search Targets Modeling

Search target is the specific object for task of multiple UAVs. Depending on the motion state, search target can be divided into static target and dynamic target. To the static target, there are fixed radar, artillery positions, buildings, roads, bridges, and so on. To the dynamic target, it can be all kinds of vehicles, aircrafts, specific people, etc.

Besides, according to whether it can attack or not, the target can be divided into antagonistic target and no antagonistic target. To the antagonistic target, it includes artillery positions, missiles, and other offensive aircrafts and ships. For this kind of targets, the UAV should avoid entering into their scope of attacks. To the no antagonistic target, there are fixed radar, aircrafts for scout, plants, and so on.

Suppose the targets will not take the initiative to escape the search of UAVs, target elements mainly considered in this study are as follows:

• Target position state x_t: It describes the specific location of different targets.

• Target velocity v_t: It describes the target speed of movement in space.

• Target movement pattern: It describes the variation law of target position and velocity in space, including stationary state, random movement, and deterministic motion (or in a particular trajectory).

Based on the description of target elements above, in the two-dimensional plane, in case the target position is x_t = (x,y) ∈ R², then for the static target, there is x_t(k) ≡ x_t(0). Otherwise, for the dynamic target in the two-dimensional space, the direction of motion is denoted as θ_t; in case of discrete time, a simple model of target motion is usually considered:

$$ {x}_t\left(k+1\right)={x}_t(k)+\varDelta x $$

(6.36)

where Δx is the displacement increment of target; in case of deterministic motion, there is Δx = v_t × t_s; otherwise, if the target moves in a random way, the displacement increment of Δx is accordingly random. A typical random motion model is Random Tours. In this case, the target starts form the initial position, x_t(0), along with a random direction, θ_t, which obeys the uniform distribution on the interval of [0, 2π), and moves for a random time, dt, which obeys the exponential distribution with parameter d. Then the position of target x_t(k) is completely random.

6.3.1.3 Environment Information Modeling

Environment information involved in search issues always includes information of targets, other UAV’s mission state information, and information of threats. The target information is a key to the multiple UAV cooperative search problem. Here, the study focuses on the description and modeling of target in search environment. Because the environment is dynamic, the uncertainty of targets decides the search problem is essentially a random question. In this situation, the search map model based on probability is a natural choice.

• (1) Basic Search Map (BSM) Model

The basic idea of search map is to represent the environment as a grid of cells. Suppose the area is divided into L_x × L_y cells, and each cell in the map associates with a certainty information strut, P_ij(k), which describes the general information of environment and target in current cell. The information strut is defined as follows:

$$ {P}_{ij}(k)=\left({p}_{ij}(k),{\chi}_i{}_j(k)\right),i\in \left\{1,\dots, {L}_x\right\},j\in \left\{1,\dots, {L}_y\right\} $$

(6.37)

where p_ij ∈ [0,1] is called target occupancy probability (TOP) in grid (i,j) and χ_ij(k) ∈ [0,1] is environment certainty (EC). We suppose each cell contains at most one target, then p_ij(k) = 0 represents that the UAV knows nothing about the target in grid (i,j) at time step k, and p_ij(k) = 1 represents high probability that a target is present in grid (i,j). The χ_ij(k) describes the UAV’s determination extent for grid (i,j) at time step k, χ_ij(k) = 0 represents UAV knows nothing about information of grid (i,j),and χ_ij(k) = 1 represents completely knows the information of this grid.

A schematic diagram of grid partition for search map is shown in Fig. 6.15, where five gray star points represent the target position.

Fig. 6.15

Regional division based on quadrilateral grid

Generally, search map describes the UAV’s belief state for existence of target in the mission area and is the direct information that UAV can understand and apply. In cooperative area searching of multiple UAVs, each UAV obtains information of external environment through not only its own sensors but also the communication equipments. The environment information, by the way of the information be obtained, can be divided into three parts: the prior information, initial intelligence from other means of reconnaissance; the probe information which got through the sensors carried by UAV; and communication information from other UAVs. All information can be expressed on the search map. During the search mission, different UAVs share the information on one search map.

• (2) Extended Search Map (ESM) Model

Based on information of BSM, single UAV can make its own path decisions. However, the BSM describes only the uncertain information of targets and environment, but not the state information of other UAVs. For cooperative area search of multiple UAVs, cooperation between different agents is a key issue. UAVs must have ability to coordinate their actions and to maximum team search efficiency. To achieve effective multiple UAV cooperation search, the BSM is extended into the extended search map (ESM) based on the Digital Hormone Model (DHM).

The ESM is on basic of BSM and it introduces the hormone information to build a mixed information strut. Then, the information strut defined in (6.37) is extended to (6.38):

$$ {P}_{ij}\hbox{'}(k)=\left({p}_{ij}(k),{\chi}_i{}_j(k),{H}_i{}_j(k)\right),i\in \left\{1,\dots, {L}_x\right\},j\in \left\{1,\dots, {L}_y\right\} $$

(6.38)

where H_ij(k) is the digital hormone information on grid (i,j) at time k step. The concentration of hormone information is a function of UAV position and time. When UAV moves to grid (i,j) at time k, it generates hormone signal on the related position of search map, and meanwhile, the digital hormone signal is sent to UAVs nearby to impact other UAV’s decision on the next time step.

Real hormone has the ability of diffusion and dissemination. The hormone information includes two types of hormones, the activator hormone H_A and inhibitor hormone H_I. The diffusion equation of H_A and H_I are given below:

$$ \begin{array}[b]{l}\varDelta {H}_A\left(i,j,k\right)=\frac{a_A}{2\pi {\sigma}^2}{e}^{-\frac{{\left(x-a\right)}^2+{\left(y-b\right)}^2}{2{\sigma}^2}}\\ {}\varDelta {H}_I\left(i,j,k\right)=-\frac{a_I}{2\pi {\rho}^2}{e}^{-\frac{{\left(x-a\right)}^2+{\left(y-b\right)}^2}{2{\rho}^2}}\end{array} $$

(6.39)

where grid (a,b) is adjacent to grid (i,j), a_A, a_I are constants, σ and ρ are the rates of diffusion, respectively, and σ < ρ to satisfy the Turing stability condition. With the diffusion of hormone in search map through communication, we get the hormone update function at time k by summing up all hormone information from neighboring UAV, where the constant τ_H ∈ [0,1] is the rate for dissipation.

$$ {H}_i{}_j\left(t+1\right)={\tau}_H\cdot {H}_i{}_j(t)+{\displaystyle \sum_{N_k}\left(\varDelta {H}_A\left(i,j,k\right)+\varDelta {H}_I\left(i,j,k\right)\right)} $$

(6.40)

The initial hormone information on the map for each grid is zero.

6.3.2 DE-Based RHC for Cooperative Area Search

The result of UAV search problem is directly reflected on the UAV’s search path. In UAV search problem, the goal of search path planning is to generate the effective trajectory along with an objective function is maximized. In this study, we consider a reward function as the objective function, and the key issue to solve the cooperative area searching of multiple UAVs with DE algorithm is to determine the reward function.

For the cooperative search problem based on RHC, the reward function is related to each UAV’s current position X(k) and the following position of track points [X(k + 1|k), X(k + 2|k), …, X(k + p|k)], in which p is the size of control window. [X(k + 1|k), X(k + 2|k), …, X(k + p|k)] is the input of the optimization problem.

The reward function for UAV search decision can be described by a composed efficacy J. As discussed above, the composed efficacy J can be represented as

$$ J\left[X(k),X\left(k+1\Big|k\right),X\left(k+2\Big|k\right),\dots, X\left(k+p\Big|k\right)\right] $$

(6.41)

During the search process, each UAV needs to determine the following p track points according to current state and search map information. The goal of search path planning for multiple UAV search is to find the most targets, gain the information on whole search area, and reduce the uncertainty of mission area. Accordingly, the optimization decisions need to reach the following aspects of subgoals:

• 1. To maximum the probability of finding the target

• 2. Tend to detect those areas with more uncertainties

• 3. To realize effective cooperation of multiple UAVs

• 4. To minimum the cost during the search process

Based on the subgoals introduced above, the composed efficacy at time k step, J(k), can be defined as

$$ J(k)={\omega}_1\cdot {J}_T(k)+{\omega}_2\cdot {J}_F(k)+{\omega}_3\cdot {J}_C(k)-{\omega}_4\cdot C(k) $$

(6.42)

where ω₁, ω₂, ω₃, and ω₄ are corresponding weight; J_T(k), J_F(k), and J_C(k) are, at time k step, three different rewards related to the subgoals introduced above; and C(k) is the search cost at time k step. Based on the ESM information, the definitions of three rewards and the cost are given below:

• (1) Target Finding Reward J_T(k)

The target finding reward describes the possibility of finding targets along the way. During optimization of the UAV path with DE, a number of alternative following track points will be considered at first. However, the algorithm tends to choose the path that reaches the biggest target finding reward as the real path that UAV will fly by.

Suppose for UAV_i(i = 1, 2, …, N_v), the whole range of sensor detection during time [k, k + p − 1] is R_i, the target finding reward J_T(k) at time k can be defined as

$$ {J}_T(k)={\displaystyle \sum_{i=1}^{N_V}{\displaystyle \sum_{\left(m,n\right)\in {R}_i}{p}_{mn}{}^i(k)}} $$

(6.43)

where p_mnⁱ(k) is the probability of target existence in UAV_i’s detection scope R_i, related only to the position of UAV_i on search map.

• (2) Reward of Expected to Detect J_F(k)

The search decision tends to make UAV detect the area with small Environment Certainty. The path with smaller EC and bigger probability of target existence gains a bigger reward. The reward of J_F(k) can be calculated by the following equation:

$$ {J}_F(k)={\displaystyle \sum_{i=1}^{N_V}{\displaystyle \sum_{\left(m,n\right)\in {R}_i}\left(1-{\chi}_{mn}{}^i(k)\left){p}_{mn}{}^i\right(k\right)}} $$

(6.44)

where χ_mnⁱ(k) is the EC of UAV_i in its detection scope R_i on search map.

• (3) Cooperation Reward J_C(k)

The cooperative of multiple UAVs can avoid excessive repetition detection on a certain area; on the other hand, it can also reduce the risk of collision and ensure the safety of multiple UAV missions. Accordingly, the cooperative reward can be defined as

$$ {J}_{C1}(k)={\displaystyle \sum_{i=1}^{N_V}{\displaystyle \sum_{n=0}^{p-1}\left[H\left({x}_i\left(k+n\right)\right)-H\left({x}_i\left(k+n+1\right)\right)\right]}} $$

(6.45)

where H(x_i(k + n)) represents the hormone information on position of UAV_i at track point x_i(k + n).

The other definition is about the overlap degree of tracks between two different UAVs:

$$ {J}_{C2}(k)={\displaystyle \sum_{i=1}^{N_V}{\displaystyle \sum_{j=1}^{N_V}{\displaystyle \sum_{n=0}^{p-1}{f}_o\left({\theta}_{ij}{}^p,{d}_{ij}{}^p\right)}}} $$

(6.46)

where θ_ij^p is the heading angle difference between UAV_i and UAV_j on their pth track points and d_ij^p is accordingly the distance between the two UAV’s pth track points. Usually, function f_o can be defined as

$$ {f}_o\left({\theta}_{ij}{}^p,{d}_{ij}{}^p\right)={ \exp}^{r_0\cdot {d}_{ij}{}^p\cdot \cos \left({\theta}_{ij}{}^p/2\right)} $$

(6.47)

where r₀ ∈ R⁺ is adjustable parameter.

The composed cooperation reward can be represented as follows:

$$ {J}_C(k)={J}_{C1}(k)+{\alpha}_C\cdot {J}_{C2}(k) $$

(6.48)

where α_C ∈ R⁺ is adjustable parameter.

• (4) Search Cost C(k)

Search cost is the comprehensive cost during process of multiple UAV search mission. It generally performs to be the time-consuming or the fuel consuming in search process. The following equation gives a certain estimate method for search cost:

$$ C(k)={\displaystyle \sum_{i=1}^{N_V}{\displaystyle \sum_{n=0}^{p-1}\Vert {x}_v{}^i\left(k+n\right)-{x}_v{}^i\left(k+n+1\right)\Vert }}/{v}_i\left(k+n\right) $$

(6.49)

where x_vⁱ(k + n) and x_vⁱ(k + n + 1) are adjacent two track points on track path of UAV_i and v_i(k + n) is the velocity between the two track points.

6.3.3 Experiments

A simulation study is included to illustrate the feasibility of our proposed method for cooperative search for multiple UAVs. The simulation scenario consists of a team of four UAVs searching a 100 × 100 (50 × 50 km) cellular environment with five targets and different kinds of threats. The threats are mainly composed of dangerous terrains and enemy threats, which can be shown by the search map in Fig. 6.16. In the search map, M1 represents the mountain, Bw1 denotes the bad weather area, and Fd1 is the forbidden fly area, which are prior information for the UAVs. It is assumed that there is some minor a priori topographical information but no other sources of information on target distribution. The initial distribution of five targets is shown in Fig. 6.16. The target information is shown in Table 6.2. The objective of UAVs is to search the environment so that they can incrementally obtain knowledge of the environment and locate targets with capability of threat avoidance. Four UAVs are initially located at four corners of the search region. For each UAV, the maximum cruise velocity is 0.1 km/s, minimum turning radius is 2 km, and the diameter of detection region for the sensor is 2 km. The search result of our experiment can be shown in Figs. 6.17 and 6.18.

Fig. 6.16

Initial information of search region

Table 6.2 Information of search targets

Fig. 6.17

Search result using our proposed method (2 dimensional)

Fig. 6.18

Search result using our proposed method (3 dimensional)

6.4 Conclusions

Multiple UAV/UGV heterogeneous cooperation provides a new breakthrough for the effective application of UAVs and UGVs. On the basis of introduction of UAV/UGV mathematical model, the characteristics of heterogeneous flocking is analyzed in detail. Two key issues are considered in multiple UGV subgroups, which are Reynolds rule and VL. RHC with PSO is proposed for multiple UGV flocking, and velocity vector control approach is adopted for multiple UAV flocking. Then, multiple UAV and UGV heterogeneous tracking can be achieved by these two approaches. The feasibility and effectiveness of our proposed method are verified by comparative experiments with artificial potential field method. Besides, we describe a time-delay compensation approach of heterogeneous network control for multiple UAVs and UGVs. The detailed updating process for status buffer of control center, UGV control input buffer, and UAV center location information buffer are also presented.

In Sect. 6.3, a DE-based RHC controller for cooperative area searching of multiple UAVs is presented. The thought of RHC in adopted to satisfy the real-time requirements. Then, the cooperative search problem can be formulated into a function, which is about designing search routes covering task areas from the perspective of maximizing the probability of target detection. Furthermore, an extended search map is used to describe the environment information on the search region. Simulation results demonstrated that the approach we proposed for area searching problem of multiple UAVs is feasible and also effective.