Research on Control Algorithm of Unmanned Vessel Formation Based on Multi-agent

Abstract

In order to improve the accuracy of the formation control of multiple unmanned vessels, reduce the construction and maintenance costs and the impact of the external environment, and improve the robustness of the formation control method. Based on the hierarchical formation control technology of the navigator method, the formation design and behavior control are carried out respectively. After the formation is formed, the artificial potential field method is used to maintain, and the gravitational field and repulsive field of the virtual navigator and other unmanned ships and the maximum communication distance are considered respectively, and an improved virtual navigator formation control algorithm is obtained. Finally, we conducted formation control experiments. The experimental results show that this paper uses the proposed improved virtual navigator algorithm to conduct formation control and obstacle avoidance experiments, and find that the algorithm has good performance. The formation control method has the characteristics of high accuracy, strong generalization ability, fast control speed, and strong environmental adaptability. It meets the needs of actual formation control of multiple unmanned ships, and the formation control method can be packaged into an intelligent control system. It has the characteristics of flexible deployment and application, strong adaptability, and has the application space for formation control in actual combat with multiple unmanned ships.

Supported by organizations of the National Natural Science Foundation of China (61572228 and 61572226),the Key Development Project of Jilin Province (20180201045GX and 20180201067GX),the Guangdong Key-Project for Applied Fundamental Research (2018KZDXM076),and the Guangdong Premier Key-Discipline Enhancement Scheme (2016GDYSZDXK036).

1 Introduction

The multi-agent system is composed of a series of interacting agents. The internal agents communicate, cooperate, and compete with each other to complete a large number of and complex tasks that a single agent cannot complete. Therefore, how to quickly, accurately and effectively control the formation of multiple agents to complete complex and important tasks is an urgent problem to be solved [1]. At present, with the in-depth research of various countries, unmanned ships have become more and more intelligent, capable of completing some tasks that are difficult for humans to complete, and are becoming more and more popular with scientists.

In view of the problem that a single unmanned ship cannot effectively complete complex tasks, the system needs to control multiple agents in formation. At present, domestic research on formation control is mainly at the stage of theoretical simulation. Shenyang Institute of Automation, Harbin Engineering University, China Shipbuilding Industry Corporation and other units have a good research foundation in this field. Aiming at the problem of unstable formation of multiple unmanned ships due to positioning accuracy, a pilot-following formation control strategy combined with extended Kalman filtering is proposed to improve positioning accuracy and form a stable formation [2]. Aiming at the unmanned ship formation system based on the multi-agent collaborative model, Lu Yu proposed a multi-vessel distributed robust adaptive formation zoom control method based on azimuth, which can solve the unmanned ship formation under the input saturation constraints and uncertain conditions. Cluster formation zoom maneuver control problem [3]. Aiming at the problem of under-driven unmanned ship formation control with uncertain model parameters, unknown marine environmental disturbances and unknown input gains, Liu Yunjie proposed a single-path guided distributed time-varying formation control method with obstacle avoidance and connectivity maintenance functions [4]. Aiming at the unmanned ship pilot-following formation system, Li He adopts a cooperative trajectory tracking control method and introduces a fixed-time control algorithm to improve the tracking control convergence performance of the unmanned ship formation system; consider unmanned vehicles with unknown environmental disturbances and unmodeled dynamics For the tracking control problem of the ship formation system, by designing a limited time precise disturbance observer, the unknown information identification and compensation in the complex environment can be realized, and the rapid and accurate tracking control of the unmanned ship pilot-following the formation can be ensured [5]. Zheng Shuai proposed the MADDPG algorithm to solve the coordination problem in the tracking process, design the local environment state space, the action space and the global reward function, and train the algorithm based on the multi-ship rescue two-dimensional plane scene, and obtain the cooperative rescue strategy model [6]. Shen Wei analyzes the mathematical model of unmanned ships, and establishes a control model of multiple unmanned ships that lead and follow. After that, the formation controller was designed using the anti-step method to make each unmanned ship reach the predetermined speed and heading. At the same time, in response to the problem of unstable formation caused by low positioning accuracy in practical applications, a position estimation algorithm based on EKF technology under the same model is proposed [7]. Liu Chang, Du Wencai, and Ren Jia proposed a distributed topology control algorithm based on directional antennas in order to solve the problem of directivity constraints of directional antennas in the formation of unmanned ships. Fast network connection. Simulation experiments prove that the algorithm can maintain the network connectivity performance at a lower energy consumption cost [8]. Liu Chang proposed an improved topology control algorithm based on directional antennas. This algorithm considers the energy consumption problem of unmanned ship formation, in order to use the minimum power consumption to make it form a connected network more quickly. The experimental results show that the algorithm effectively solves the problems that arise in the topological structure of the unmanned ship formation in a dynamic situation, and reduces the energy consumption [9]. Xiang Xianbo uses a microcontroller to receive data from the GPS sensor to obtain latitude and longitude information, and to receive data from the attitude sensor to obtain heading angle information. The actuator adjusts the heading angle and speed of the unmanned ship in real time. Under the polling control of the host as the master node, multiple unmanned ships form a fleet of unmanned ships [10]. A new multi-unmanned vessel formation and path tracking control method uses a single unmanned vessel as the leader to track the required path; other unmanned vessels act as followers and only use the position of the leader to maintain the formation and track Navigator [11]. For multi-following ships, the outer loop is designed with a distributed time-varying formation tracking control law that only uses adjacent relative information, and the inner loop is designed with a robust attitude tracking law. In attitude tracking, the sliding mode control method is used to control the heading angle and sailing speed, which eliminates the influence of model parameter perturbation and external disturbances such as wind, waves and currents on the ship’s motion [12]. The formation uses leader-follower architecture, the controller uses fuzzy control, and uses intelligent algorithms to optimize the entire controller [13]. The autonomous navigation control of the unmanned ship is realized by the deep reinforcement learning algorithm, which improves the adaptability of the unmanned ship under complex and changeable real-time simulation conditions. Through the continuous improvement and optimization of the machine reinforcement learning, it can successfully avoid obstacles and avoid obstacles in complex environments. Arrive at the destination [14]. The above-mentioned methods have problems such as high construction and maintenance costs, easy to be affected by the on-site environment, and poor adaptability. In order to improve the accuracy of formation control, reduce construction and maintenance costs and external environmental impact, and improve the adaptability of formation control methods. This paper proposes an improved virtual navigator formation control algorithm to control the formation. After the formation is formed, this article uses an improved artificial potential field method to maintain the formation, and finally analyzes it through field experiments.

2 Hierarchical Formation Control Technology Based on the Navigator Method

Since formation control is divided into three phases: formation change, formation maintenance and formation avoidance, it is necessary to design these three phases separately. First, we need to divide the entire formation system; then, we use ordered arrays and control matrices to achieve accurate formation of the formation; finally, we perform formation maintenance and formation avoidance separately.

2.1 Framework Diagram of Layered Formation System

The division of the entire formation system is shown in Fig. 1.

Fig. 1.

Frame diagram of layered formation system.

2.2 Formation Design

Ordered array and control matrix to achieve accurate formation of formation. Among them, the four-element ordered array is used to determine the relationship between the connecting edges of the formation, and the control matrix is used to determine the hierarchical relationship between the unmanned ships in the entire formation system.

The characteristics of the formation system are as follows:

• (1) There is one and only one leader in the formation system.

• (2) For the navigator, its tracking degree is ${d_{\overline{L} }} = 0$, for the follower, its tracking degree is ${d_{\overline{F} }} = 1$, for the trailing person, its tracking degree is ${d_{\overline{T} }} = 0$, the tracking degree is defined as: whether a certain unmanned ship has a forward connection For an unmanned ship, if there is an unmanned ship before it, the tracking degree is 1, otherwise it is 0. The tracking degree is defined as: whether an unmanned ship has a follow-up unmanned ship, and if there is no follow-up unmanned ship, it will be The tracking degree is 0.

• (3) In the formation system of N unmanned ships, there are N−1 connecting edges in the hierarchical structure of the formation.

• (4) The connecting edge is represented by a four-element ordered array ${E_N}$, ${V_i}$ represents the node, and, ${R_{ij}}$ and ${\sigma _{ij}}$ respectively represent the distance and orientation between the two nodes of the formation:

$$\begin{aligned} {E_N} = \{ ({V_1},{V_2},{R_{12}},{\sigma _{12}}),({V_2},{V_3},{R_{23}},{\sigma _{23}}),...,({V_{N - 1}},{V_N},{R_{N - 1\;N}},{\sigma _{N - 1\;N}})\} \end{aligned}$$

(1)

The corresponding formation control matrix can be derived through the quaternary ordered array. According to the logical relationship between each node, the formation control matrix is defined according to the following ideas:

$$\begin{aligned} {M_N} = \left\{ {\begin{array}{*{20}{c}} {{m_{ij}} = 1,}&{}{\mathrm{\text {i is a follower of j,\,i}} \ne \mathrm{{j}}}\\ {0,}&{}{\mathrm{{others}}} \end{array}} \right. \end{aligned}$$

(2)

The element characteristics of the control matrix are:

• (1) If the i-th unmanned ship is the navigator, then $\sum \nolimits _j^N {{m_{ij}}} = 0$.

• (2) If the j-th unmanned ship is a follower, then $\sum \nolimits _i^N {{m_{ij}}} \ge 0$.

• (3) If the j-th unmanned ship is a follower, then $\sum \nolimits _i^N {{m_{ij}}} \mathrm{{ = }}0$.

2.3 Behavior Control

For an unmanned ship, other unmanned ships in the team and obstacles in the environment will have a repulsive effect on it, and the real-time sub-target point will have a gravitational effect on it. There are mainly the following formation control behaviors: formation maintaining behavior, formation avoidance and obstacle avoidance behavior, and formation regeneration behavior. The safety of the formation system of the unmanned ship s is the most important. Therefore, under the constraints of the complex environment, the priority of the collision avoidance and obstacle avoidance behaviors of the formation is higher than other behaviors.

(1) Formation maintenance

During the entire formation operation, each unmanned ship needs to keep track of its corresponding sub-target point. Using the gravitational field of the artificial potential field method, the sub-target point is attractive to the unmanned ship and urges it to run to the sub-target point. , So as to achieve the expected formation geometry.

The formation remains as shown in Fig. 2. The gravity function is defined as a function of the relative distance and azimuth between the actual position of the unmanned ship and the sub-target point. The gravity function and the potential function are shown in the following formulas.

$$\begin{aligned} {F_{PK}} = - \sin \delta \cdot \nabla {P_{PK}} \end{aligned}$$

(3)

$$\begin{aligned} {P_{PK}} = \frac{1}{2}{\rho _{PK}} \cdot {\left\| {{d_{jj'}}} \right\| ^2},\forall j = 1,2,...,N - 1 \end{aligned}$$

(4)

Fig. 2.

Schematic diagram of formation keeping.

Where ${\rho _{PK}}$ is the positive gain to keep track of the sub-target point; $\delta = |{\delta _{ij'}} - {\delta _{ij}}|$ represents the deviation between the actual position of the unmanned ship and the ideal position; $\left\| {{d_{jj'}}} \right\|$ represents the relative distance between the actual position of the first unmanned ship and the ideal position; ${\sigma _{ij}}$, ${\sigma _{ij'}}$ respectively represent the first unmanned ship The ideal and actual position between the ship and the first unmanned ship.

It can be known from the foregoing formula that when $\delta = 0$ or $\left\| {{d_{ij}}} \right\| = 0$, ${F_{PK}} = 0$. Therefore, when the unmanned ship reaches the corresponding sub-target point, it will maintain the same speed and orientation to complete the expected formation.

(2) Formation to avoid collision and obstacle

In the formation operation environment of unmanned ships, when the distance between the unmanned ships and the distance between the unmanned ships and obstacles is less than the maximum safe distance of the unmanned ships, there will be a danger of collision. When the distance is greater than a certain value, it can be considered that the unmanned ship is in a normal and safe state.

Figure 3 is a schematic diagram of formation avoidance and obstacle avoidance. The repulsion function is defined as the function of the relative distance and azimuth between the actual position of the unmanned ship and the obstacle or other unmanned ships. The repulsion function and the potential function are shown in the following equations.

Fig. 3.

Schematic diagram of formation avoidance and obstacle avoidance.

$$\begin{aligned} {F_{CAi}} = - \exp (\alpha ) \cdot \sin \alpha \cdot \nabla {P_{CAi}} \end{aligned}$$

(5)

$$\begin{aligned} {P_{CAi}} = {\rho _{CA}} \cdot \sum \limits _{j = 1}^M {{P_{CAi,j}}} ,\forall i = 1,2,...,N \end{aligned}$$

(6)

$$\begin{aligned} {P_{CAi,j}}\left\{ {\begin{array}{*{20}{c}} {\frac{1}{2}{{(\frac{1}{{{d_{ij}}}} - \frac{1}{{{d_{safe}}}})}^2},{d_{ij}} \le {d_{safe}},\forall i,j = 1,...,N}\\ {0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;{d_{ij}} > {d_{safe}}} \end{array}} \right. \end{aligned}$$

(7)

Where ${d_{ij}}$ is the distance between the unmanned ship and the obstacle or unmanned ship; ${d_{safe}}$ is the maximum safety distance of the unmanned ship; ${\rho _{CA}}$ is the positive collision avoidance gain) $\alpha = {\cos ^{ - 1}}\frac{{{V_i} \cdot {V_j}}}{{\left\| {{V_i}} \right\| \left\| {{V_j}} \right\| }}$, is the angle of repulsion, that is, the angle between the current heading of the unmanned ship and the tangent plane of obstacles or other unmanned ships;

$$\begin{aligned} \sin \alpha = \left\{ {\begin{array}{*{20}{c}} {\sin \alpha \;\;\alpha \in [0,\frac{\pi }{2}]}\\ {0\;\;\;\;\;\;\;\;\;\;\;\;others} \end{array}} \right. \end{aligned}$$

(8)

This formula means that only when $\alpha \in \left[ {0\frac{\pi }{2}} \right]$, Obstacles and other unmanned ships will pose a threat to the current unmanned ships. At this time, collision avoidance and obstacle avoidance are required. Under other circumstances, the unmanned ships do not need to perform collision avoidance and obstacle avoidance behaviors.

(3) Formation regeneration

When the formation runs into a complex environment, it will be disrupted. At this time, some or all of the unmanned ships will first complete collision avoidance and obstacle avoidance behaviors, and then enter a relaxed environment, the formation will be restored or converted to other formation types. During this period, because the position of some or all of the unmanned ships has a relatively large range, it is necessary to complete the task of regenerating the formation.

From this process, it can be seen that the nature of the formation regeneration behavior is different from the formation maintenance behavior:

• 1) When the formation maintenance is completed, the speed of the unmanned ship changes very little, and the orientation change is not too large, and the formation regeneration is completed. At that time, some or all of the unmanned ships will reach the new sub-target point at full speed, and the orientation adjustment will be large.

• 2) When the unmanned ship s is in the formation holding state, the overall shape is in a good state, and when in the formation regeneration state, the formation shape is in an irregular or completely disrupted state.

Figure 4 is a schematic diagram of formation regeneration. The gravity function is defined as a function of the relative distance and azimuth between the actual position of the unmanned ship and the new sub-target point. The gravity function and the potential function are as follows:

$$\begin{aligned} {F_{PA}} = - \exp (\sin {\phi _{jj'}}) \cdot \nabla {P_{PA}} \end{aligned}$$

(9)

$$\begin{aligned} {P_{PA}} = \frac{1}{2}{\rho _{PA}} \cdot {\left\| {{d_{jj'}}} \right\| ^2},\forall j = 1,2,...,N - 1 \end{aligned}$$

(10)

Among them, $\left\| {{d_{jj'}}} \right\|$ is the relative distance between the actual position of the following unmanned vessel and the actual ideal position planned by the pilot unmanned vessel, $\left\| {{d_{jj'}}} \right\| \mathrm{{ = }}\sqrt{{{({x_j} - {x_{j'}})}^2} + {{({y_j} - {y_{j'}})}^2}}$; ${\phi _{jj'}}$ is the azimuth difference between the actual heading of the unmanned vessel and the ideal heading, ${\phi _{jj'}} = |{\alpha _j} - {\alpha _{j'}}|$; ${\rho _{PA}}$ is the sub-target point Positive gain; d is the motion vector of each unmanned ship, and $\alpha$ is the heading of each unmanned ship.

Fig. 4.

Schematic diagram of formation regeneration.

3 Improved Virtual Navigator Formation Control Algorithm

After the formation is formed, the gravitational field, repulsion field and communication distance will affect the formation of the unmanned ship. Therefore, the artificial potential field method in this chapter is improved to maintain the formation.

3.1 The Gravitational Field or Repulsion Field of the Virtual Navigator

The force analysis of the unmanned ship is shown in Fig. 5.

Fig. 5.

Force analysis of unmanned ship.

The unmanned ship needs to maintain a proper distance and angle with the virtual navigator in order to maintain the formation. As shown in Fig. 5, the corresponding gravitational or repulsive force generated by the gravitational field or repulsion field provided for the virtual navigator is ${F_1}$. Gravity or repulsion is limited by the distance between the unmanned ship and the virtual navigator. Assuming that the theoretical value of the distance between the unmanned ship and the pilot is ${R_1}$, and the actual distance is ${R_0}$, the following model is available:

$$\begin{aligned} {F_1} = \left\{ {\begin{array}{*{20}{c}} { - \xi {K_1}\sqrt{\left| {{R_1} - {R_0}} \right| } ,{R_0} > {R_1}}\\ {\xi {K_1}\sqrt{\left| {{R_1} - {R_0}} \right| } ,{R_0} < {R_1}} \end{array}} \right. \end{aligned}$$

(11)

Among them, $\sqrt{\left| {{R_1} - {R_0}} \right| }$ represents the distance value between the theoretical position and actual position of the unmanned ship relative to the navigator; ${K_1}$ is the proportional coefficient; ${R_0}$ represents the constant of the potential field.

By ${F_1}$ ensuring that the distance between the unmanned ship and the virtual navigator is stable, and then obtaining angle information through the sensor, the vector position of the unmanned ship and the virtual navigator can be kept consistent.

3.2 Gravitational Field or Repulsion Field of Other Unmanned Ships

When the unmanned ship is moving in the system, it is necessary not only to clarify its role in the entire formation, but also to ensure that it does not collide with other unmanned ships. This requires accurate relative positions between unmanned ships. Suppose the theoretical distance of the unmanned ship i to the unmanned ship is ${R_i}$, and the actual distance is ${R_{i0}}$, so the following model is obtained:

$$\begin{aligned} {F_i} = \left\{ {\begin{array}{*{20}{c}} { - \xi {K_2}\sqrt{\left| {{R_i} - {R_{i0}}} \right| } ,{R_{i0}} > {R_i}}\\ {\xi {K_2}\sqrt{\left| {{R_i} - {R_{i0}}} \right| } ,{R_{i0}} < {R_i}} \end{array}} \right. \end{aligned}$$

(12)

Among them, $\sqrt{\left| {{R_i} - {R_{i0}}} \right| }$ represents the distance value between the theoretical position and the actual position of the unmanned ship relative to the unmanned ship i; ${K_2}$ is the proportional coefficient; $\xi$ is the constant of the potential field.

In the same way, obtaining angle information on this basis can ensure that the vector positions of the unmanned ships are consistent.

3.3 Maximum Communication Distance Limit

Due to the limited communication capability of the unmanned ship formation control system, it is necessary to add the communication distance constraint to the potential field model to prevent some unmanned ships from leaving the formation beyond the communication range. Assuming that the maximum communication distance is ${R_{\max }}$, and the distance between the unmanned ship and the nearest unmanned ship is ${R_s}$, the following model is available:

$$\begin{aligned} {F_i} = \left\{ {\begin{array}{*{20}{c}} { - \xi {K_3}{R_{\max }},{R_s} > {R_{\max }}}\\ {0,{R_s} < {R_{\max }}} \end{array}} \right. \end{aligned}$$

(13)

Among them, ${K_3}$ is the proportional coefficient, $\xi$ is the potential field constant.

When the unmanned vessel leaves the maximum communication distance, the unmanned vessel cannot receive the signal. This requires the unmanned vessel to record the direction of the adjacent unmanned vessel at the moment when it is about to leave the formation, and regard this direction as the direction of gravity.

4 Calculation and Analysis of Results

The details of the test parameters are as follows, as shown in Table 1:

Table 1. Test parameter selection

Randomly deploy 5 unmanned ships at the starting position, randomly select one of the unmanned ships as the leader, and the remaining unmanned ships as the followers. After that, the five unmanned ships began to execute the formation algorithm, transform the formation into a triangle, and then maintain the formation to sail. If unmanned ships encounter obstacles (represented by black rectangles, 60 m long and 15 m wide) during navigation, they will trigger the execution of obstacle avoidance algorithms, passing between two relatively distant obstacles. After crossing the obstacle, they continue to execute the formation algorithm and change back to the triangular formation in time. The trajectory of the entire formation control process is shown in Fig. 6. During formation control, the relative distance between the four followers and the leader is shown in Fig. 7 below. The relative distance between the four followers is shown in Fig. 8 below.

Fig. 6.

Trajectory diagram of the whole process of unmanned ship formation control.

Fig. 7.

The relative distance between the four followers and the leader.

Fig. 8.

The relative distance between the four followers.

It can be seen from the trajectory of the entire formation control process in Fig. 6 that the formation algorithm can quickly transform multiple unmanned ships into a triangular formation. After encountering an obstacle, the obstacle avoidance algorithm can be executed in time to avoid the obstacle. At the same time, after passing through obstacles, it can revert to a triangular formation. It can be seen from Figs. 7 and 8 that the formation starts at 0 s and transforms into a triangle formation in about 20 s. During the period from 20 s to 50 s, the formation is in the holding phase. Since the formation is symmetric, the distance between the symmetrical unmanned ships is 0, and the relative distance to other unmanned ships remains unchanged. Obstacle avoidance starts at 50 s and ends at 120 s. Obstacle avoidance time is about 70 s. About 35 s after the obstacle avoidance is over, the formation can revert to the triangle formation, and then the formation is in the holding phase again.

From the above results, it can be concluded that the improved virtual leader formation control algorithm designed in this paper has good obstacle avoidance ability, and can quickly change to the corresponding formation after crossing obstacles, and has good robustness, rapidity and precision.

5 Conclusion

Aiming at the formation and obstacle avoidance problems in the multi-formation control system, the formation control algorithm based on the navigator is adopted to carry out the formation design and behavior control respectively. Then, this article improves on this basis. After the formation is formed, the artificial potential field is used to maintain, and the gravitational field and repulsion field of the virtual navigator and other unmanned ships and the maximum communication distance are considered respectively, and the improved virtual navigator is obtained. Formation control algorithm, and then we conducted a formation maintaining experiment. The experimental results show that this paper uses the proposed improved virtual navigator algorithm to conduct formation control and obstacle avoidance experiments, and find that the algorithm has good performance.

The formation control method has the characteristics of high accuracy, strong generalization ability, fast control speed, strong environmental adaptability, etc., and the control method can be packaged into an intelligent control system. Use Java interface to provide third-party services, flexible deployment and application, and strong adaptability. In the future, we can continue to dig deeper into the artificial potential field to improve the robustness of formation control.