Improved Tau-Guidance and Vision-Aided Navigation for Robust Autonomous Landing of UAVs

Abstract

In many unmanned aerial vehicle (UAV) applications, flexible trajectory generation algorithms are required to enable high levels of autonomy for critical mission phases, such as take-off, area coverage, and landing. In this paper, we present a guidance approach which uses the improved intrinsic tau guidance theory to create spatio-temporal 4-D trajectories for a desired time-to-contact with a landing platform tracked by a visual sensor. This allows us to perform maneuvers with tunable trajectory profiles, while catering for static or non-static starting and terminating motion states. We validate our method in both simulations and real platform experiments by using rotary-wing UAVs to land on static platforms. Results show that our method achieves smooth landings within 10 cm accuracy, with easily adjustable trajectory parameters.

1 Introduction

Unmanned aerial vehicles (UAVs) play a significant role in providing services and enhancing safety thanks to their flexible and low-cost surveillance, monitoring, and risk assessment capabilities. In many emerging applications, including environmental monitoring [3], industrial inspection [4], and emergency response [13], high levels of system autonomy are vital to guarantee safe and reliable operation. This is especially true when UAVs act as sentinels monitoring large areas, which take-off from a nest and return to it for recharging before performing further tasks.

A typical mission can be divided into the following phases: (1) take-off, (2) mid-course, (3) area scanning [14, 15], (4) visual tracking, and (5) landing. Often, the final phase is the most critical as it involves performing delicate maneuvers; e.g., landing on a station for re-charging [2] or on a ground carrier for transportation [8]. These procedures are subject to constraints on time and space, and must be robust to changes in environmental conditions, such as visibility and wind disturbances [6]. To achieve smooth landings, precise sensing and accurate control techniques are therefore required.

In this paper, we address these problems by integrating, within the end-to-end software system developed at the ETH Zürich, a trajectory generation algorithm based on a visual tracking system and a bio-inspired guidance method for autonomously landing on a specified target. Our motivation is to increase the reliability and versatility of landings in the example scenarios mentioned above. We use the improved intrinsic tau guidance theory [10, 19] to generate spatio-temporal (4-D) trajectories for a desired time-to-contact (TTC) based on the estimate of the relative pose of the UAV with respect to the target. This approach enables us to perform a maneuver with arbitrary initial and final motion states, and tailor its trajectory profile for various types of rotary- or fixed-wing tasks, such as landing, in-flight obstacle avoidance, and object-picking. The advantage of this approach is an “user-oriented” trajectory generation method, where fundamental parameters can be tuned based on the mission requirements. As such, the user may be interested in assigning predefined mission requirements for the trajectory, such as to maintain the course within the boundary of two intersecting planes (e.g., flying within a natural or man-made canyon), controlling the total mission time or energy consumption, or specifying the initial and final landing angles.

We simulate and experimentally validate our approach in outdoor and indoor experiments using rotary-wing UAV equipped with a downward-looking camera for detecting a static target. The main contributions of this work are:

• 1. A bio-inspired landing method which:

  • generates easily tunable 4-D trajectories to the target,

  • provides guidance starting or terminating with static or non-static motion states,

  • is applicable with different sensor configurations.

• 2. The validation of our method in both simulation and real platform experiments.

The paper is structured as follows: Sect. 2 begins by outlining the state-of-the-art in autonomous landing methods. The general landing problem is formulated in Sects. 3, and 4 describes our bio-inspired approach. We present our experimental set-ups and results in Sects. 5 and 6 before concluding in Sect. 7.

2 Related Work

Significant work has been done recently on autonomous landing methods for UAVs in various environments. As discussed by [6], landing navigation frameworks mainly use a Global Positioning System (GPS) and an Inertial Measurement Unit (IMU), with small, light-weight visual sensors often integrated for improved accuracy, especially in outdoor applications.

Landing strategies can be categorized based on the guidance techniques used to create trajectories to the landing position. We consider (i) position-based [18, 20] and (ii) biologically-inspired [9, 11, 17] techniques. Traditional position-based guidance approaches, such as the pursuit [20] and proportional [18] laws, leverage Line-of-Sight (LoS) to navigate the UAV towards the target. While these methods provide precise tracking, they are limited to position sensors only as positions and/or velocities must be accurately computed [8]. Moreover, they may require complex control algorithms and lack controllability of the UAV trajectory pattern and profile, thus restricting their applicability.

Bio-inspired guidance paradigms overcome these limitations by using visual information such as the TTC [11] or optical flow [17] to generate 4-D trajectories [9, 17], enabling the control of both their spatial and temporal components. In this category, the general tau theory [10, 11] has been popularly postulated to describe goal-directed movements, e.g., collision avoidance [19], docking, and landing [9]. [9] recently introduced a tau-based UAV autopilot and implemented it to perform high-accuracy maneuvers. Similarly, we apply the tau principles to generate a 4-D trajectory for landing. However, by using elements of improved intrinsic tau guidance [19], our method is not restricted to static initial and final states and is thus also suitable for landing on moving platforms.

3 Problem Definition

As outlined in Sect. 1, there are various UAV missions requiring large area coverage. For safe and robust autonomous operation, it is necessary to generate successive and consecutive trajectories by maintaining position and/or velocity continuities on the boundary waypoint between the nth and the nth + 1 trajectories. Adjourning the trajectories is also required during the successive steps of following or landing on a moving object, taking into account the error deriving by the tracking algorithm, including noise. Consequently, the main problem is to develop and apply a fast and flexible algorithm enabling the UAV to perform the above-mentioned phases autonomously, while maintaining a small number of intuitive turning parameters.

The general goal-directed trajectory generation problem is defined as follows. We describe a spatio-temporal 4-D UAV trajectory by the state:

$$\begin{aligned} \mathbf S (t) = \{x(t),\, y(t),\, z(t), \dot{x}(t),\, \dot{y}(t),\, \dot{z}(t)\} \end{aligned}$$

The aim is to guide the UAV from arbitrary initial states $\mathbf S (t_0)$ to goal states $\mathbf S (t_0 + T)$ in the execution time T. The UAV dynamics are:

$$\begin{aligned} {\left\{ \begin{array}{ll} \dot{x} = u_x\textit{,} \\ \dot{y} = u_y\textit{,} \\ \dot{z} = u_z\textit{,} \end{array}\right. } \end{aligned}$$

(1)

where $\mathbf u = \{u_x,\, u_y,\, u_z\}$ are the velocity components along each co-ordinate axes used as commands for the trajectory tracker, which is discussed for our application in Sect. 5.

Although the algorithm is applicable to a generic trajectory, this paper focuses on the most complex mission scenario involving landing on a stationary or moving target. The trajectory is adjourned based on visual information provided by a camera during the approach phase. The specific requirements are to:

• 1. reach the target point in a specified time T,

• 2. arrive and stop at the target point with zero velocity at contact, such that $\mathbf S (t) = \{x(t),\, y(t),\, z(t),\, 0,\, 0,\, 0\}$ at $t = t_0 + T$,

• 3. reach the target point from a specific approach direction, as discussed in Sect. 4.1.

4 Approach

This section overviews the proposed bio-inspired guidance strategy for autonomously landing on a platform, where the perching trajectory is generated assuming that the target can be tracked with a camera. In brief, we use the intrinsic tau guidance strategy to generate tunable 4-D trajectories for smooth landings with static or non-static starting and terminating states. First, we present our method of trajectory parametrization as the basis of our approach. We then summarize the basic principles of general tau theory before detailing our specific guidance technique.

4.1 Trajectory Parametrization

With reference to Fig. 1, in the East-North-Up (ENU) frame, the landing maneuver usually requires arriving at a destination with a final state of the trajectory depending on the target morphology and dynamics. The main gaps are:

• the distance gap d(t),

• the relative speed,

• the approaching angle of the trajectory $\alpha (t)$, between d(t) and the normal to the EN plane,

• the approaching angle of the trajectory $\beta (t)$, which permits changing the UAV heading during perching.

d(t) is the instantaneous distance along the LoS to the target, given by the difference between the UAV position $\mathbf p _u(t)$, and the target position $\mathbf p _t(t)$. The relative speed is given by $\dot{\mathbf{p }}_u(t) - \dot{\mathbf{p }}_t(t)$ which, in order to avoid collisions, must be zero at the touch-down. The approaching angle $\alpha (t)$ is based on the landing approach, and consequently depends on the UAV platform and application, e.g., along the tangent to the surface for a fixed-wing UAV, near-vertical for a rotary-wing UAV, or inclined at a certain angle to enter an opening. The approaching angle $\beta (t)$ ensures that the body frames of the UAV and the target respect a required orientation at the touch-down, e.g., when recharging requires docking between the two.

To introduce these parameters in the mission design, we use the intrinsic tau guidance approach presented in Sect. 4.3.

Fig. 1

Reference frame and main gaps for autonomous tau-landing

4.2 General Tau Theory

Our guidance method for landing is based on the general tau theory [10], which uses the tau function to represent the TTC of a goal-directed movement. We define the tau variable of a motion gap $\chi$ as:

$$\begin{aligned} \tau _\chi = {\left\{ \begin{array}{ll} \frac{\chi (t)}{\dot{\chi }(t)} &{} |\dot{\chi }(t)| \ge \dot{\chi }_{min}{} \textit{,} \\ {\text {sgn}}\left( \frac{\chi (t)}{\dot{\chi }(t)}\right) \tau _{max} &{} |\dot{\chi }(t)| < \dot{\chi }_{min}{} \textit{,} \end{array}\right. } \end{aligned}$$

(2)

where $\chi (t)$ can be the gap of any state in $\mathbf S (t)$.

4.3 Improved Intrinsic Tau Guidance Strategy

Specifically, our landing method uses the improved intrinsic tau guidance strategy [19] to allow for goal-directed movements and, in particular, to start the trajectory with an initial flight condition. We formulate the intrinsic guidance gap of a movement $G_v(t)$ as:

$$\begin{aligned} {\left\{ \begin{array}{ll} G_v(t) = -0.5at^2 + V_Gt + G_0\textit{,} \\ \dot{G}_v(t) = -at + V_G\textit{,} \\ \ddot{G}_v(t) = -a\textit{,} \end{array}\right. } \end{aligned}$$

(3)

where a is the acceleration, $V_G$ is an initial velocity, and $G_0$ specifies an initial intrinsic gap. In particular, the intrinsic tau guidance strategy (3) represents the vertical component of a projectile motion. Once the acceleration a is assigned, the initial velocity $V_G$ and the initial gap $G_0$ must be computed according to the initial ($t=0$) and the final ($t=T$) conditions of the actual movement, as described in the following.

Considering the movement along a generic x-axis from time 0 to T as an example, the position and velocity gaps can be expressed as $\varDelta x = x_T - x$ and $\varDelta \dot{x} = \dot{x}_T - \dot{x}$, respectively, where $x_T$ and $\dot{x}_T$ denote the goal states at time T. We apply the tau coupling strategy [9, 19] for synchronous gap-closing to obtain the movement states:

$$\begin{aligned} {\left\{ \begin{array}{ll} x(t) = x_T + \dot{x}_T(t - T) - \frac{\chi _{x0}}{G_0^{1/k_x}}G_v^{1/k_x}{} \textit{,} \\ \dot{x}(t) = \dot{x}_T - \frac{\chi _{x0}}{k_xG_0^{1/k_x}}\dot{G}_vG_v^{1/k_x-1}{} \textit{,} \\ \ddot{x}(t) = -\frac{\chi _{x0}}{k_xG_0^{1/k_x}}G_v^{1/k_x-2}\left( \frac{1-k_x}{k_x}\dot{G}_v^2 + G_v\ddot{G}_v\right) \textit{.} \end{array}\right. } \end{aligned}$$

(4)

where $k_x$ is a gain parameter controlling gap convergence along the x-axis, as discussed below.

From the definition of $G_v$, Eq. 4, and Eq. 3, it can be shown that:

$$\begin{aligned} {\left\{ \begin{array}{ll} G_0 = \frac{\chi _{x0}gT^2}{2(\chi _{x0}+k_x\varDelta \dot{x}_0T)}{} \textit{,} \\ V_G = \frac{k_x\varDelta \dot{x}_0gT^2}{2\left( \chi _{x0}+k_x\varDelta \dot{x}_0T\right) }{} \textit{.} \end{array}\right. } \end{aligned}$$

(5)

$G_0$ and $V_G$ can be viewed as bonding actual and intrinsic movements due to gravitational effects. If $k_x \in (0,\,0.5)$, $(x,\, \dot{x},\, \ddot{x}) \rightarrow (x_T,\, \dot{x}_T,\, 0)$ and the position and velocity can be steadily guided to the target values, as required. Hence, varying the components of $\mathbf k = \{k_x,\,k_y,\,k_z\}$ within this range allows for modifying the trajectory profile along each axis.

In addition, using the tau coupling strategy described by Eqs. (2) and (3), we can apply a more “user-oriented” approach by coupling to the intrinsic movements the gaps d(t), $\alpha (t)$ and $\beta (t)$ as follows:

$$\begin{aligned} d(t) = \frac{d(0)}{G_0^\frac{1}{k_d}}G_v^\frac{1}{k_d} \end{aligned}$$

(6)

$$\begin{aligned} \alpha (t) = \frac{\alpha (0)}{d(0)^\frac{1}{k_\alpha }}d(t)^\frac{1}{k_\alpha } \end{aligned}$$

(7)

$$\begin{aligned} \beta (t) = \frac{\beta (0)}{d(0)^\frac{1}{k_\beta }}d(t)^\frac{1}{k_\beta } \end{aligned}$$

(8)

where each gap has its own gain parameter ($k_d$, $k_\alpha$, $k_\beta$). Combining these equations the position vector at time t of the UAV is simply given by:

$$\begin{aligned} \mathbf p({ t}) = \mathbf p({ T}) + \begin{bmatrix} \ -d(t)sin\alpha (t)cos\beta (t) \\ \ -d(t)sin\alpha (t)sin\beta (t) \\[0.3em] \ d(t)cos\alpha (t) \end{bmatrix} \end{aligned}$$

(9)

5 Experimental Set-Up

This section details physical system architectures used to test the proposed method (Sect. 6.2). To demonstrate applicability, we conducted experiments in both outdoor and indoor environments.

The outdoor experiments are conducted on an empty $20\times 20$ m farmland plot in clear weather conditions (Fig. 2a), where we demonstrate our landing algorithms running in real-time on an AscTec Firefly (Fig. 2b). Our UAV is equipped with an autopilot providing a low- and high-level control, and on-board computer (AscTec Mastermind), and the following sensors:

• 100 Hz IMU,

• 10 Hz Piksi V2 (RTK) differential GPS,

• 20 Hz (VI-)sensor,

• 50 Hz downward-looking Point Grey Chameleon 2.0 camera.

The landing platform target (Fig. 2c) is an A3 ($297\times 420$ mm) arrangement of variable-dimension tags, obtained from the ar_track_alvar library. The nested tag layout on the platform allows for detecting and estimating the camera-to-target relative pose from different altitudes. Note that, in this paper, we use a static platform for proof of concept and leave the study of moving targets to future work.

The logical architecture of the algorithms running on the UAV is described in Fig. 3, which depicts the different interacting modules. In particular, the input data (Fig. 3, left) include: GPS measurements, images from the downward-looking camera and the VI-sensor, and accelerometer and gyroscope measurements from the VI-sensor and the on-board IMU.

For navigation, the UAV pose is estimated by integrating VI-sensor and GPS data within the RObust Visual Inertial Odometry (ROVIO) framework [1]. This is then integrated with IMU data in the Multi-Sensor Fusion (MSF) framework [12] to obtain a refined state estimate Fig. 3. Guidance is based on the improved tau guidance strategy (red block in Fig. 3), which uses the actual UAV state and the camera-to-target relative pose to generate reference trajectories that are then tracked by a non-linear Model Predictive Controller (MPC) [7].

Fig. 2

a Shows our experimental set-up on the field, with the GPS RTK base station visible on the right. b Depicts the AscTec Firefly positioned on the landing platform. c Exemplifies the tag arrangement on the platform as seen by the on-board camera

Fig. 3

Systems diagram for our outdoor experiments. A camera is used to track the landing platform pose, and localization is provided by the MSF framework output. The guidance unit passes reference trajectories to the MPC. Note that our RTK GPS requires a ground base station (Fig. 2a) to receive satellite signals and transmit position corrections to the UAV

We also performed indoor experiments in an empty $20\,\times \,20$ m environment (Fig. 4) using the same landing platform. Here, our algorithms run in real-time on a DJI Matrice 100 with a 100 Hz IMU and Intel RealSense ZR300 camera providing 30 Hz images [16]. Our system comprises the general pipeline in Fig. 3; however, in the absence of GPS data, only images and IMU data are input to ROVIO and MSF.

Fig. 4

A side view of our indoor experimental set-up showing the UAV and the landing platform

6 Results

This section demonstrates the usage of the improved tau guidance strategy to land smoothly on a target platform. We first validate our method in simulation before presenting results from outdoor and indoor experiments.

6.1 Simulations

The performance of the proposed landing method is validated preliminarily in simulation. The simulation environment is developed in the RotorS framework [5], which realistically replicates the flight dynamics of an Asctec Firefly and its on-board sensors. For the IMU, the orders of magnitude of biases and random errors are set to be consistent with Micro Electro-Mechanical Systems (MEMS) used on the AscTec Firefly (Table 1). Furthermore, the GPS position uncertainty in the ENU frame is modeled as a constant bias plus Gaussian white noise to account for the time correlation in GPS errors. For vision sensing, only the downward-looking camera was simulated with an Instantaneous Field of View (IFoV) uncertainty modeled as Gaussian white noise with a standard deviation of $0.05^{\circ }$. This camera is used to detect the landing target on the ground and to obtain its relative pose with respect to the UAV. Consequently, navigation is achieved here without simulating ROVIO, but only integrating simulated position and attitude measurements within the MSF framework.

Table 1 Assumptions on the simulated bias and random errors for our simulation trials

The simulations serve to validate and integrate each component of the software framework before the experimental tests (Sect. 6.2). Since the focus is set on rotary-wing UAVs, the interest is also to land on the platform with a desired orientation $\alpha (t)$ and zero velocity. Hence, simulations have been performed to evaluate the behavior of trajectory shapes for varying coefficients $\mathbf{k }$ with an emphasis on vertical landing. Figure 5a shows a comparison of different landing trajectories where the maneuver starts from an height of 1.5 m and EN components of 0.8 m each. This figure evidences how the 3-D shape of each trajectory is determined by coupling the coefficients $\mathbf{k }$, In addition, Fig. 5b shows the velocity and acceleration profiles during the near-vertical landing trajectory (blue curve in Fig. 5a), demonstrating that zero velocity is reached at the end of the landing maneuver.

Fig. 5

a Depicts landing trajectories with different elements in $\mathbf k$. These parameters enable modifying the trajectory profile for LoS (black), near-vertical (blue), and near-horizontal (red) shapes. b Shows the velocity and acceleration norm profiles for the near-vertical (blue) curve in (a)

6.2 Experimental Tests

To test our approach in practical scenarios, we conducted multiple experiments using the set-ups described in Sect. 5. In each trial, the UAV was commanded to:

• 1. take-off and perform its mission,

• 2. fly back to the landing area,

• 3. fly a search path to detect the target, and

• 4. safely land on the target point in a specified time.

Figures 6 and 7 illustrate the two autonomous landing stages for an outdoor test using an AscTec Firefly. The first stage is target detection (at an height of about 2 m, as shown in Fig. 6c) followed by a refinement phase for noise reduction. Once the target is detected and tracked, the UAV starts the approaching stage during which the camera continues tracking the target to update its relative pose. If the difference between successive target positions is larger than a certain tolerance, the trajectory is re-planned accordingly and passed to the non-linear MPC.

These aspects are evident in Fig. 6, which shows the UAV position components in the ENU frame. A comparison of the reference trajectory generated by our guidance strategy with the UAV position output from the MSF framework confirms that the controller follows the reference throughout the maneuver until the touch-down.

Figure 7 compares the UAV position and velocity along the three co-ordinate axes. The plots illustrate the optimum velocity profile of the maneuver to reach the softest touch-down while achieving the accuracy required to land near the target center.

As shown in Fig. 8, the final landing position remains within a maximum dispersion of several centimeters with respect to the target center. It is worth noting that, if the target is lost for a certain time period, the UAV is commanded to climb to increase the camera footprint, and returns on the landing path only after the target is detected and tracked again. Figure 8 demonstrates this effect as the target is lost at $\sim$20 cm height mainly due to the UAV shadow causing occlusions impeding robust tracking. As a result, the UAV ascends, landing smoothly upon target re-detection.

Fig. 6

Differences between the reference and MSF trajectories during the final phases of landing

Fig. 7

Velocity profile during the landing maneuver with the position profile as a reference

Fig. 8

UAV landing position with respect to the target position in the EN plane

In addition, we executed 63 successful indoor landings with the DJI Matrice 100. State estimation was obtained by integrating ROVIO within the MSF framework, the relative pose between the UAV and the landing platform was obtained by images acquired by the Intel RealSense ZR300 and the VICON system was used for ground truth reference. These tests show the repeatability and robustness of the proposed landing approach as well as its achievable accuracy. To this end, six case studies were considered varying in the $\mathbf{k }$ coefficients and the height of the target detection stage. The approaching heights tested were 1.5, 2 and 2.5 m, with two different sets of $\mathbf{k }$ coefficients: $k_x=k_y=0.2, k_z=0.4$, and $k_x=k_y=k_z=0.2$. Statistical results are summarized in Table 2. Here, errors are computed based on UAV and target ground truth positions acquired from the VICON system.

Table 2 Landing mean errors and standard deviations for the considered case studies

7 Conclusions and Future Work

This paper presented a guidance approach based on the improved intrinsic tau guidance law for autonomous UAV landing on a static platform. The guidance theory generates smooth and computationally efficient 4-D trajectories that are both suitable for fixed- and rotary-wing UAV platforms. The framework was validated in simulations and multiple outdoor and indoor experiments with different platforms, showing that trajectories can be easily designed by varying the guidance coefficients. Results from over 60 indoor tests, using a VICON system only to provide ground truth reference, demonstrate landing with centimeter-level accuracy.

Future work will examine methods of developing a reliable target tracker based on the current visual target detector as well as GPS and IMU measurements to estimate a target pose with respect to the UAV. This would allow more robust target-tracking in challenging conditions, such as dynamic motion, and even in cases where the target is outside the camera field-of-view. An accurate target-tracker would also enable landing on a moving platform. In addition, we will consider developing a target detector more suitable for poorly lit environments.

Finally, we are interested in applying the proposed guidance law to fixed-wing landing missions.