Keywords

1 Introduction

Unmanned aerial vehicles came out in the United Kingdom in 1917. They have significant characteristics such as no risk of casualties, low cost, light weight, good maneuverability, and strong concealment capabilities. Therefore, its development is highly valued by many countries. UAVs can be used in aerial photography, transportation, intelligent irrigation, etc., and have broad civilian prospects. At the same time, UAVs have played an important role in several high-tech local wars in recent years, and their military application value is equally significant.

Today, positioning, route planning, control, and environmental perception are still major problems in the field of drone technology. Among these problems, positioning is the primary link in autonomous flight and an important prerequisite for the successful completion of various tasks by drones. GPS positioning is currently the most commonly used method for outdoor drone positioning. However, in some mission scenarios, such as cities, forests, or environments with complex electromagnetic fields, GPS signals are blocked by tall obstacles or electromagnetic interference, and cannot be used for drones. Provide continuous and effective positioning information. In addition, GPS itself also has accuracy problems. This poses a huge challenge for the UAV to achieve full and reliable positioning during the mission. Therefore, the UAV positioning technology that does not rely on GPS has important research value and has received widespread attention in recent years.

This paper introduces several UAV positioning methods that do not rely on GPS in popular research fields, analyzes their advantages and disadvantages, the current challenges in UAV positioning and finally looks forward to the future development.

2 Ground-Based Binocular Drone Positioning

According to different sensor configurations, the UAV positioning system can be divided into two working modes: ground-based and airborne. The ground-based positioning method is usually used in the UAV guided take-off and landing stage. The position and attitude data of the target UAV is obtained through a combination of ground-based sensors, and the image coordinates of the UAV target are extracted from it, and the spatial location of the UAV is calculated in real time according to the principle of vision measurement. At this time, as long as the ground servo turntable is driven according to the calculated space position of the drone, the camera can track the aerial drone target in real time. Ground-based visual positioning mode has been widely used in some specific scenarios, such as UAV take-off and landing, with its sufficient computing resources and stable operating performance.

The UCARS [1] system developed by Sierra Nevada of the United States in 2006 uses a millimeter-wave radar that is installed on land or ships to automatically locate and track the transponder of the drone. The DT-ATLS [2] system developed by the company in 2008 combines the advantages of various sensors such as differential GPS, millimeter-wave radar, and infrared imager. It not only achieves improved guidance for drones landing on the ground and aircraft carriers, but also It can be applied to various types of drones. The Deck Finder [3] system developed by the Austrian Siebel Electronic Equipment Company in 2013 uses another method-radio frequency ranging, which can achieve precise spatial positioning of the UAV through six radio frequency transmitters installed on the ship.

After studying these systems, we can learn that a simple implementation plan is to set up two 2-degree-of-freedom turntables (PTU) on the ground, and one camera is fixed on each of them, as shown in Fig. 1.

In order to ensure that the UAV remains in the field of view of the two cameras during the flight, it is controlled by the ground station computer. The rotation of the turntable causes the camera to rotate in conjunction to track the flight of the drone. The ground station computer collects the image data of the two cameras and the attitude data of the turntable in real time, and uses the fusion positioning algorithm to calculate the space position of the drone, and sends it to the drone in real time through the data transmission link as the positioning data for the autonomous flight of the drone.

Fig. 1.
figure 1

Ground-based binocular drone positioning

Full size image

The UAV spatial position calculation algorithm based on the ground-based binocular vision system usually requires input of the following parameters: image data, camera internal and external parameters, attitude data of the two turntables, estimated initial contour center and radius, estimated ROI (region of interest) Center and side length, the state of the drone at the previous moment. Through image preprocessing, region segmentation and filtering, and introducing an extended Kalman filter model for parameter estimation, the state of the UAV at time k can be iteratively obtained.

3 Airborne Positioning

Aerial positioning is often used in the flight phase of drones. It uses a set of sensors (including inertial sensors, visible light vision sensors, laser sensors, etc.) on the drone to obtain the drone’s position and movement status information. At the same time, a certain data fusion model is used to correct and predict the collected data samples to ensure small measurement errors. This positioning method can be performed in a small indoor area, and some methods can also obtain indoor environment information at the same time, which has good application value.

3.1 Airborne Visual/Laser Fusion Positioning

Due to the limitations of the UAV’s flight environment and load capacity, higher requirements are placed on the sensors on board. Especially small and micro UAVs have weak load capacity, which further limits the types of sensors that can be mounted. In addition to traditional micro drone airborne sensors such as IMU and barometer, visible light vision sensors can collect environmental images, obtain the orientation information of environmental targets, and can obtain a large amount of environmental information at a very small load cost. The drone is very suitable for airborne sensors. However, only using the image data collected by the visible light vision sensor for positioning is very dependent on the lighting conditions of the environment and the richness of textures. The laser sensor can obtain the distance information of the target in the environment, which is not only complementary to the information collected by the visible light vision sensor, but also does not depend on the lighting conditions and the richness of the texture of the environment. Combining the data of these two kinds of sensors can enhance the environmental adaptability of the positioning system, which is the current research hotspot of micro-UAV positioning methods.

The flow of the UAV spatial positioning algorithm [4] proposed by researchers from the National University of Defense Technology is shown in Fig. 2. It can be seen from the figure that the algorithm is divided into four algorithm modules as a whole, namely key point tracking, initial pose solution, pose filter optimization, and update of map, key points, and key frame. First, use the key frames and key points updated in the previous frame to track the key points in the image frame at this moment, and then calculate the initial pose according to the key points. Then, the pose filter optimization module uses the barometer and IMU data to optimize the calculated initial pose and output the UAV’s spatial positioning results. Finally, add optical flow information to the laser scanning points according to the calibration parameters, add it to the incremental 3D map, and update the keyframe and key point.

Fig. 2.
figure 2

Algorithm flow of airborne UAV spatial positioning

Full size image

3.2 Fusion Positioning of Visual SLAM and IMU

Visual SLAM

The problem of visual SLAM (simultaneous localization and mapping) originates from the field of robotics. It can perceive and model the surrounding environment in an unknown environment to obtain the robot’s own pose information. The classic visual SLAM framework mainly includes four aspects: visual odometry [5], back-end optimization, loop detection and mapping [6]. The flowchart is shown in Fig. 3.

Fig. 3.
figure 3

Visual SLAM flowchart

Full size image
  1. 1.

    Sensor data collection: mainly image data in visual SLAM.

  2. 2.

    Visual odometer: The function of the visual odometer is to estimate the motion of the camera corresponding to two adjacent frames of images. The visual odometer is also called the visual SLAM front end.

  3. 3.

    Back-end optimization: The back-end receives odometer information and loop information, optimizes the camera’s motion trajectory, and obtains a globally consistent trajectory and map.

  4. 4.

    Loop detection: It is judged whether the robot has moved to the vicinity of the previous position. If a loop is detected, the loop information is provided to the back end for optimization processing.

  5. 5.

    Map building: According to the estimated trajectory, create a map corresponding to the task requirements. According to different processing methods, visual SLAM is usually divided into two methods: filter-based and optimization-based. Filter-based methods are generally based on Markov assumptions, using two steps of prediction and update to achieve real-time positioning and environment modeling. Based on the optimization method, the system is generally described as a nonlinear least squares problem, and the objective function is minimized by continuously optimizing the state quantity.

SLAM Integrating Vision and IMU

If only binocular vision is used for UAV positioning, because the visual information is not robust enough under harsh conditions such as moving objects, obstructing the line of sight, and drastic changes in illumination, there will usually be large positioning errors or even positioning failures. Therefore, we usually need to use the drone itself or external auxiliary sensors to achieve positioning in a GPS rejection environment [7]. Inertial measurement units (IMU), cameras, laser scanners, ultra-sonics, etc. are some commonly used drone positioning sensors. These sensors calculate their own position and attitude by sensing themselves or outside information and using some geometric constraints and other relationships. The IMU is mainly used to measure the acceleration and angular velocity information of the carrier. There is basically no requirement for the environment. The IMU can provide high-frequency (200‒1000 Hz) pose estimation, while the vision can only provide low-frequency (10‒50 Hz) pose estimation. But if only relying on the IMU for positioning, there will be movement drift in a short time (2‒5 s). Image visual information and IMU information have good complementarity, and fusion positioning can achieve accurate, reliable, high-frequency UAV pose estimation.

A binocular and IMU fusion positioning model [8] proposed by researchers at Zhejiang University is shown in Fig. 4.

Fig. 4.
figure 4

Binocular vision and IMU fusion positioning system framework

Full size image

The model is composed of a motion equation and an observation equation, and converts the fusion positioning problem into restoring state quantities from noisy data. For all motions and arbitrary observation data, define the error between the measured data and the predicted data, and use the LM algorithm to iteratively solve to obtain the state to be solved. In order to improve the robustness of the results, the binocular and IMU data are fused at the end.

3.3 Fusion Algorithm of UWB Positioning and Inertial Navigation

UWB (Ultra-Wide Band) technology is essentially a wireless communication technology, and the positioning technology suitable for wireless signals mainly includes methods based on ranging and direction finding. According to the parameters measured in the positioning process, the ultra-wideband positioning method is mainly divided into the Received Signal Strength Indication (RSSI) method, the Angle of Arrival method (AOA) and the signal receiving time method. The method of receiving time can be divided into time-of-arrival method (TOA) and time-difference-of-arrival method (TDOA).

The positioning algorithm based on ranging can be divided into two steps, ranging and position calculation. Having used the TDOA algorithm to calculate the arrival time difference between the node to be located and multiple positioning base stations, the next step is how to use these TDOA data solves the position coordinates. If the Chan algorithm [9] or Taylor algorithm [10] is directly used to analyze the TDOA measurement data, it will be found that the error of the TDOA measurement value in some areas is large, which will cause the algorithm to fail to converge in the area with large errors or the solution result is insufficiently consistent. Considering that the UAV has an IMU unit, if the multi-sensor fusion technology is adopted, when the measurement error of individual sensors is large, the fusion algorithm can still more accurately reflect the system state. Therefore, the ultra-wideband positioning system and IMU inertial measurement unit are essential to provide stable and reliable positioning results for the UAV formation in the indoor environment through the EKF fusion algorithm.

Extended Kalman Filter (EKF) is essentially a nonlinear version of Kalman Filter [11]. The original intention of the Kalman filter algorithm is to use the state equation of the linear system to obtain the optimal estimation of the system state from a series of noise-containing measurements. Therefore, the Kalman filter is aimed at the linear system, and the noise of the system is required to be Gaussian white noise. However, in practical engineering applications, many systems are nonlinear, and the Kalman filter algorithm is not effective in dealing with these nonlinear systems. In order to make Kalman filter applicable to nonlinear systems, extended Kalman filter is derived. At present, when the state equation is determined, EKF has become the de facto standard in the industry for state estimation of nonlinear systems. The extended Kalman filter first uses Taylor’s first-order expansion to linearize the state equation and measurement equation of the nonlinear system, and then uses the partial derivative matrix (Jacobian matrix) of the equation to update the covariance matrix.

Researchers at Xiamen University draw on the idea of time division and design a TDOA algorithm based on time division [12], as shown in Fig. 5. Each UWB positioning base station is assigned a time slot. Each base station can only broadcast TODA data packets when it is its time slot, otherwise it will continue listening. When the base station finishes sending data, it will automatically switch to the receiving state, and then it will update the time slot when it receives broadcast data packets from other base stations or when the reception times out (a base station does not send packets in its time slot). By allocating time slots and making reservations for receiving and sending, we have achieved time synchronization between different base stations to ensure the reliability of information transmission. After the drone receives the data packet broadcast by the base station, it first calculates the TDOA value, and then combines the information from the IMU sensor to use EKF for data fusion to calculate the positioning coordinates. Each time EKF is executed, the average value of the accelerometer and gyroscope values of the IMU sensor will be obtained (this is because the update period of the IMU is greater than the update period of the EKF), and then use the state equation to predict the movement state of the drone based on these data, calculate covariance, and then introduce process noise to the covariance matrix. Since TDOA is geometrically the difference between the two sides of a triangle, the wrong data can be filtered out by using the relationship between the sides of the triangle. Then use the measurement equation of TDOA to calculate the Jacobian matrix, calculate the extended Kalman gain, and update the state information, and finally update the error covariance, enter the next calculation, and so on.

Fig. 5.
figure 5

UWB and IMU fusion positioning algorithm

Full size image

3.4 Fusion Positioning of Lidar and Inertial Navigation

Lidar navigation is usually achieved through synchronous positioning and composition technology. After years of development, the Lidar SLAM method [13] has been widely used in the field of robotics. Lidar is divided into two-dimensional lidar and three-dimensional lidar. Limited by factors such as volume, load, and cost, micro-small UAVs usually use two-dimensional lidar for SLAM.

At present, LIDAR SLAM often uses scan matching method to estimate the carrier’s pose. The so-called scan matching is to register the current environmental laser scan points with reference scan data. As far as ground robots are concerned, the methods for estimating pose through two-dimensional lidar scan matching are quite mature, but for UAVs with six degrees of freedom, the use of these scan matching methods has greater limitations. On the one hand, there is a change in the attitude of the aircraft during the flight, which causes the environmental information detected by the lidar to be out of the same plane at certain moments, which may cause errors in SLAM positioning. On the other hand, because the two-dimensional radar can only scan a plane of the environment, and the aircraft is moving in the height direction, the two-dimensional environmental structure scanned by the two-dimensional lidar will undergo abrupt changes, and the reference scan data in the matching will be compared with the current. The scanning points are not in the same plane, which results in large errors in matching. The SLAM with the aid of inertial information proposed by researchers from Nanjing University of Aeronautics and Astronautics can perform reliable and accurate navigation and positioning in a three-dimensional environment [14], as shown in Fig. 6.

Fig. 6.
figure 6

Inertial navigation and lidar fusion positioning method

Full size image

First, use IMU to predict the attitude quaternion, velocity, position, accelerometer zero offset and gyroscope zero offset of the UAV at the current time according to the state of the UAV at the previous moment. Then the pose estimation is performed according to the projection points of the lidar scanning points on the same horizontal plane, so as to compensate the pose estimation errors caused by the scanning points not on the same horizontal plane caused by the attitude movement of the aircraft. And use IMU information to predict the initial value of the UAV’s pose. At the same time, in order to avoid changes in altitude during the flight of the aircraft, the two-dimensional environmental structure sensed by the two-dimensional lidar mounted on the aircraft will undergo abrupt changes. This mutation will cause a large error in the pose estimated by the scan matching, and it needs to be detected and eliminated by combining the information of the IMU and the scan matching. Subsequently, the overlap between the current scan data and the reference scan data is used for registration, the pose is estimated, and the UAV state is updated according to the EKF model. Finally, make a three-position composition.

Compared with the traditional method, the inertial assisted lidar mainly adds three modules: state prediction, mutation detection and filter update. Its main advantages are:

  1. 1.

    Use IMU information to compensate for the influence of carrier attitude changes on lidar scanning, and perform scan matching on the basis of IMU prediction of carrier attitude, which speeds up the matching iteration process and can avoid the matching process from falling into the local optimum to a certain extent.

  2. 2.

    The mutation detection link is introduced to detect whether the Lidar sensing environment has a sudden change, and the update strategy of the filter and reference scan data is determined according to the detection result, which can overcome the problem of large or even impossible pose estimation errors when the two-dimensional environment structure changes suddenly. Therefore, we can realize precise and robust navigation.

  3. 3.

    A method is proposed to detect whether a sudden change occurs in the sensing environment of the lidar, combining the IMU state prediction information and the scanning matching result for accurate detection.

  4. 4.

    Filtering using the posture estimation result of scan matching and the predicted state, and updating the reference scan data and constructing a three-dimensional map according to the updated state of the filter, with higher positioning and composition accuracy.

4 Challenges and Future Development Trend

In the actual working environment, the positioning of drones will be restricted by various factors.

  1. 1.

    In the actual working environment, the positioning of drones will be restricted by various factors. The limited load of the UAV has greatly restricted the configuration of the sensor and the computing power of the onboard processor, which limits the performance of all aspects of the positioning system. For micro-rotor drones, their load generally does not exceed 5 kg, and there are not many sensors that can be mounted on them. These low-mass sensors have certain limitations in range, range, and accuracy, which greatly limits the working airspace and working environment of the positioning system.

  2. 2.

    Each positioning method has its own advantages and disadvantages.

  3. •

    Using the image data collected by the visible light vision sensor for positioning can obtain a large amount of environmental information at a very small load cost, but it is very dependent on the lighting conditions of the environment and the richness of textures.

  4. •

    The IMU is mainly used to measure carrier acceleration and angular velocity information, but if only the IMU is used for positioning, there will be movement drift after a period of time.

  5. •

    UWB positioning uses the TDOA method and does not rely on synchronization between base stations, but requires a large number of base stations and is sensitive to occlusion in the channel.

  6. •

    Although lidar does not rely on illumination and is accurate in two-dimensional positioning, for a six-degree-of-freedom UAV, there is a large positioning error.

  7. 3.

    The fusion model itself has certain limitations [15]. At present, the most commonly used model in the data fusion of UAV positioning is the EKF model, but when the prediction function and update function of this model are nonlinear, (especially in SLAM problems), EKF cannot guarantee the global optimum. If the estimated value at the time when the processing of the previous frame is completed is not yet accurate, the prior information transmitted to the next frame will contain errors. Since the state of the previous frame does not change anymore, the error in the prior information cannot be eliminated, and the error is continuously transmitted backwards causing error accumulation.

In order to solve these problems in the positioning of drones in the GPS denial environment, the future development direction may have the following aspects:

  1. 1.

    The introduction of new positioning methods. It includes the invention of new positioning technology and the miniaturization of existing positioning technology, making it easy to carry on drones.

  2. 2.

    Reduction of errors in traditional positioning methods. For example, reduce the degree of dependence of visual positioning on ambient light, or reduce the time accumulation error of IMU.

  3. 3.

    The update of the fusion model of multiple positioning technologies. For example, Mourikis improved EKF in 2007 and proposed MSCKF [16] to better solve the SLAM problem.

  4. 4.

    In addition, deep learning, which has developed rapidly in recent years, can be used to further improve the accuracy of positioning. Or use the idea of a priori map to further eliminate errors.

5 Conclusions

With the development of science and technology and the diversification of demand, UAVs have been applied in more and more occasions. In many cases, only relying on GPS for positioning can no longer meet the positioning needs of different scenarios. Focusing on the positioning technology of drones in GPS denial environments, this paper mainly discusses ground-based binocular vision positioning, fusion positioning of airborne vision/lidar/inertial navigation and UWB base station, analyzes the current challenges in UAV positioning and future development trends. Since the existing positioning method is limited by its own robustness and the loading conditions of the UAV itself, fusion positioning is required. The introduction of new positioning methods, the optimization of traditional positioning methods, and the updating of fusion models will always be valuable research directions in this field.